JPH07264577A - Vehicle periphery monitoring device - Google Patents

Abstract

PURPOSE:To immediately recognize the position of an obstacle by marking an object interfering with vehicle traveling on an image in a monitoring area. CONSTITUTION:When there is no object on a road, a luminescent point(LP) obtained from a photographed spot is previously recorded in a reference data recording part 13. When image pickup is executed by a camera 11, an LP extracting part 14 successively reads out respective picture elements recorded in a frame memory 12, compares the picture elements with a prescribed threshold and extracts a low LP part. Then the extracting part 14 calculates the center-of-gravity position of luminance of each picture element surrounded by low luminance picture elements and records the center-of-gravity position as an LP position. Then a moving LP extracting part 15 compares the LP position recorded in the recording part 13 with a position obtained by photographing the periphery of a vehicle and records a moving LP. An object position calculating part 16 calculates the three-dimensional position of the moving LP, and when an obstacle judging part 17 judges the existence of danger of collision, an obstacle marking part 18 marks an obstacle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery monitoring device which is effectively applied to monitor the periphery of a vehicle such as an automobile and assist the driver's safety confirmation in driving the vehicle.

[0002]

2. Description of the Related Art A conventional vehicle peripheral device is disclosed in Japanese Patent Application No. 4-17.
As described in No. 0559, the position of the object is calculated from the amount of movement of the bright spot position where the same spot as the captured bright spot of the spot illuminated on the road surface is reflected by the object, and the presence of an obstacle I was trying to detect.

That is, as shown in FIG. 5, a floodlight mounted on a vehicle irradiates a large number of spots on a road surface where no object exists as shown in FIG. 6A, and the illuminated spots are imaged by a camera. Then, as shown in FIG. 6B, when the road surface on which the object exists is imaged, the imaged spot position changes with respect to the spot irradiated on the object.

The position of the object is calculated from the amount of change in the spot position and displayed on the display unit as shown in FIG.
Further, in addition to this, a camera is installed in place of the floodlight, the position of the object is calculated from both image positions picked up by two left and right cameras, and the object position is displayed on the display unit as shown in FIG. It was

[0005]

As described above, the conventional vehicle periphery monitoring device has a height higher than the amount of movement of the bright spot position of the spot imaged by the camera or the difference between the image positions imaged by both cameras. The position of the object is calculated, and the obstacle that hinders the traveling of the vehicle is displayed on the display unit using a graphic figure.

For this reason, the display on the display unit is different from the image of the actual monitoring area, so that the safety confirmation is not sufficient, so that the graphic figure and the captured image are switched and monitored. It took a long time to recognize which one of the captured images the object displayed as a figure corresponds to.

The present invention provides a vehicle periphery monitoring device improved so that an object which interferes with the traveling of a vehicle can be marked in an image obtained by picking up a monitoring area displayed on a monitor so that the position of the obstacle can be immediately recognized. The purpose is to

[0008]

Means adopted by the present invention for solving the above-mentioned problems will be described with reference to FIG. FIG. 1 is a basic configuration diagram of the present invention. Image pickup means (1) for picking up an image of a monitoring area around the vehicle, and the image pickup means (1)
Display means (2) for displaying the image captured by the image capturing means, object position calculating means (3) for calculating the position of the object from the image data captured by the image capturing means (1), and the object position calculating means (3 ) Obstacle determining means (4) for determining whether or not the object calculated in () interferes with the running of the vehicle, and when the obstacle determining means (4) determines that the object will interfere with the running of the vehicle, Obstacle marking means (5) for marking an obstacle in the captured image displayed on the display means (2).

Further, the image pickup means (1) picks up an image of a plurality of spots illuminated by a light projector. Further, for the image displayed by the display means (2), a bandpass filter that passes light having the wavelength of the irradiation light of the spot is removed from the image pickup path, and the image is picked up by the image pickup means (1).

Further, the image pickup means (1) picks up an image of the monitoring area with two left and right cameras.

[0011]

The image pickup means 1 picks up an image of the monitoring area around the vehicle. The object position calculation means 3 calculates the position of the object from the image data picked up by the image pickup means 1.

The obstacle determining means 4 determines whether or not the object calculated by the object position calculating means 3 interferes with the running of the vehicle. When the obstacle determination means 4 determines that the traveling of the vehicle is an obstacle, the obstacle marking means 5 marks an obstacle object in the image captured by the image capturing means 1 and displayed on the display means 2.

When the image pickup means 1 picks up an image of the spot irradiated by the light projector to calculate the object position, the image pickup means 1 is used.
The band-pass filter installed at is removed outside the imaging path and the imaged image is displayed on the display unit 2 to mark an obstacle object. As described above, since the object that interferes with the traveling of the vehicle in the image displayed by capturing the image of the monitoring area around the vehicle is marked, it is possible to immediately recognize the position of the obstacle. Therefore, the vehicle can be driven safely and smoothly.

Further, when the method of calculating the object position by imaging the spot irradiated by the projector, the band-pass filter is removed and the imaged image is displayed.
A clear image is displayed and the marked obstacle can be easily recognized.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described with reference to FIGS. FIG. 2 is a block diagram of the first embodiment of the present invention, and FIG. 3 is an operation flowchart of the same embodiment. Figure 2
In FIG. 10, 10 is a projector, 11 is an imaging camera, 12 is a frame memory, 13 is a reference data recording unit, 14 is a bright spot extraction unit, 15 is a moving bright spot extraction unit, 16 is an object position calculation unit,
Reference numeral 17 is an obstacle determination unit, 18 is an obstacle mark unit, 19 is a display unit, 20 is a band filter, 21 is a filter insertion / removal unit that inserts or removes the band filter 20 into or out of the imaging path, and 22 performs other processing. A processing unit, 23 is an interface (I / O), and 24 is a processor (CPU) that executes all processes.

The projector 10 is composed of a laser light source 10b for emitting a laser beam and a fiber grating (FG) 10a, and irradiates a grid-shaped laser beam spot on a monitoring area around the vehicle. As shown in FIG. 4, the FG 10a is configured by arranging about 100 optical fibers each having a diameter of several tens of μm and a length of 10 mm in a sheet shape, and stacking these two sheets at right angles.

When laser light is incident on the FG 1a from the laser light source 10b as indicated by an arrow, the laser light is condensed at the focal points of the individual optical fibers and then spreads as a spherical wave while interfering with each other. A grid-like spot is irradiated on the projection surface. The camera 11 captures an image of the spot illuminated by the light projector 10, and the captured image signal is temporarily recorded in the frame memory 12.

The bandpass filter 20 installed on the front surface of the camera 11 allows the light emitted by the laser light source 10b of the projector 10 to pass, for example, light having a wavelength near 830 nm in the infrared region. The reason why the bandpass filter 20 is installed in front of the camera 11 is that when the camera 11 captures an image of the spot illuminated by the projector 10, sunlight or illumination light is incident on the camera 11 in addition to the spot, and the spot is masked. Like The bandpass filter 20 passes the light of the radiated spot, removes the sun and illumination light, and improves the S / N ratio.
It is installed to ensure that the bright spots of the spot can be extracted.

Further, as shown in FIG. 5, the projector 10 and the camera 11 have an angle θ with respect to a normal line to the ground at the rear of the vehicle.
It is fixedly installed at. FIG. 6 shows an image of the illuminated spot taken by the camera 11, and FIG.
Shows the case where there is no object on the road surface, and FIG. 6 (B)
Indicates the case where there is an object.

When the object is illuminated with the spot emitted from the projector 10, the spot position of the image captured by the camera 11 moves. That is, in a place where there is no object, the spot position indicated by a circle in FIG. 6A and the black circle in FIG.
The spot positions indicated by the marks match, but when the object is irradiated with the spots, the O mark positions and the ● mark positions do not match.

The vehicle periphery monitoring device calculates the object position based on the amount of movement of the positions marked with a circle and the positions marked with a circle. In the reference data recording unit 13, as shown in FIG. 6A, the bright spot position obtained from the spot imaged when no object is present on the road surface is recorded in advance. The bright spot position is determined by the bright spot extraction unit described below.

First, the bright spot extracting section 14 sequentially reads each pixel in the horizontal direction (V axis) recorded by the camera 11 and recorded in the frame memory 12, and based on the background luminance, as shown in FIG. When the brightness is lower than the threshold value that is determined, the brightness I of the pixel is set to O.

When the above process is performed for all the pixels recorded in the frame memory 12, only the imaged portion of the illuminated spot can be extracted as shown in FIG. Note that FIG. 8 shows one spot as a representative. Then, the bright spot extraction unit 14 determines that the brightness value is ◯ as shown in FIG.
For the pixels surrounded by, the brightness I is weighted at the coordinate position of each pixel to calculate the barycentric position, and the calculated barycentric position is recorded in a memory (not shown) as a bright spot position.

In the reference data recording unit 13, the bright spot positions which are picked up by the bright spot extracting unit 14 and which are obtained by imaging the road surface without any object are recorded in advance. Moving bright spot extraction unit 15
Compares the position of the bright spot recorded in the reference data recording unit 13 with the position obtained by imaging the periphery of the vehicle, and records the bright spot whose position is moving in a memory (not shown).
That is, a bright spot obtained by capturing an image of the spot illuminated on the object among the spots illuminated by the projector 10 is extracted.

The object position calculation unit 16 calculates the position of the object from the bright spot position. FIG. 9 is an explanatory diagram for calculating the object position of the object position calculation unit 16, and FIG. 9A shows X, Y and Z with the origin at the point where the optical axis of the lens of the camera 11 intersects the road surface. It is expressed in Cartesian coordinates and is shown in FIG.
(B) is represented by the Y and Z axis planes.

Further, L is the focal length of the lens of the camera 11, D is the distance between the center of the lens and the projector 10, and θ is shown in FIG.
The installation depression angle of the camera described above, H is the height on the road surface at the center of the lens. Further, a point P (x, y, z) is a point where the spot from the light projector 10 is applied to the object, and a point P (u, v + δ).
Is a bright spot obtained by the camera 11 capturing a spot P (x, y, z) irradiated on the object, and a point P _A (x _A , y _A ,
0) is the spot position where the road surface is illuminated when there is no object,
_A point P _A (u, v) indicates a bright spot position obtained by irradiating the point P _A (x _A , y _A , 0).

As is clear from FIG. 9A, the projector 1
When the object is irradiated with a spot irradiated from 0, the bright spot position moves in the v-axis (horizontal direction) direction from the bright spot position obtained when there is no object, that is, when the spot is irradiated on the road surface. Also, when a spot is irradiated at a point lower than the road surface, such as a groove, the spot moves in the opposite direction.

Therefore, the point P (x, y, z) is z = z ′ cos θ + y ′ sin θ−y ″ sin θ cos θ y = y ′ / cos θ−z tan θ x = (v + δ) (h ′) according to the principle of triangulation. -z ') / L ··· (1 ) ^{However, z' = h '2 δ} / (DL + Hδ) y' = u (h'-z ') / L h' = H / cosθ + y "sinθ y" = H (Tan φ−tan θ) φ = θ + tan ⁻¹ (u / L) (2), the object position calculation unit 16 calculates the object positions x, y, and z by the equation (1).

The operation of the first embodiment will be described below with reference to FIG. In process S1, the processing unit 22 instructs the filter insertion / removal unit 21 to insert the bandpass filter 20 into the imaging path on the front surface of the camera 11. In process S2,
The camera 11 images the monitoring area around the vehicle, and the image data imaged in step S3 is recorded in the frame memory 12.

In process S4, the bright spot extraction unit 14 extracts the bright spot position from the brightness value recorded in the frame memory 12 by the above-described process and records it in a memory (not shown).
In the process S5, the moving bright spot extraction unit 15 checks the positions of the bright spots irradiated on the road surface recorded in the reference data recording unit 13 and the bright spots extracted in the process S4, and there is a position shift. The bright spots are recorded in a memory (not shown).

In process S6, the processing unit 21 determines whether or not there is a bright spot recorded as having been moved in position in process S5. If the determination is NO, the process proceeds to process S1 and YES.
In case of, it moves to the processing S7. In process S7, the object position calculation unit 16 calculates the three-dimensional position of the object corresponding to the moved bright point by performing the calculation process of the equation (1) as described above.

In step S8, the obstacle determining section 17 determines the I / O ratio.
The steering angle of the vehicle is taken in from O23 to predict the vehicle traveling path. In process S9, the obstacle determination unit 17 determines whether there is a danger of collision based on the object position calculated in process S7 and the route prediction predicted in process S8. If the determination is NO, the process proceeds to process S1. At the end, if YES, process S1
Move to 0 and warn with a buzzer (not shown).

In step S11, the obstacle mark unit 18 instructs the filter insertion / removal unit 21 to remove the bandpass filter 20 outside the imaging path. In step S12, the camera 11 captures an image of the monitoring area and records it in the frame memory 12.

Since the band-pass filter 20 has been removed in the process S11, the image captured in the process S12 becomes a natural captured image. The emission of the laser light source 10b may be stopped at the same time when the bandpass filter is removed in the process S11. When the laser light source 11b stops emitting light,
No spot appears in the image captured in step S12, resulting in a more natural image.

In step S13, the obstacle mark portion 18
The image recorded in the frame memory 12 is displayed on the display unit 19
To display. In step S14, the obstacle mark portion 18
Regarding the object determined to be in danger of collision in the process S9,
Obstacle marks are added as shown in FIG.

As a method of marking an obstacle, the process S
In step 12, the image data value recorded in the frame memory 12 is changed or superposed on the recorded image data. In addition, for marking, as shown in FIG. 10, a circle mark or a rod shape in which these marks are connected is used to mark, and the mark color is changed according to the height of the obstacle, and the size of the detected obstacle.・ Various methods such as marking with shape can be considered.

FIG. 10 shows an example in which an obstacle mark of O is attached to the bright spot position where the spot shown in FIG. 6B has moved. Next, a second embodiment of the present invention will be described with reference to FIGS. FIG. 11 shows the second aspect of the present invention.
FIG. 12 is a configuration diagram of this embodiment, and FIG. 12 is an operation flowchart of the embodiment.

In FIG. 11, 31 and 32 are cameras for picking up images around the vehicle, 33 and 34 are frame memories for temporarily recording the image signals picked up by the cameras 31 and 32, and 35 is recorded in the frame memories 33 and 34. The distortion aberration correction unit that corrects the distortion aberration of the lenses of the cameras 31 and 32 and records the image data in the memory (right) 38 and the memory (left) 39, respectively. Road surface image removing unit for removing the road surface image from the stored image signal, 3
Reference numeral 7 denotes an object edge detection unit that detects the edge of the object.

Further, 40 is an object position calculation unit for calculating the position of an object, 41 is an obstacle determination unit, 42 is an obstacle mark unit, 43 is a display unit, 44 is a processing unit for performing other processing, and 45 is an interface. (I / O) and 46 are processors (CPU) that execute processing.

First, the distortion correction unit 35 will be described. When the lenses of the cameras 31 and 32 have distortion, the image signals captured by the cameras 31 and 32 are
The road surface image removing unit 36 and the object edge detecting unit 3 described below
7. An error occurs in the processing result of the object position calculation unit 40.

That is, when there is no aberration in the lens of the camera or when a lattice pattern is imaged, lattice image data is recorded in the frame memory as shown in FIG. 13 (A). However, in the case where the lens has aberration, FIG.
Alternatively, a distorted lattice-shaped image is recorded as shown in (C).

The amount of aberration increases in proportion to the cube of the distance of the point P from the center of the optical axis. That is, as shown in FIG. 13D, when the point P (x ₀ , y ₀ ) when there is no aberration is imaged at the point P ′ (x, y) due to the aberration, the aberration amount D is D ^{_{= [(x 0 -x) 2}} + (y 0 -y) 2] 0.5 ··· (3) , and this D is proportional to the cube of the distance to the point P from the center point of the lens. That is, the ^{_{[(x 0 -x) 2 +}} (y 0 -y) 2] 0.5 = k [(x 0 2 -y 0 2) 0.5] 3 ··· (4) where k is a proportional constant.

Therefore, when aberration is present in the lens, for example, x ₀ ≈x (1-k (x ² + y ² )) y ₀ ≈y (1-k (x ² + y ² )) (5 ) Is performed to correct distortion.

The distortion correction unit 35 is provided in the frame memory 3
The pixel data recorded in 3 and 34 are subjected to the calculation of the equation (5) to correct the lens aberration and recorded in the memories 38 and 39, respectively. In addition, when the calculation of the equation (5) is performed to correct the pixels of the frame memory, data omission occurs in the corrected memories 38 and 39. Interpolation is performed for such pixels without data based on the data of adjacent pixels.

Similar to the first embodiment, the cameras 31 and 32 are installed at a height H above the road surface at the rear of the vehicle to image the rear of the vehicle, as shown in FIG. Therefore, in order to facilitate the subsequent description, as shown in FIG. 14, the coordinates at the camera installation point are represented by X ′, Y ′, and Z ′, and the coordinates on the road surface are represented by X, Y, and Y. To do. Therefore, when the installation depression angle θ _{S of the} camera is 90, X = X ′, Y = Y ′ + H, Z = Z ′ (6)

The optical axes of the lenses of the cameras 31 and 32 are
As shown in FIG. 15, both optical axes are aligned with the Z ′ axis, and are installed at a distance L. First, the object position calculation unit 30
A method of calculating the object position in will be described. In FIG. 15, the X ′ axis is taken in the direction connecting the center points of the lenses of both cameras, and the cameras 31 and 32 are installed so that the X ′ axis is parallel to the road surface.

The Y'axis is an axis perpendicular to the Z'axis and the X'axis, and is an axis perpendicular to the road surface. In order to facilitate the following description, the O point on the X ′, Y ′ and Z ′ axes is the camera (left) 3
Take the center of the second lens. Images are taken by the cameras 31 and 32 thus installed, and the memories 38 and 3 are taken.
The recording of the point P (X _P ′, Y _P ′, Z _P ′) recorded at 9 is P _L (x _LP , y _LP ) and P, respectively.
_{If R} (x _RP , y _RP ), then the distance Z _P ′ to the point P is Z _P ′ = Lf / (x _LP −x _RP ) ... (7) where L is the distance between both lenses and f is the lens It is expressed by the focal length of.

Therefore, the object position calculator 30 reads x _LP from the memory 39 and x _RP from the memory 38 to calculate Z _P ′. In addition, the point P (X _P ′, Y _P ′,
Z _P ') of the Y _P', from FIG. _{15, Y P '= Atan Φ} L = Z P' tan Φ L / sin θ L = Z P 'y LP / f ··· (8) = Ly LP / (X _LP −x _RP ) ... Represented by (9).

From FIG. 15, the point P (X _P ′, Y _P ′, Z _P ′) of X _P ′ is as follows: X _P ′ = A cos θ _L = Z _P ′ cos θ _L / sin θ _L = Z _{P 'x LP / f ··· (10} ) = Lx LP / (x LP -x RP) represented by (11).

As shown in FIG. 5, when the camera is oriented θ _S downward from the vertical direction, correction is necessary.
When the depression angle of the camera is θ _S , it becomes as shown in FIG. Position X of point P represented by X ', Y'and Z'coordinates
_P ', Y _P' and Z _P 'is as shown in Figure 14 is represented by a road surface coordinate X, Y and Z.

Therefore, equations (7), (9) and (1
Z calculated in 1)_P′, Y_P'And X_P'Using
The value of the point P represented by X, Y and Z coordinates is X._P, Y_POh
And Z _PThen, X_P= X_P′ ・ ・ ・ (12) Y_P= H-Z_P′ Cos θ_S+ Y_P′ Sin θ_S (13) Z_P= Z_P′ Sin θ_S+ Y_P′ Cos θ_S (14)

When the left and right imaging angles of the camera are θ _R , correction is performed by rotating θ _R with the Y axis as the rotation axis. The object position calculation unit 30 calculates the object position by performing the calculations of Expressions (12) to (14).

Next, the road surface image removing section 36 will be described. When a white line or a character is drawn on the road surface, the white line or the character is determined to be an object and the distance is calculated. But,
White lines and characters on the road surface do not hinder the running of the vehicle at all, and such an object having a height of 0 can be removed in advance to simplify the object position calculation process.

The principle of removing the road surface image will be described with reference to FIG. FIG. 16A shows a right image captured by the camera 31 and recorded in the memory 38. FIG. 16 (A)
In the figure, 30a and 30b represent white lines drawn on the road surface, and 30c represents an object (pole).

Therefore, it is assumed that all the right images recorded in the memory 38 have a height of 0, that is, an image drawn on the road surface. An image obtained by projecting the right image on the left image is calculated by assuming that the left camera 32 has captured the right image (FIG. 16B). When the projection image calculated in this way is superimposed on the left image (memory 39), it becomes as shown in FIG.

That is, when the image picked up by the right camera 31 is projected, the pattern such as a white line drawn on the road surface coincides with the position picked up by the left camera 32 in terms of position and brightness, and the object is viewed from the road surface. The higher the height, the larger the difference. Therefore, as shown in FIG. 16D, the brightness value of the pixels forming the road surface other than the pixels forming the object having the height is 0 or close to 0 by obtaining the difference between the left image data and the projection image data. When the value equal to or smaller than the predetermined threshold value is set to 0, all the values become 0. As a result, only the height portion is extracted as a value other than 0, and the road surface image can be removed.

Then, next, the right image is thrown with all the heights set to 0.
A method of calculating the shadow image will be described. Point P on the right image
_R(X_RP, Y_RP), The point of the projection image corresponding to_L′ (X
_LP′, Y _LP′)

As shown in FIG. 7, the X ′ axis of the camera coordinates and the X axis of the road surface coordinates are parallel to each other, and the x axis of the sweep line imaged by the camera (x _L axis and x _R axis in FIG. 15). ) Are also parallel, the y _L and y _R values of the captured image when the same object is captured are the same.

Therefore, y _LP ′ = y _RP (15) Further, x _LP ′, if all of the image is on the road surface, the value of Y _P shown in equation (13) is 0, and 0 = H _P −Z _P ′ cos θ _S + Y _P ′ sin θ _S (16)

Therefore, Z _P ′ and Y _P ′ in equation (16) are
Substituting Z _P ′ in equation (7) and Y _P ′ in equation (9) into x _LP ′, x _LP ′ = (Lfcos θ _S −Ly _RP sin θ _S ) / H + x _RP ... (17)

The road surface image removing unit 36 calculates the projected image (FIG. 16B) by performing the operations of the equations (15) and (17). After the projection image is created, the difference from the left image, that is, the data recorded in the memory 39 is obtained, and the road surface image is removed (FIG. 16D).

Next, the object edge detector 37 will be described. In FIG. 17A, the left camera 32 images the white lines 30a and 30b and the object (pole) 30c on the road surface drawn on the road surface similar to that described in FIG.
9 shows the image data recorded in No. 9.

The brightness value I _{m, n} of the image data of m rows and n columns recorded in the memory 39 is swept in the horizontal direction, and if | I _{m, n + 1} −I _{m, n} | ≧ E ₀ If E _{m, n} = 1 | I _{m, n + 1} −I _{m, n} | <E _0, then E _{m, n} = 0 (18) where E ₀ is a threshold value and differentiated Obtaining the image, FIG. 17 (B)
As shown in, the vertical edge portion of an object or a character drawn on the road surface is "1", and the other portions are "0".

The differential image thus obtained (see FIG. 17)
(B)) and the difference image (FIG. 16 (D)) obtained by the road surface image removing unit 16 described above are overlapped and the AND operation is performed, and only the edge portion of the object shown in FIG. 17 (C) is extracted. .
Therefore, the calculation processing can be significantly reduced by calculating the three-dimensional position of the object with respect to the edge portion of the extracted object.

Next, the operation of the embodiment will be described with reference to FIG. In process S21, the processing unit 44 determines that the camera 31
The image data captured by and 32 are recorded in the frame memories 33 and 34, respectively. In step S22, the distortion aberration correction unit 35 uses the cameras 31 and 32 for the data recorded in the frame memories 33 and 34.
Correcting the aberration of the lens of,
It is recorded in the memories 38 and 39.

In the process S23, the object edge detector 37
Is not shown by sweeping the left image data recorded in the memory 39 in the horizontal direction and performing the calculation shown in Expression (18) to create a differential image of the left image (FIG. 17B). Record in memory. In the process S24, the road surface image removing unit 36
Creates a projection image (FIG. 16 (B)) obtained by projecting the right image recorded in the memory 38 into a left image on the assumption that the right image is an image on the road surface and records it in a memory (not shown).

In the process S25, the road surface image removing unit 36
A difference image obtained by subtracting the projection image data created in step S24 from the left image recorded in the memory 38 (see FIG. 16).
(D)) is created and recorded in a memory (not shown). In process S26, the object edge detection unit 37 uses the differential image (FIG. 17B) created in process S23 and the difference image (FIG. 16D) created in process S25 to represent an object edge image ( FIG. 17C is created and recorded in a memory (not shown).

In the process S27, the processing section 44 determines the process S2.
For the edge portion of the object edge image data created in 6, the corresponding points of the left and right images of the object to be measured are extracted from the memories 38 and 39. In process S28, the object position calculation unit 30 calculates the object position corresponding to the point extracted in process S27.

In process S29, the obstacle determining unit 41 determines whether or not there is an obstacle from the processing result of process S28.
If the determination is NO, the process proceeds to step S30, and the image recorded in the frame memory 33 or 34 is displayed on the monitor (not shown). In step S31, the obstacle determination unit 41 determines that the I
The steering angle signal is read via / O45, and the process proceeds to step S32 to estimate the trajectory of the vehicle on the basis of the read steering angle.

In the process S33, the obstacle determining unit 41 determines whether or not the vehicle may collide with the detected obstacle by comparing it with the traveling locus in the process S32, and the determination is Y.
If it is ES, the process proceeds to step S34. If there is no possibility of collision, the process proceeds to step S30 to display the captured image on the monitor and then proceeds to process S21.

In the process S34, the obstacle judging section 41 sounds a buzzer to give an alarm. In step S35, the obstacle mark unit 42 displays the image recorded in the frame memory 33 or 34 on the display unit 43. In process S36, the obstacle mark unit 42 adds an obstacle mark to the object determined to have a possibility of collision in process S33 as shown in FIG. 10, and proceeds to process S21.

The obstacle marking method is the same as that described in the first embodiment. The process S3
The image in 5 is displayed using the image data recorded in the frame memory 34, and the object edge (FIG. 17C) detected by the object edge detection unit 37 in step S23 is superimposed on the display image. By doing so, marking of obstacles can be easily performed.

In the embodiment described above, the projection image is created from the right side image and superimposed on the left side image, but the projection image may be created from the left side image and superimposed on the right side image.

[0074]

As described above, according to the present invention, the following effects can be obtained. Since the object that interferes with the traveling of the vehicle in the image displayed by imaging the monitoring area around the vehicle is marked, it is possible to immediately recognize the position of the obstacle, safely and The vehicle can be driven smoothly.

Further, when the method of calculating the object position by imaging the spot irradiated by the projector, the band-pass filter is removed and the imaged image is displayed.
A clear image is displayed and the marked obstacle can be easily recognized.

[Brief description of drawings]

FIG. 1 is a basic configuration diagram of the present invention.

FIG. 2 is a configuration diagram of a first embodiment of the present invention.

FIG. 3 is an operation flowchart of the same embodiment.

FIG. 4 is an explanatory diagram of fiber grating of the projector of the embodiment.

FIG. 5 is a diagram showing an installation example of a floodlight and a camera of the embodiment.

FIG. 6 is an explanatory diagram of imaging spots.

FIG. 7 is a diagram showing a luminance distribution on one scanning line of a captured image signal.

FIG. 8 is an explanatory diagram for calculating a bright spot position from a captured image signal.

FIG. 9 is an explanatory diagram for calculating an object position from a bright spot position.

FIG. 10 is a diagram showing a display example.

FIG. 11 is a configuration diagram of a second embodiment of the present invention.

FIG. 12 is an operation flowchart of the embodiment.

FIG. 13 is an explanatory diagram of lens aberration correction of the same example.

FIG. 14 is an explanatory diagram of camera depression angle correction of the same embodiment.

FIG. 15 is an explanatory diagram of three-dimensional position measurement of the same example.

FIG. 16 is an explanatory diagram of road surface image removal according to the embodiment.

FIG. 17 is an explanatory diagram of object edge detection according to the same embodiment.

FIG. 18 is a diagram showing a conventional display example.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image pickup means 2 Display means 3 Object position calculation means 4 Obstacle determination means 5 Obstacle mark means 10 Projectors 11, 31, 32 Cameras 10a Fiber grating (FG) 10b Laser light source 12, 33, 34 Frame memory 13 Reference data recording Part 14 Bright spot extraction part 15 Moving bright spot extraction part 16,40 Object position calculation part 17,41 Obstacle determination part 18,42 Obstacle mark part 19,43 Display part 20 Bandpass filter 21 Filter insertion removal part 22,44 Processing Part 23, 45 Interface (I / O) 24, 46 Processor (CPU) 35 Distortion aberration corrector 36 Road surface image remover 37 Object edge detector 38, 39 Memory

Claims (4)

[Claims] 1. An image pickup unit for picking up an image of a surveillance area around a vehicle, a display unit for displaying an image picked up by the image pickup unit, and an object position for calculating a position of an object from image data picked up by the image pickup unit. Calculating means, an obstacle determining means for determining whether or not the object calculated by the object position calculating means interferes with the traveling of the vehicle, and the obstacle determining means for determining with the obstacle for traveling of the vehicle In this case, the vehicle periphery monitoring device is provided with: obstacle marking means for marking an object that obstructs the captured image displayed on the display means. 2. The vehicle periphery monitoring device according to claim 1, wherein the image pickup means picks up an image of a plurality of spots illuminated by a light projector. 3. The image displayed by the display means is imaged by the image pickup means by removing a bandpass filter that passes light having the wavelength of the irradiation light of the spot from the image pickup path. The vehicle periphery monitoring device according to claim 2. 4. The vehicle periphery monitoring device according to claim 1, wherein the image capturing means captures an image of the monitoring area with two left and right cameras.