JPH06325298A - Device for monitoring vehicle periphery - Google Patents

Abstract

PURPOSE:To provide a vehicle periphery monitoring device which is improved so as to surely detect a luminescent point even if distortion aberration exists in an image-pickup machine lens and to easily detect an obstacle. CONSTITUTION:A pattern optical projector 3 projects a luminescent matrix on a projecting surface in a square grid shape as against incident laser light. The image-pickup machine image-pickups the luminescent pattern of the luminescent matrix. A data processing part 10 adds luminescent locating direction data corresponding to lens aberration to respective kinds of luminescent data which are obtained from a flat road surface and from the image-pickup machine, scanning and locating are executed based on luminescent locating direction data at the time of locating a luminescent obtained from the inspection road surface and an image is processed so that the existance of the obstacle, a groove, a person, etc., is detected.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery monitoring device which is effectively applied to monitor the periphery of a vehicle such as an automobile and assist the driver's safety confirmation in driving the vehicle.

[0002]

2. Description of the Related Art As a device for monitoring the periphery of a vehicle such as an automobile, a grid-like bright spot projected by a pattern light projector is imaged by an image pickup device, and a bright spot pattern projected on a flat road surface and an inspection road surface are projected. Obstacles and the like are detected based on the deviation from the bright spot pattern projected on the screen.

[0003]

As described above, in the conventional vehicle periphery monitoring device, the bright spot pattern projected on the flat road surface is obtained by capturing the image of the grid-like bright spots projected by the pattern light projector. An obstacle or the like is detected based on the deviation of the bright spot pattern projected on the inspection road surface.
In order to expand the monitoring range of obstacles, etc., it is necessary to change the lens attached to the imaging device to a wide-angle lens, and when changing to a wide-angle lens, distortion of the lens occurs, and the bright spot position increases with distance from the center. Change and no longer form a grid pattern.

For this reason, when there is no distortion, the comparison between the bright spots projected on the flat road surface and the bright spots projected on the inspection road surface is searched in the horizontal direction, and an obstacle or the like is three-dimensionally measured from the difference. However, if there is distortion, there is no bright spot even when searching in the horizontal direction, and a three-dimensional obstacle cannot be detected. An object of the present invention is to provide a vehicle periphery monitoring device improved so that a bright spot can be reliably detected even if a lens of an image pickup device has distortion and an obstacle can be easily detected.

[0005]

In order to solve the above-mentioned problems, a vehicle periphery monitoring device according to the present invention is provided with a pattern light projector for projecting a bright spot pattern onto a surveillance region by incidence of laser light, and a bright spot pattern for the bright spot pattern. It is characterized by including an image pickup device for picking up an image and a data processing unit for processing an image obtained from the image pickup device to detect the presence of an obstacle, a groove, a person or the like.

The data processing unit extracts a bright spot pattern from pixel data based on an image signal from the image pickup device that picks up a bright spot pattern projected on a flat road surface, and creates reference data including each bright spot coordinate. Reference data creation means, bright spot search direction adding means for adding bright spot search direction data for each bright spot of the reference data, and image signals from the image pickup device that picks up the bright spot pattern projected on the inspection road surface It has a detection means for detecting the presence of an obstacle, a groove, a person or the like by comparing the bright spots extracted by scanning the pixel data based on the bright spot search direction data with each bright spot of the reference data. It has a feature.

[0007]

In the above structure, the pattern light projector projects a bright spot matrix in a square lattice pattern on the projection surface (in the embodiment, the ground is shown as a monitoring area) in response to the laser light incident.
The reference data creation means extracts the bright spot pattern projected on the flat road surface picked up by the image pickup device, and creates reference data including each bright spot coordinate.

The bright spot search direction adding means adds data indicating the direction in which the bright spot search is performed for each bright spot created by the reference data creating means. In the detection means, the pixel data of the image signal of the image of the bright spot pattern projected on the inspection road surface is scanned based on the bright spot search direction data, and the extracted bright spot is compared with each bright spot of the reference data to cause an obstacle. Detects the presence of objects, grooves, people, etc.

[0009]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing an embodiment of a vehicle periphery monitoring device according to the present invention, in which 1 is a laser light source, 2 is a laser light source driving device for driving a laser light source 1, and 3 is a laser light from the laser light source 1. 1a is input and a pattern light projecting element for projecting a bright spot matrix on a monitoring area by the inputted laser light 1a, 4 is a CCD camera as an image pickup device for photographing the bright spot matrix, and 5 is obtained from the CCD camera 4. A frame memory that temporarily stores a video signal, 6 is a data processing unit configured by a computer that operates according to a predetermined program, and 7 is image data in the case of a flat ground with no obstacles or the like stored in advance as reference data. Reference data storage unit, 8 is a rudder angle detector that detects the steering angle of the steering wheel, 9 is a buzzer that emits an alarm sound, and 10 is a display device.

As the pattern light projecting element 3 shown in FIG.
Fiber grating (FG) 1 as shown in (a)
3 or a multi-beam projector (MBP) 23 as shown in FIG. 2B is applied.

The FG 13 shown in FIG. 2A has a diameter of several tens μ.
About 100 optical fibers each having a length of 10 mm and a length of 10 m are arranged in a sheet shape, and two sheets of the optical fibers are stacked orthogonally to each other. When the laser light 1a generated by the laser light source 1 is incident on the FG 13, the laser light is condensed at the focal points of the individual optical fibers and then spreads as a spherical wave while interfering, resulting in a square lattice on the projection surface. The bright spot matrix 14 is projected in a uniform manner.

The MBP 23 shown in FIG. 2 (b) is a thin transparent plate in which a large number of microlenses are integrated. The laser light 1a incident from the laser light source 1 becomes a multiple beam by the MBP 23 and has a square lattice pattern on the projection surface. The bright spot matrix 14 is projected onto the.

With the above configuration, the pattern light projecting element 3
The bright spot matrix 14 projected onto the monitoring area by the (FG 13 or MBP 23) is imaged by the CCD camera 4, and the video signal obtained from the CCD camera 4 is temporarily stored in the frame memory 5 and then stored in the data processing unit 6. It is captured. The data processing unit 6 compares the image data obtained from the frame memory 5 with the reference data stored in the reference data storage unit 7 in advance to calculate the movement amount of the bright spot on the image plane 4b of the CCD camera 4. Ask for.

FIG. 3 shows the optical arrangement when the lens in the embodiment described above with reference to FIG. 1 has no distortion, and CC
With the lens 4a of the D camera 4 as the origin, the pattern light projecting element 3 is placed at the position of the distance d on the y-axis, and the image plane 4
b is arranged at the position of the distance I on the Z axis. In this optical arrangement, the monitoring area 4c (field of view of the CCD camera)
Is a flat ground surface (road surface) without obstacles and the like, a point P _n
The bright point projected on (X _n , Y _n , 0) is the monitoring area 4c.
Due to the existence of the object O inside the point P _B on the object O
It is projected on (X _B , Y _B , Z _B ). As a result, the image plane 4 of the CCD camera 4 that takes an image of this bright spot
On b, point P _n in FIG. _{(X n, Y n, 0} ) corresponding points A (u, v) is the point _{P B (X B, Y B} , Z B) points corresponding to B ( u, v + δ). That is, the bright spot moves in a fixed direction.

Therefore, the movement amount δ is detected by obtaining the distance between the position of the point A and the position of the point B. The data processing unit 6 also uses the distances d and I, the distance h from the monitoring area 4c to the y axis, the CCD camera optical axis, and the monitoring area 4c.
The three-dimensional position of the bright spot [point P _B (X _B , Y _B , Z _{B in} FIG. 3)] is detected by performing calculation processing using the angle θ formed with the normal line of To do. Then, the three-dimensional position is detected for all the bright points in the input image, and the approximate size and position of obstacles, grooves, people, etc. are detected by performing arithmetic processing on the bright points whose three-dimensional position has changed, Display is performed on the display device 10 according to the detection result.

FIG. 4 shows a sensor section S composed of a pattern light projecting element 3, a laser light source 1 and a CCD camera 4.
An example of installation in the case where the vehicle is fixed at an angle θ with respect to the normal to the ground at the rear of the vehicle is shown.

The processing in the data processing unit 6 outlined above is roughly divided into a standard screen data (reference data) creation processing and an obstacle detection processing shown in the flowcharts of FIGS. 5 and 6.

In the reference data creating process, as shown in the flowchart of FIG. 5, in step S1, C
The image signal of the bright spot matrix 14 captured and output by the CD camera 4 is input, and this is converted into pixel data having 512 × 512 pixels and a brightness of 0 to 255 gradations and temporarily stored in the frame memory 5. The pixel data temporarily stored in the frame memory 5 is first subjected to bright point extraction in the data processing unit 6 in step S2. Then, in step S3, a process of determining the coordinate position of the center of gravity of the bright spot is performed. After that, in step S4, a process of determining the brightness of the bright spot, that is, a process of determining a threshold value for extracting the bright spot at the time of inspection is performed. Step S
In 5, the bright spot search direction data applied when the lens has distortion is added to each bright spot of the reference data. In step S6, background luminance data required for brightness correction is obtained.

The bright spot extraction processing in step S2 is as shown in FIG.
In order to extract the bright spot by comparing the brightness on one scanning line as shown in FIG. 8 of the bright spot projection image in the monitoring area as shown in FIG. For each pixel data of 5, if the gradation value of the pixel is larger than a preset threshold value, the value is left, and if smaller, the gradation value of the pixel is set to zero. The above process is 1
This process is performed for all the pixels by shifting them pixel by pixel, and a cluster (bright spot) of pixels as shown in FIG. 9 is extracted by this processing.

If there is a large difference in luminance between the bright spots and the bright spots cannot be extracted with a fixed threshold value, the brightness shown in FIG.
The average brightness value in a window of m × n pixels centered on a certain pixel as shown in Fig. 3 is obtained, this average value is used as a threshold value to decide whether or not to leave that pixel, and the same applies to other pixels. It suffices to obtain a threshold value by processing and extract a bright spot with a different threshold value for each pixel.

Next, the process of step S3, that is, the process of determining the coordinate position of the center of gravity of the bright spot will be described. In this determination processing, for example, the barycentric position (U, V) of the bright point as shown in FIG. 11 is obtained by weighting the brightness from the coordinates of the bright point of each pixel in the bright point to obtain the barycentric position.

The subsequent processing of step S4, that is,
The process of determining the brightness (luminance threshold) of the bright spot will be described. In this processing, for example, the minimum value of the pixels forming the bright spot is set as the brightness of the center of gravity of the bright spot. At the bright spot shown in FIG. 11, since I (min) = 50, the brightness of the bright spot centroid is 50.

Next, in the process of step S5, bright spot search direction data is added. When the lens of the CCD camera 4 has no distortion aberration, the square lattice shown in FIG. 12A is imaged, and a square lattice without distortion is obtained as a captured image. However, if the lens has distortion, distortion occurs in the image pickup screen as shown in FIGS. 12 (B) and 12 (C). Therefore, the data of the search direction when searching for the bright spot obtained from the inspection road surface, which will be described later, is calculated and added from the bright spot position of the reference data.

As shown in FIG. 13, the bright spot search direction data shows that for a bright spot P (u ₁ , v ₁ ), the aberration curve passing through the point P is a quadratic curve u = av ² + u ₀ . (1) However, a = ku ₀ is expressed by (2).

Therefore, by differentiating the equation (1), u '= 2av = 2ku ₀ v (3) is obtained, and the tangent tangent T _P of the tangent line at the point P (u ₁ , v ₁ ) is obtained.
Is expressed by T _P = 2ku ₀ v ₁ (4) ≈ 2ku ₁ v ₁ (5)

Therefore, in step S5, the tangential direction is calculated by performing the calculation process of equation (5) for each bright point, and the number of pixels until one pixel is shifted in the vertical direction when the pixels are sequentially moved in the horizontal direction. Is added as bright spot search direction data.

By carrying out the above-mentioned processing, FIG.
As shown in FIG. 4, the final standard luminescent spot data (reference data) consisting of the luminescent spot centroid of each luminescent spot No., its brightness, and background data (position, brightness) is obtained, and this is stored in the reference data storage unit 7. Remembered. By executing the flow chart of FIG. 5 described above, the data processing unit 6 determines each bright spot coordinate, each brightness threshold value from the pixel data based on the image signal from the image pickup device that picks up the bright spot pattern projected on the flat road surface. Create reference data including search direction data and background data.

In the obstacle detection process, as shown in the flow chart of FIG. 6, first, in step S11, a process of fetching reference data from the reference data storage unit 7 is performed. Then, in a succeeding step S12, a correction process for the height change of the sensor is performed. Then, step S1
After performing the process of capturing the image in step 3, step S
Proceeding to 14, the detection / correction processing of the change with respect to the brightness of the background is performed on the captured image. Further, the process proceeds to step S15, and the moving spot is extracted based on the bright spot search direction data. Subsequently, in step S16, calculation processing of the three-dimensional coordinate position of each spot, step S1
Processing for displaying the monitoring area in step 7, processing for fetching the steering angle value in step S18, processing for predicting the vehicle path in step S19, processing for predicting collision with an obstacle in step S20, step S2
The display process is performed in 1 and the process returns to step S13.

The correction process for the height change of the sensor in step S12 is performed for the following reason. That is, the reference data consisting of the three-dimensional coordinates on the image plane 4b used together with the distance h to measure the above-mentioned three-dimensional position point P _B (X _B , Y _B , Z _B ) is fetched. In consideration of the reproducibility of the height h of the vehicle and the like, the coordinate is used when there is no heavy object on the vehicle. For this reason, reference data composed of three-dimensional coordinates on the image plane 4b of each bright spot on the flat road surface is stored and used as a reference value at each measurement. Then, in the three-dimensional measurement of the uneven road surface in the actual running state, the three-dimensional coordinates of each bright spot are obtained from the deviation of the bright spot coordinates at the time of measurement with respect to the reference value of each bright spot obtained without placing a heavy object at all. become.

However, in actual driving, an occupant, luggage, and the like ride on the vehicle, and the total weight of these is several tens kg to several hundreds kg, and as a result, the vehicle sinks several cm. In addition, the vehicle does not sink uniformly, but the vehicle sinks at a considerable angle due to the arrangement of heavy objects.

Since the perimeter monitoring device is fixed to the vehicle as described above with reference to FIG. 4, the monitoring area 4c on the road surface is approached as the vehicle sinks due to the load. In this case, when viewed from the surroundings monitoring device, the area 4c located at the distance h from the CCD lens 4a is in a state of protruding upward by the amount of the sinking of the vehicle, so that the surroundings monitoring device indicates that the entire road surface is protruding by several cm. to decide. Here, if the vehicle-mounted weight is large, the vehicle sinks too much, and as a result, the monitoring device gives an erroneous alarm because there is a convex surface of a height that is problematic for vehicle traveling.

In order to solve such a problem, the amount of road protrusion due to the vehicle sinking may be first measured on a road surface which is considered to be substantially flat in a state where an occupant, luggage, etc. are on. To this end, as shown in FIG. 15, the road surface protrusion amounts of the bright spots at the four corners are obtained from the moving amounts of the bright spots 1, 2, 3 and 4 at the four corners of the detection area, and from this, the straight line approximation of each bright spot is performed. The amount of protrusion can be calculated. These detected amounts of protrusion are generated due to the sinking of the vehicle, and serve as correction values when obtaining the three-dimensional coordinates of the actual road surface unevenness.

At the time of actual measurement, the three-dimensional coordinates of the road surface obtained from each bright spot are corrected (subtracted) by using the above-mentioned correction values to obtain accurate three-dimensional coordinates in which the subduction of the vehicle is corrected. I want it.

In the above-mentioned example, the correction values are obtained for the bright spots 1 to 4 at the four corners of the monitoring area 4c. However, if the inclination of the vehicle is smaller than the sinking of the vehicle, one point in the monitoring area 4c, for example, The road surface height of the point 5 can be obtained and this value can be used as a correction value for all bright points. In any case, the three-dimensional coordinates obtained from several bright points on a flat road surface with an occupant or the like mounted thereon may be used as a correction value for the vehicle subsidence during measurement.

The correction process for the height change of the sensor in step S12 is specifically executed by the flowchart of FIG. In the flowchart of FIG. 16, when it is detected that the door is closed and the ignition switch is turned on in steps S12a and 12b, it is determined in step S12c whether the correction data acquisition switch is turned on, and the determination is NO.
In case of, it is confirmed in step S12d that the vehicle speed is between 5 km / h and 40 km / h.

As a result of this confirmation, the vehicle speed was 5 km / h and 40
If it is between km / h and step S12e, it is calculated whether or not the monitoring area is flat in accordance with the amount of movement of the bright spots at several places, and if it is determined in step S12f that it is flat,
In step S12g, vehicle subduction correction data is calculated from the amount of movement of the bright spots at several locations. When the correction data acquisition switch is turned on and the determination in step S12c is YES, the process proceeds to step S12g, the vehicle subduction correction data is calculated from the bright spot movement amounts of several places, and then the process proceeds to the subsequent step. By executing the flowchart of FIG. 16, the data processing unit 6 functions as a height correction unit that corrects the reference data according to the height change of the image pickup device from the road surface.

As a timing of fetching the correction value, the driver may confirm the flat road surface and perform a manual operation such as a switch operation. Further, as described above, the air resistance may affect the vehicle. If the processing device can determine that the road surface detected by the monitoring device is substantially flat when traveling at a constant speed at a low speed (for example, 5 to 40 km / h) that does not give a value, in that case, even if the processing device automatically captures the correction value. Good. In addition to this, when the processing device can determine that the road surface is flat, the correction value may be loaded at that time.

In the image capturing process in step S13, the bright spot projection image in the monitoring area 4c as shown in FIG. 7 is captured. One scanning line segment of the image signal captured and output by the CCD camera 4 is shown in FIG. 8. However, since the brightness of the bright spot and the background are not uniform, the brightness threshold value for detecting the bright spot is the same. The threshold value (value different for each bright point) is shown by the broken line in the figure. This brightness threshold value is stored in advance as a part of the reference data as described above.

The detection of the change in the background brightness of the image and the correction process of the luminance threshold value in the step S14 are performed for the following reason. Now, when trying to detect an obstacle or the like by capturing a luminance projection image to obtain the amount of movement of the bright spot, the bright spot must first be extracted from the captured image. At this time, if the inside of the monitoring area 4c is illuminated by the light of the brake lamp of the own vehicle or the headlights of another vehicle,
As shown in FIG. 17, the brightness distribution changes, and it becomes impossible to extract bright points with the brightness threshold value stored in advance. In this case, the obstacle cannot be detected or a large detection error occurs.

In order to solve such a problem, the brightness of the pixel corresponding to the background position previously recorded in the reference data is obtained when the bright spot projection image is captured and the movement amount of the bright spot is obtained. Next, a correction coefficient for the brightness threshold value is calculated from the obtained background brightness values of several points. The brightness threshold value of the reference data is corrected based on this correction coefficient, and the bright spot is extracted.

Specifically, for example, as shown in FIG.
Of the bright spots in the monitoring area 4c, the brightest spot on the upper left and the bright spot on the lower right are focused, and the following formula is used based on the luminance data of the pixels (marked by x) that are considered to be the background around these two bright spots. As shown in (6), the correction coefficient A (luminance gradient in the u coordinate direction) is obtained. The brightness threshold value of each bright spot of the reference data is corrected by the following expression (7). _{A = (I RL -I LU)} / (U RL -U LU) ......... (6) Th '(i) = Th (i) + A (U (i) -U LU) + (I LU -I BK ) ………… (7)

In the above formula, A: correction coefficient u _LU : u coordinate of a predetermined pixel around the leftmost bright point I _LU : luminance of a predetermined pixel around the left uppermost bright point u _RL : around the right lowermost bright point U _RL of a predetermined pixel of ‘I _RL’ : the brightness of a predetermined pixel around the bottom rightmost bright point Th ′ _(i) : a brightness threshold after correction of the i th bright point Th _(i) ： the i th bright point Before correction of u _(i) : u coordinate of i-th bright spot I _BK : Background brightness at the time of reference data acquisition

By executing step S14 described above, the data processing unit 6 determines the brightness of the background other than the bright spot from the pixel data based on the image signal from the image pickup device that picks up the bright spot pattern projected on the inspection road surface. The brightness threshold value for extracting the bright spot of the pixel data is corrected according to the amount of change of the background brightness of the reference data.

The subsequent extraction of the moving spot by the bright spot search in step S15 is performed for each bright spot by the corrected luminance threshold Th ′ obtained for each bright spot. Then, in step S16, the movement amount of each bright spot is obtained from the extracted bright spot coordinates and the bright spot coordinates of the reference data, and the three-dimensional position of the obstacle is calculated. A general interpolation method or the like can be used as the method of calculating the correction coefficient in step S14 described above. Further, if all the bright spots are corrected using the brightness data of the background around each bright spot, the processing time becomes slow, but the correction can be performed with high accuracy.

The coordinates of each bright spot on the image plane 4c can be found by finding the barycentric coordinates of each bright spot by the pixels constituting the bright spot extracted as described above. The calculation of the coordinates of each bright spot is applied both when obtaining the reference value on a flat road surface and when performing three-dimensional measurement on an uneven road surface or the like. The calculation of the bright spot coordinates is performed by all pixels (for example, 512
X512) Since the output is the calculation target, the calculation time becomes long. There is no problem in taking a reference value for a flat road before measurement, but there is a problem that the measurement speed is slow during measurement.

In the geometric layouts shown in FIGS. 3 and 4, the coordinates of each bright spot on the image plane 4b are the points A (u, v) in the case of a flat road surface when there is no lens distortion. On the other hand, if there is unevenness on the road surface, the bright spot coordinates B
(U, v + δ) moves only in a fixed direction. In the case of the figure,
The moving direction of the bright spot is only the v direction, and v when the road surface is convex
In the positive direction, and in the negative direction of v in the case of the concave. When the lens has an aberration, the lens moves in the direction indicated by the equation (4) or (5). By utilizing this property, the calculation of the three-dimensional coordinates can be speeded up as follows.

In FIG. 19, A (u, v) indicates a point corresponding to a bright point on the flat road surface. The frame F shows the scanning area at the time of measuring the three-dimensional coordinates, and the sub-areas + S ₁ and −S ₂ can be set based on the size of the road surface unevenness which becomes a problem in the actual traveling state.
For example, in the case of an ordinary passenger car, 1.5 m above the flat road and 0 below the road.
It can be set to 5 m, etc., and + S ₁ and -S ₂ can be set accordingly. It should be noted that all the points shown in the figure represent the center of gravity of each bright point, and the distance between adjacent bright points in the v direction overlaps even if the bright points move under the above conditions. Is set to never happen. Further, in the figure, in order to effectively utilize the space, each bright spot is arranged in the u direction in every other pattern.

Next, at the time of measurement, for each bright spot,
The bright spot coordinates may be obtained only for the scanning area. The bright spot coordinates may be obtained by calculating the center of gravity or the geometric center of the bright spots, for example, assuming that pixels having a predetermined threshold value or more form the bright spots.

Specifically, when there is no lens distortion, as shown in FIG. 20, V ₁ and V ₂ are detected on the same line as the bright spot barycenter of the reference data, and the detected V is detected.
_The bright center of gravity (V ₁ + V ₂ ) / 2 is obtained from ₁ and V ₂ , and the amount of movement ΔV of the bright spot is obtained from the distance between the centers of gravity. In addition,
If there is no bright spot on the same line, the upper and lower lines are scanned. Further, the detection of V ₁ and V ₂ is performed by detecting the brightness threshold I of the reference bright spot. The width W of the scanning region may be one pixel line, or may be several pixel lines, for example, 5 pixel lines in consideration of improvement in detection accuracy.

The bright spot search when there is distortion is
After searching for the search direction data in the reference data on the same line as the center of gravity of the bright spot, the search is performed for the search direction data in the v direction again by shifting by one pixel in the u direction, and the bright spot is searched by repeating this search. To do. In the above example, the barycentric table is replaced with the geometrical center coordinates in order to increase the detection speed, but in order to improve the accuracy, the barycentric coordinate detection method at the time of creating the reference data is preferable.

As described above, at the time of measurement, it is only necessary to obtain the luminance coordinate for the set scanning area, so that the measurement can be speeded up. Further, FIG. 21 shows the monitoring area 4c.
The three-dimensional position detection result of the bright spot in the case where there is a wall and a groove within shows an example of displaying on the display device 10, in FIG., O ₁
Is a wall and O ₂ is a groove.

Next, a monitoring method when the steering angle detector 8 is used will be described. In the embodiment, the steering angle detector 8
The moving path of the vehicle is obtained by predicting the course of the vehicle using the steering angle detected by, and the vehicle is considered as an obstacle by superimposing the trajectory on the two-dimensional map of the monitoring area 4c where obstacles are shown. The contact or collision is detected in advance, and a warning sound such as a buzzer sound or a warning message formed by a voice synthesizing unit configured in the data processing unit 6 is generated in the speaker 9 as a sounding unit,
The driver is warned by causing the display device 10 to display.

FIG. 21 shows a point where the edge of the car body Bo of an automobile comes into contact with an obstacle such as the wall O ₁ and the groove T of the tire T.
An example in which the point falling at ₂ is displayed on the display device 10 is shown. In the backward movement of the vehicle shown in FIG. 20, the data processing unit 6 processes the images successively obtained from the CCD camera 4, whereby the obstacle O ₁ and the groove O ₂ in the monitoring area 4c are detected almost in real time, In addition, the driver's safety confirmation can be effectively supported by warning in advance of contact with an obstacle or the like by predicting the route of the vehicle using the steering angle obtained from the steering angle detector 8.

In the illustrated embodiment described above, the detection signal of the steering angle of the steering wheel by the steering angle detector 8 is used to obtain the locus of the automobile, but instead of this, a rotational angular velocity sensor such as a gyro is used. A direction sensor or the like may be provided, signals from these sensors may be processed to detect the traveling direction of the automobile, and the course thereafter may be predicted.

Further, in the above-mentioned embodiment, the case where the driver of the automobile is informed of the existence of the obstacle, the groove and the like in the monitoring area has been described. However, it can be applied to the automatic guided vehicle in the factory or the industrial robot in the assembly process. The present invention can be applied.

[0056]

As described above, according to the present invention,
When there is distortion in the lens of the imager, the bright spot search direction data is added to each bright spot obtained by imaging the flat road surface, so when searching for a bright spot obtained from the inspection road surface. Further, if the scan is performed based on the bright spot search direction data, the search can be easily performed, and the obstacle can be easily detected in a short time.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of a vehicle periphery monitoring device according to the present invention.

FIG. 2 is a diagram showing a fiber grating and a multi-beam projector as pattern light projecting elements.

FIG. 3 shows details of the optical arrangement of the device of FIG.

FIG. 4 is a pattern projecting device, a laser light source, of the apparatus of FIG.
It is a figure which shows the example of installation of the sensor part comprised by a CCD camera.

5 is a flowchart showing one process performed by a data processing unit in FIG. 1. FIG.

FIG. 6 is a flowchart showing another processing performed by the data processing unit in FIG.

FIG. 7 is a diagram showing a bright spot projection image in a monitoring area.

FIG. 8 is a diagram showing a luminance distribution on one scanning line.

FIG. 9 is a diagram showing pixel data of bright spots obtained by the bright spot extraction processing.

FIG. 10 is a diagram showing pixel data in a frame memory.

FIG. 11 is a diagram for explaining how to obtain a bright spot centroid.

FIG. 12 is an explanatory diagram when a lens has aberration.

FIG. 13 is an explanatory diagram of a bright spot search direction data calculation method.

FIG. 14 is a diagram showing created reference data.

FIG. 15 is a diagram for explaining a method of height correction.

FIG. 16 is a flowchart showing a specific process of a height correction process.

FIG. 17 is a diagram for explaining the necessity of brightness correction.

FIG. 18 is a diagram for explaining a method of brightness correction.

FIG. 19 is a diagram illustrating a method of setting a scanning area.

FIG. 20 is a diagram illustrating a method of obtaining a bright spot centroid and a moving distance in a scanning region during three-dimensional coordinate measurement.

FIG. 21 is a diagram showing a display example of a three-dimensional position detection result of a bright spot.

FIG. 22 is a diagram showing an example of a display of a result of carrying out peripheral monitoring of a vehicle.

[Explanation of symbols]

1 laser light source 3 pattern light projector 4 imager 6 data processing unit (reference data creation means, detection means,
Height correction means, brightness correction means) 8 Steering angle detector (track detection means) 9 Buzzer, voice synthesizer (speaker) 10 Display device

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] August 10, 1993

[Procedure Amendment 1]

[Document name to be amended] Statement

[Name of item to be corrected] 0027

[Correction method] Change

[Correction content]

By executing the above-described flowchart of FIG. 5, the data processing unit 6 obtains each bright spot coordinate and each luminance from pixel data by the image signal from the image pickup device which picks up the bright spot pattern projected on the flat road surface. Create reference data including threshold, search direction data, and background data. This reference
The data is stored in the reference data storage unit 7.

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0028

[Correction method] Change

[Correction content]

In the obstacle detection process, as shown in the flow chart of FIG. 6, first, in step S11, a process of fetching reference data from the reference data storage unit 7 is performed. Then, in a succeeding step S12, a correction process for the height change of the sensor is performed. Then, step S1
After performing the process of capturing the image in step 3, step S
Proceeding to 14, the detection / correction processing of the change with respect to the brightness of the background is performed on the captured image. Further, the process proceeds to step S15, and the moving spot is extracted based on the bright spot search direction data. Subsequently, in step S16, calculation processing of the three-dimensional coordinate position of each spot, step S1
Detection result calculated in S16 in the monitoring area in 7
Is displayed , the steering angle value acquisition process is performed in step S18, the vehicle course prediction device is performed in step S19, the collision prediction process with an obstacle is performed in step S20, the display process is performed in step S21, and the process returns to step S13.

[Procedure 3]

[Document name to be amended] Statement

[Correction target item name] 0050

[Correction method] Change

[Correction content]

The bright spot search when there is distortion is
After searching for the search direction data in the reference data on the same line as the center of gravity of the bright spot, the search is performed for the search direction data in the v direction again by shifting by one pixel in the u direction, and the bright spot is searched by repeating this search. To do. In the above example, the barycentric coordinate is replaced by the geometrical center coordinate in order to increase the detection speed, but in order to improve the accuracy, the barycentric coordinate detecting method at the time of creating the reference data is preferable. ─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] August 10, 1993

[Procedure Amendment 1]

[Document name to be corrected] Drawing

[Name of item to be corrected] Figure 5

[Correction method] Change

[Correction content]

[Figure 5]

[Procedure Amendment 2]

[Document name to be corrected] Drawing

[Name of item to be corrected] Figure 6

[Correction method] Change

[Correction content]

[Figure 6]

Claims (1)

[Claims] 1. A pattern light projector for projecting a bright spot pattern on a monitoring area by laser light incidence, an image pickup device for picking up the bright spot pattern, and an image pickup device for picking up the bright spot pattern projected on a flat road surface. Reference data creating means for creating a reference data including each bright spot coordinate by extracting a bright spot pattern from pixel data based on the image signal of the above, and a bright spot search for adding bright spot search direction data for each bright spot of the reference data. Direction adding means and each of the bright spots and the reference data extracted by scanning the pixel data based on the image signal from the image pickup device that picks up the bright spot pattern projected on the inspection road surface based on the bright spot search direction data. A vehicle periphery monitoring device comprising: a data processing unit having a detection unit that detects the presence of an obstacle, a groove, or a person by comparing bright spots.