JP3123303B2 - Vehicle image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle image processing apparatus for recognizing an environment around a vehicle. Such a vehicle image processing device is, for example, a preceding vehicle detection device applied to an automatic traveling control device for a vehicle or an approach warning device for another vehicle, that is, detects a position of a preceding vehicle traveling ahead of the own vehicle. It is used for devices that perform

[0002]

2. Description of the Related Art As a conventional image processing apparatus for a vehicle, for example, there is one as shown in FIG. In FIG.
(A) is an image processing apparatus for a vehicle that uses a single image sensor to take in as much information as possible around the vehicle and recognizes the information. Also, (b) is an image processing apparatus for a vehicle that uses two image sensors, installs them at a predetermined angle and distance, and extracts the distance, direction, and feature to the object from the parallax of each information. It is.

[0003]

The above-mentioned conventional image processing apparatus for a vehicle has the following problems. That is, since the image of a distant object is small, it is difficult to extract its features. Since all the captured images are processed, the amount of information is large. As a result, the processing becomes complicated, the processing speed becomes slow, and the scale of the processing device becomes large, which makes it difficult to use as vehicle equipment. In addition, since the distance to an object cannot be measured accurately in a running vehicle, even if image information is obtained, it cannot be used effectively to warn the driver or control the running of the vehicle. There was a problem.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems of the prior art, and an image processing apparatus for a vehicle which can perform appropriate image processing and can be practically realized for a vehicle. The purpose is to provide.

[0005]

Means for Solving the Problems In order to achieve the above object, the present invention is configured as described in the claims. That is, according to the first aspect of the present invention, an image sensor that is mounted on a vehicle and inputs image information around the vehicle and a distance measurement that obtains information on the distance and direction to one or more objects existing outside the vehicle. Means and the presence of the object based on the distance and direction information
For determining the periphery of the region to be processed as the region for image processing
Image processing of any of the above objects
Determining means for determining and setting an image processing method including any one of processing to be determined as an object; and image information obtained by the image sensor based on the image processing method set by the determining means. Image processing means for performing processing. In addition, the above-mentioned determination means is, for example,
As described in claim 2, enlargement, reduction, redirecting images
An image processing method including at least one of the focus adjustment processes is determined and set. Also,
For example, the image sensor may include an enlargement / reduction / direction change / focus set by the determination means.
Detailed image processing in the image is required depending on the point adjustment processing.
It is for enlarging, reducing, turning, and adjusting the focus of a predetermined object. The invention according to claim 4, in addition to the above configuration, comprising a measuring means for detecting a traveling state of the vehicle, said determining means is a car according to a detection result of the measuring means
Judgment of both driving environments
It is obtained by configured to determine a defined object as an object for performing an image processing. The invention according to claim 5 shows a specific configuration in a case where the image processing device for a vehicle is applied to a preceding vehicle detection device.

[0006]

As described above, in the present invention, the judgment means determines the image processing method based on the measurement result of the distance measurement means. The image processing method includes, for example, a region for performing image processing around an area where an object exists , as described in claim 1.
Processing to be determined as an area or any of the above objects
Any of the processes that determine
One of the processes, or as described in claim 2, enlarges the image.
Small, large, shrink, change direction, focus adjustment processing
This is one process . The image processing means performs image processing based on the set image processing method.
Therefore, the process of extracting image information is focused on an important object that requires image processing, for example, an object that hinders traveling of the own vehicle, and the range in which data processing is performed by the image processing means is limited. Therefore, the number of data to be handled is greatly reduced, and the processing speed is increased, so that the feature extraction of the object to be watched can be executed at high speed. According to a third aspect of the present invention, in an apparatus for enlarging / reducing / redirecting / focusing a predetermined object that requires detailed image processing in an image by an image sensor, an optimal image processing is performed. Since an image is obtained, detailed information which cannot be obtained from a small amount of pixel data, for example, information for identifying a license plate of a vehicle can be obtained from the image processing means. Further, as in the invention as set forth in claim 4, there is provided a measuring means for detecting a running state of the own vehicle, and the judging means judges and judges the running environment of the vehicle according to the detection result .
Image processing of a predetermined object for the driving environment
In what determines the elephant was, if there are a plurality of objects in front of the vehicle, which object can be selected whether to watch in accordance with the running state of the vehicle. Also,
According to a fifth aspect of the present invention, there is provided an image processing area setting means for detecting a distance and a direction to the reflector by the reflector detecting means, and setting a processing area in the image processing recognizing means according to the result. The image processing area is determined according to the distance and direction to the body. As a result, the position of the preceding vehicle can be reliably detected, and the region for performing image processing can be limited to a narrow range, so that calculation processing can be performed in real time without using an ultra-high-speed computer. It becomes possible.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a first embodiment of the present invention. In FIG. 1, a distance measuring means 60 is a radar using, for example, light, radio waves, ultrasonic waves, and the like, and can obtain information on the distance to an object and direction. Some of the devices that use light measure the distance by combining a laser and a photo sensor and measuring the elapsed time until the laser light is reflected back to the object. If the emitted laser light is scanned up, down, left, and right, the distance in that direction can be measured, so that information on the distance and direction can be obtained. Even those using radio waves or ultrasonic waves can be realized with almost the same configuration as described above. The determining means 61 is a means for determining and setting an image processing method based on the measurement result of the distance measuring means 60. Note that the above image processing method is
In this embodiment, it is a process of determining an area for image processing. The image sensor 62 is means for inputting image information around the vehicle, and is, for example, a video camera device using a CCD. The image processing means 63 is means for performing, for example, pattern recognition. The image processing means 63 and the determination means 61 can be constituted by, for example, a computer. S1 is a distance / direction signal, S1
2 is an image signal, S3 is a judgment signal, and S4 is a feature extraction signal.

Next, FIGS. 2A and 2B are diagrams showing the positional relationship between the vehicle and an object outside the vehicle, wherein FIG. 2A is a top view and FIG. 2B is a side view. In FIG. 2, reference numeral 64 denotes a vehicle, and 65 denotes an object outside the vehicle. FIG. 3 is a chart showing an example of a distance measurement result,
FIG. 4 is a flowchart showing a processing procedure in the apparatus of FIG. The operation of the embodiment of FIG. 1 will be described below with reference to FIGS. First, an object 65 existing in front of the vehicle
Is measured by distance measuring means 60 (not shown in FIG. 2) provided at the front end of the host vehicle 64. The distance measuring means 60 calculates θx in the lateral direction of the vehicle and θx in the height direction.
When y is set (see FIG. 2), data indicating the distance for each direction can be obtained on the matrix L (θx, θy). The example of the distance measurement result illustrated in FIG. 3 is a result in a case where the object 65 is in the right direction as illustrated in FIG. The judging means 61 estimates the position of the object based on the measurement result. This estimation method is performed, for example, as follows. That is, for all L (θx, θy), the values of adjacent L (θx ′, θy ′) are compared. Then, at the boundary between the place where the object exists and the place where the object does not exist, L (θx, θy) and L (θx ′, θy ′)
Are greatly different from each other, which indicates that an object exists at that position. In the example described above, it is estimated that an object is present in the right direction. Further, the judgment means 61 sets an image processing method based on the above result. In the case of this embodiment, a process of determining a region to be subjected to image processing is used as an image processing method. Therefore, the determination means 61 determines the area where the image processing is to be performed around the object. The determination method at this time is to multiply the estimated size of the object by a certain appropriate value to obtain a region larger than the object. Next, the image processing means 63 sets an area for image processing based on the data sent from the determination means 61 with respect to the image information ahead of the vehicle obtained by the image sensor 62, and focuses on the area. Perform data processing. In the example above,
Image processing is mainly performed on the rightward object. As described above, the determination means 61 detects the distance and direction of an object that hinders the travel of the host vehicle, and performs processing for mainly extracting image information for the object. Since the range in which data processing is performed is limited, the number of data to be handled is significantly reduced, the processing speed is increased, and the feature extraction of the object to be watched can be performed at high speed.

Next, FIG. 5 is a block diagram showing a second embodiment of the present invention. In this embodiment, the present invention is applied to an image processing apparatus that requires detailed image information on an object. In FIG. 5, the distance measuring means 66 is the same as that in FIG. Further, the determining means 67 estimates the distance and direction to the object as in the case of FIG.
Further, a signal for controlling the image sensor 68 is output. The image sensor 68 is an image sensor that can control enlargement, reduction, focus, and rotation of image information according to a control signal given from the determination unit 67. As such an image sensor, a video camera device having a lens having a zoom mechanism and a mechanism capable of rotating the entire camera can be used. The image processing means 69 is a detailed image processing means for identifying a license plate of a vehicle, for example. S1 is a distance / direction signal, S2 is an image signal, S
3 is a judgment signal, S4 is a feature extraction signal, and S5 is an enlargement / reduction / direction change signal. In the environment shown in FIG. 2, an object exists rightward. In the distance measuring means 66, distance data is obtained on a matrix as in FIG. The judging means 67 determines, based on the above result, the first
As in the embodiment, the distance and direction to the object are estimated, and the pan, tilt, zoom, and focus adjustment mechanisms of the image sensor 68 are controlled. For example, when the data shown in FIG. 3 is obtained, the camera is moved to the right and the focus is adjusted according to the distance. By performing such processing, an optimal image for image processing can be obtained. Therefore, detailed information which cannot be obtained from a small amount of pixel data, for example, information for identifying a license plate of a vehicle can be obtained from the image processing means 69. The detailed information obtained by the image processing means 69 includes:
There are the brake lamp of the preceding vehicle, the lighting of the direction indicator, the size of the vehicle, the traveling direction, and the like, and these characteristics can be extracted. Further, the image processing unit 69 performs image processing on the gazing object set by the determination unit 67, so that the characteristics of the pedestrian and the building can be extracted.
It is also possible to recognize the sign, the position of the traffic light, the presence / absence of a railroad crossing, the status of road facilities, and the like.

Next, FIG. 6 is a block diagram of a third embodiment of the present invention. In FIG. 6, the distance measuring means 70
This is the same as that used in the first embodiment. The vehicle speed sensor 71 measures the running speed of the vehicle. For example, the vehicle speed sensor 71 detects the number of rotations of the wheels per unit time, and measures the speed of the vehicle from the length of the circumference of the wheels. There are those that reflect radio waves to the road surface and measure the speed by the Doppler effect. The steering angle sensor 72 detects a steering angle, and for example, a steering wheel having a built-in rotary encoder can be used. The human interface 73 is a device for exchanging information between the image processing device and the occupant, and is, for example, an operation switch or a display device provided in the passenger compartment. The occupant operates this operation switch to input a signal to the image processing apparatus, and displays information of the image processing apparatus on a display device to give to the occupant. The image sensor 74 is the same as that used in the second embodiment. The criterion setting means 75 is means for setting the criterion of the determining means 76 (details will be described later). Reference numeral 76 denotes a determination unit, and 77 denotes an image processing unit. The judgment criterion setting means 75, the judgment means 76, and the image processing means 77 can be constituted by, for example, a computer. S1 is a distance / direction signal, S2 is an image signal, S3 is a determination signal, S4 is a feature extraction signal, S5 is an enlargement / reduction / direction change signal, S6 is a vehicle speed signal, S7 is a steering angle signal, and S8 is a reference value. Setting signal, S
9 is a reference signal.

Next, the operation will be described. When a plurality of objects are present in front of the vehicle, it is necessary to set a plurality of image processing regions. In this case, in this case, the size and distance of the objects, It is configured such that an object to be watched is changed according to a traveling state or the like. The judgment criterion setting means 75 is
Set criteria for That is, in the judgment means 76,
The environment in which the host vehicle is currently determined is determined, and an object to be watched is determined accordingly. The determination criterion setting unit 75 sets the determination criterion in the above determination. For example, the detection of the state of the own vehicle is performed by detecting a traveling speed and a steering angle by a vehicle speed sensor and a steering angle sensor, and when the traveling speed is equal to or more than a predetermined value and the steering angle is equal to or less than a predetermined value, the highway is detected. When it is determined that the vehicle is traveling, and when the steering angle is equal to or more than the predetermined value and the vehicle speed is equal to or less than the predetermined value, it can be determined that the vehicle is traveling near the intersection.
Reference numeral 5 is used to set a threshold value of a vehicle speed value or a steering angle value, which is a reference for the determination. The setting of the reference value is input by the occupant using the human interface 73 such as an operation switch. The judging means 76 judges that an obstacle in front is a gazing object when it is judged that the vehicle is traveling on the highway as described above, and when it is judged that the vehicle is traveling near an intersection,
The traffic signal ahead is determined to be the gazing object. Then, the image processing means 77 performs image processing on the watched object.
In the above description, the case where the reference value of the determination reference setting means 73 is set using the human interface 73 has been described. However, the environment setting may be automatically performed by inputting the image processing result. It is possible.
For example, when the present invention is applied to a system in which the image processing unit 77 detects a white line on a road, a characteristic that a continuous white line is not detected near an intersection is used. Judgment can be made and the gaze object can be set to be a traffic light ahead. The above is summarized as follows. First, the distance and direction of the object are measured by the distance measuring means 70. Based on the measurement result and the traveling state of the vehicle, the criterion setting means 75 and the deciding means 76 determine an object to be subjected to image processing. For example, when it is determined that the vehicle is traveling on a highway, the preceding vehicle is set as the target object, and when it is determined that the vehicle is running near the intersection, the traffic light is set as the target object. However, it is not limited to the above example, since it can be set appropriately according to the situation. Further, the determination means 76 adjusts (enlarges, reduces, changes direction, etc.) the image sensor 74 so as to obtain an optimal image in performing the image processing. Decide whether to process the part. The image processing unit 77 that has captured the image of the image sensor 74 performs feature extraction of the object according to the target. The feature extraction includes, for example, detection of an obstacle, detection of lighting of a brake lamp of a vehicle at risk of collision, recognition of a speed limit indicated on a sign, and the like. Further, by calculating the temporal change of the distance information, the relative speed between the host vehicle and the target object can be detected. In this embodiment, as in the second embodiment, the detailed information obtained by the image processing means 77 includes the brake lamp of the preceding vehicle, the lighting of the direction indicator, the size of the vehicle, and the traveling direction. Etc., and those features can be extracted. The image processing means 77 performs image processing on the gazing object set by the judging means 76, so that the characteristics of pedestrians and buildings can be extracted. It is also possible to recognize such situations. Further, the distance measuring means 7
0 and the image sensor 74 can be added as needed, such as the front, rear, left and right of the vehicle. In this case, the feature of the object can be extracted in independent directions.

Next, an embodiment in which the present invention is applied to detection of a preceding vehicle will be described. 2. Description of the Related Art As a conventional preceding vehicle detection device, there is one that uses electromagnetic wave reflection as described in, for example, Japanese Patent Application Laid-Open No. 61-23985. In this method, an electromagnetic wave (for example, a laser beam) is emitted while sweeping a predetermined angle in front of the vehicle, and a reflected wave from a reflector is received.
By calculating the distance to the reflector based on the propagation delay time from the emission of the electromagnetic wave to the reception at each predetermined sweep angle, the distance and direction from the own vehicle to the reflector are detected. Information on the direction and distance to the reflecting object can be obtained. In general, a vehicle is provided with a reflex reflector at the rear in order to improve visibility from the rear, and the reflex reflector serves as a reflector, and the electromagnetic wave such as a laser beam is used. Is reflected, so that it can be easily detected. In addition, a device that recognizes a preceding vehicle or a road by inputting a scene in front of the vehicle using a television camera or the like directed toward the front of the vehicle and performing image processing on the input image has been considered. I have.

However, such a conventional preceding vehicle detection device has the following problems. First, in a preceding vehicle detection device using reflection of electromagnetic waves, such as a laser radar, not only a reflex reflector provided at the rear of the vehicle as a reflector, but also a corner reflector on a guardrail installed on the roadside zone of the road. Therefore, there is a problem that it is very difficult to select only the reflection from the reflex reflector of the preceding vehicle from many reflectors. Also, comparing the amount of change in the detection distance to the reflector in a unit time with the traveling vehicle speed of the own vehicle, determining whether the reflector is a stationary object or a moving object, and recognizing only the moving object as a vehicle. Is relatively easy. However, according to this method, for example, a vehicle stopped at the end of traffic congestion on an expressway is determined to be a stationary object and therefore not a vehicle, so that the position of the preceding vehicle can be detected. For this reason, there is a problem that it is not possible to perform high-precision traveling control or to issue an appropriate warning for an excessive approach to another vehicle. Also, in a preceding vehicle detection device that recognizes a preceding vehicle by processing an image from a television camera, it is relatively easy to recognize a white line indicating a roadside or a lane using image processing with current technology. However, it is not possible to expect very high accuracy in identifying the preceding vehicle.
In addition, since a large number of processes are required for identifying the preceding vehicle, a large amount of image data needs to be processed. When the present invention is applied to automatic running of a vehicle, a rear-end collision warning device, and the like, it is essential to perform such processing at a high speed (real time), and there is a problem that a very high-speed computer is required. Further, in a technique using normal image processing other than the purpose of detecting a preceding vehicle, an area (attention area) where image processing needs to be performed is limited from the entire screen, and various processing is performed within the area. Although a method of reducing the total calculation amount can be applied, in the case of the preceding vehicle detection, the preceding vehicle may exist in any area with respect to the screen of the camera of the own vehicle, and simply Since the attention area cannot be set, it is difficult to increase the speed by this method. This embodiment solves the problems of the prior art as described above, and provides a preceding vehicle detection device that can reliably detect a preceding vehicle and that can be practically realized without an ultra-high-speed computer. is there.

FIG. 7 is a functional block diagram of this embodiment. In FIG. 7, the reflector detecting means 1 emits an electromagnetic wave in the traveling direction of the vehicle while sweeping a predetermined angle range, receives a reflected wave from the reflector, and receives from the emission of the electromagnetic wave at each predetermined sweep angle. The distance and direction from the host vehicle to the reflector are detected by calculating the distance to the reflector based on the propagation delay time up to the vehicle. The reflector detecting means 1 corresponds to, for example, a laser light sweep direction detecting device 11, a laser light sweeping device 13, and a laser radar device 21 in the embodiment of FIG. The photographing means 3 photographs a scene ahead of the own vehicle in the traveling direction. This photographing means 3
For example, it corresponds to the television camera 23 in the embodiment of FIG. Further, the image recognizing means 7 processes the image ahead of the vehicle photographed by the photographing means 3 to detect a preceding vehicle existing in front. This image recognition means 7 corresponds to, for example, a portion of an image processing circuit 25 in the embodiment of FIG. Further, the image processing area setting means 5 sets the image processing area in the image recognition means 7 according to the distance and direction detected by the reflector detecting means 1. The image processing area setting means 5 corresponds to, for example, a signal processing circuit 31 in the embodiment of FIG. Further, the preceding vehicle position output means 9 outputs the distance and / or direction to the reflector recognized as the preceding vehicle by the image processing recognition means 7. The preceding vehicle position output means 9 corresponds to, for example, a part of the preceding vehicle position output circuit 27 in the embodiment of FIG. The contents of the image processing area setting means 5 include setting an image processing area to perform image processing only in an area within a predetermined angle range from the direction in which the distance detection from the reflector detecting means 1 is performed; Reflector detecting means 1
It is conceivable that the image processing area is set to the upper side of the screen as the detection distance is larger, and the area of the image processing area is set to be smaller as the detection distance by the reflector detector 1 is larger. As described above, in this embodiment, the image processing area setting means 5 for detecting the distance and direction to the reflector by the reflector detecting means 1 and setting the processing area in the image processing recognizing means 7 according to the result is provided. The image processing area is determined according to the distance and direction to the reflector. As a result, the position of the preceding vehicle can be reliably detected, and the region for performing image processing can be limited to a narrow range, so that calculation processing can be performed in real time without using an ultra-high-speed computer. It becomes possible.

FIG. 8 is a block diagram of the present embodiment. FIG. 8 illustrates a case where the present invention is applied to an inter-vehicle distance control device that automatically controls the inter-vehicle distance between the host vehicle and the preceding vehicle to be equal to or longer than a predetermined safety distance. FIG.
, The laser radar device 21 includes a light transmitter 15 that emits laser light, a light receiver 17 that receives reflected light of the laser light output from the light transmitter 15, and a light receiver 15 that receives light from the laser light emitted from the light transmitter 15. A distance detecting circuit for detecting a distance to the reflector based on a propagation delay time of the laser beam until the reflected light is received by the detector. The output of the distance detection circuit 19 is sent to a signal processing circuit 31. The laser light emitted from the light transmitter 15 of the laser radar device 21 is controlled by the laser light sweeping device 13 so as to be swept right and left around the front direction of the vehicle.
The laser light sweeping device 13 sweeps laser light in the left-right direction, for example, with an amplitude of about ± 10 degrees and a frequency of about 10 Hz using a mirror, for example. The sweep angle of the laser light sweep device 13 is detected by the laser light sweep direction detection device 11 and supplied to the signal processing circuit 31. As described above, the signal processing circuit 31 receives the distance signal from the distance detection circuit 19 and the sweep angle signal from the laser light sweep direction detection device 11. Therefore, the signal processing circuit 3
The signal input to 1 is a signal having information on the distance to the reflector at each predetermined sweep angle, that is, the distance and direction from the host vehicle to the reflector. When the laser light sweep direction detecting device 11 uses, for example, a step motor as a means for controlling the mirror of the laser light sweep device 13, the sweep is performed in accordance with the number of pulses for driving the step motor. An apparatus that outputs an angle can be used, and when, for example, a galvanometer type is used as the laser light sweeping apparatus 13, an apparatus that outputs a control signal for controlling the galvanometer as a signal corresponding to the sweep angle is used. Can be used. Next, the signal processing circuit 31 performs an image area to be subjected to image processing based on the distance supplied from the distance detection circuit 19 of the laser radar device 21 and the sweep angle supplied from the laser light sweep direction detection device 11. Is set, and the image processing device 29 is set.
To the image processing circuit 25. In addition, the image processing device 29 outputs a television camera 23 for capturing an image ahead of the host vehicle, an image processing circuit 25 that processes the captured image to recognize a preceding vehicle, and a result of the image processing circuit 25. It consists of a preceding vehicle position output circuit 27,
The distance, direction, and speed to the detected preceding vehicle are output. The signal processing circuit 31 and the image processing circuit 25
The portion of the preceding vehicle position output circuit 27 can be constituted by a microcomputer, for example. Further, the vehicle motion control circuit 33 outputs a command value to the actuator 35 based on the given distance, direction, and speed to the preceding vehicle so that the vehicle can follow the preceding vehicle while maintaining a safe inter-vehicle distance. I do. A steering device 37 such as a throttle valve, a brake, and a steering wheel is operated by the actuator 35.
, It is possible to safely follow the vehicle.

Next, before describing the overall operation of this embodiment, a method of setting an image processing area will be described first. FIG. 9 is a plan view schematically showing a state in which the host vehicle is traveling on a gentle right curve. In FIG. 9, a laser beam emitted from a laser radar device 21 (not shown)
5 is swept left and right within the range of the angle θM. The television camera 23 (not shown) is set so that the angle of view in the horizontal plane coincides with the sweep range −θM to θM of the laser radar device 21, and captures images in this range. In addition, 41 and 43 are reflectors (such as cat's eyes) provided on the roadside belt, 45 is a preceding vehicle, 47a and 47b are reflex reflectors attached to the preceding vehicle, 49 is a center line, and 51 and Reference numeral 53 denotes a line indicating a road end.

Next, FIG. 10 is a diagram showing an image in front of the vehicle taken from the television camera 23 in the state of FIG. 9, and images of the same ones as those in FIG. I have. That is, 45 'is an image of the preceding vehicle, 41' and 43 'are reflectors provided on the left and right roadside zones, 47a' and 47b 'are reflex reflectors attached to the preceding vehicle, 49' Indicates an image of a center line, and 51 'and 53' indicate images of lines indicating roadsides.

Next, FIG. 11 is a diagram showing a detection distance L from the laser radar device 21 with respect to the sweep direction θ output from the laser beam sweep direction detection device 11, and the same image as FIG. Symbols are indicated with “″”. FIG. 11 shows reflex reflectors 47a "and 47b" attached to the preceding vehicle, reflectors 41 "and 43" provided on the left and right roadside zones, and a center line 55 ".

A description will be given below with reference to FIGS.
In the front image shown in FIG.
Is displayed in the size shown in the figure at the position shown in the figure, and the displayed position and size may be different from those of the preceding vehicle 45 with respect to the center axis of the TV camera 23 if the angle of view range and magnification of the TV camera 23 are fixed. Determines the position in the image left-right direction (X direction), and the distance Lc determines the position in the image up-down direction (Y direction). The size of the image 45 'is inversely proportional to the distance Lc. Therefore, to detect the preceding vehicle by performing image processing only on the attention area, the signal processing circuit 3
In 1, the attention area to be subjected to the image processing in the image ahead of the vehicle may be determined from the distance and the angle to the reflectors 47a and 47b by the following calculation method as shown in FIG. (1) The center point P of the attention area according to the detection angle θ of the reflection object (the angle from the vehicle body center line 55, corresponding to the above θc).
Is determined in the left-right direction (X direction). (2) The position (Y0) in the vertical direction (Y direction) of the center point P of the attention area is determined according to the detection distance L (corresponding to Lc) of the reflection object. (3) The width w in the left-right direction and the width h in the vertical direction of the attention area are determined in inverse proportion to the detection distance L of the reflection object. (4) The attention area is a rectangular area having the following two points as two diagonal vertices. Point A (X0-w / 2, Y0-h / 2) Point C (X0 + w / 2, Y0 + h / 2) As described above, the determination of the attention area (image processing area) An area is set within a predetermined angle range from the direction in which the detection is performed (X direction), and the image processing area is set above the screen as the detection distance increases (Y direction).
The larger the detection distance is, the smaller the area of the image processing area is set.

FIG. 13 is a flowchart showing a processing procedure in the embodiment of FIG. Hereinafter, the overall operation of the embodiment shown in FIG. 8 will be described based on FIG. FIG.
The processing procedure shown in FIG. 3 is roughly divided into an angle / distance calculation unit 101 that calculates and stores a detection distance for each sweep angle, an image capture unit 103 that captures an image ahead of the vehicle from a television camera, A preceding vehicle detecting unit 105 that determines an area where image processing of an image is to be performed, performs image processing in the area, and recognizes a preceding vehicle, and a preceding vehicle that outputs the calculated position of the preceding vehicle if there is a preceding vehicle It is divided into a position output unit 107.

Hereinafter, the processing procedure will be described in detail. angle·
In the distance calculation unit 101, first, steps 121 and 123
Sets the variables k and j for processing to 0. That is, the laser sweep is performed in the range of angles −θM to θM,
During this time, the laser is output every 2θM / Kmax, and the output is performed Kmax times per sweep. The variable k is the number of the laser output in this one sweep, and the maximum value is K
max. The variable j is the number of times a distance has been detected in one sweep. Next, at step 125, the traveling vehicle speed of the own vehicle is read from the vehicle speed sensor 39. Next, steps 127 to 141 form a loop. This loop is performed for each laser output. When one sweep is completed, the process proceeds from step 141 to the next vehicle determination unit 103. In this loop, first, the sweep angle θ is read from the laser beam sweep direction detecting device 11 (steps 127 and 12).
9) The detection distance L is read from the distance detection circuit 19 (step 131). If no distance is detected, the process proceeds to step 139. If there is a distance detection, the process proceeds to step 135 (step 133). In steps 135 and 137, data is taken into the array D (j) only when the distance is detected. The array D (j) is an array for storing two data of the angle θ and the distance L, and inputs the obtained angle θ and the distance L. Thereafter, the next sweep angle command value is output (step 13).
9) After that, in step 141, the value of k is compared with Kmax, and if k is smaller than Kmax, the current sweep is not over yet and the process returns to step 127. Otherwise, all data by the current sweep is detected. Assuming that it has been acquired, the number of times the distance has been detected in the current sweep is substituted for Jmax (step 143), and the process proceeds to the next image capturing unit 103.

Next, the image capturing section 103 captures an image ahead of the vehicle from the television camera 23. Next, the preceding vehicle detection unit 105 first sets a variable j for processing to 0 (step 145). Next steps 147-1
55 constitutes a processing loop for each detected reflector. First, in step 147, an attention area to be subjected to image processing is determined from the detected array of angle / distance information D (j) = (θ, L). This determination method is as described above, and is calculated as follows. (1) X coordinate of the central point P: X0 = K1 × θ (K1 is a constant) (2) Y coordinate of the central point P: Y0 = K2 × L + T1 (K2 and T1 are constants) (3) Width in the left-right direction: w = K3 / L Width in the vertical direction: h = K4 / L (K3 and K4 are constants) (4) Calculation of coordinates of each vertex of the attention area Point A (X0-w / 2, Y0-h / 2) Point C (X0 + w / 2, Y0 + h / 2) In the next step 149, image processing is performed on the determined attention area. Here, for example, processing such as edge detection and binarization may be performed, and processing may be performed in the next step 151 to facilitate recognition of the shape (substantially square) of the vehicle. Next, in step 151, it is detected whether or not the image after the image processing includes the shape of the preceding vehicle. For this, a shape pattern (for example, a rectangle) that can be regarded as a vehicle is stored in advance, and it is possible to recognize the preceding vehicle by determining whether or not the image after image processing includes the shape pattern. Become. For example, when the reflector from which the distance data is obtained is other than a vehicle such as the reflectors 41 and 43 provided on the roadside zone of the road, image processing and preceding vehicle recognition are performed with the surrounding area as the attention area. Even if this operation is performed, an image matching the stored shape pattern cannot be obtained, and thus it is recognized that the vehicle is not the preceding vehicle. When the preceding vehicle recognition processing (Step 151) is completed, the same processing is performed on the next reflector data, and the processing is repeated until the number of data in the current sweep is reached.

Finally, proceed to the preceding vehicle position output unit 107,
In step 157, it is determined whether or not a preceding vehicle has been detected. If there is a preceding vehicle, the position of the preceding vehicle, that is, angle / distance information D (j) is output.

As described above, according to the present embodiment,
By detecting the distance and direction to the reflector and setting the processing area in image recognition according to the result, it is possible to reliably detect the position of the preceding vehicle and perform image processing. Since the area to be performed can be limited to a narrow range, it can be realized practically without an ultra-high-speed computer. Therefore, by applying the present invention to an automatic traveling device of a vehicle, a warning device for approaching another vehicle, and the like, it is possible to obtain an effect that more accurate automatic traveling and an appropriate alarm can be generated.

[0025]

As described above, according to the present invention, (1) the image processing method in the image processing means is set based on information on the distance and direction to an object outside the vehicle. ) An object on which image processing is to be performed is specified, and processing according to the object may be performed, so that the processing system is simplified. (2) Improvement in processing speed is expected.
(3) Since image information can be adjusted so that image processing is optimized, more accurate object feature extraction becomes possible.
(4) Since the change of the processing method can be arbitrarily set, a processing method according to the environment can be selected. (5) It is possible to accurately obtain a warning at the time of vehicle driving and information necessary for vehicle driving control. (6) It is possible to detect the behavior of an approaching vehicle (for example, intention to change lanes) that cannot be detected only by distance measurement or image processing. (7) By applying the image processing device of the present invention, Since the distance, direction, and characteristics to the object can be obtained, effects such as improvement in the performance of a driving support device such as a collision prevention device and an image processing system for an autonomous vehicle can be obtained. Further, in a device in which the present invention is applied to a preceding vehicle detection device, by detecting a distance and a direction to a reflector and setting a processing region in image recognition according to the result, the preceding vehicle is detected. Since it is possible to reliably detect the position and to limit the area for performing image processing to a narrow range,
It can be realized practically without an ultra-high-speed computer. Therefore, by applying the present invention to an automatic traveling device of a vehicle, a warning device for approaching another vehicle, and the like, it is possible to obtain an effect that more accurate automatic traveling and an appropriate alarm can be generated.

[Brief description of the drawings]

FIG. 1 is a block diagram of a first embodiment of the present invention.

FIGS. 2A and 2B are diagrams showing a positional relationship between a vehicle and an object outside the vehicle, wherein FIG. 2A is a top view and FIG.

FIG. 3 is a table showing an example of a distance measurement result.

FIG. 4 is a flowchart showing a processing procedure in the apparatus of FIG. 1;

FIG. 5 is a block diagram of a second embodiment of the present invention.

FIG. 6 is a block diagram of a third embodiment of the present invention.

FIG. 7 is a functional block diagram when the present invention is applied to a preceding vehicle detection device.

FIG. 8 is a block diagram of an embodiment when the present invention is applied to a preceding vehicle detection device.

FIG. 9 is a plan view schematically showing a state in which the host vehicle is traveling on a gentle right curve.

FIG. 10 is a view showing an image in front of the vehicle taken from the television camera in the state of FIG. 9;

FIG. 11 is a diagram showing a detection distance L from a laser radar device with respect to a sweep direction θ output from a laser light sweep direction detection device.

FIG. 12 is a view for explaining a method of determining a region of interest on which image processing is to be performed.

FIG. 13 is a flowchart showing a processing procedure in the embodiment of FIG. 8;

FIG. 14 is a diagram illustrating an example of a conventional vehicle image processing apparatus.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Reflector detection means 11 ... Laser light sweep direction detection apparatus 3 ... Imaging means 13 ... Laser light sweeping apparatus 5 ... Image processing area setting means 15 ... Light transmitter 7 ... Image processing recognition means 17 ... Light receiver 9 ... Preceding vehicle Position output means 19 ... Distance detecting device 21 ... Laser radar device 29 ... Image processing device 23 ... TV camera 31 ... Signal processing circuit 25 ... Image processing circuit 33 ... Vehicle motion control circuit 27 ... Previous vehicle position output circuit 35 ... Actuator 37 ... Control devices such as throttles, brakes, steering wheels, etc. 60, 66, 70 distance measuring means 65 objects outside the vehicle 61, 67, 76 determining means 71 vehicle speed sensors 62, 68, 74 image sensors 72 steering angle sensors 63 69, 77: Image processing means 73: Human interface 64: Vehicle 75: Judgment standard setting means

Continuation of the front page (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/00, 7/00 G08G 1/16 G01B 11/00-11/30

Claims (5)

(57) [Claims] 1. An image sensor mounted on a vehicle for inputting image information around the vehicle, distance measuring means for obtaining information on the distance and direction to one or more objects existing outside the vehicle, and the distance and direction based on the information, the presence of the object
A process of determining the periphery of the region as a region for performing image processing,
Or the object to perform image processing on any of the above objects
Determining means for determining and setting an image processing method including any one of processing for determining an object; and performing image processing on the image information obtained by the image sensor based on the image processing method set by the determining means. An image processing device for a vehicle, comprising: Wherein said determination means is characterized in that it is used for setting to determine an image processing method including at least one processing of processing to scale, turning, focusing the images The vehicle image processing device according to claim 1. 3. An image sensor according to claim 1, wherein said image sensor enlarges / reduces / orients a predetermined object requiring detailed image processing in accordance with the enlargement / reduction / direction change / focus adjustment processing set by said determination means. 3. The image processing apparatus for a vehicle according to claim 2 , wherein the apparatus performs conversion and focus adjustment . 4. A vehicle according to claim 1, further comprising a measuring unit for detecting a traveling state of the host vehicle, wherein said determining unit determines a vehicle state according to a detection result of said measuring unit.
Judge the driving environment of the
The image processing system as recited in claim 2 or 3, characterized in that to determine the meta-object as an object for performing an image processing. 5. An electromagnetic wave is emitted in a traveling direction of the vehicle while sweeping a predetermined angle range, and a reflected wave from a reflector is received.
A reflector detecting means for detecting a distance and a direction from the host vehicle to the reflector by calculating a distance to the reflector based on a propagation delay time from emission of the electromagnetic wave to reception at each predetermined sweep angle; Photographing means for photographing a scene in front of the traveling direction of the vehicle; image recognizing means for processing an image in front of the vehicle photographed by the photographing means to detect a preceding vehicle existing ahead; image processing in the image recognizing means An image processing area setting means for setting an area according to the distance and direction detected by the reflector detection means; and a distance and / or a direction to the reflector recognized as a preceding vehicle by the image processing recognition means. A vehicle image processing device, comprising: a preceding vehicle position output means for outputting.