JP4980970B2 - Image pickup means adjustment device and object detection device - Google Patents

Description

  The present invention relates to an imaging device adjustment device and an object detection device, and more particularly to an imaging device adjustment device capable of adjusting an exposure amount of the imaging device and an object detection device for detecting an object or the like using the adjustment device.

  In recent years, a preceding vehicle traveling in front of the host vehicle is detected from an image captured by an imaging unit such as a camera having a built-in CCD (Charge Coupled Device) image sensor, and constant speed traveling control of the host vehicle is performed based on the detected vehicle. Cruise control device to perform, ACC to perform cruise control with preceding vehicle tracking function (Adaptive Cruise Control, hereinafter abbreviated as ACC) for performing tracking control for the preceding vehicle in addition to constant speed traveling control of the host vehicle An apparatus has been developed (see, for example, Patent Document 1).

In the ACC device, when the preceding vehicle does not exist, the host vehicle travels at a set speed at a set speed, and when the preceding vehicle exists, the accelerator throttle or brake of the own vehicle follows the preceding vehicle. Control for automatically operating the mechanism and the like is performed automatically. Therefore, the ACC device is strongly required to detect the preceding vehicle with high accuracy. Further, even in a cruise control apparatus that does not involve follow-up traveling control for a preceding vehicle, detection of the preceding vehicle is usually performed for the purpose of preventing a rear-end collision with the preceding vehicle.
Japanese Patent Laid-Open No. 2005-1566

  By the way, when the preceding vehicle is detected using the imaging means as described above during night driving, the main body such as the body of the preceding vehicle existing in a dark background is imaged to detect the preceding vehicle. When adjustments such as increasing the exposure amount of the means are made, lights of lights such as tail lamps and brake lamps of the preceding vehicle are widely diffused and imaged, and the light is captured at the end (edge portion) of the body of the preceding vehicle, etc. ) And the image is taken out.

  In such a situation, since the degree of diffusion of the light captured in the image captured by the image capturing unit is different and is also different for each frame, for example, stereo matching is performed on a pair of images captured by the stereo image capturing unit. Spatial variations and temporal variations occur in the parallax and distance information of the lights such as tail lamps and brake lamps obtained, and the distance between the host vehicle and the preceding vehicle varies and is detected.

  Therefore, even though the preceding vehicle actually travels in front of the host vehicle stably, the preceding vehicle is detected as if the preceding vehicle is moving back and forth in time. When the follow-up traveling control for the preceding vehicle is operating, a phenomenon occurs in which the host vehicle accelerates or decelerates behind the preceding vehicle that is traveling stably.

  Such a phenomenon occurs not only in the case of detecting a preceding vehicle, but also when detecting an object having lights such as a signal light of a traffic light at night. If the image is projected to a large extent, the parallax or distance to the image cannot be detected stably, and for example, a signal that is stationary with respect to the road surface is detected as a moving object.

  Therefore, in such a case, on the contrary, adjustments such as reducing the exposure amount of the image pickup means are performed to take an image so that lights of lights such as a tail lamp of a preceding vehicle and a signal light of a traffic light do not diffuse around. For example, by appropriately performing stereo matching, it is possible to accurately detect the position of lights such as a tail lamp and accurately detect the distance from the result to the preceding vehicle, traffic light, or the like.

  However, if adjustments such as reducing the exposure amount of the imaging means are performed, parts other than lights will be imaged very dark this time, so roadside pedestrians, lanes marked on the road surface, etc. Is difficult to detect. Even if the exposure amount or the like of the image pickup means is adjusted, it is practically impossible to accurately detect lights such as tail lamps and signal lights of a preceding vehicle and pedestrians and lanes simultaneously. In the present invention, a continuous line or a broken line marked on a road surface such as a road center line such as an overtaking prohibition line, a vehicle traffic zone boundary line, a lane line dividing a roadside zone and a roadway is referred to as a lane.

  Therefore, at night, when detecting preceding vehicles, traffic lights, etc., when detecting pedestrians and lanes, the amount of exposure is reduced so that the lights of the lights such as tail lamps and signal lights do not diffuse around. In order to increase the exposure amount, it is necessary to switch the exposure amount of the image pickup means accurately and adjust the image so that they are appropriately imaged.

  The present invention has been made in view of such circumstances, and automatically switches and adjusts the exposure amount and the like of the imaging means so that a pedestrian, a lane, and lights can be accurately captured at night. It is an object of the present invention to provide an adjustment device for an imaging means that can be used and an object detection device that detects an object using the adjustment device.

In order to solve the above problem, the first invention is an adjustment device for an imaging means,
Object detection means for detecting an object from the image captured by the imaging means;
Night detection means for detecting that it is night,
When it is detected by the night detection means that it is nighttime, the imaging mode of the imaging means is at least a first imaging mode capable of imaging the contour part of the lit lamps of an object having lamps; An imaging mode switching means for switching between a second imaging mode capable of detecting an object having no lights;
It is characterized by providing.

  According to a second aspect of the present invention, in the adjusting device for an image pickup means according to the first aspect of the present invention, the lighting of the object having the lighting is a lighting of a lighting or a traffic light provided on the back of the preceding vehicle. It is characterized by being a signal light.

  According to a third aspect of the present invention, in the adjusting device for an imaging means according to the first or second aspect, the object that does not have the lights is a pedestrian.

According to a fourth aspect of the present invention, in the adjusting device for an imaging means according to any one of the first to third aspects,
Furthermore, lane detection means for detecting the left and right lanes of the host vehicle from the image,
The object that does not have the lights is the lane.

  According to a fifth aspect of the present invention, in the adjustment device for an imaging unit according to any one of the first to fourth aspects, the imaging mode switching unit is configured to operate when the preceding vehicle follow-up control or the constant speed traveling control is activated in the host vehicle. Switching the imaging mode of the imaging unit from the second imaging mode to the first imaging mode, and switching the imaging mode of the imaging unit from the first imaging mode to the second imaging mode when the operation is stopped. Features.

  According to a sixth aspect of the present invention, in the apparatus for adjusting an imaging means according to any one of the first to fifth aspects, the imaging mode switching means is configured to capture an image of the imaging means when the speed of the host vehicle exceeds a predetermined speed. The mode is switched from the second imaging mode to the first imaging mode, and the imaging mode of the imaging means is switched from the first imaging mode to the second imaging mode when the speed becomes lower than the predetermined speed. It is characterized by.

According to a seventh aspect of the invention, in the adjustment device for an imaging means according to any one of the first to sixth aspects of the invention,
It further comprises travel detection means for detecting that the host vehicle is traveling on an automobile road,
The imaging mode switching means switches the imaging mode of the imaging means from the second imaging mode to the first imaging mode when the travel detection means detects that the host vehicle is traveling on a road dedicated to automobiles. When the traveling detection unit does not detect that the host vehicle is traveling on a road dedicated to automobiles, the imaging mode of the imaging unit is switched from the first imaging mode to the second imaging mode.

  According to an eighth aspect of the present invention, in the adjustment device for an imaging unit according to any one of the first to seventh aspects, the imaging mode switching unit sets the imaging mode of the imaging unit when the wiper of the host vehicle is operated. The imaging mode of the imaging unit is switched from the first imaging mode to the second imaging mode when the operation of the wiper is stopped when switching from the second imaging mode to the first imaging mode.

  According to a ninth aspect of the present invention, in the adjustment device for an imaging unit according to any one of the first to eighth aspects, the imaging mode switching unit is configured to switch an exposure amount of the imaging unit and the imaging unit in accordance with the switching of the imaging mode. In this case, at least one of the luminance values of the pixels output from is adjusted.

According to a tenth aspect of the present invention, in the adjustment device for an imaging means of the ninth aspect,
The imaging means is a stereo imaging means for simultaneously imaging an object with a pair of cameras and outputting a pair of images,
The image processing apparatus further includes a stereo matching unit that performs a stereo matching process on the pair of images captured by the image capturing unit and calculates a parallax for each pixel block of the image.

  The eleventh invention is the same as the tenth invention, wherein the imaging mode switching means is the same when the imaging mode of the imaging means is switched from the second imaging mode to the first imaging mode. The adjustment is performed so that the variation of each data of the parallax corresponding to the lighting of the object having the lighting calculated by the stereo matching unit in the imaging cycle is within a predetermined threshold value. To do.

  According to a twelfth aspect of the present invention, in the adjustment device for an imaging means according to the tenth or eleventh invention, the imaging mode switching means switches the imaging mode of the imaging means from the second imaging mode to the first imaging mode. The absolute value of the temporal change amount of the parallax data corresponding to the lighting of the object having the lighting calculated by the stereo matching means in the temporally adjacent imaging cycle is within a predetermined threshold value The adjustment is performed so that

  According to a thirteenth aspect of the present invention, in the adjustment device for an imaging unit according to any one of the ninth to twelfth aspects, the imaging mode switching unit changes the imaging mode of the imaging unit from the second imaging mode to the first imaging mode. In the case of switching, based on the distribution of luminance values of the image area in which the lamps that are illuminated of the objects having the lamps in the image are captured, the luminance values of the pixels in the image area and the pixels adjacent to the pixel The adjustment is performed such that the maximum absolute value of the difference from the luminance value is equal to or greater than a predetermined threshold value.

  According to a fourteenth aspect of the present invention, in the adjustment device for an imaging unit according to any one of the tenth to thirteenth aspects, the imaging mode switching unit changes the imaging mode of the imaging unit from the first imaging mode to the second imaging mode. In the case of switching, the adjustment is performed so that the number of data points of the parallax corresponding to an object that does not have the lights calculated by the stereo matching unit in the same imaging cycle is equal to or greater than a predetermined threshold value. .

  According to a fifteenth aspect, in the adjustment device for an imaging means according to any one of the first to fourteenth aspects, the imaging mode of the imaging means is set to the first imaging mode with respect to the driver of the host vehicle. An informing means for informing is provided.

A sixteenth aspect of the invention is an object detection apparatus,
The imaging means for imaging the surroundings of the host vehicle;
An adjustment device for an imaging means according to any one of claims 1 to 15,
The object detecting means detects an object having the lights when in the first imaging mode, and an object not having the lights when in the second imaging mode. .

  According to the first to fourth aspects of the present invention, when the object is imaged by the imaging means during night driving, the object to be detected is an object having lights such as a tail lamp of a preceding vehicle or a signal light of a traffic light. The imaging mode of the imaging means is switched between the case where the lighting is turned on and the case where the subject does not have lighting such as a pedestrian or lane.

  For this reason, when driving at night, the following driving control for the preceding vehicle is activated to try to detect those objects that are virtually impossible to detect accurately at the same time. To avoid the situation where the host vehicle accelerates or decelerates behind the preceding vehicle, or detects a traffic signal that should be stationary with respect to the road surface as a moving object Is possible.

  In addition, when detecting an object having lights, the exposure amount of the imaging means is reduced so that the light of the lit lights is not diffused to the surroundings, and when an object having no lights is detected. Since it is possible to automatically and accurately switch the exposure amount and the like of the imaging means so as to increase the exposure amount and the like, it is possible to adjust so that the detection target is appropriately imaged.

  According to the fifth to eighth inventions, in addition to the effects of the above-mentioned inventions, when the preceding vehicle follow-up control or constant speed traveling control is operated on the own vehicle, or the speed of the own vehicle becomes equal to or higher than a predetermined speed. In this case, when it is detected that the host vehicle is traveling on an automobile-only road, the imaging mode of the imaging unit is changed from the second imaging mode to the first imaging mode during a rain or snowing when the wiper of the host vehicle is operated. By switching to the mode, it is possible to adjust the exposure of the imaging means so that the tail lamp of the preceding vehicle can be accurately detected.

  In addition, when those conditions disappear, the imaging mode of the imaging unit is switched from the first imaging mode to the second imaging mode, so that pedestrians, lanes, and the like are detected and imaging is performed so as to improve vehicle safety. It is possible to adjust the exposure of the means.

  According to the ninth invention, in addition to the effects of the respective inventions, the imaging mode is switched by adjusting at least one of the exposure amount of the stereo imaging means and the luminance value of the pixel output from the stereo imaging means. Thus, it is possible to easily and appropriately adjust the exposure of the stereo image pickup means, and to adjust quickly according to the sampling period of the stereo image pickup means.

  According to the tenth and eleventh aspects of the invention, in addition to the effects of the above-described inventions, it corresponds to a lighting of an object having a lamp having a lamp calculated by stereo matching processing for a pair of images picked up by the stereo image pickup means. By adjusting so that the variation of each parallax data to be within a predetermined threshold value, it becomes possible to make adjustment based on the variation in the parallax data that can be easily detected, and easy adjustment of the exposure of the stereo imaging means And it becomes possible to carry out quickly.

  According to the twelfth invention, by further adjusting based on temporal variations in parallax data, it becomes possible to easily and more reliably adjust the exposure of the stereo imaging means, and the effects of the above inventions can be achieved. Exhibited accurately.

  According to the thirteenth aspect, by performing the adjustment based on the distribution of the luminance value of the image area in which the lamps are imaged, it is possible to perform more reliable exposure adjustment of the imaging unit, The effect of each invention is exhibited accurately.

  According to the fourteenth invention, in addition to the effects of the above inventions, when the imaging mode of the imaging means is switched from the first imaging mode to the second imaging mode, the parallax corresponding to an object having no lights By adjusting so that the number of data points is equal to or greater than a predetermined threshold, it is possible to adjust the exposure of the imaging means so that pedestrians, lanes, etc. can be accurately detected even in a dark environment during night driving. .

  According to the fifteenth aspect, when the imaging mode of the imaging means is the first imaging mode, a target that does not have lights such as a pedestrian is hardly detected. Therefore, in addition to the effects of the above inventions, if the driver is informed of such a state by displaying it on a display screen or turning on a lamp, etc. It is possible to call attention.

  According to the sixteenth aspect of the invention, since the image pickup means adjustment apparatus having the above functions is provided, the light emitted from the lights such as the tail lamps of the preceding vehicle is diffused in the automatic exposure adjustment provided in the image pickup means. Even during nighttime travel where the image is taken, the imaging mode of the imaging unit is appropriately switched by the imaging mode switching unit.

  Therefore, when detecting an object having lamps, the contour of the lit lamp is clearly imaged, and the object can be detected accurately. In addition, when detecting an object such as a pedestrian that does not have lights, the exposure amount of the imaging unit is automatically switched to increase the exposure amount, so that the object can be accurately detected. .

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an imaging device adjusting device and an object detecting device according to the present invention will be described below with reference to the drawings.

  As shown in FIG. 1, an object detection apparatus 1 including an imaging device adjustment device according to the present embodiment is an imaging device 2, a conversion device 3, a stereo matching device 6, and the like, and an adjustment device for the imaging device 2. And processing means 9. In the present embodiment, the case where the imaging unit 2 is a stereo imaging unit that simultaneously captures an object with a pair of cameras and outputs a pair of images will be described, but the present invention is not limited thereto.

  In this embodiment, the configuration from the image pickup means 2 to the stereo matching means 6 is disclosed in Japanese Patent Laid-Open Nos. H5-114099, H5-265547, and H6-266828 previously filed by the applicant of the present application. The vehicle exterior monitoring device described in JP-A-10-283461, JP-A-10-283477, JP-A-2006-72495, and the like is used as a base. A brief description is given below.

  The imaging means 2 includes a main camera 2a and a sub camera 2b, which are a pair of imaging means arranged at a certain distance in the vehicle width direction. Is to be imaged. The pair of captured images are converted into digital images by A / D converters 3a and 3b, which are conversion means 3, and image correction such as displacement and noise removal and luminance value correction is performed by the image correction unit 4. Thus, it is stored in the image data memory 5 and transmitted to the processing means 9.

  In the present embodiment, the main camera 2a and the sub camera 2b are each composed of a camera (hereinafter referred to as a CCD camera) incorporating a CCD image sensor. As a function normally provided in the CCD camera, the brightness around the host vehicle is determined. It is configured to automatically perform exposure adjustment for determining appropriateness and obtaining appropriate exposure. Here, the exposure adjustment in the present embodiment includes at least shutter time adjustment, amplifier gain switching, LUT (Look Up Table) selection and luminance value conversion based on the adjustment, and these are adjusted comprehensively. It has become.

  Since the CCD camera used in this embodiment does not include an iris, the iris diaphragm adjustment will not be described. However, when the imaging unit 2 has other functions for adjusting the exposure of the iris or the like, these are included. Thus, automatic exposure adjustment is performed to obtain a comprehensively optimal exposure.

  Further, the automatic exposure adjustment level of the image pickup means 2 and the shutter level indicating the degree of adjustment of the exposure amount of the image pickup means 2 are transmitted to the processing means 9 described later.

  On the other hand, the pair of captured images that have undergone image correction are also transmitted to the stereo matching means 6. The stereo matching means 6 includes an image processor 7 and a distance data memory 8, and the image processor 7 performs a stereo matching process.

  Specifically, as shown in FIG. 2, the image processor 7 sets a reference pixel block PB having a predetermined number of pixels such as 3 × 3 pixels or 4 × 4 pixels on the reference image T, so that the reference pixel block For each comparison pixel block PBc of the same shape as the reference pixel block PB on the epipolar line EPL in the comparison image Tc corresponding to PB, an SAD value that is a difference in luminance pattern with the reference pixel block PB is calculated according to the following equation (1). The comparison pixel block PBc having the smallest SAD value is specified.

  Note that p1st represents the luminance value of each pixel in the reference pixel block PB, and p2st represents the luminance value of each pixel in the comparison pixel block PBc. Further, the above sum is obtained when the reference pixel block PB and the comparison pixel block PBc are set as a 3 × 3 pixel region, for example, a range of 1 ≦ s ≦ 3, 1 ≦ t ≦ 3, and 4 × 4 pixels. When set as a region, calculation is performed for all pixels in the range of 1 ≦ s ≦ 4 and 1 ≦ t ≦ 4.

  For each reference pixel block PB of the reference image T in this way, the image processor 7 calculates the parallax dp from the position on the comparison image Tc of the identified comparison pixel block PBc and the position on the reference image T of the reference pixel block PB. It comes to calculate. In the present embodiment, the image processor 7 assigns the parallax dp calculated as described above to each reference pixel block PB of the reference image T illustrated in FIG. 3, for example, and forms a distance image Tz as shown in FIG. The data is transmitted to the distance data memory 8 and stored, and information on the parallax dp is transmitted to the processing means 9.

The parallax dp and the coordinates (i, j) on the distance image Tz and the position (X, Y, Z) in the real space can be associated as follows based on the principle of triangulation. Specifically, a point on the road surface directly below the center of the main camera 2a and the sub camera 2b in the real space is set as the origin, and the vehicle width direction of the own vehicle, that is, the X axis in the left and right direction, the Y axis in the vehicle height direction, Taking the Z axis in the long direction, that is, the distance direction, the relationship between the point (X, Y, Z) in the real space, the parallax dp, and the coordinates (i, j) on the distance image Tz is
X = CD / 2 + Z * PW * (i-IV) (2)
Y = CH + Z × PW × (j−JV) (3)
Z = CD / (PW × (dp−DP)) (4)
(X, Y, Z) and (i, j, dp) are associated one-to-one.

  Here, CD is the distance between the main camera 2a and the sub camera 2b, PW is the viewing angle per pixel, CH is the mounting height of the main camera 2a and the sub camera 2b, and IV and JV are infinity points in front of the host vehicle. The i-coordinate and j-coordinate, and DP represent an offset amount also called vanishing point parallax.

  Further, as described above, the parallax dp and the distance Z in the real space are uniquely associated through the above equation (4). Therefore, at the time of forming the distance image Tz, the parallax dp for each reference pixel block PB is converted into the information on the distance Z, and the information on the distance Z is assigned to each reference pixel block PB of the reference image T, respectively. It is also possible to configure so as to form.

  The processing means 9 (see FIG. 1) is configured as a computer configured by connecting a CPU, ROM, RAM, input / output interface, etc. (not shown) to the bus. The processing means 9 is connected to sensors Q such as a vehicle speed sensor, a yaw rate sensor, and a steering angle sensor for measuring the steering angle of the steering wheel.

  It is also possible to use a device that estimates the yaw rate from the vehicle speed of the host vehicle or the like instead of the yaw rate sensor. In this embodiment, the sensors Q include information on the operation and stop of an ACC device (not shown) mounted on the host vehicle, that is, on / off information transmitted from a switch of the ACC device, Stop information, that is, on / off information transmitted from the wiper switch is also included.

  The processing means 9 is an adjustment device for the imaging means 2 in the present embodiment, and includes a lane detection means 10, a lights detection means 11, an object detection means 12, a night detection means 13, and an imaging mode switching means 14. And a memory (not shown). In addition, necessary data from the sensors Q is input to each means of the processing means 9.

  The lane detection means 10 detects the left and right lanes of the host vehicle from the reference image T imaged by the imaging means 2. Specifically, as shown in FIG. 5, the lane detection unit 10 searches the horizontal line j having a width of one pixel using the reference image T, for example, from the center of the reference image T in the left-right direction, and determines the luminance value. Are detected as lane candidate points cl and cr that greatly change from the luminance value of adjacent pixels to a threshold value or more.

  Then, while shifting the horizontal line j on the reference image T upward by one pixel, lane candidate points are detected on each horizontal line j in the same manner. At this time, if the lane detection unit 10 determines that the lane candidate point is not on the road surface based on the parallax dp of the detected lane candidate point, the lane detection point 10 excludes the lane candidate point from the lane candidate point. Then, of the remaining lane candidate points, the lane is approximated by a straight line by Hough transformation or the like based on the lane candidate point closer to the own vehicle, and detected on the left and right sides of the own vehicle, respectively.

  In this case, if multiple lanes are detected on one side (for example, the right side) of the host vehicle while improving the reliability of detection by excluding lane candidate points that cannot be consistent with the straight line Select a lane that is consistent with the lane detected on the other side of the vehicle (for example, the left side) or a lane that is consistent with the lane detected in the previous sampling cycle, etc. To do.

  In this way, when the lanes are respectively detected in a straight line on the side close to the host vehicle, the lane candidate points are selected and connected based on the positional relationship with the straight line based on the straight line on the far side, thereby FIG. As shown in the figure, the lanes Ll and Lr are detected on the left and right sides of the host vehicle, respectively. The processing configuration of the lane detection means 10 described above is described in detail as a lane recognition device in Japanese Patent Application Laid-Open No. 2006-331389 previously filed by the applicant of the present application. For details, see the same publication. .

  The lamp detection means 11 detects lamps that are turned on by an object having lamps. Illuminated lights of objects having lights are, for example, illuminated lights provided on the back of a preceding vehicle such as a tail lamp, a brake lamp, a turn signal, a fog lamp, a back lamp, a number lamp that illuminates a license plate, A signal light with a traffic light on.

  Hereinafter, a case where the tail lamp of the preceding vehicle Vah is detected will be described as an example. However, the following description is also applied to a case where other lights are detected. In the following, detection of the left tail lamp of the preceding vehicle Vah will be mainly described. However, detection of the right tail lamp is simultaneously performed in the same manner as the left tail lamp.

  First, as shown in FIG. 7, the lamp detection unit 11 detects the coordinates (cl_i, cl_j) of the pixel on the reference image T detected as the center of the tail lamp of the preceding vehicle Vah in the previous sampling cycle stored in the memory. It is read out and it is determined whether or not the luminance value p1ij at the current sampling period of the coordinates is equal to or greater than a predetermined luminance value br1. The brightness value br1 is set to a brightness value as high as 250, for example, with 256 brightness gradations from 0 to 255. If the current luminance value p1ij of the coordinates is equal to or higher than the luminance value br1, the pixel at the coordinates (cl_i, cl_j) on the reference image T is considered to be in the high luminance region representing the tail lamp even in the current sampling cycle. .

  If the current luminance value p1ij of the coordinates (cl_i, cl_j) is less than the luminance value br1, the upper and lower sides of the pixel of the coordinates (cl_i, cl_j) on the reference image T are searched, or the right side detected last time A luminance value br1 or more is obtained by searching for a point on the left side of the tail lamp center by the distance between the left and right lamps and the upper and lower sides thereof, or searching for the upper and lower sides of a predetermined position from the left and right ends of the frame line Fr of the preceding vehicle Vah detected previously. The pixel having the luminance value p1ij is searched.

  As described above, it is difficult to detect the preceding vehicle Vah itself at night. Therefore, in such a case, the position of the preceding vehicle Vah is estimated from the detected positions of the left and right tail lamps on the reference image T, and a rectangular frame line Fr surrounding the preceding vehicle Vah is set. Yes.

  Next, as shown in FIG. 8, the lamp detection unit 11 includes the previous tail lamp center (cl_i, cl_j) (or a pixel having a luminance value equal to or higher than the luminance value br1 detected by searching the vicinity thereof). The luminance value p1ij of each pixel is searched while shifting one pixel upward on the pixel row pls having a width of one pixel extending in the vertical direction in the reference image T, and the luminance value p1ij is obtained from the luminance value br1 described above. The coordinates (cl_i, jmax) of the pixel immediately before the pixel that is less than the low luminance value br2 are detected. Similarly, the lights detection means 11 searches downward on the pixel row pls, and detects the coordinates (cl_i, jmin) of the pixel immediately before the pixel whose luminance value p1ij is less than the luminance value br2. The luminance value br2 is set to a luminance value of 230, for example, lower than the luminance value br1.

  The lamp detection means 11 stores the upper end coordinates (cl_i, jmax) and the lower end coordinates (cl_i, jmin) of the detected high-intensity area in the memory, respectively, and then searches the pixel row pls (or the above search). The pixel columns are shifted in the right and left directions on the reference image T from the detected pixel value having a luminance value equal to or higher than the luminance value br1, and a high luminance area is searched for in each pixel column.

  Specifically, the j coordinate jmid of the middle point of the upper and lower j coordinates jmax and jmin of the high luminance region in the pixel column pls is calculated, and the pixel column is plunged to the right as shown in FIG. , And the upper and lower ends of the high brightness area are detected in the same manner as described above with the pixel at the coordinates (cl_i + 1, jmid) in the pixel column pl as the start point, and the coordinates (cl_i + 1, jmax), (cl_i + 1, jmin) are detected. ) In each memory. The search is repeated while further shifting the pixel column pl in the right direction, the same search is performed in the left direction of the first pixel column pls, and the coordinates of the upper end and the lower end of the high luminance region are stored in the memory.

  At that time, the lights detection means 11 monitors the number of pixels R (hereinafter referred to as the length R in the vertical direction) from the uppermost pixel to the lowermost pixel of the high luminance area of each pixel row pls, pl. As shown in FIG. 10A, the vertical length R of the high luminance areas of the pixel columns pls and pl once becomes the maximum value Rmax, and the high luminance area of the pixel column pl as shown in FIG. 10B. The vertical length R of the pixel column pl is less than the maximum value Rmax, and as shown in FIG. 10C, the vertical length R of the high luminance region of the pixel column pl continues to decrease, and the pixel column pl has a high luminance. If the area cannot be found, the search in the right or left pixel column is stopped at that time.

  In addition, as shown in FIG. 11, when the length R in the vertical direction of the high luminance region of the pixel row pl decreases to once reach the minimum value Rmin, then when the value again becomes larger than the minimum value Rmin, The search in the right or left pixel column in the column pl is stopped, and the range up to the pixel column immediately before is set as the range in which the high luminance region is detected.

  Then, the lights detection means 11 reads out the coordinates of the upper end and the lower end of each high brightness area in each pixel column pls, pl detected in this way from the memory, and as shown in FIG. The midpoint of the highest point Jmax of the upper end coordinates and the lowest point Jmin of the lower end coordinates is the j coordinate cl_j of the tail lamp center of the tail lamp TLl on the left side of the preceding vehicle Vah, and the vertical length R of the high luminance area The coordinate (cl_i, cl_j) of the left tail lamp center is detected with the i coordinate of the pixel column pl having the maximum value Rmax as the i coordinate cl_i of the left tail lamp center. Further, the maximum value Imax and the minimum value Imin of the i coordinate of the pixel row in which the high luminance region is detected are also detected. Similarly, for the tail lamp TLr on the right side, the coordinates (cr_i, cr_j) of the tail lamp center are detected.

  The lamp detection means 11 stores the coordinates (cl_i, cl_j), (cr_i, cr_j) of the left and right tail lamp centers detected in this way in the memory as the center of the tail lamp of this time, and the highest of the left and right tail lamps TLl, TLr. The point Jmax, the lowest point Jmin, the maximum value Imax of the i coordinate, the minimum value Imin, etc. are stored in the memory, respectively.

  The object detection unit 12 detects an object from the reference image T imaged by the imaging unit 2. In the present embodiment, the object detection processing from the reference image T in the object detection means 12 is configured based on the processing in the outside monitoring apparatus and the like described in the publications. A brief description is given below.

  For example, the object detection unit 12 divides the distance image Tz shown in FIG. 3 into vertical strip-shaped sections Dn having a predetermined width as shown in FIG. Then, as shown in FIG. 14, a histogram Hn is created for each section Dn, and among the parallaxes dp included in each section Dn, the parallax dp existing above the road surface is voted on the histogram Hn, and its mode value Let dpn be the representative parallax of that segment. Do this for all categories.

  Then, the object detection means 12 calculates coordinates (X, Y, Z) of the object in real space from the equations (2) to (4) based on the representative parallax of each section Dn. When the calculated coordinates of the object are plotted on the real space, the coordinates of each object are plotted as points with some variation in the part corresponding to the part facing the host vehicle A of the object ahead as shown in FIG. Is done.

  For each point plotted in this way, the object detection unit 12 determines the distance in the X-axis direction and the distance in the Z-axis direction from the adjacent points in the real space, the left end point to the right end when grouped While searching for the total length in the X-axis direction up to the point, the points whose values are within the set threshold are grouped together, and each point in each group is linearly approximated as shown in FIG. The object is detected.

  Further, in the present embodiment, the object detection means 12 detects each object detected in this manner by surrounding it with a rectangular frame line on the reference image T as shown in FIG. Yes. 16 and 17, the label O and the label S indicate the type of the surface of the object facing the own vehicle A, the label O indicates the back surface of the object, and the label S indicates that the side surface of the object is detected. .

  In the present embodiment, the object detection means 12 further detects a preceding vehicle from the detected objects.

  In detecting the preceding vehicle, the object detection means 12 first estimates a trajectory that the host vehicle A will travel in the future based on the behavior of the host vehicle A as shown in FIG. An area corresponding to the vehicle width of the host vehicle A centering on Lest is calculated as the traveling path Rest of the host vehicle A.

The travel path Lest of the host vehicle A is calculated according to the following formula (5) or the following formulas (6) and (7) based on the vehicle speed V, the yaw rate γ, the steering angle δ of the steering wheel, etc. It can be calculated based on the turning curvature Cua of A. In the following equations, Re is a turning radius, Asf is a vehicle stability factor, and Lwb is a wheelbase.
Cua = γ / V (5)
Re = (1 + Asf · V ² ) · (Lwb / δ) (6)
Cua = 1 / Re (7)

  Then, the object detection means 12 detects an object closest to the host vehicle A among the objects existing on the traveling path Rest of the host vehicle A as a preceding vehicle that travels in front of the host vehicle A. . For example, in FIGS. 17 and 18, the vehicle O3 is detected as the preceding vehicle Vah.

  In the present embodiment, the object detection means 12 calculates the probability that the preceding vehicle detected in the previous sampling cycle and the object detected as the preceding vehicle in the current sampling cycle are the same three-dimensional object, etc. The preceding vehicle is tracked while maintaining consistency. Further, the object detection means 12 can detect that the detected preceding vehicle has left the front of the host vehicle and that the preceding vehicle becomes a new leading vehicle, or that another vehicle interrupts between the host vehicle and the preceding vehicle. Thus, the replacement of the preceding vehicle due to the other vehicle becoming a new preceding vehicle can be detected.

  The above is the configuration of normal object detection and preceding vehicle detection of the object detection means 12.

  However, in a state in which the imaging mode of the imaging unit 2 described later is switched to the first imaging mode, the lights that are provided on the rear surface of the preceding vehicle Vah such as the tail lamps TLl, TLr, and the brake lamp of the preceding vehicle Vah and are lit. In addition, the traffic light is in a state where only a lit blue, yellow or red signal lamp is imaged, and as described above, stereo matching cannot be appropriately performed on the edge portion of the preceding vehicle Vah itself. For this reason, in the distance image Tz, data on the parallax dp effective for the edge portion of the preceding vehicle Vah itself cannot be obtained, or even if it is obtained, it is difficult to be data sufficient to detect the preceding vehicle Vah etc. sufficiently effectively. .

  Therefore, in this embodiment, in such a case, the object detection unit 12 detects the preceding vehicle Vah, the traffic signal, and the like based on information on the tail lamps TL1, TLr, signal lights, and the like detected by the lights detection unit 11 described above. It is supposed to be.

  Specifically, the object detection unit 12 stores, for example, the maximum point Jmax, the minimum point Jmin, the maximum value Imax, the minimum value Imin, and the like of the left and right tail lamps TLl and TLr detected by the lamp detection unit 11 from the memory. Based on the information of the left and right tail lamps TL1 and TLr detected in the past, the positions at which the left and right tail lamps TLl and TLr are imaged on the reference image T in the current sampling cycle are estimated, and these are detected from the distance image Tz. By extracting the parallax dp of the image part of the predetermined range including them, calculating their average value, mode value, etc., and substituting the average value, mode value, etc. of the parallax dp into dp in the above equation (4) The distance Z between the host vehicle and the preceding vehicle Vah is calculated.

  The object detection means 12 outputs information on each object detected as described above, information on the preceding vehicle Vah, and the like to the outside of the object detection apparatus 1 as necessary.

  The night detection means 13 detects that the environment in which the host vehicle is traveling is nighttime. In this embodiment, the night detection means 13 detects lighting of the headlight of the host vehicle and detects that the environment in which the host vehicle is traveling is nighttime. In addition to this, for example, it is possible to detect that it is night by time, or to obtain the time of sunrise and sunset and detect that it is night based on it.

  The imaging mode switching unit 14 can capture the imaging mode of the imaging unit 2 at least when the night detection unit 13 detects that it is nighttime, and can capture at least the contour portion of the lit lighting object. It is configured to switch between the first imaging mode and the second imaging mode capable of detecting an object having no lights.

  As described above, when driving at night, it is practically possible to accurately detect lights such as tail lamps and signal lights, and pedestrians and lanes of the preceding vehicle, even if the exposure amount of the imaging means is adjusted. Impossible.

  Therefore, in this embodiment, in the first imaging mode for detecting lights such as the tail lamp of the preceding vehicle, the level is further increased from the shutter level indicating the degree of automatic exposure adjustment of the imaging means 2 described above, and the imaging means The adjustment is made so that at least one of the exposure amount 2 and the luminance value of the pixel output from the image pickup means 2 becomes small.

  By adjusting in this way, it becomes possible to prevent the light of the lights that have been lit by an object having lights such as the tail lamp of the preceding vehicle from diffusing to the surroundings, and to clearly capture the contours of the lights. . However, although pedestrians and lanes are not necessarily detected, their detection is basically sacrificed.

  Also, in the second imaging mode for detecting objects such as pedestrians and lanes that do not have lights, the level of the imaging means 2 is lowered to an automatic shutter level, and is adjusted as necessary. Adjustment is made so that at least one of the exposure amount of the image pickup means 2 and the luminance value of the pixel output from the image pickup means 2 is increased.

  By adjusting in this way, it becomes possible to image the object which does not have lights, such as a pedestrian and a lane, in the state which can be detected. However, since the light of the lamps lit by the object having the lamps is diffused and imaged, the parallax and the distance variation usually obtained by stereo matching based on them are increased.

  Hereinafter, the conditions, switching method, and the like for switching the imaging mode of the imaging unit 2 in the imaging mode switching unit 14 when night detection is detected by the night detection unit 13, that is, during night driving will be described.

[First condition]
(1) The imaging mode switching means 14 changes the imaging mode of the imaging means 2 to the second imaging mode when the preceding vehicle follow-up control or constant speed traveling control (control by the ACC device in the present embodiment) is operated in the host vehicle. To the first imaging mode. When the operation is stopped, the first imaging mode is switched to the second imaging mode.

  This condition is that, if at least the distance between the host vehicle and the preceding vehicle Vah is not accurately detected when the preceding vehicle follow-up control or the constant speed traveling control is activated, the preceding vehicle that is traveling stably as described above. This is a condition for giving priority to the detection of the preceding vehicle in such a case, because the host vehicle accelerates or decelerates behind and causes an abnormal behavior. Therefore, in this case, the imaging mode switching unit 14 gives up detection of a pedestrian or a lane, switches the imaging mode of the imaging unit 2 from the second imaging mode to the first imaging mode, and determines the exposure amount and imaging of the imaging unit 2. Adjustment is made so that at least one of the luminance values of the pixels output from the means 2 becomes small.

  In addition, when the operation of the preceding vehicle tracking control or the constant speed traveling control is stopped, the detection of the preceding vehicle Vah is continued, but the vehicle is crossing the roadside or the road rather than accurately detecting the preceding vehicle Vah. It is possible to improve vehicle safety by detecting a pedestrian or the like or by detecting a lane to prevent lane departure. Therefore, in this case, the imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 from the first imaging mode to the second imaging mode in order to facilitate detection of pedestrians, lanes, and the like, and exposes the imaging unit 2. Adjustment is made so that at least one of the amount and the luminance value of the pixel output from the imaging means 2 is increased.

[Second condition]
(2) The imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 from the second imaging mode to the first imaging mode when the speed of the host vehicle becomes equal to or higher than a predetermined speed. Further, when the speed becomes less than the predetermined speed, the first imaging mode is switched to the second imaging mode.

  If the speed of the host vehicle is 80 km / h, for example, it is considered that the host vehicle is traveling on an automobile exclusive road, and at least there are no pedestrians on the automobile exclusive road. I can concentrate on it. This condition is a condition for determining that the host vehicle is traveling on an automobile exclusive road. Also in this case, if this condition is satisfied, the imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 from the second imaging mode to the first imaging mode, regardless of other conditions, and exposes the imaging unit 2. Adjustment is made so that at least one of the amount and the luminance value of the pixel output from the imaging means 2 is small.

  Further, in a situation where the speed of the host vehicle is lower than, for example, 80 km / h, the host vehicle may be traveling on a general road instead of an automobile-only road. Therefore, when the speed of the host vehicle is less than a predetermined speed (for example, 80 km / h), the imaging mode switching unit 14 is configured to capture the imaging mode of the imaging unit 2 in order to easily detect pedestrians, lanes, and the like. Is switched from the first imaging mode to the second imaging mode, and at least one of the exposure amount of the imaging unit 2 and the luminance value of the pixel output from the imaging unit 2 is adjusted.

  In addition, when the own vehicle is provided with the car navigation apparatus etc., it can be determined whether the own vehicle is drive | working the road only for motor vehicles using the map information etc. Therefore, travel detection means (not shown) is provided for obtaining map information and detecting that the host vehicle is traveling on an automobile road, and the host vehicle is traveling on an automobile road by the travel detection means. It is also possible to configure so that the imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 in the same manner as described above.

[Third condition]
(3) The imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 from the second imaging mode to the first imaging mode when the wiper of the host vehicle is operated. Further, when the operation of the wiper is stopped, the first imaging mode is switched to the second imaging mode.

  When there is rainfall or snowfall that activates the wiper during night driving, if the imaging mode of the imaging unit 2 remains in the second imaging mode, the windshield disposed in front of the lens of the imaging unit 2 The light of the lights such as tail lamps TLl, TLr and brake lamps of the preceding vehicle Vah is greatly diffused (ie, blurred) in the surroundings due to the influence of water adhering to the vehicle. Etc. becomes difficult to detect.

  Therefore, in such a case, it is necessary to adjust so that at least one of the exposure amount of the imaging unit 2 and the luminance value of the pixel output from the imaging unit 2 becomes small. However, by adjusting the exposure amount of the imaging means 2 to be small, it becomes more difficult to detect pedestrians and lanes.

  In such a case, this condition is a condition for focusing on detection of the preceding vehicle so that the tail lamps TLl and TLr of the preceding vehicle Vah can be accurately detected. Therefore, in this case as well, if this condition is satisfied, the imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 from the second imaging mode to the first imaging mode regardless of other conditions, and the imaging unit 2 Is adjusted so that at least one of the exposure amount and the luminance value of the pixel output from the image pickup means 2 becomes small.

  Also, if the wiper operation is stopped, it is considered that rain or snowfall has stopped, so that the imaging mode switching means 14 makes it easy to detect pedestrians, lanes, etc., unless other conditions are met. The imaging mode of the imaging unit 2 is switched from the first imaging mode to the second imaging mode, and adjustment is performed so that at least one of the exposure amount of the imaging unit 2 and the luminance value of the pixel output from the imaging unit 2 is increased.

  Next, when any of the above conditions is satisfied and the imaging mode of the imaging unit 2 is switched, the imaging mode switching unit 14 switches the imaging unit 2 when switching from the second imaging mode to the first imaging mode. Then, an exposure adjustment level higher than the automatic exposure adjustment level is transmitted to forcibly reduce the exposure amount of the image pickup means 2.

  When switching from the first imaging mode to the second imaging mode, the automatic exposure adjustment level or an exposure adjustment level close to the automatic exposure adjustment level is transmitted to the imaging means 2 and the imaging means 2 as required. The exposure amount is adjusted.

  The imaging unit 2 changes at least one of its exposure amount and the luminance value of the pixel to be output according to the level transmitted from the imaging mode switching unit 14. How to adjust the exposure amount and the luminance value of the pixel is set in advance by the configuration of the imaging means 2 and the like.

  When the imaging mode switching unit 14 switches from the second imaging mode to the first imaging mode, the variation in the data of the parallax corresponding to the lit lights such as the tail lamp of the preceding vehicle Vah calculated in the same sampling period is changed. The exposure amount of the image pickup means 2 is adjusted while monitoring the degree.

  Here, for example, when the tail lamps TLl and TLr of the preceding vehicle Vah are imaged at night in the second imaging mode, as shown in FIG. 19, the light of the tail lamps TLl and TLr is greatly diffused to the surroundings and the edge of the body of the preceding vehicle Vah Images are taken out of the area. In this state, the stereo matching process is performed by the stereo matching unit 6 to calculate the parallax dp, and the representative parallax is calculated as described above. As shown in FIG. It varies widely for each Dn.

  The variation in the representative parallax dpn of each section Dn is caused by switching the imaging mode of the imaging unit 2 from the second imaging mode to the first imaging mode, and the exposure amount of the imaging unit 2 and the luminance of the pixel output from the imaging unit 2. It becomes smaller by adjusting so that at least one of the values becomes smaller.

Therefore, when the imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 from the second imaging mode to the first imaging mode, the variance degree of the representative parallax dpn of each section Dn is calculated, for example, the variance σ ^2. And the adjustment is performed so as to increase the level of forced exposure adjustment for the image pickup means 2 until the variance σ ² falls within a predetermined threshold value set in advance.

  Further, when the lights of the tail lamps TLl and TLr of the preceding vehicle Vah are imaged as shown in FIG. 20, the representative parallax dpn of each section Dn also varies in each sampling period, and the actual distance between the own vehicle and the preceding vehicle Or, a variation that cannot occur with the corresponding parallax occurs. Therefore, in this embodiment, the imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 from the second imaging mode to the first imaging mode for each sampling period of the representative parallax dpn of each section Dn. The degree of variation is also monitored.

  Specifically, for example, with respect to the representative parallax dpn of one or a plurality of sections Dn, the imaging mode switching unit 14 determines, for each section Dn, the representative parallax dpn in the previous sampling period and the representative parallax dpn in the current sampling period. The absolute value of the difference (change amount) is calculated, and the adjustment is performed by increasing the level of forced exposure adjustment for the image pickup means 2 until all the absolute values are within a preset threshold value. Yes.

  In this embodiment, in order to determine the level of forced exposure adjustment for the imaging means 2 based on the above two monitoring criteria, imaging is performed so that light protrudes from the edge portion of the body of the preceding vehicle Vah as shown in FIG. The tail lamps TLl and TLr of the preceding vehicle Vah that have been made are imaged in a state where there is almost no diffusion of light to the surroundings as shown in FIG.

  Therefore, the tail lamps TLl and TLr of the preceding vehicle Vah are accurately detected by the lights detection means 11, and based on this, the distance Z between the host vehicle and the preceding vehicle Vah can be accurately calculated by the object detection means 12. .

Note that, for example, the absolute value of the variation of the variance σ ² and the representative parallax dpn for each sampling period and the exposure adjustment level forcibly set in the imaging unit 2 are tabulated in advance, and the table is referred to. Thus, the exposure adjustment level forcibly set in the imaging unit 2 can be determined.

  In addition to this, for example, the level of forced exposure adjustment for the imaging unit 2 can be determined based on the distribution of luminance values in a state where light is diffused. Specifically, for example, in the case of a signal light with a traffic light lit, as shown in FIG. 21, the light detection means 11 provides a high luminance to the signal light SL of the traffic light B in a state where light is diffused. The highest point Jmax, the lowest point Jmin, the maximum value Imax and the minimum value Imin of the i coordinate are detected.

  Based on the highest point Jmax and the like of the high luminance area, an image area PT including a diffused light portion is set as shown in FIG. Then, when the luminance value p1ij of each pixel belonging to the pixel row Jmid having a width of 1 pixel extending in the horizontal direction at the middle position between the upper end and the lower end of the image region PT is extracted, as shown in the graph of FIG. The luminance value p1ij is high in the vicinity of the i-coordinate c_i of the center point, and a distribution shape of the luminance value p1ij is obtained such that the luminance value p1ij gradually decreases with increasing distance from the center.

  Therefore, as shown in FIG. 24, the imaging means 2 is compulsory until the maximum absolute value of the difference between the luminance value of a certain pixel on the pixel row Jmid and the luminance value of the adjacent pixel becomes equal to or greater than a predetermined threshold value. Adjustment is performed so as to increase the level of proper exposure adjustment. If comprised in this way, as shown in FIG. 25, it will come to image in the state which hardly diffuses the light of the signal lamp SL of the traffic light B to the circumference | surroundings, and the distance of the own vehicle and the traffic light B is made like the above. It becomes possible to detect accurately.

  On the other hand, when switching from the first imaging mode to the second imaging mode, it is possible to accurately detect an object having no lights such as the left and right lanes LL and LR shown in FIG. 6 and the pedestrian M shown in FIG. Thus, it is necessary to adjust the exposure adjustment level of the imaging means 2. Therefore, the imaging mode switching unit 14 sets the exposure adjustment level of the imaging unit 2 to a normal automatic level so that at least one of the exposure amount of the imaging unit 2 and the luminance value of the pixel output from the imaging unit 2 is increased. The exposure adjustment level is restored.

  However, since it cannot be said that the lanes LL, LR and the pedestrian M can be accurately captured at the level of automatic exposure adjustment of the imaging unit 2, in this embodiment, the imaging mode switching unit 14 is the same. The number of data points of the parallax dp in the image area in which the lanes LL and LR and the pedestrian M are detected in the sampling cycle is monitored.

  This monitoring area is set to the lanes LL and LR themselves if the object having no lights is the lanes LL and LR, for example. Then, the imaging mode switching unit 14 determines whether the number of data points of the lane candidate points cl and cr (see FIG. 5) used for detecting the lanes LL and LR is predetermined at the level of the automatic exposure adjustment of the imaging unit 2. If the threshold value is greater than or equal to the threshold value, the level is maintained. If the predetermined threshold value is not reached, the exposure adjustment level of the imaging unit 2 is forcibly increased or decreased so that the number of data points of the lane candidate points cl and cr is greater than or equal to the predetermined threshold value. It is supposed to be.

  Further, when the object having no lights is, for example, the pedestrian M shown in FIG. 26, the imaging mode switching means 14 displays a frame line (not shown) set so as to surround the detected pedestrian M. If the number of data points of the parallax dp corresponding to the pedestrian M in the monitoring area is equal to or greater than a predetermined threshold at the level of automatic exposure adjustment of the imaging means 2 as the monitoring area, the level is maintained and reaches the predetermined threshold If not, the exposure adjustment level of the imaging means 2 is forcibly increased or decreased so that the number of data points of the parallax dp corresponding to the pedestrian M is equal to or greater than a predetermined threshold.

  In addition, when the object which does not have lights is the pedestrian M, the number of pixels contained in the frame line which surrounds the pedestrian M decreases as the position of the pedestrian M from the own vehicle moves away. Therefore, it is preferable to set the predetermined threshold value as a value that varies according to the parallax dp, instead of a constant value.

  As described above, according to the imaging device adjusting device (processing device) 9 according to the present embodiment, the object to be detected when the imaging device 2 captures an image of the preceding vehicle Vah when traveling at night. In the case of a lamp that is lit by an object having lights such as the tail lamps TLl and TLr and the signal light SL of the traffic light B, and in the case of a target that does not have lights such as the pedestrian M and lanes LL and LR Thus, the imaging mode of the imaging unit is switched.

  Therefore, when traveling at night, the following traveling control for the preceding vehicle Vah is activated in an attempt to simultaneously detect those objects that are virtually impossible to detect at the same time. It is possible to avoid that the host vehicle accelerates or decelerates behind the preceding preceding vehicle Vah and detects the traffic light B that should be stationary with respect to the road surface as a moving object. Become.

  Further, when detecting the preceding vehicle Vah, the traffic light B, etc., the exposure amount of the imaging means 2 is reduced so that the light of the lights such as the tail lamps TL1, TLr, the signal light B, etc. is not diffused to the surroundings. When detecting lanes LL, LR, the exposure amount of the imaging means is automatically and accurately switched so as to increase the exposure amount, etc., and adjustment is performed so that the target to be detected is appropriately imaged. It becomes possible.

  Furthermore, since the object detection apparatus 1 according to the present embodiment includes the image pickup means adjustment device (processing means) 9 having the above-described functions, the preceding vehicle Vah is used in the automatic exposure adjustment provided in the image pickup means 2. The imaging mode switching unit 14 can appropriately switch the imaging mode of the imaging unit 2 even in an environment where light from the left and right tail lamps TLl, TLr, brake lamp, and the like is diffused and imaged.

  Therefore, when the preceding vehicle Vah or the like is detected, the contours of the tail lamps TLl and TLr and the like are clearly imaged, so that it is possible to accurately detect the preceding vehicle Vah and the like. When detecting M or the like, the exposure amount or the like of the imaging means is automatically switched to increase the exposure amount or the like, so that the pedestrian M can be accurately detected.

  In the present embodiment, as described above, when the preceding vehicle follow-up control or the constant speed traveling control is operated, when the speed of the host vehicle exceeds a predetermined speed, or the wiper of the host vehicle is operated. In this case, the imaging mode switching unit 14 switches the imaging mode of the imaging unit 2 between the first imaging mode and the second imaging mode. Therefore, if the driver is careful, it can be known whether the imaging mode of the imaging means 2 is the first imaging mode or the second imaging mode.

  However, since the driver concentrates on driving, even if, for example, the imaging mode of the imaging unit 2 is in the first imaging mode and a pedestrian or the like is not easily detected, this is usually not noticeable. . Therefore, for example, a notification unit that notifies the driver that the imaging mode of the imaging unit 2 is the first imaging mode by displaying on a display screen or lighting a lamp is provided. Is preferred.

It is a block diagram which shows the structure of the object detection apparatus containing the adjustment apparatus of the imaging means which concerns on this embodiment. It is a figure explaining the method of the stereo matching process in an image processor. It is a figure which shows an example of a reference | standard image. It is a figure which shows the distance image formed based on the reference | standard image etc. of FIG. It is a figure explaining the method of the detection of the lane candidate point on a reference | standard image. It is a figure which shows the left and right lane detected on the reference | standard image. It is a figure showing the preceding vehicle detected on the reference | standard image, and the pixel of the taillight center of the last time. It is a figure showing the high-intensity area | region detected by searching on the pixel row | line containing the last tail lamp center, and the upper end and lower end. It is a figure explaining the method of the detection of the high-intensity area | region in the pixel column on the right of the pixel column of FIG. It is a figure explaining the change of the length of the vertical direction of a high-intensity area | region, When (A) becomes the maximum value, (B) is shorter than a maximum value, (C) is still shorter than (B) Represents. It is a figure explaining the case where the length of the vertical direction of a high-intensity area | region increased from the minimum value. It is a figure explaining the tail lamp center, the highest point, the lowest point, the maximum value of i coordinate, and the minimum value detected about each of the left and right tail lamps. It is a figure which shows each division which divides | segments a distance image. It is a figure which shows an example of the histogram produced for every division of FIG. It is the figure which plotted the coordinate on real space based on the representative parallax for every division. It is a figure showing the object detected in real space based on each point of FIG. It is a figure showing each object and preceding vehicle which were detected by being enclosed by the frame line on the reference | standard image. It is a figure showing the driving | running | working locus | trajectory of the own vehicle in real space, a travel path, and a preceding vehicle. It is a figure showing the right and left tail lamps of the preceding vehicle in which light diffused. It is a figure explaining a mode that the representative parallax calculated based on FIG. 19 varies for every division Dn. It is a figure explaining the center point, the highest point, the lowest point, the maximum value of i coordinate, and the minimum value detected about the signal lamp of the traffic light. It is a figure explaining the image area | region containing the signal lamp set on the reference | standard image. It is a graph which shows the distribution shape of the luminance value in the image area | region of FIG. It is a graph which shows the distribution shape of the luminance value after forced adjustment of the exposure amount etc. with respect to an imaging means was performed. It is a figure showing the signal lamp of the traffic light imaged in the state where light does not diffuse. It is a figure which shows an example of the reference | standard image by which the pedestrian was imaged.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Object detection apparatus 2 Imaging means 6 Stereo matching means 9 Imaging means adjustment apparatus 12 Object detection means 13 Night detection means 14 Imaging mode switching means B Object (signal)
dp Parallax LL, LR Target (lane)
M Target (pedestrian)
p1ij Luminance value PB Pixel block SL Lights (Signal light)
T image (reference image)
TLl, TLr Lamps (tail lamp)
Vah object (leading vehicle)

Claims (16)

Object detection means for detecting an object from the image captured by the imaging means;
Night detection means for detecting that it is night,
When it is detected by the night detection means that it is nighttime, the imaging mode of the imaging means is at least a first imaging mode capable of imaging the contour part of the lit lamps of an object having lamps; An imaging mode switching means for switching between a second imaging mode capable of detecting an object having no lights;
A device for adjusting imaging means, comprising:   2. The imaging unit according to claim 1, wherein the lighting of the object having the lighting is a lighting of a light provided on a rear surface of a preceding vehicle or a signal light of a traffic light. Adjustment device.   The apparatus according to claim 1 or 2, wherein the object not having the lights is a pedestrian. Furthermore, lane detection means for detecting the left and right lanes of the host vehicle from the image,
The apparatus for adjusting an imaging unit according to any one of claims 1 to 3, wherein the object not having the lights is the lane.   The imaging mode switching means switches the imaging mode of the imaging means from the second imaging mode to the first imaging mode when the preceding vehicle follow-up control or constant speed traveling control is activated in the host vehicle, and the operation stops. The imaging device adjusting apparatus according to any one of claims 1 to 4, wherein the imaging mode of the imaging unit is switched from the first imaging mode to the second imaging mode when the imaging mode is set.   The imaging mode switching unit switches the imaging mode of the imaging unit from the second imaging mode to the first imaging mode when the speed of the host vehicle becomes equal to or higher than a predetermined speed, and the speed is less than the predetermined speed. 6. The image pickup unit adjustment apparatus according to claim 1, wherein the image pickup mode of the image pickup unit is switched from the first image pickup mode to the second image pickup mode. . It further comprises travel detection means for detecting that the host vehicle is traveling on an automobile road,
The imaging mode switching means switches the imaging mode of the imaging means from the second imaging mode to the first imaging mode when the travel detection means detects that the host vehicle is traveling on a road dedicated to automobiles. The imaging mode of the imaging unit is switched from the first imaging mode to the second imaging mode when the traveling detection unit no longer detects that the host vehicle is traveling on an automobile exclusive road. The imaging device adjusting apparatus according to any one of claims 1 to 6.   The imaging mode switching means switches the imaging mode of the imaging means from the second imaging mode to the first imaging mode when the wiper of the host vehicle is operated, and the imaging when the wiper operation is stopped. The imaging device adjusting apparatus according to any one of claims 1 to 7, wherein the imaging mode of the unit is switched from the first imaging mode to the second imaging mode.   The imaging mode switching unit adjusts at least one of an exposure amount of the imaging unit and a luminance value of a pixel output from the imaging unit in accordance with the switching of the imaging mode. The adjusting device for an imaging unit according to claim 8. The imaging means is a stereo imaging means for simultaneously imaging an object with a pair of cameras and outputting a pair of images,
The imaging means according to claim 9, further comprising stereo matching means for performing a stereo matching process on a pair of images taken by the imaging means and calculating a parallax for each pixel block of the image. Adjustment device.   When the imaging mode switching unit switches the imaging mode of the imaging unit from the second imaging mode to the first imaging mode, the imaging mode switching unit of the object having the lamps calculated by the stereo matching unit in the same imaging cycle 11. The adjustment device for an imaging unit according to claim 10, wherein the adjustment is performed so that variation in each data of the parallax corresponding to the lit lights is within a predetermined threshold.   When the imaging mode switching unit switches the imaging mode of the imaging unit from the second imaging mode to the first imaging mode, the lamps calculated by the stereo matching unit in an imaging period that is temporally adjacent are used. 12. The adjustment according to claim 10, wherein the adjustment is performed so that an absolute value of a temporal change amount of the parallax data corresponding to a lamp that is lighted by an object is within a predetermined threshold. The imaging device adjusting apparatus according to claim.   The imaging mode switching unit is an image obtained by capturing an image of lighting of an object having the lamps in the image when the imaging mode of the imaging unit is switched from the second imaging mode to the first imaging mode. Based on the luminance value distribution of the region, the adjustment is performed so that the maximum absolute value of the difference between the luminance value of the pixel in the image region and the luminance value of the pixel adjacent to the pixel is equal to or greater than a predetermined threshold value. The imaging device adjusting apparatus according to claim 9, wherein the imaging device adjusting apparatus is performed.   The imaging mode switching means is an object that does not have the lights calculated by the stereo matching means in the same imaging cycle when the imaging mode of the imaging means is switched from the first imaging mode to the second imaging mode. The imaging device adjustment apparatus according to any one of claims 10 to 13, wherein the adjustment is performed so that the number of data points of the parallax corresponding to a predetermined threshold value or more.   The information processing device according to any one of claims 1 to 14, further comprising notification means for notifying a driver of the host vehicle that the imaging mode of the imaging means is the first imaging mode. The imaging device adjusting apparatus according to claim. The imaging means for imaging the surroundings of the host vehicle;
An adjustment device for an imaging means according to any one of claims 1 to 15,
The object detecting means detects an object having the lights when in the first imaging mode, and an object not having the lights when in the second imaging mode. Object detection device.

Images