JP2010006270A - Vehicle behavior detecting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle behavior detecting device detecting behavior of an own vehicle with a simple method. <P>SOLUTION: The vehicle behavior detecting device for detecting the behavior of the own vehicle is provided with: an imaging means for imaging the periphery of the own vehicle; a comparison means (S3) for comparing pixel information in a comparison region having a smaller proportion of a moving object than other regions among a plurality of images respectively imaged at different imaging time points by the imaging means; and a detection means (S3) for detecting a change in behavior of the own vehicle based on the comparison result. The comparison means compares luminance waveforms in the vertical direction of a vehicle in the comparison region among a plurality of images respectively imaged at different imaging time points by the imaging means. It is preferable that the detection means detect pitching of the own vehicle, based on the comparison result by the comparison means. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle behavior detection device.

  In recent years, driving support devices such as a collision prevention device, an inter-vehicle distance control device, and a following traveling device have been developed. In these driving assistance devices, it is important to detect a vehicle traveling in front of the host vehicle. In order to improve detection accuracy, an object detection apparatus includes an apparatus provided with detection means having two different characteristics: a detection means using a radar such as a millimeter wave radar and a detection means using an image from a camera. In this object detection device, an image processing area is set according to the position information of the object detected by the radar, image processing is performed on the set image processing area in the image captured by the camera, and the image processing area Some of them detect object information.

The behavior of a vehicle changes (pitching, rolling, yawing) depending on road surface conditions, acceleration / deceleration or turning during traveling. Therefore, for example, if the image processing area is set according to the position information by the radar during the occurrence of pitching, the radar cannot detect the information in the height direction. Therefore, all or part of the object is included in the set image processing area. May only exist. In such a case, information about the object cannot be detected from the image processing area. Therefore, it is necessary to correct the position of the image processing area in accordance with the behavior of the host vehicle. To that end, it is important to detect the behavior of the host vehicle. In the apparatus described in Patent Document 1, the same feature amount and its displacement amount are detected in different images taken by the cameras, and the turning amount of the host vehicle is calculated from the displacement amount of the feature amount.
JP 2008-82932 A

  In the above-described apparatus, processing for extracting a feature amount from an image is necessary, and the processing is complicated. Therefore, the processing load increases. Therefore, it cannot be realized with an inexpensive image processing chip, and the responsiveness also deteriorates.

  Then, this invention makes it a subject to provide the vehicle behavior detection apparatus which can detect the behavior of the own vehicle with a simple method.

  The vehicle behavior detection apparatus according to the present invention is a comparison in which the proportion occupied by a moving object is small compared to other regions between an imaging unit that captures the periphery of the host vehicle and a plurality of images captured at different imaging times by the imaging unit. Comparing means for comparing pixel information in the region, and detecting means for detecting a change in the behavior of the host vehicle based on the comparison result of the comparing means.

  In this vehicle behavior detection device, the periphery of the host vehicle is imaged by the imaging unit, and pixel information (for example, luminance information, RGB information) in a predetermined comparison region is compared between a plurality of images having different imaging times by the comparison unit. This comparison area is an area in which the moving object occupies a smaller ratio than other areas. For example, when there is no moving object in the comparison area, the movement of the stationary object between the comparison areas at different imaging times corresponds to the movement of the imaging means (and thus the own vehicle). Therefore, by comparing the luminance information between the comparison areas in which the moving object is not included as much as possible, the movement of the host vehicle can be estimated from the difference in pixel information corresponding to the movement of the stationary object. Therefore, in the vehicle behavior detection apparatus, the behavior of the host vehicle is detected by the detection means based on the comparison result of the comparison means. As described above, the vehicle behavior detection device can detect the behavior of the host vehicle by a simple method of simply comparing pixel information in a comparison region between a plurality of images having different imaging times. For this reason, even a cheap image processing chip with a low processing load can be processed with good responsiveness.

  In the vehicle behavior detection apparatus of the present invention, the comparison region may be a region located outside by a predetermined amount from the center of the image captured by the imaging unit.

  In general, the imaging means mounted on the vehicle is applied as a means for detecting an obstacle (for example, another vehicle, a pedestrian), so that an imaging position or the like so that such an obstacle enters near the center of the image. The imaging direction is set. Therefore, there is a high possibility that a moving object such as another vehicle exists near the center of the image. Therefore, in order to avoid such an image center area, an area located outside a predetermined amount is set as a comparison area. The predetermined amount is set to an appropriate amount in consideration of the horizontal and vertical ranges that enter the image, the size of the object to be detected, and the like.

  In the vehicle behavior detection apparatus of the present invention, the comparison means compares the luminance waveform in the vehicle vertical direction in the comparison region between a plurality of images captured at different imaging times by the imaging means, and the detection means is the comparison means. It is preferable to detect the pitching of the host vehicle based on the comparison result.

  In this vehicle behavior detection apparatus, the comparison means compares luminance waveforms extending in the vertical direction generated from the luminance values of the pixels in the comparison region between a plurality of images having different imaging times. For example, if each image with different imaging time is shifted in the vertical direction due to pitching of the host vehicle, and there is no moving object in the comparison area of each image, each luminance waveform generated from each comparison area is The shape of the waveform is substantially the same, but shifted in the vertical direction. Therefore, in the vehicle behavior detection device, the vertical movement (that is, pitching) of the host vehicle is detected from the vertical shift of the luminance waveform between the comparison regions by the detection means. Thus, in the vehicle behavior detection device, the processing load can be further reduced by comparing the pixel information using the luminance waveform.

  The present invention can detect the behavior of the host vehicle by a simple method of simply comparing pixel information in a comparison region between a plurality of images having different imaging times.

  Hereinafter, an embodiment of a vehicle behavior detection device according to the present invention will be described with reference to the drawings.

  In the present embodiment, the object detection device according to the present invention is applied to a periphery monitoring device mounted on a vehicle. The periphery monitoring device according to the present embodiment detects obstacles in front of the host vehicle (for example, moving objects such as other vehicles, bicycles, and pedestrians, stationary objects such as falling objects), and the detected obstacle information is detected. Provided to the driver by output or voice or display to a driving assistance device (such as a collision prevention device). The detection direction is front, but may be other directions such as side and rear.

  With reference to FIGS. 1-3, the periphery monitoring apparatus 1 which concerns on this Embodiment is demonstrated. FIG. 1 is a configuration diagram of the periphery monitoring device according to the present embodiment. FIG. 2 is an explanatory diagram of a pitching correction amount estimation method in the ECU of FIG. FIG. 3 is an example of a captured image showing the relationship between the pitching of the host vehicle and the image processing area.

  The periphery monitoring device 1 includes a millimeter wave sensor and a camera as detection means in front of the host vehicle, and sets an obstacle sensing area (image processing area) in an image captured by the camera by using obstacle sensing information by the millimeter wave sensor. To do. In particular, the periphery monitoring device 1 detects the pitching of the host vehicle in order to reliably detect obstacle information from the sensing area even when the behavior of the host vehicle changes (especially pitching), and according to the pitching. Correct the sensing area. The periphery monitoring device 1 includes a millimeter wave sensor 2, a camera 3, and an ECU [Electronic Control Unit] 4.

  In the present embodiment, the camera 3 corresponds to the imaging unit described in the claims, and the function in the ECU 4 corresponds to the comparison unit and the detection unit described in the claims.

  The millimeter wave sensor 2 is a radar sensor that detects an object using millimeter waves. The millimeter wave sensor 2 is attached to a predetermined height position in the center of the front side of the host vehicle (a height position at which an obstacle to be detected can be reliably detected). The millimeter wave sensor 2 transmits the millimeter wave forward while scanning the millimeter wave in the left-right direction, and receives the reflected millimeter wave. The angle in the vertical direction when transmitting the millimeter wave is fixed, and an angle that is parallel to the road surface when the vehicle is stopped is set. In the millimeter wave sensor 2, a millimeter wave signal composed of millimeter wave information (scanning azimuth angle in the left and right direction, transmission time, reception time, reflection intensity, etc.) for each reflection point (detection point) that has been able to receive the reflected millimeter wave is converted into an ECU Send to.

  The camera 3 is a camera that images the front of the host vehicle. The camera 3 is attached to the front center of the host vehicle. The camera 3 images the front of the host vehicle and transmits the captured image information to the ECU 4 as an image signal. This captured image is an image of a frame every certain time (for example, 1/30 second). The camera 3 has a wide imaging range in the left-right direction, and when the number of lanes is small, the camera 3 can capture images up to a range that sufficiently includes the side lanes and shoulders outside the driving lane and the opposite lane. Since the ECU 4 uses the right side end region of the image in order to detect the pitching of the host vehicle, the right side (opposite lane side) range may be imaged at a wider angle. Since each pixel of the captured image only needs to have at least luminance information, the camera 3 may be a color camera or a monochrome camera.

  The ECU 4 is an electronic control unit including a CPU [Central Processing Unit], a ROM [Read Only Memory], a RAM [Random Access Memory], an image processing chip, and the like, and comprehensively controls the periphery monitoring device 1. In the ECU 4, when an application program stored in the ROM is loaded into the RAM and executed by the CPU, the millimeter wave obstacle detection function, the image processing area setting function, the own vehicle momentum estimation function, the image processing area correction function, the image obstacle A detection function is configured. The ECU 4 takes in the millimeter wave signal from the millimeter wave sensor 2 and the image signal from the camera 3 at regular time intervals, and stores the millimeter wave information and the captured image information in time series. Then, the ECU 4 performs processing in each function using these pieces of information, and outputs detected obstacle information (for example, relative distance, relative speed, lateral position, size information, type information) to the driving support device or driving To provide

  The millimeter wave obstacle detection function will be described. The ECU 4 uses the millimeter wave information at the current time (t), and calculates the relative distance to the obstacle ahead based on the time from transmission to reception of the millimeter wave. Further, the ECU 4 calculates the relative speed with the obstacle ahead based on the change of the distance calculated at the current time (t) and the distance calculated at the previous time (t−1). Further, the ECU 4 uses the millimeter wave information at the current time (t) to detect the direction of the millimeter wave that is reflected most strongly among the reflected millimeter waves, and from the direction, the traveling direction of the host vehicle and the obstacle The angle formed with the direction of the object is obtained, and the lateral position is calculated from the angle. When calculating the relative speed, past relative distance data may be used further than the previous time (t−1).

  The image processing area setting function will be described. The ECU 4 sets the image processing area based on the obstacle information (particularly relative distance and lateral position) detected by the millimeter wave obstacle detection function. Here, the center position of the image processing area is determined based on the relative distance, the horizontal position, and the like, and the image processing area that secures the preset vertical and horizontal widths from the central position is set.

  The own vehicle momentum estimation function will be described. The ECU 4 extracts the right side edge region (comparison region) from the frame image at the current time (t) and the frame image at the previous time (t−1). This right side edge region is a region farthest from the center of the image, and is a background region that does not include the traveling lane or oncoming lane of the own vehicle (and thus the vehicle traveling in each lane) as much as possible. Therefore, the side edge area is an area where there is a low possibility that the running vehicle, which is the largest moving object in the image, is included, and the ratio occupied by the moving object is a smaller area than the other areas. However, when the number of lanes is large, the side edge region may include lanes, but the size of the traveling vehicle is smaller than the vicinity of the center of the image. The lateral width (number of pixels) of the side edge region is set in advance by experiment or the like in consideration that a sufficient difference can be detected when luminance waveforms are compared.

  The ECU 4 generates a luminance waveform from information on all pixels in the side end region of the two frames. Here, the luminance values of all the pixels included in the horizontal column are integrated for each vertical column of the side edge region, and a luminance waveform extending in the vertical direction is formed from the integrated luminance value for each vertical column. Is done. For example, when the host vehicle is traveling straight and no moving object is included in the side end region, the luminance waveforms of the two frames have substantially the same waveform shape, The waveform is shifted in the vertical direction in accordance with the vertical movement (pitching) of the host vehicle. However, the vertical position shift corresponding to the movement of the host vehicle in the front-rear direction is smaller than the vertical position shift corresponding to the vertical movement of the host vehicle, and is within an error range. Therefore, the vertical shift between the luminance waveforms of the two frames can be assumed as a shift corresponding to the pitching of the host vehicle.

  The ECU 4 compares the luminance waveforms of the two frames and obtains the amount of deviation (number of pixels) in the vertical direction of the two luminance waveforms. This comparison method may be any method, for example, using SAD [Sum of Absolute Difference] or moving one luminance waveform in the vertical direction to move one luminance waveform and the other luminance waveform. To find the amount of movement that best matches. The calculated vertical shift amount corresponds to the vertical shift amount of the image between two frames according to the pitching (pitch angle) of the host vehicle, and is the vertical correction amount of the position of the image processing area. In addition, when comparing a luminance waveform, you may use the image of the past frame further from previous time (t-1).

FIG. 2 shows an example of a frame image P _t frame image P _t-1 and the current time before being displaced in the vertical direction by pitching the vehicle time (t-1) (t) . From this each image _{P t-1, P t,} in order to compare the luminance waveform, the right side end area _{A t-1, A t} is extracted. Side end region A _t-1, A _t includes a post T with characteristic luminance information of the other background portion. Therefore, the luminance waveform _{W t-1, W t} that is generated from the side end area _{A t-1, A t,} part _{B t-1} integrated value of luminance corresponding to the portion of the post T increases, B Each has _t . By comparing the two luminance waveforms W _t−1 and W _t , the amount of vertical shift (number of pixels PW) is obtained. By the way, even if there is no object characterized by luminance information such as this post, the side edge area such as the boundary between the ground side and the sky, the boundary between the sidewalk and the building, etc. unless the side edge area is a uniform image throughout If there is a portion where the luminance information changes, the amount of deviation in the vertical direction can be obtained.

  The image processing area correction function will be described. In ECU4, it is determined whether the pitching correction amount calculated | required by the own vehicle momentum estimation function is larger than a threshold value. The threshold value is a pitching correction amount that affects the image processing result (obstacle information) when image processing is performed in an image processing region (region set by the image processing region setting function) that is displaced due to the pitching of the host vehicle. This is a threshold value for determining whether or not (the number of pixels), and is set in advance by an experiment or the like. When the pitching correction amount is larger than the threshold value, the ECU 4 translates the position of the image processing region set by the image processing region setting function in the vertical direction by the pitching correction amount, and finally determines the image processing region.

  On the other hand, when the pitching correction amount is equal to or smaller than the threshold value, the ECU 4 finally determines the image processing area set by the image processing area setting function as the image processing area.

  The image obstacle detection function will be described. The ECU 4 extracts an image of the image processing area that is finally determined from the image at the current time (t). The ECU 4 detects obstacle information from the image in the image processing area. Obstacle information that can be detected from an image includes obstacle size information (width, height), obstacle type information using pattern recognition (for example, a vehicle, a pedestrian), and the like.

  The example shown in FIG. 3 is a case where an image processing area is set based on obstacle information based on millimeter wave information when the host vehicle is pinching. In this case, an image processing area IA that is shifted downward with respect to the preceding vehicle FV in the captured image P is set. At this time, the vertical correction amount (pixel number PW) corresponding to the pitching of the host vehicle is obtained, and the image processing area RA is reset by moving the image processing area IA upward in accordance with the pixel number PW. The The front vehicle FV is sufficiently included in the image processing area RA.

  With reference to FIGS. 1-3, operation | movement of the periphery monitoring apparatus 1 is demonstrated. In particular, the image processing area determination process in the ECU 4 will be described with reference to the flowchart of FIG. FIG. 4 is a flowchart showing a flow of image processing area determination processing in the ECU of FIG.

  The millimeter wave sensor 2 scans with a millimeter wave at regular time intervals, and transmits a millimeter wave signal indicating millimeter wave information about each detection point to the ECU 4. Every time the millimeter wave signal is received, the ECU 4 stores millimeter wave information in time series.

  The camera 3 captures the front of the host vehicle at regular time intervals and transmits an image signal indicating the image information to the ECU 4. Each time the image signal is received, the ECU 4 stores the image information of the frame in time series.

  The ECU 4 reads out the millimeter wave information at the current time (t) and detects obstacle information (relative distance, relative speed, lateral position, etc.) based on the millimeter wave information (S1). Then, the ECU 4 sets an image processing area based on the obstacle information (S2).

  The ECU 4 reads the images of the frame at the current time (t) and the frame at the previous time (t−1), respectively, and extracts the right side end region (comparison region) from the images of the two frames. Then, the ECU 4 calculates the luminance waveform of each side edge region, and estimates the correction amount according to the pitching of the host vehicle from the vertical difference between the two luminance waveforms (S3).

  The ECU 4 determines whether or not the pitching correction amount is larger than a threshold value (S4). If it is determined in S4 that the pitching correction amount is equal to or smaller than the threshold value, the ECU 4 determines the image processing area set in S2.

  On the other hand, when it is determined in S4 that the pitching correction amount is larger than the threshold value, the ECU 4 corrects the position of the image processing region according to the pitching correction amount, and determines the corrected image processing region (S5).

  Then, the ECU 4 detects obstacle information (size information, type information, etc.) from the determined image processing area in the image at the current time (t).

  The ECU 4 provides the detected obstacle information (relative distance, relative speed, lateral position, size information, type information, etc.) to the driver by outputting to the driving support device or by voice or display.

  According to the periphery monitoring device 1, it is possible to detect the pitching of the host vehicle (particularly, the correction amount corresponding to the pitching) by a simple method of simply comparing the luminance waveforms in the comparison region between the plurality of images having different imaging times. . For this reason, even an inexpensive image processing chip with a low processing load can be processed with good responsiveness.

  In particular, since the peripheral monitoring device 1 uses the side edge area in the image as a comparison area, there is little possibility of a large moving object such as a traveling vehicle entering, and the pitching correction amount is obtained by comparing the luminance waveform in the side edge area. Can be obtained with high accuracy. Moreover, in the periphery monitoring apparatus 1, the processing load can be further reduced by comparing the luminance waveforms.

  Note that the position variation of the image processing area becomes the largest due to the change in the behavior of the host vehicle due to the effect of pitching.By performing position correction according to pitching as in this embodiment, the position of the image processing area The effect of position correction is very high. By the way, when performing position correction according to the roll, the position is corrected by rotating the image center as the rotation axis, so the position of the image processing area that exists near the center of the image does not change so much. The correction effect is low. As described above, in this embodiment, in order to reduce the processing load, only the correction for the pitching with the highest effect of the position correction of the image processing region is performed.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is implemented in various forms, without being limited to the said embodiment.

  For example, in the present embodiment, the present invention is applied to the periphery monitoring device, but it can also be applied to the vehicle behavior detection device itself or a vehicle behavior detection function in a driving support device or a vehicle behavior control device.

  Further, although the millimeter wave sensor is applied in this embodiment, the millimeter wave sensor is applied, but other means such as other radar sensors such as a laser sensor and an ultrasonic sensor, a sensor using an image of a camera, and an infrastructure. May be applied.

  In this embodiment, the luminance is used as the pixel information and the luminance waveform is compared. However, other pixel information such as RGB may be used, and other information besides the luminance waveform may be used. You may compare.

  Further, in the present embodiment, the luminance waveform in the vertical direction of the image is compared and the pitching is detected as the behavior. Other behavior may be detected.

  In the present embodiment, the side edge of the predetermined image is used as the comparison area. However, other areas such as upper and lower edges may be used, and based on the position information of the moving object detected from the millimeter wave information. Thus, an area that does not include a moving object may be set dynamically.

  In the present embodiment, the pitching correction amount (number of pixels) for correcting the image processing region is calculated as the amount of movement of the host vehicle. However, the pitch angle directly indicating the amount of movement of the host vehicle may be calculated. . Incidentally, since the distance between adjacent pixels and the pitch angle are determined in advance, the obtained pitching correction amount (number of pixels) can be easily converted into the pitch angle.

It is a block diagram of the periphery monitoring apparatus which concerns on this Embodiment. It is explanatory drawing of the estimation method of the pitching correction amount in ECU of FIG. It is an example of the captured image which shows the relationship between the pitching of the own vehicle, and an image processing area. It is a flowchart which shows the flow of the image process area | region determination process in ECU of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Perimeter monitoring apparatus, 2 ... Millimeter wave sensor, 3 ... Camera, 4 ... ECU

Claims (3)

Imaging means for imaging the periphery of the vehicle;
Comparison means for comparing pixel information in a comparison region in which the proportion of moving objects is smaller than other regions between a plurality of images captured at different imaging times by the imaging unit,
A vehicle behavior detection apparatus comprising: a detection unit that detects a change in behavior of the host vehicle based on a comparison result of the comparison unit.   The vehicle behavior detection device according to claim 1, wherein the comparison region is a region located outside a predetermined amount from the center of the image captured by the imaging unit. The comparison means compares the luminance waveform in the vehicle vertical direction in the comparison area between a plurality of images with different imaging times captured by the imaging means,
The vehicle behavior detection device according to claim 1, wherein the detection unit detects pitching of the host vehicle based on a comparison result of the comparison unit.

Images