JP2012014553A - Apparatus for monitoring surrounding of vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus for monitoring surrounding of vehicle in which no synchronous circuit is needed and the detection timing of a plurality of detection means can be adjusted.SOLUTION: An apparatus for monitoring surrounding of a vehicle comprises: first detection means for continually detecting object information of surrounding of a vehicle at a prescribed period, second detection means for detecting the object information of surrounding of a vehicle in a different manner from that of the first detection means at a prescribed timing; and time correction means for correcting the detection timing at which the first detection means have detected the object information within one cycle as one timing of the one cycle. The one timing within the one cycle is made to match one of predetermined timings of the second detection means.

Description

  The present invention relates to a vehicle periphery monitoring apparatus, and more specifically to synchronization correction of a radar and a camera in a vehicle periphery monitoring apparatus.

  Patent document 1 discloses the target object recognition apparatus which recognizes the target object ahead of a vehicle. This object recognition device uses a scanning radar and an imaging device (camera) in combination to identify an object in an infrared image based on detection information of an object candidate obtained from the radar.

Japanese Patent No. 4205825

  There is a difference in the data acquisition timing between the radar and the camera (hereinafter also referred to as “synchronization error”). If this synchronization error is not corrected, the detection range (position) of both will greatly deviate, corresponding to the detection information of the radar. There are cases where an object in a camera image cannot be detected. This synchronization shift is caused by a shift in data acquisition timing between a radar that requires time correction of a predetermined scanning time and detection timing and a camera that captures images at each predetermined timing.

  In addition, when correcting this synchronization shift, it is necessary to simultaneously correct data fluctuations due to vehicle movement during the timing shift time (positional shift of an object on the image, etc.).

As a method for correcting the synchronization error, a synchronization signal is output from a radar with a slow (long) detection cycle by synchronizing the detection start timing of both with an external synchronization circuit, and the timing of the camera with a fast (short) detection cycle is set to the radar timing. There is a known method to synchronize the detection start timing in accordance with.
However, the conventional method may require a new synchronization circuit, or may require a dedicated design for a radar having a long processing time to drive in accordance with a camera having a short processing time. Furthermore, since radar with a long processing time does not operate at the original processing cycle, it is difficult to deploy the data in another system, and the original performance (detection speed, etc.) of the radar or camera is utilized. There is a fear that you can not.

  Therefore, the present invention does not require a new synchronization circuit, and can adjust (correct) the detection timing of the radar and the camera, and simultaneously correct the data fluctuation due to the vehicle movement during the detection timing shift time. An object of the present invention is to provide a vehicle periphery monitoring device.

  The present invention provides a first detection means for continuously detecting object information around the vehicle at a predetermined cycle, and a method different from the first detection means for detecting object information around the vehicle at every predetermined timing. 2 detection means and time correction means for correcting the detection timing of the object information detected by the first detection means within one cycle as one timing within the one cycle, and one timing within the one cycle is: The vehicle periphery monitoring device is characterized in that it coincides with one of the predetermined timings of the second detection means.

  According to the present invention, without using a synchronization signal, the inherent timing control of each of the first detection means (for example, radar) and the second detection means (for example, camera) is utilized as it is, that is, asynchronous control of both is performed. It is possible to correct the acquisition timing of both data (correlation between data) while using them.

  According to one aspect of the present invention, a radar that detects an object around the vehicle, a camera that captures an image around the vehicle, and a corresponding image captured by the camera that corresponds to the position or size of the object detected by the radar There is provided a vehicle periphery monitoring device comprising: means for setting a predetermined area on the top and identifying an object in the predetermined area. The periphery monitoring device includes a detection unit that detects a difference between the detection timing of the position or size of the object detected by the radar and the acquisition timing of the corresponding image acquired by the camera, and movement of the vehicle corresponding to the difference. Calculation means for calculating the amount, and correction means for correcting the position or size of the object detected by the radar according to the amount of movement of the vehicle.

  According to an aspect of the present invention, the position or size of an object detected by a radar is corrected according to the amount of movement of a vehicle from the timing at which a camera image is acquired until the radar information is acquired. Thus, the matching accuracy between the camera image and the radar information can be improved.

  According to one aspect of the present invention, the radar detects an object every predetermined scanning time, the detection timing by the radar is a timing at the start or end of the predetermined scanning time, and the image acquisition timing by the camera is predetermined. Is the acquisition timing within the scanning time.

  According to one aspect of the present invention, the detection timing of the radar is matched with the image acquisition timing of the camera within the radar scanning time without requiring a new synchronization circuit, thereby correcting the detection timing of both. can do.

It is a block diagram which shows the structure of the vehicle periphery monitoring apparatus according to one Example of this invention. It is a figure which shows the processing flow in an image processing unit according to one Example of this invention. It is a figure which shows the correction processing flow of the acquisition time of object information according to one Example of this invention. It is a figure which shows the correction processing flow of the moving amount | distance of the own vehicle during the radar scanning time according to one Example of this invention. FIG. 5 is a diagram illustrating a timing flow of radar scanning and image acquisition according to an embodiment of the present invention. It is a figure which shows the mode of the object detection ahead of a vehicle according to one Example of this invention.

  Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention. The periphery monitoring device is mounted on a vehicle and is based on a radar 10, a camera 12, an image processing unit 14 for detecting an object around the vehicle based on image data captured by the camera 12, and a detection result thereof. A speaker 16 that generates an alarm by sound or voice, and a display device 18 that displays an image obtained through imaging of the camera 12 and makes the driver recognize an object around the vehicle are provided.

  The radar 10 corresponds to, for example, a scanning laser radar, and receives the reflected light from the object of the beam scanned in each direction, thereby detecting the position and size (area) of the object (candidate) in each direction as a detection point. ) And the distance to it. The radar 10 may be another type of radar (for example, a millimeter wave radar).

  The camera 12 includes an infrared camera, a camera that uses light in a wavelength band other than infrared (for example, visible light), and the like. In the following description, the camera 12 is a grayscale image taken by an infrared camera, but is not limited to this. Furthermore, in a vehicle provided with a navigation device, the corresponding function provided in the navigation device may be used as the speaker 16 and the display device 18.

  The image processing unit 14 in FIG. 1 includes (executes) means (functions) indicated by blocks 141 to 148. That is, the image processing unit 14 receives a signal from the radar 10 and obtains information (detection point data) such as positions of objects around the vehicle and an image around the vehicle captured by the infrared camera 12. Image acquisition means 142 to acquire as a gray scale image, means 143 to detect the acquisition timing (time) of the acquired image, and detection time (timing) of information (detection point data) such as the position of the object detected by the radar 10 Function as time correction means 144 for correcting the above.

  The image processing unit 14 further includes detection means 145 for detecting a difference between the detection timing of the position or size of the object detected by the radar 10 and the acquisition timing of the corresponding image acquired by the infrared camera 12, and the difference Corresponding to the detected object position, the calculating means 146 for calculating the movement amount of the vehicle corresponding to the above, the correcting means 147 for correcting the position or size of the object acquired by the radar 10 according to the movement amount of the vehicle, and The unit 148 functions as a unit 148 that identifies a predetermined area on the acquired corresponding grayscale image and identifies an object in the predetermined area.

  The function of each block is realized by a computer (CPU) included in the image processing unit 14. The configuration of the image processing unit 14 may be incorporated in the navigation device.

  The image processing unit 14 includes, for example, an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, and a central processing unit (CPU) that performs various arithmetic processes. ), RAM used by the CPU to store data for computation, ROM to store programs executed by the CPU and data to be used (including tables and maps), drive signals for the speakers 16, display signals to the display device 18, etc. An output circuit for outputting is provided. The output signals of the radar 10 and the infrared camera 12 are converted into digital signals and input to the CPU.

  Next, a method for correcting the synchronization between the output signals of the radar and the camera in one embodiment of the present invention will be described.

  FIG. 2 is a diagram illustrating a processing flow executed by the image processing unit 14. The processing flow in FIG. 2 is executed at predetermined time intervals by calling the processing program stored in the memory by the CPU of the image processing unit 14.

  In step S1, an output signal (that is, captured image data) of the infrared camera 12 is received as an input, A / D converted, and stored in an image memory. The stored image data is a grayscale (monochrome shade) image including luminance information. In an infrared camera in which the higher the temperature of the target, the higher the brightness is, the image data has a higher brightness value in the higher brightness portion, and the image data has a lower brightness value as the temperature becomes lower (lower brightness).

  In step S2, scanning of the radar 10 is started. This scanning is performed in parallel with the image data acquisition in step S1.

  In step S3, detection point data (point sequence data) indicating the position of a predetermined object around the vehicle is acquired in response to a signal from the radar 10. An arbitrary method can be used for this data acquisition, for example, as follows.

  A scanning laser radar is used as the radar 10, and the position of an object, for example, a pedestrian candidate is specified from the width of the detection point group. Specifically, a scanning laser radar measures the distance to an object in each direction as a detection point by receiving reflected light from the object of a beam scanned in a predetermined range. A set of detection points is obtained by plotting the positions of these detection points in a two-dimensional coordinate system in which the position of the laser radar is the origin and the Y axis is the front direction of the radar. From these set of detection points, detection point groups are grouped as detection point groups whose mutual distance between detection points is equal to or smaller than a predetermined value, and among the grouped detection point groups, those having a spread width equal to or smaller than a predetermined value are grouped as predetermined objects. Each position is regarded as a candidate (for example, a pedestrian candidate) and stored in the memory as detection point data (point sequence data).

  In step S4, time correction is performed. The time correction mentioned here means two corrections: correction of the acquisition time (timing) of the object information (point sequence data) acquired by the radar 10 and correction of the movement amount of the own vehicle during the radar scanning time. These two corrections will be described with reference to FIGS.

  FIG. 3 is a diagram illustrating a processing routine (flow) for correcting the acquisition time of object information, which is executed by the image processing unit 14. In step S41 of FIG. 3, the acquisition timing of the image acquired by the infrared camera 12 is detected. In step S42, the acquisition time of the object information (point sequence data) acquired by the radar 10 is corrected. The contents of these two steps will be described with reference to FIG.

  FIG. 5 is a diagram illustrating a timing flow of radar scanning and image acquisition according to an embodiment of the present invention. The radar 10 detects the position of the object candidate and the like every predetermined scanning time TS (for example, about 60 ms). In this case, the object candidate is detected at some timing within the scanning time TS, but since the timing changes at the position of the object candidate, etc., it is conventionally detected at the start of scanning (P1 to P4 in FIG. 4). As a result, the time was corrected. On the other hand, the infrared camera 12 captures an image in front of the vehicle at predetermined timings T1 to T6 (for example, about 33 ms). These timings T1 to T6 are timings acquired in step S41 in FIG. Therefore, the detection timing of the object candidate by the radar 10 and the timing of image acquisition by the infrared camera 12 do not basically coincide with each other, that is, there is no principle that both timings can be synchronized.

  According to an embodiment of the present invention, instead of performing control so as to synchronize both timings, time correction is performed so that the detection timing of an object candidate by the radar 10 matches the timing of image acquisition by the infrared camera 12. . In the example of FIG. 5, for example, time correction is performed so that the timing of the object candidate detected between the radar scans P <b> 1 and P <b> 2 matches the timing T <b> 2 of image acquisition sent from the infrared camera 12. That is, object candidate data (position, size, etc.) detected between the scans P1 and P2 by the radar 10 corresponds (related) as data at the same timing as the image data acquired by the infrared camera 12 at the timing T2. Attached. This time (timing) correction corresponds to the time correction executed in step S42 of FIG.

  At this time, if the radar 10 cannot acquire the image acquisition timing from the infrared camera 12 within the scanning time TS, it is assumed that the object candidate data has been acquired at the timing of the end of scanning. In the example of FIG. 5, when the radar 10 cannot acquire the image acquisition timings T3 and T4, it is assumed that the object candidate data has been acquired by the radar 10 at the timing of P3 (T4 ′) at the end of scanning. . In this case, as the image data acquired by the infrared camera 12, the image data at the timing T4 closest to the end of scanning P3 (T4 ') is used, and the image data at the T timing 4 is used as object candidate data. Applicable (related).

  As described above, according to an embodiment of the present invention, without using a synchronization signal, the timing control of each of the radar and the camera is utilized as it is, that is, while using the asynchronous control of both, It is possible to correct the acquisition timing (correlation between data).

  FIG. 4 is a diagram showing a processing routine (flow) for correcting the movement amount of the host vehicle during the radar scanning time, which is executed by the image processing unit 14. In step S401 of FIG. 4, the acquisition timing of the image acquired by the infrared camera 12 is detected as in step S41 of FIG.

  In step S <b> 402, the difference between the position detection point data (point sequence data) of the object detected by the radar 10 and the image acquisition timing acquired by the infrared camera 12 is detected. As described with reference to FIG. 5, the detection timing of the object candidate detected by the radar 10 and the acquisition timing of the image acquired by the infrared camera 12 are in principle shifted, that is, a difference (time difference) occurs. Will do. In step S402, a difference (time difference) between the two acquisition timings is detected. In the example of FIG. 5, for example, when matching with the image acquisition timing T2, the difference (time difference) between the timing T2 and the detection timing of the object candidate by the radar 10 is obtained. In this case, since one image acquisition timing always falls within the scanning time TS, the difference is a maximum value smaller than the scanning time TS (0 <t <TS).

  Next, in step S403, the movement amount of the vehicle at the difference (time) in the acquisition timing obtained in step S401 is calculated. The calculation of the movement amount will be described with reference to FIG.

  FIG. 6 is a diagram showing a state of object detection in front of the vehicle according to one embodiment of the present invention. In FIG. 6, it is assumed that the vehicle 20 is traveling along the direction 21. Lines indicated by reference numerals 23, 24, and 25 are ranges that can be detected by the radar 10, and are ranges in which scanning is repeated along the direction indicated by the arrow 22. Times t0, t1, and t2 indicate vehicle movement times, respectively, and it is assumed that images are acquired by the infrared camera 12 at times t0 and t2. That is, times t0 and t2 correspond to timings T1 to T5 in FIG.

  Assume that three object candidates A, B, and C in the figure are detected as detection data (point sequence data) between times t0 and t2 by radar 10 scanning performed in parallel with image acquisition by the infrared camera 12. Assume that the object candidate A is detected by the radar 10 at time t0 or a timing near it, the object candidate B is detected at time t1 or a timing near it, and the object candidate C is detected by time t2 or a timing near it. It is assumed that the object candidates A, B, and C are all stationary between the times t0 and t2. As described with reference to FIG. 5, in the embodiment of the present invention, the detection timing of the object candidate by the radar 10 is matched with the timing of the image acquisition by the infrared camera 12. The time correction is performed assuming that all object candidates A to C have been detected.

  In this case, since the vehicle 20 is moving forward at a predetermined speed, when performing the time correction, the movement amount of the vehicle corresponding to the time difference is calculated at the same time, and the acquisition data of the object candidates A, B, and C Therefore, it is necessary to correct the movement amount. In step S403, the movement amount of the vehicle for that purpose is calculated. Specifically, for example, data of the vehicle speed V is acquired, the object candidate B is multiplied by the time difference Δt (= t1−t0), and the object candidate C is multiplied by the time difference Δt (= t2−t0). To calculate respective movement amounts (distances) d1 and d2 (d = V × Δt).

  Returning to the flow of FIG. 2, in step S5, the object position on the image is set. Specifically, a predetermined area indicating the position of the object candidate is set on the acquired grayscale image. This predetermined area is an area including the detection point (the position of the object candidate) acquired by the laser radar 10 in step S3. At that time, the detection point (object candidate position) and the like are corrected using the movement amount (distance) obtained in step S403 of FIG.

  Specifically, in the example of FIG. 6, the movement amount (distance) d1 is added to the data of the object candidate B to correct its position (distance from the vehicle 20), and on the image of the object candidate B Is corrected (reduced or enlarged) for a predetermined area for the object candidate B. The same applies to the object candidate C as in the case of the object candidate B.

  An arbitrary method can be used for setting the predetermined area. For example, when the position of the pedestrian is specified as the position of the object, an area of a predetermined size including the position of the pedestrian candidate (detection point) is used. Group) is converted into the camera coordinate system of the infrared camera 12, mapped (superimposed) in the gray scale image, and set as a predetermined area on the corresponding gray scale image.

  Thus, according to an embodiment of the present invention, the position or size of the object on the corresponding image according to the amount of movement of the vehicle from the timing when the camera image is acquired until the radar information is acquired. By correcting (predetermined region), it is possible to improve the matching accuracy between the camera image and the radar information.

  In step S6, it is specified whether the object candidate on the image is a predetermined object. This specification is performed for object candidates in the area or in the vicinity of the predetermined area on the image obtained in step S5. This specification can be performed by an arbitrary method. For example, an object candidate (object) is collated from a black and white image obtained by binarizing a grayscale image.

  Specifically, first, it is checked whether or not it has the characteristics of the excluded object stored in the memory in advance. The excluded object means, for example, an object (such as a quadruped animal or a vehicle) that is a heating element but has a clearly different shape. If it does not correspond to the excluded object, it is checked whether it corresponds to the characteristic of the predetermined object. For example, when the predetermined object is a pedestrian, a rounded white portion exists as a feature of the pedestrian's head, and an elongated white portion corresponding to the body and leg portions is connected below the white portion. Etc. are included. Further, the similarity with a predetermined pattern representing a pedestrian stored in advance may be calculated using known pattern matching, and the similarity may be used as a material for determining whether or not the user is a pedestrian. Furthermore, since the walking speed of the pedestrian is substantially constant, information on the moving speed of the object obtained from the radar 10 may be used as one of judgment materials.

  The embodiment of the present invention has been described above, but the present invention is not limited to such an embodiment, and can be modified and used without departing from the spirit of the present invention.

10 Radar,
12 cameras (infrared cameras),
14 Image processing unit 16 Speaker,
18 display device,
20 Vehicles 23, 24, 25 Detectable (scanning) range of radar

As a method for correcting the synchronization error, a synchronization signal is output from a radar with a slow (long) detection cycle by synchronizing the detection start timing of both with an external synchronization circuit, and the timing of the camera with a fast (short) detection cycle is set to the radar timing. There is known a method of synchronizing the detection start timing so as to match.
However, the conventional method may require a new synchronization circuit, or may require a dedicated design for driving a camera with a short processing time according to a radar with a long processing time. Furthermore, since a camera with a short processing time does not operate at the original processing cycle, it is difficult to deploy the data in another system, and the original performance (detection speed, etc.) of the radar or camera is utilized. There is a fear that you can not.

Claims (3)

First detection means for continuously detecting object information around the vehicle at a predetermined period;
Second detection means for detecting object information around the vehicle at a predetermined timing in a method different from the first detection means;
A time correction unit that corrects the detection timing of the object information detected by the first detection unit within one cycle as one timing within the one cycle;
The vehicle periphery monitoring device according to claim 1, wherein one timing in the one cycle coincides with one of the predetermined timings of the second detection means. A predetermined area is set on a corresponding image captured by the camera corresponding to the position or size of an object detected by the radar, a radar that detects an object around the vehicle, a camera that captures an image around the vehicle And a vehicle periphery monitoring device comprising means for identifying the object in the predetermined area,
Detection means for detecting a difference between the detection timing of the position or size of the object detected by the radar and the acquisition timing of the corresponding image acquired by the camera;
Calculating means for calculating a movement amount of the vehicle corresponding to the difference;
A vehicle periphery monitoring device comprising: correction means for correcting the position or size of an object detected by the radar according to the amount of movement of the vehicle.   The radar detects an object every predetermined scanning time, the detection timing by the radar is a timing at the start or end of the predetermined scanning time, and the acquisition timing by the camera is the predetermined scanning time. The periphery monitoring device according to claim 2, which is an acquisition timing within a time.

Images