JP2011232139A - Periphery monitoring device for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a periphery monitoring device for a vehicle, comprising a radar that detects an object around the vehicle, a camera that captures an image around the vehicle, and means for identifying a predetermined region corresponding the position of the object detected by the radar in the corresponding image captured by the camera, and identifying the object within the predetermined region.SOLUTION: The periphery monitoring device comprises means for calculating a phase shift amount between the time variation of the movement amount of the prescribed region in the identified corresponding image and the time variation of the movement amount of the predetermined object in the image captured by the camera, phase correction means that performs phase correction of the output signal of the radar or the camera so that the phase shift amount becomes zero, and a determination means that determines that the radar or the camera has an optical axis deviation if the direction of the position deviation of the prescribed region with respect to the predetermined object in the corresponding image is always the same, and determines that there is a synchronization shift between the radar and the camera if the direction of the position deviation of the prescribed region is opposite to the moving direction of the predetermined object.

Description

  The present invention relates to a vehicle periphery monitoring apparatus, and more specifically, to detecting and correcting a synchronization shift (phase shift) or an optical axis shift between a laser and a camera in a vehicle periphery monitoring apparatus.

  Patent document 1 discloses a pedestrian detection device. This pedestrian detection device uses a scanning laser radar and an infrared camera together, and detects a pedestrian in an infrared image based on detection information of a pedestrian candidate obtained from the laser radar.

JP 2003-302470 A

  However, if there is a synchronization shift (phase shift) between the output signals of the laser radar and the infrared camera, the detection range (position) of the two shifts, and a pedestrian in the infrared image corresponding to the detection information of the laser radar is detected. There are cases where it cannot be done. This synchronization shift occurs because the detection time (for example, about 120 ms) of the object by the laser radar accompanied by scanning becomes longer than the acquisition time (for example, about 33 ms) of the image including the object by the infrared camera.

  In the apparatus described in Patent Document 1, no consideration is given to the correction for the synchronization deviation between the laser radar and the infrared camera.

  Also, when there is a deviation in the optical axes of the laser radar and the infrared camera, the detection ranges of the two will be different, and it may not be possible to detect a pedestrian in the infrared image corresponding to the detection information of the laser radar. Arise.

  Therefore, it is necessary to make an appropriate correction after ascertaining whether the deviation between the detection ranges of the laser radar and the infrared camera is caused by the synchronization deviation (phase deviation) or the optical axis deviation between the two.

  In view of the above, the present invention has an object to determine whether the deviation of the detection range of the laser radar and the infrared camera is caused by the synchronization deviation (phase deviation) or the optical axis deviation from the obtained image information. And

  Further, the present invention is specified by the radar by detecting and correcting the synchronization shift (phase shift) between the output signals of both after confirming that there is a synchronization shift (phase shift) between the laser radar and the infrared camera. An object of the present invention is to improve the extraction accuracy of a predetermined object on a camera image.

  Furthermore, the present invention confirms that there is an optical axis shift between the laser radar and the infrared camera, and then detects and corrects the optical axis shift between the two, thereby detecting a predetermined object on the camera image specified by the radar. The purpose is to improve the extraction accuracy.

  According to the present invention, a radar that detects an object around the vehicle, a camera that captures an image around the vehicle, and a predetermined area on the corresponding image captured by the camera that corresponds to the position of the object detected by the radar is specified. Thus, there is provided a vehicle periphery monitoring device including means for specifying an object in a predetermined area. The periphery monitoring device calculates a phase shift amount between a temporal change in the movement amount of the predetermined area on the identified corresponding image and a temporal change in the movement amount of the predetermined object on the image captured by the camera; The phase correction means for correcting the phase of the output signal of the radar or camera so that the phase shift amount becomes zero, and the radar or camera when the position shift direction of the predetermined area with respect to the predetermined object on the corresponding image is always the same direction. And determining means for determining that there is a synchronization shift between the radar and the camera when the position shift direction of the predetermined region is opposite to the moving direction of the predetermined object.

  According to the present invention, it is possible to determine from the obtained image information whether the deviation of the detection range of the laser radar and the infrared camera is caused by the synchronization deviation (phase deviation) or the optical axis deviation between the two. Thus, appropriate correction can be performed according to the determination result. As a result, the accuracy of specifying (extracting) a predetermined object on the camera image specified by the radar can be improved.

  According to an aspect of the present invention, the means for calculating the phase shift amount calculates the phase shift amount when the determination unit determines that there is a synchronization shift.

  According to one aspect of the present invention, after confirming that there is a synchronization shift (phase shift) between the laser radar and the infrared camera, the synchronization shift (phase shift) between the output signals of the two is detected and corrected. Thus, it is possible to improve the extraction accuracy of the predetermined object on the camera image specified by.

  According to an aspect of the present invention, the optical system further includes means for calculating the optical axis deviation amount between the radar and the camera from the positional deviation amount of the predetermined area with respect to the predetermined object on the corresponding image and the focal length of the camera. The means for calculating the amount of axial deviation calculates the amount of optical axis deviation when the determining means determines that there is an optical axis deviation.

  According to one aspect of the present invention, after confirming that there is an optical axis shift between the laser radar and the infrared camera, the optical axis shift between the two is detected and corrected, thereby allowing a predetermined image on the camera image specified by the radar to be detected. It is possible to improve the object extraction accuracy.

It is a block diagram which shows the structure of the periphery monitoring apparatus of the vehicle of this invention. It is a figure which shows the processing flow in the image processing unit of this invention. It is a figure which shows the position shift amount calculation flow of this invention. It is a figure which shows the predetermined area | region and pedestrian image on the image of this invention. It is a figure for demonstrating the position shift direction of the predetermined area | region of this invention. It is a figure for demonstrating the position shift direction of the predetermined area | region of this invention. It is a figure which shows the flow of the optical axis offset correction | amendment of this invention. It is a figure which shows the flow of phase shift correction of this invention. It is a figure for demonstrating acquisition of the time series data of the variation | change_quantity of this invention. It is a figure for demonstrating acquisition of the time series data of the variation | change_quantity of this invention. It is a figure for demonstrating the calculation method of the phase shift amount of this invention.

  Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention. The periphery monitoring device is mounted on a vehicle, and includes a laser radar 10, an infrared camera 12, an image processing unit 14 for detecting an object around the vehicle based on image data captured by the infrared camera 12, and a detection result thereof. And a display device 18 for displaying an image obtained through imaging by the infrared camera 12 and for causing the driver to recognize an object around the vehicle. With.

  The laser radar 10 corresponds to, for example, a scanning laser radar, and receives the reflected light from the object of the beam scanned in each direction, thereby detecting the position (region) of the object (candidate) in each direction as a detection point. Detect the distance to it. The laser radar 10 may be another type of radar (for example, a millimeter wave radar). The infrared camera 12 may also be a camera that uses light in a wavelength band other than infrared (for example, visible light). Therefore, in the following description, a gray scale image taken by the infrared camera 12 is targeted, but the present invention is not limited to this. Furthermore, in a vehicle provided with a navigation device, the corresponding function provided in the navigation device may be used as the speaker 16 and the display device 18.

  The image processing unit 14 in FIG. 1 includes (executes) means (functions) indicated by blocks 141 to 147. That is, the image processing unit 14 receives a signal from the laser radar 10 and specifies an object position specifying unit 141 for specifying the position of an object around the vehicle, and an image around the vehicle captured by the infrared camera 12 as a grayscale image. An image acquisition unit 142 to acquire, and a unit 143 that specifies a predetermined region on the acquired corresponding grayscale image corresponding to the position of the object specified by the object position specifying unit 141 and specifies an object in the predetermined region. including. The object position specifying unit 141 may be configured as a part of the function of the laser radar 10, that is, integrated with the laser radar 10.

  The image processing unit 14 further includes a phase change between the temporal change in the movement amount of the predetermined area on the corresponding image specified by the laser radar 10 and the temporal change in the movement amount of the predetermined object on the image captured by the infrared camera 12. Means 144 for calculating the amount of deviation, means 145 for calculating the amount of optical axis deviation between the radar and the camera from the positional deviation amount of the predetermined area with respect to the predetermined object on the corresponding image, and the focal length of the camera, and radar and camera 146, a phase correction unit 147 for correcting the phase of the output signal of the radar or the camera so that the phase shift amount becomes zero, and a positional shift direction of the predetermined region with respect to the predetermined object on the corresponding image. Is always in the same direction, it is determined that there is a deviation of the optical axis of the radar or camera, and the position displacement direction of the predetermined area is opposite to the moving direction of the predetermined object And a determination unit 148 for determining that there is a synchronization error of the radar and the camera.

  The image processing unit 14 also needs to receive detection signals from a vehicle speed sensor that detects the speed (vehicle speed) of the host vehicle, an acceleration sensor, and a yaw rate sensor that detects a yaw rate (change speed of the rotation angle in the turning direction). Has a function to perform various processing.

  The function of each block is realized by a computer (CPU) included in the image processing unit 14. The configuration of the image processing unit 14 may be incorporated in the navigation device.

  The image processing unit 14 includes, for example, an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, and a central processing unit (CPU) that performs various arithmetic processes. ), RAM used by the CPU to store data for computation, ROM to store programs executed by the CPU and data to be used (including tables and maps), drive signals for the speakers 16, display signals to the display device 18, etc. An output circuit for outputting is provided. The output signals of the laser radar 10 and the infrared camera 12 are converted into digital signals and input to the CPU.

  Next, a method for correcting optical axis deviation or phase deviation of a radar or a camera according to an embodiment of the present invention will be described. FIG. 2 is a diagram illustrating a processing flow executed by the image processing unit 14. The processing flow in FIG. 2 is executed at predetermined time intervals by calling the processing program stored in the memory by the CPU of the image processing unit 14.

  2, the position of a predetermined region (range) on the image captured by the infrared camera 12 corresponding to the position (region) of the object candidate detected by the laser radar 10 and the image captured by the infrared camera 12. The amount of positional deviation from the position of the object candidate above is calculated.

  FIG. 3 is a diagram showing a positional deviation amount calculation flow (routine) in step S1 of FIG. In step S11 of FIG. 3, the position of a predetermined object around the vehicle is specified by receiving a signal from the laser radar 10. An arbitrary method can be used for the identification, for example, as follows.

  A scanning laser radar is used as the laser radar 10, and the position of an object, for example, a pedestrian candidate is specified from the width of the detection point group. Specifically, the scanning laser radar measures the distance to the object in each direction as a detection point by receiving reflected light from the object of the beam scanned in each direction. A set of detection points is obtained by plotting the positions of these detection points in a two-dimensional coordinate system in which the position of the laser radar is the origin and the Y axis is the front direction of the radar. From these set of detection points, detection point groups are grouped as detection point groups whose mutual distance between detection points is equal to or smaller than a predetermined value, and among the grouped detection point groups, those having a spread width equal to or smaller than a predetermined value are grouped as predetermined objects. The position is identified as a candidate (for example, a pedestrian candidate).

  In step S12, an output signal (that is, captured image data) of the infrared camera 12 is received as input, A / D converted, and stored in the image memory. The stored image data is a grayscale (monochrome shade) image including luminance information. In an infrared camera in which the higher the temperature of the target, the higher the brightness is, the image data has a higher brightness value in the higher brightness portion, and the image data has a lower brightness value as the temperature becomes lower (lower brightness).

  In step S13, a predetermined area that is a range in which a predetermined object is detected is set in the acquired grayscale image. This predetermined area is an area including the position (area) of the predetermined object candidate specified by the laser radar 10 in step S12. An arbitrary method can be used for setting the predetermined area. For example, when the position of the pedestrian is specified as the position of the object, an area of a predetermined size including the position of the pedestrian candidate (detection point) is used. Group) is converted into the camera coordinate system of the infrared camera 12, mapped (superimposed) in the gray scale image, and set as a predetermined area on the corresponding gray scale image.

  In step S14, a predetermined object on the image is specified. This specification is performed in the area or in the vicinity of the predetermined area on the image set in step S13. This specification can be performed by an arbitrary method. For example, an object is collated from a black and white image obtained by binarizing a grayscale image. Specifically, first, it is checked whether or not it has the characteristics of the excluded object stored in the memory in advance. The excluded object means, for example, an object (such as a quadruped animal or a vehicle) that is a heating element but has a clearly different shape. If it does not correspond to the excluded object, it is checked whether it corresponds to the characteristic of the predetermined object. For example, when the predetermined object is a pedestrian, a rounded white portion exists as a feature of the pedestrian's head, and an elongated white portion corresponding to the body and leg portions is connected below the white portion. Etc. are included. Further, the similarity with a predetermined pattern representing a pedestrian stored in advance may be calculated using known pattern matching, and the similarity may be used as a material for determining whether or not the user is a pedestrian. Furthermore, since the walking speed of the pedestrian is substantially constant, information on the moving speed of the object obtained from the laser radar 10 may be used as one of judgment materials.

  In step S15, it is determined whether a predetermined object has been identified. If this determination is No, the process proceeds to step S19, where the amount of positional deviation is set to zero, and the process ends (returns). When the determination in step S15 is Yes, the process proceeds to step S16.

  With reference to FIG. 4, the calculation of the amount of positional deviation between the position of the predetermined area on the image in steps S16 to S18 and the position of the specified predetermined object will be described. FIG. 4 is a diagram illustrating an example of a predetermined area and a predetermined object (pedestrian) on a black and white image after binarization processing. In FIG. 4, in order to recognize the line, the whole is white except for the line part, but the background part around the pedestrian images 22 and 24 that are the predetermined objects is actually displayed in black.

  In FIG. 4, pedestrian images 22 and 24 are shown as predetermined objects in the binarized black and white image 20 acquired by the infrared camera 12. The image 22 of (a) is a case where the pedestrian is stationary, and the image 24 of (b) is equivalent to the case where the pedestrian is moving in the right direction in the figure. Frames indicated by reference numerals 21 and 23 are predetermined areas specified by the laser radar 10 as pedestrian positions (areas). While the pedestrian image 22 in (a) is within the predetermined area 21 and the positional deviation between the two is small, the positional deviation between both is large in (b). The positional shift in (b) is caused by either or both of the synchronization shift (phase shift) and the optical axis shift between the radar and the camera. This separation of synchronization deviation or optical axis deviation will be described later. On the other hand, the case (a) corresponds to a case where there is almost no or little synchronization shift and optical axis shift between them.

  In step S16 in FIG. 3, the centroids of the predetermined areas 21 and 23 in FIG. 4 are calculated. In step S17, the center of gravity of the pedestrian images 22, 24 is calculated. In step S18, the difference between the calculated positions P1 and Q1 of the center of gravity or P2 and Q2 in the horizontal direction (horizontal direction) is calculated as the shift amounts d1 and d2 on the camera image. Note that the position for calculating the amount of deviation is not limited to the position of the center of gravity, but may be a position determined by another arbitrary reference, for example, the midpoint of the predetermined area and the maximum horizontal width of the pedestrian image.

  Returning to FIG. 2, in step S2, it is determined whether or not the amount of displacement calculated in step S1 is greater than a predetermined amount. The predetermined amount is set as one threshold value for determining whether synchronization deviation correction or optical axis deviation correction is necessary. If this determination is No, the process ends. In the example of FIG. 4, the positional deviation amount d <b> 1 in the case of (a) corresponds to a case where it is smaller than a predetermined amount. If the determination is Yes, the process proceeds to the next step S3.

  In step S3, the position shift direction of the predetermined area with respect to the predetermined object on the image is calculated. This calculation determines whether the position of the predetermined region is shifted to the right or left in the horizontal direction with respect to the position of the predetermined object on the image. At this time, it is also determined whether the position of the predetermined area is shifted in the same direction or the reverse direction with respect to the moving direction of the predetermined object. Specifically, for example, it is determined whether the center of gravity of the predetermined area determined in step S16 is shifted to the right or left with respect to the position of the center of gravity of the pedestrian determined in step S17 of FIG. At the same time, it is also determined whether the center of gravity of the predetermined area is shifted in the same direction or the opposite direction with respect to the moving direction of the center of gravity of the pedestrian.

  In step S4, it is determined whether or not the position of the predetermined area is shifted in the same direction with respect to the position of the predetermined object on the image. FIG. 5 is a diagram for explaining the direction of displacement of a predetermined area. FIG. 5 shows a predetermined region and a predetermined object (pedestrian) on the black and white image 20 after the binarization process as in FIG. 4. In order to recognize the line, the whole is white except for the line portion. However, the background part of the pedestrian image is actually displayed in black.

  An image 22 in FIG. 5A is a case where the pedestrian is stationary, an image 24 in FIG. 5B is a case where the pedestrian is moving in the right direction in the figure, and an image 25 in FIG. This corresponds to the case where the pedestrian is moving in the left direction in the figure. A frame indicated by reference numeral 23 is a predetermined area specified as the position (area) of the pedestrian by the laser radar 10. In FIG. 5, the predetermined area 23 is shifted to the right side of the pedestrian images 22, 24, 25 in any case where the pedestrian is stationary or moved left and right. Therefore, this corresponds to the case where the position of the predetermined area is shifted in the same direction with respect to the position of the predetermined object (pedestrian) on the image.

  The positional deviation in the same direction illustrated in FIG. 5 occurs as a constant constant deviation regardless of whether or not the predetermined object (pedestrian) has moved, and therefore is caused by the optical axis deviation of the radar or camera. It is determined that Therefore, if the determination in step S4 is Yes, the process proceeds to the next step S5, and optical axis deviation correction is performed. Details of this optical axis deviation correction will be described later.

  When the determination in step S4 is No, the process proceeds to step S6, and it is determined whether or not the position of the predetermined area is shifted in the reverse direction with respect to the moving direction of the predetermined object on the image. FIG. 6 is a diagram for explaining the direction of displacement of a predetermined area. FIG. 6 shows a predetermined region and a predetermined object (pedestrian) on the black and white image 20 after the binarization processing, as in FIG. 5, and the same reference numerals in both drawings indicate the same object.

  In FIG. 6, when the pedestrian image 22 in (a) is stationary, there is no position shift of the predetermined region 23. When the pedestrian image 24 of (b) moves rightward, the predetermined area 23 is shifted to the left side of the pedestrian image 24. On the other hand, when the pedestrian image 24 in (c) moves leftward, the predetermined area 23 is shifted to the right side of the pedestrian image 24. Therefore, the example of FIG. 6 corresponds to the case where the position of the predetermined region is shifted in the opposite direction with respect to the moving direction of the predetermined object (pedestrian) on the image.

  The positional deviation in the direction opposite to the movement direction illustrated in FIG. 6 is mainly caused by a delay in the output signal (processing) of the laser, and is a case where the synchronization deviation between the radar and the camera is relatively large. Determined. Therefore, if the determination in step S6 is Yes, the process proceeds to the next step S7 to perform phase shift correction. Details of the phase shift correction will be described later. A process is complete | finished when determination of step S6 is No.

  Next, details of the optical axis deviation correction in step S5 of FIG. 2 will be described with reference to FIG. FIG. 7 is a diagram showing a flow of optical axis deviation correction. In step S51 of FIG. 7, when the position of the predetermined region is shifted in the same direction with respect to the position of the predetermined object on the image, that is, due to the optical axis shift of the radar or camera exemplified in FIG. The positional deviation amount d in the case of the positional deviation is acquired. The acquisition of the positional deviation amount d is obtained from the position of the center of gravity of the predetermined object and the predetermined area as in the case of the positional deviation amounts d1 and d2 described with reference to FIG.

  In step S52, it is determined whether or not the obtained shift amount d is larger than a predetermined amount ΔL1. When this determination is No, it is determined that the amount of deviation of the optical axis is small and within the allowable range, and the process is terminated without performing the correction. If the determination is Yes, it is determined in step S53 whether or not the shift amount d is smaller than a predetermined amount ΔL2 (> ΔL1). When this determination is No, it is determined that the optical axis shift amount (angle) is within a large range, and the process proceeds to step S55, where it is determined that the optical axis of the infrared camera 12 has a defect that cannot be adjusted by correction.

If the determination in step S53 is Yes, in step S54, the optical axis deviation angle α between the radar and the camera is calculated. The shift angle α can be obtained from equation (1) from the shift amount d on the camera image acquired in step S51 and the focal length f of the camera.

α = tan ⁻¹ (d / f) (1)

  Finally, in step S56, correction corresponding to the obtained optical axis deviation angle α is performed. The correction method is performed using any conventional method such as correction in image processing or adjustment of the optical axis itself.

  Next, details of the phase shift correction in step S7 of FIG. 2 will be described with reference to FIGS. FIG. 8 is a diagram showing a flow of the phase shift correction. In step S71, time-series data is acquired in order to specify a data section for performing synchronization (phase) shift correction. Phase correction is performed by detecting the amount of phase shift between the radar and the camera during the period when the phase shift between the radar and the camera is relatively large, that is, the amount of temporal position change of the object detected by the radar or camera is relatively large. It is preferable from the viewpoint of detection accuracy and the like to detect the amount of phase shift in an easy period. However, for example, when the vehicle is traveling on a general road, it is difficult to continuously and stably observe changes necessary for the correction. Therefore, for example, the following method is used to acquire time-series data in which data changes over time over a certain period of time.

  FIG. 9 is a diagram for explaining that the sections A ′ to D ′ are formed by gathering only the sections of the sections A to D where the temporal position change amount of the object detected by the radar or the camera is large. . In FIG. 9, the sections A to D having a large temporal position change amount shown on the upper side are gathered together to create continuous sections A ′ to D ′ at the time T1 on the lower side of the figure. In the section where the temporal position change amount of the object in FIG. 9 is large, various detection signals indicating the running state of the vehicle when the change in the vehicle state is large, such as the yaw rate during turning of the vehicle or the vehicle speed during acceleration / deceleration, for example. It is desirable to obtain from In this way, by combining only sections where the amount of change in the position of the object over time is large, and detecting and correcting the phase shift between the laser and camera in the combined section, the accuracy and reliability of the correction Can be improved.

  FIG. 10 is a diagram for explaining another example of acquiring time-series data of a predetermined amount or more. In an in-vehicle environment in which the host vehicle moves (runs), stationary targets are frequently replaced, so it is difficult to continuously and stably observe changes necessary for correction on a general road. Therefore, in the example of FIG. 10, temporally continuous data is generated by connecting the moving speeds of different stationary targets (pedestrians and the like).

  In FIG. 10A, there are three stationary targets 1 to 3 in the traveling direction of the vehicle 30. As the vehicle 30 travels, the three stationary targets 1 to 3 gradually approach the vehicle 30. (B) shows the speed change 32 of the vehicle 30 and the speed change 33 of the stationary targets 1 to 3 in that case. As illustrated, the speed change 33 of the stationary targets 1 to 3 is obtained by connecting the speed changes A to C of the stationary targets 1 to 3, and is symmetric (negative) with respect to the speed change 32 of the vehicle 30. Relationship (graph). As a result, it is possible to obtain a section where the relative speed of the vehicle and the stationary target changes. Similarly to the case of FIG. 9, by detecting and correcting the phase shift amount of the laser and the camera in the section, The accuracy and reliability of the correction can be improved.

  Next, in step S72 of FIG. 8, the phase shift amount is calculated. FIG. 11 is a diagram for explaining a method of calculating the phase shift amount. FIG. 11A shows a binarized black and white image 20 obtained by extracting the portion shown in FIG. 4B, and the same reference numerals as those in FIG. As already described in the description of FIG. 4, the predetermined region 23 specified as the position (region) of the pedestrian by the laser radar 10 is more than the pedestrian image 24 after the binarization processing acquired by the infrared camera 12. Displayed out of position.

  (B) and (c) of FIG. 11 show the time change A of the movement amount of the pedestrian image 24 on the black and white image 20 and the time changes B and B ′ of the movement amount of the predetermined area 23 corresponding to the case of (a). FIG. (B) shows a time change when the positional deviation of (a) occurs due to the synchronization deviation of the radar and the camera. (C) shows a time change when the positional deviation of (a) occurs due to both the optical axis deviation of the radar or camera and the synchronization deviation of both. The data of (b) and (c) is a part of data obtained by cutting out each movement amount data in a continuous section having a relatively large change as obtained by the example of FIG. 9 or FIG. .

  From the graph of (b), it can be seen that the movement amount change B of the predetermined region 23 is delayed in phase (signal rising) by ΔT1 time than the movement amount change A of the pedestrian image 24. Therefore, the phase shift amount in this case is ΔT1. This phase shift amount ΔT1 corresponds to the positional shift amount d2 on the image of (a).

  From the graph of (c), it can be seen that the movement amount change B ′ of the predetermined region 23 is delayed in phase (signal rising) by ΔT2 time from the movement amount change A of the pedestrian image 24. In this case, the phase shift amount is ΔT2. At the same time, a steady amplitude deviation ΔP1 caused by the optical axis deviation is observed. Therefore, in the case of (c), the positional deviation amount d2 on the image of (a) is caused by the phase deviation amount ΔT2 and the amplitude deviation ΔP1.

  As described above, by monitoring the temporal change in the movement amount of the predetermined area on the image and the predetermined object image in the predetermined section (time), the phase shift amount of both can be obtained. This amount of phase shift corresponds to the amount of synchronization shift between the output signals of the laser radar 10 and the infrared camera 12. Instead of looking at the difference between the rising times of the two movement amount changes, the phase deviation amount may be obtained from the period of the obtained function by obtaining the cross-correlation coefficient between them or using the phase-only correlation method. .

  In step S73, in the processing of the output signal of the laser radar 10 and the image processing of the output signal of the infrared camera 12, correction is made so that the phase shift amounts ΔT1 and ΔT2 in FIGS. 11B and 11C are eliminated (become zero). (adjust. Specifically, for example, the rising time of the output signal of the infrared camera 12 is delayed by an amount corresponding to the phase shift amounts ΔT1 and ΔT2, or the laser radar 10 and the infrared camera 12 are synchronized with a predetermined clock (timing) signal. Processing such as rising of the output signal is performed.

  The embodiment of the present invention has been described above, but the present invention is not limited to such an embodiment, and can be modified and used without departing from the spirit of the present invention.

10 Laser radar,
12 Infrared camera,
14 Image processing unit 16 Speaker,
18 display device,
20 Camera image 21, 23 Predetermined area 22, 24, 25 detected by radar Pedestrian image 30 Vehicle

Claims (3)

A radar that detects an object around the vehicle, a camera that captures an image around the vehicle, and a predetermined area on the corresponding image captured by the camera corresponding to the position of the object detected by the radar A vehicle surroundings monitoring device comprising means for identifying the object in a predetermined area,
Means for calculating a phase shift amount between a time change of the movement amount of the predetermined area on the identified corresponding image and a time change of the movement amount of the predetermined object on the image captured by the camera;
Phase correction means for performing phase correction of the output signal of the radar or the camera so that the phase shift amount becomes zero;
When the positional deviation direction of the predetermined area with respect to the predetermined object on the corresponding image is always the same direction, it is determined that there is an optical axis deviation of the radar or the camera, and the positional deviation direction of the predetermined area is the predetermined object. Determining means for determining that there is a synchronization error between the radar and the camera when the moving direction is opposite to the moving direction;
A vehicle periphery monitoring device having   The periphery monitoring device according to claim 1, wherein the unit that calculates the phase shift amount calculates the phase shift amount when the determination unit determines that there is the synchronization shift. And a means for calculating an optical axis deviation amount between the radar and the camera from a positional deviation amount of the predetermined area with respect to the predetermined object on the corresponding image and a focal length of the camera.
The periphery monitoring device according to claim 1, wherein the means for calculating the optical axis deviation amount calculates the optical axis deviation amount when the determination unit determines that there is the optical axis deviation.

Images