JP2008292278A - Optical deviation detection method of distance detection device, and distance detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a deviation detection method capable of detecting deviation simply even when a vehicle is stopped without requiring a different a distance detection means such as a mileage sensor in a distance detection device, and the distance detection device. <P>SOLUTION: In this deviation detection method of the distance detection device, when detecting deviation in the distance detection device for detecting a distance based on stereo images acquired by a plurality of cameras constituted of an optical system and an imaging element, the first distance value from a known object image imaged by the imaging element through the optical system to a known object is determined, and the first distance value is compared with the second distance value from the stereo images to the detected known object, to thereby detect deviation in the distance detection device. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an optical axis misalignment detection method and a distance detection device for a distance detection device using a stereo camera.

  When a stereo camera is mounted on a car in order to detect the distance to a preceding vehicle or an obstacle, the optical system of the camera may be displaced due to vibration. In order to obtain the distance from the parallax based on the image obtained by the stereo camera, if the optical axes and positional relationships of the left and right cameras are shifted, the accuracy of distance detection is greatly affected. Therefore, it is necessary to detect and correct an optical system shift at any time for accurate distance detection.

Patent Literature 1 discloses a calibration method for an in-vehicle stereo camera and an in-vehicle stereo camera to which the method is applied. The same known object is photographed at two different points, and the camera deviation is corrected based on the information. Patent Document 2 discloses a distance detection device for a vehicle, photographs a stationary object, compares a distance value calculated from the self-travel distance with a distance value measured by a stereo camera, and corrects the difference as an offset value. is doing. The distance based on the stationary object is calculated using the information of the self-travel distance sensor.
Japanese Patent Laid-Open No. 10-341458 JP 2003-329439 A

  As in Patent Document 2, there is a method of detecting a shift in combination with another sensor such as a self-traveling distance measurement sensor in an optical system of a stereo camera. However, another sensor is necessary and must be linked with another sensor. There is a problem that control must be complicated, and a method that can be detected more easily is desired. Further, in Patent Documents 1 and 2, since it is necessary to calculate the self-travel distance and can be corrected only during travel, correction of optical system deviation cannot be performed while the vehicle is stopped.

  In view of the above-described problems of the prior art, the present invention provides a displacement detection method and distance that can be easily detected even when the vehicle is stopped without requiring another distance measurement means such as a travel distance sensor in the distance detection device. An object is to provide a detection device.

  In order to achieve the above object, the distance detection device shift detection method according to the present invention provides a shift in a distance detection device that detects a distance based on stereo images obtained by a plurality of cameras including an optical system and an image sensor. A first distance value from the known object image picked up by the image sensor via the optical system to the known object, and detected from the first distance value and the stereo image The shift in the distance detection device is detected by comparing the second distance value to the known object.

  According to the shift detection method of the distance detection device, the first distance value is obtained from the known object image photographed by the optical system of the camera, the second distance value is detected from the stereo image, and the first distance value and the second distance value are detected. Therefore, it is possible to easily detect the deviation even when the vehicle is stopped without requiring another distance measuring means such as a travel distance sensor.

  In the displacement detection method of the distance detection device, a relationship between the size of the known object image and the distance is obtained in advance, and the first distance value is based on the size of the known object image obtained from the imaging surface of the imaging device. Can be obtained. The size of the known object image can be, for example, the number of pixels on the imaging surface of the imaging device.

  In addition, by performing the comparison a plurality of times and detecting the deviation based on the data obtained by averaging the comparison results, more accurate deviation detection can be performed.

  A distance detection device according to the present invention includes a plurality of image input devices for inputting an object image for distance detection, a means for generating distance data from the data of the image input device, and a known object image input to the image input device. A means for obtaining a distance value based on the information obtained from the information, a means for comparing the distance data with a distance value obtained from the information of the known object image, and detecting a shift in the image input device based on the comparison result. And means for performing.

  According to this distance detection device, the distance value from the known object image input to the image input device to the known object is obtained, the distance data to the same known object is obtained from the data of the image input device, the distance value and the distance data, Since the deviation can be detected based on the comparison result, the deviation can be easily detected even when the vehicle is stopped without requiring another distance measuring means such as a travel distance sensor. The above-described deviation detection method can be executed by this distance detection device.

  The distance detection device may include means for maintaining a relationship between the information on the known object and the distance value, and the distance value may be obtained from the information on the known object obtained by the image input device.

  In addition, by performing the comparison by the comparison means a plurality of times and performing the deviation based on the data obtained by averaging the comparison results, more accurate deviation detection becomes possible.

  Further, by performing calibration of the distance detection device based on the deviation detection, it is possible to correct an error in the detection distance caused by the deviation in the image input device and perform accurate distance detection.

  Note that the image input device may include an optical system such as a lens and a camera serving as an image sensor.

  According to the deviation detection method and the distance detection apparatus of the present invention, the distance detection apparatus can easily detect deviation even when the vehicle is stopped without requiring another distance measuring means such as a travel distance sensor.

  The best mode for carrying out the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram schematically showing a distance detection apparatus according to the present embodiment. FIG. 2 is a schematic diagram for explaining the principle of distance detection in the distance detection device 10.

  As shown in FIG. 1, the distance detection device 10 includes a stereo camera 11 including a sensor A and a sensor B, a deviation detection circuit 12, a memory 13, a parameter correction circuit 14, a memory 15, and an image processing circuit 16. A calibration circuit 17.

  As shown in FIG. 2, the stereo camera 11 of FIG. 1 has a left camera 11a composed of an optical system of a lens E having a focal length f and a sensor A, and a right composed of an optical system of a lens F having a focal length f and a sensor B. And a camera 11b. Sensors A and B are constituted by an image sensor such as a CCD or a CMOS image sensor. Image data taken from each of the sensors A and B is output, and a reference image can be obtained from the sensor A of the left camera 11a and a reference image can be obtained from the sensor B of the right camera 11b.

  As shown in FIG. 2, the imaging surfaces C and D of the sensors A and B are arranged on the surface g. The lenses E and F are arranged such that the optical axis a and the optical axis b passing through the lens centers O1 and O2 are parallel to each other with a lens center interval L in the lateral direction, and orthogonal to the optical axes a and b. It is arranged on a lens surface h passing through the centers O1 and O2. The surfaces g of the imaging surfaces C and D and the lens surface h are parallel to each other by a focal length f. Further, the horizontal interval between the reference points C0 and D0 where the optical axes a and b of the imaging surfaces C and D are orthogonal to each other is equal to the lens center interval L.

As shown in FIG. 2, it is assumed that the subject I as a distance measurement target is on the optical axis a of the lens E of the left camera 11a, and the distance from the lens surface h to the subject I is H. The subject I passes through the center O1 of the lens E of the left camera 11a and forms an image at the reference point C0 on the imaging surface C, while the subject I passes through the center O2 of the lens F of the right camera 11b and is on the imaging surface D. It is assumed that an image is formed at the position D1. The distance x from the reference point D0 on the imaging surface D to the position D1 is a shift amount (parallax) resulting from the left camera 11a and the right camera 11b being arranged at an interval L. From FIG. 2, H / L = f / x holds, and the following expression (1) is obtained.
H = (L · f) / x (1)

  From the above equation (1), since the lens center interval L and the focal length f are constant, the distance H from the shift amount x to the subject I can be measured. In this way, the distance H to the subject I can be detected based on the image information from the stereo camera 11.

  The deviation detection circuit 12 in FIG. 1 includes a known pattern detection circuit 12a that detects a known object from a reference image captured by the sensor A, a size measurement circuit 12b that measures the size of the detected known object, and a measured size. A distance value calculation circuit 12c that calculates a distance value from the distance value, and a distance value comparison circuit 12d that compares the distance value measured from the size with the actually measured distance value calculated by the image processing circuit 16.

  The memory 13 stores known pattern information and pixel number-distance value conversion information. As the known pattern information, for example, shape data such as a traffic light, a sign, a license plate, and a guard rail are recorded. In addition, the relationship between the number of pixels and the distance value calculated from the relationship between the optical image and the number of pixels in the optical system of the stereo camera 11 is recorded.

  The memory 15 stores the distance image generation parameters corrected by the parameter correction circuit 14 as information necessary for generating the data relating to the above-described distance value. For example, the number of pixels, the pixel pitch, the camera baseline length, Record misalignment information. When a shift of the stereo camera 11 is detected, the shift information is recorded in the distance image generation parameter. Note that the memories 13 and 15 may be integrated instead of separate memories, and may be constituted by a rewritable nonvolatile semiconductor memory or the like.

  The image processing circuit 16 receives the image data captured by the sensors A and B of the stereo camera 11 and generates distance data indicating the distance to the subject. At this time, the distance image generation parameter is read from the memory 15. To generate and output distance data. In addition to the distance data, image data subjected to various signal processing is also output. The distance data and the image data are sent to a subsequent processing circuit and processed according to the application.

  The image processing circuit 16 performs a parallax calculation based on the image data from the stereo camera 11 to generate distance data. For this parallax calculation, a correlation method based on a sum of absolute differences (SAD) or phase limitation is performed. A correlation method (Phase-Only Correlation: POC) is used. Specifically, the image processing circuit 16 performs hardware processing on an SAD method or POC method using an integrated element or the like, but may perform software processing on a CPU (Central Processing Unit). In this case, the CPU executes a predetermined calculation according to a predetermined program.

  The deviation detection circuit 12 calculates the deviation in the stereo camera 11 from the image data obtained from the sensors A and B and the data recorded in the memory 13, records the information in the memory 13, and the image processing circuit 16 To the effect that a deviation has occurred.

  Here, the shift in the stereo camera 11 is the positional shift between the camera 11a and the camera 11b, the inclination of the optical axis a and the optical axis b, the parallelism of the optical axis a and the optical axis b, and the lens center interval L in FIG. It means that the distance value detected by the distance detection device 10 has an error due to deviation or the like.

  As a result of the deviation detection by the deviation detection circuit 12, when correction is necessary, calibration is performed by the calibration circuit 17 based on information from the image processing circuit 16. The calibration circuit 17 performs a correction process so that the deviation detected based on the deviation information recorded in the memory 15 is canceled.

  Calculation of the distance value to the known object in the deviation detection circuit 12 of FIG. 1 will be described with reference to FIGS. FIG. 3 is a diagram schematically illustrating a relationship between an optical image of a known object and pixels on the imaging surface of the sensor in FIG. FIG. 4 is a schematic diagram illustrating an example of a conversion table between the number of pixels of the known pattern and the distance value of the known object in FIG.

  As shown in FIG. 3, a large number of pixels are formed in a grid in the vertical and horizontal directions on the imaging surface C of the sensor A in FIGS. 1 and 2, and the horizontal pixels are P1, P2, P3,. Px, and so on. In the conversion table of FIG. 4, for example, a known object is used as a traffic light, the diameter of the circular signal lamp G1 is known, and the relationship between the number of pixels on the imaging surface C of the signal lamp G1 and the distance to the signal lamp G1 is shown in advance. It is obtained by finding and stored in the memory 13 as the pixel number-distance value conversion information.

  As shown in FIG. 3, the known pattern detection circuit 12 a in FIG. 2 recognizes the optical image G formed on the imaging surface C as a traffic light image from the known patterns of known objects recorded in the memory 13.

  Since the image of the circular signal lamp G1 in the optical image G on the imaging surface C in FIG. 3 is located between the left end of the pixel Px and the right end of the pixel (Px + N), the size measuring circuit 12b The number of pixels corresponding to the diameter is calculated and calculated as (Px + N) − (Px) = N. Then, the distance value calculation circuit 12c refers to the conversion table in FIG. 4 and calculates the distance value Y from the number N of pixels measured.

  Next, basic processing steps S01 to S09 in the distance detection device 10 of FIGS. 1 to 4 will be described with reference to the flowchart of FIG.

  First, a standard image and a reference image are taken with the left and right cameras having sensors A and B (S01), and a known pattern detection operation recorded in advance from the standard image of sensor A is performed (S02). When a traffic light pattern such as that shown in FIG. 3 is detected as the known pattern (S03), the number N of pixels corresponding to the diameter of the signal lamp G1 is calculated as the known pattern size as shown in FIG. 3 (S04). A distance value (first distance value) to the traffic light is calculated from the number N of pixels using a conversion table as shown in FIG. 3 (S05).

  On the other hand, an actual measurement distance value from the stereo image (standard image / reference image) by the sensors A and B to the traffic signal is calculated as a second distance value by normal processing in the image processing circuit 16 (S06). Then, the second distance value is compared with the first distance value calculated from the known patterns in steps S04 and S05 (S07). If the distance values do not match (S08), the image processing circuit 16 is corrected. Is notified that it is necessary (S09). Thereby, the calibration of the distance measurement can be performed by the calibration circuit 17.

  As described above, the known object is detected from the image captured by the sensor A of the left camera 11a, and the first distance value from the camera 11a to the known object is obtained based on the detected size of the known object. The one distance value and the second distance value actually obtained by the stereo method are compared. As a result, if both distance values are the same, it is determined that there is no deviation in the distance detecting device 10, and both distances are determined. If the values are different, it is determined that the distance detection device 10 has a deviation, and the distance detection device 10 can be calibrated according to the determination result.

  Therefore, according to the present embodiment, by calculating the distance from the size (size) of the known object such as a traffic light on the imaging surface to the known object, another distance measuring unit such as a travel distance sensor is separately provided. Even if it is not prepared, the distance value can be calculated, and the deviation of the optical system of the camera can be easily detected by comparing with the result of normal distance measurement.

  In addition, deviation detection is possible even when the vehicle is stopped. Since the comparison is performed based on the standard image / reference image obtained in step S01, it is possible to detect deviation even when the vehicle is running. Moreover, the general-purpose distance detection system which is not limited to a vehicle model can be constructed | assembled by making the object which detects distance into a known object (general structure).

  Next, the processing steps S01 to S07 and S11 to S15 in which the detection of the known object in FIG. 5 is performed a plurality of times, the difference from the actual measurement value is averaged, and the deviation is determined after reducing the error are shown in the flowchart of FIG. A description will be given with reference to FIG. FIG. 7 is a diagram for explaining averaging of differences in steps S11 to S13 of FIG.

  Steps S01 to S07 similar to those in FIG. 5 are executed, and the difference between the actual measurement distance value (second distance value) in the normal process and the first distance value calculated from the known pattern is obtained in the comparison in step S07. It is recorded and held in the memory 13 or 15 (S11).

  Step S11 for obtaining the difference is performed a predetermined number of times set in advance (S12), and for example, as shown in FIG. 7, an average value of the differences based on the number of detection times is obtained (S13). Then, it is determined whether or not the average value of the differences is zero (S14). If it is not zero, the image processing circuit 16 is notified that correction is necessary (S15). Thereby, calibration of the distance detection apparatus 10 can be performed.

  As shown in FIGS. 6 and 7, the deviation detection can be performed more accurately by detecting the deviation based on the average value of the difference from the actual measurement value.

  Note that in steps S07 and S08 in FIG. 5, similarly to step S14 in FIG. 6, the difference between the first distance value and the second distance value may be taken and the determination criterion may be set to zero. 5 and 6, the determination criterion may be set to a predetermined value z (> 0) instead of zero, and the difference or the average value of the differences is −z or less or + z or more. In some cases, it may be determined that correction is necessary.

  As described above, the best mode for carrying out the present invention has been described. However, the present invention is not limited to these, and various modifications are possible within the scope of the technical idea of the present invention. For example, the distance detection device 10 of FIG. 1 can be used by being mounted on a self-propelled vehicle such as an automobile, but is not limited thereto, and of course may be mounted on a self-propelled robot or the like. Moreover, a known pattern is not limited to a traffic light, For example, various signs, a license plate, a guardrail, etc. may be sufficient.

  Further, the calibration of the distance detection device 10 is not limited to the electrical correction processing of the distance data or the like by the calibration circuit 17. For example, the cameras 11 a and 11 b can adjust the inclinations of the optical axes a and b, respectively. In this case, the calibration circuit 17 may control the drive unit to perform calibration when a deviation is detected.

It is a block diagram which shows schematically the distance detection apparatus by this Embodiment. It is a schematic diagram for demonstrating the principle of the distance detection in the distance detection apparatus 10 of FIG. It is a figure which shows typically the relationship between the optical image and pixel of a known object in the imaging surface of the sensor of FIG. It is a schematic diagram which shows the example of the conversion table of the pixel number and distance value of the known pattern of the known object of FIG. 5 is a flowchart for explaining basic processing steps S01 to S09 in the distance detection device 10 of FIGS. 6 is a flowchart for explaining processing steps S01 to S07 and S11 to S15 in which the detection of the known object in FIG. 5 is performed a plurality of times, the difference from the actual measurement value is averaged, and the deviation is determined after reducing the error. It is a figure for demonstrating the averaging of the difference in step S11-S13 of FIG.

Explanation of symbols

10 Distance detector 11 Stereo camera (image input device)
11a Left camera (image input device)
11b Right camera (image input device)
12 Displacement detection circuit 12a Known pattern detection circuit 12b Size measurement circuit 12c Distance value calculation circuit 12d Distance value comparison circuit 13, 15 Memory 16 Image processing circuit 17 Calibration circuit A, B Sensor, Image sensor C, D Image plane E, F Lens G Optical image of a known object G1 Signal lamp (known pattern)

Claims (7)

A method for detecting a shift in a distance detection device that detects a distance based on a stereo image obtained by a plurality of cameras including an optical system and an image sensor,
Determining a first distance value from a known object image captured by the imaging device via the optical system to the known object;
A displacement detection method for a distance detection device, wherein the displacement in the distance detection device is detected by comparing the first distance value with a second distance value to the known object detected from the stereo image. .   The distance detection according to claim 1, wherein a relationship between a size and a distance of the known object image is obtained in advance, and the first distance value is obtained based on the size of the known object image obtained from the imaging surface of the imaging element. Device deviation detection method.   The deviation detection method of the distance detection apparatus according to claim 1, wherein the comparison is executed a plurality of times, and the deviation is detected based on data obtained by averaging the comparison results. A plurality of image input devices for inputting object images for distance detection;
Means for generating distance data from data of the image input device;
Means for obtaining a distance value based on information obtained from a known object image input to the image input device;
Means for comparing the distance data with a distance value obtained from information of the known object image;
And a means for detecting a shift in the image input device based on the comparison result. Means for maintaining a relationship between the information of the known object and the distance value;
The distance detection device according to claim 4, wherein the distance value is obtained from information on a known object obtained by the image input device.   The distance detection device according to claim 4 or 5, wherein the comparison by the comparison unit is performed a plurality of times, and the determination is performed based on data obtained by averaging the comparison results.   The distance detection device according to claim 4, wherein the distance detection device is calibrated based on the deviation detection.

Images