JP2016149611A - Optical axis deviation detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an optical axis deviation detection device capable of easily detecting deviation of an optical axis of a camera.SOLUTION: An optical axis deviation detection device (1) is provided that comprises: an image acquisition unit (9) which acquires an image (27) using a camera (5); a target recognition unit (11) which recognizes a target (29) in the acquired image; a first distance calculation unit (13) which calculates a first distance as a distance to the target based upon positional relationship between a longitudinal-direction position (Y) of the target in the image and a longitudinal-direction reference position (Y) in the image; a second distance calculation unit (15) which calculates a second distance as a distance to the target using a time-of-flight ranging method; and an optical axis deviation detection unit (17) which detects a deviation of an optical axis (K) of the camera based upon the distance difference between the first distance and second distance.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an optical axis deviation detection device.

  Conventionally, a technique for recognizing a white line or an obstacle in front of a vehicle based on an image acquired using a camera mounted on the vehicle is known. In this technique, in order to accurately recognize the position of a white line or the like, it is necessary to accurately determine the deviation of the optical axis in the camera.

  The invention described in Patent Document 1 recognizes left and right white lines in an image, and uses the intersection of the left and right white lines as a vanishing point. Then, the optical axis of the camera is detected based on the position of the vanishing point in the vertical direction.

JP 2000-242899 A

  In the invention described in Patent Document 1, when the left and right white lines cannot be recognized (for example, when the white line is hidden by a preceding vehicle in a traffic jam or when there is no white line on the road), the optical axis of the camera is detected. Can not.

  The present invention has been made in view of these problems, and an object of the present invention is to provide an optical axis deviation detection device that can easily detect an optical axis deviation in a camera.

  An optical axis deviation detection device of the present invention includes an image acquisition unit that acquires an image using a camera, a target recognition unit that recognizes a target in the image, a position of the target in the vertical direction in the image, Based on the positional relationship with the reference position in the vertical direction, a first distance calculation unit that calculates a first distance that is a distance to the target, and a distance to the target using the time-of-flight ranging method A second distance calculating unit for calculating the second distance; and an optical axis deviation detecting unit for detecting an optical axis deviation in the camera based on a distance difference between the first distance and the second distance.

  The optical axis deviation detecting device of the present invention can easily detect the optical axis deviation in the camera even if the white line on the road is not necessarily recognized.

1 is a block diagram illustrating a configuration of an optical axis deviation detection device 1. FIG. FIG. 3 is an explanatory diagram showing the arrangement of a camera 5 and a millimeter wave sensor 7 in a host vehicle 21. It is a flowchart showing the process which the optical axis deviation detection apparatus 1 performs. It is explanatory drawing showing the method of calculating a 1st distance. It is explanatory drawing showing the shift | offset | difference of the optical axis K. FIG. And deviation of the optical axis K, the position of the target 29 is an explanatory diagram showing the relationship between the coordinate Y _s, and [Delta] Y. FIG. 7A is an explanatory diagram illustrating a correction method when the optical axis K is shifted downward, and FIG. 7B is an explanatory diagram illustrating a correction method when the optical axis K is shifted upward. It is a flowchart showing the process which the optical axis deviation detection apparatus 1 performs. It is a flowchart showing the process which the optical axis deviation detection apparatus 1 performs.

Embodiments of the present invention will be described with reference to the drawings.
<First Embodiment>
1. Configuration of Optical Axis Deviation Detection Device 1 The configuration of the optical axis deviation detection device 1 will be described with reference to FIGS. The optical axis deviation detection device 1 is an in-vehicle device mounted on a vehicle. Hereinafter, the vehicle on which the optical axis deviation detection device 1 is mounted is referred to as the own vehicle. As shown in FIG. 1, the optical axis deviation detection apparatus 1 includes a control unit 3, a camera 5, and a millimeter wave sensor 7.

  The control unit 3 is a known computer including a CPU, RAM, ROM, and the like. The control unit 3 executes processing to be described later with a program stored in the ROM. The control unit 3 functionally includes an image acquisition unit 9, a target recognition unit 11, a first distance calculation unit 13, a second distance calculation unit 15, an optical axis deviation detection unit 17, and a correction unit 19. The function of each unit will be described later.

The camera 5 is a monocular camera. As shown in FIG. 2, the camera 5 is fixed to the front side of the passenger compartment of the host vehicle 21. The camera 5 can photograph a landscape in front of the host vehicle 21 and create an image. The height of the camera 5 from the ground is constant. The optical axis K of the camera 5 is essentially an optical axis K ₁ shown in FIG. However, the optical axis K is some reason, it may be shifted from the optical axis K _1.

  The millimeter wave sensor 7 emits millimeter waves in front of the host vehicle. Further, the millimeter wave sensor 7 receives a millimeter wave (reflected wave) reflected from a target in front of the host vehicle. The millimeter wave sensor 7 uses a time of flight (TOF) method to calculate the distance from the vehicle to the target reflecting the millimeter wave. That is, ½ of the distance traveled by the millimeter wave from the time when the millimeter wave is emitted to the time when the reflected wave is received is calculated as the distance from the host vehicle to the target. Hereinafter, the distance calculated in this way is referred to as a second distance. Examples of the target detected by the millimeter wave sensor 7 include other vehicles (for example, preceding vehicles, stopped vehicles, oncoming vehicles, etc.), fixed objects on the road, pedestrians, and the like.

  As shown in FIG. 2, the millimeter wave sensor 7 is attached to the front end of the host vehicle 21 and can emit a millimeter wave in front of the host vehicle 21. Therefore, the millimeter wave sensor 7 can detect a target existing in front of the host vehicle 21 and measure the distance (second distance) from the host vehicle to the target.

  As shown in FIG. 1, the host vehicle includes a notification device 23 and a vehicle control device 25 in addition to the optical axis deviation detection device 1. As will be described later, when the deviation of the optical axis K is detected, the notification device 23 notifies the driver of the host vehicle by an image, sound, or the like.

The vehicle control device 25 controls the host vehicle based on the distance to the target calculated by the first distance calculation unit 13 and the second distance calculation unit 15 as described later.
For example, when the distance to the target existing in front of the host vehicle is equal to or less than a predetermined threshold, the vehicle control device 25 performs a collision avoidance process (automatic braking, automatic steering, etc.). In addition, the vehicle control device 25 performs cruise control so that the distance from the preceding vehicle (an example of a target) is kept constant.

2. Processing Performed by Optical Axis Deviation Detection Device 1 The processing that optical axis deviation detection device 1 (especially control unit 3) repeatedly executes every predetermined time will be described with reference to FIGS.

In step 1 of FIG. 3, the image acquisition unit 9 captures the front of the host vehicle using the camera 5 and acquires an image.
In step 2, the target recognition unit 11 performs a known image recognition process on the image acquired in step 1 to recognize the target. Examples of the target to be recognized include other vehicles, fixed objects on the road, pedestrians, and the like.

In step 3, the first distance calculation unit 13 calculates the distance D from the host vehicle to the target recognized in step 2. This calculation method will be described with reference to FIG. FIG. 4 shows the image 27 acquired in the step 1 and the target 29 recognized in the step 2 existing in the image 27. The first distance calculation unit 13 stores the position of the vanishing point 31 in the image 27 in advance. The position of the vanishing point 31 in the image 27 is fixed. The vertical axis with the lower end 27A of the image 27 as the origin is taken as the Y axis. The coordinate of the vanishing point 31 on the Y axis is Y ₀ (fixed value).

The position of the vanishing point 31 in the image 27, when the optical axis K of the camera 5 is coincident with the original optical axis K _1, coincides with the vanishing point of truth. Meanwhile, when the optical axis K of the camera 5 is displaced from the original optical axis K _1, the vanishing point 31 is offset from the true vanishing point.

The coordinates in the Y-axis of the target 29 and _{Y s,} the difference between _{Y 0} and _{Y s} and [Delta] Y. There is a correlation between the value of ΔY and the distance D from the host vehicle to the target 29. When ΔY is determined, the corresponding distance D is uniquely determined. That is, as the distance D is large, _{Y s} is high, [Delta] Y is small. Also, as the distance D is small, _{Y s} is low, [Delta] Y increases.

The first distance calculation unit 13 includes a map that defines the relationship between the distance D and ΔY in advance. In the image 27, the first distance calculation unit 13 first obtains Y _s , then subtracts Y _s from Y ₀ to obtain ΔY, and finally substitutes the ΔY into the above-described map, and the distance D is obtained. calculate. The distance D calculated in this way is hereinafter referred to as a first distance.

The first distance varies according to the deviation of the optical axis K in the camera 5. This will be described with reference to FIGS. The _{Y s} when the optical axis K is the original optical axis _{K 1} and _{Y s1,} the _{Y s} where the optical axis K is downward optical axis _{K 2} than the original optical axis _{K 1} and _{Y s2, Let} Y _s3 be Y _s when the optical axis K is the optical axis K ₃ upward from the original optical axis K ₁ .

Further, the [Delta] Y (difference between _{Y 0} and _{Y s1)} when the optical axis K is the original optical axis _{K 1} and [Delta] Y _1, downward optical axis _{K 2} from the optical axis K is the original optical axis _{K 1} ΔY (difference between Y ₀ and Y _s2 ) is ΔY _2, and ΔY (Y ₀ and Y _s3 is the optical axis K ₃ upward from the original optical axis K _1. the difference) of the ΔY _3.

Since ΔY ₂ is smaller than ΔY ₁ as shown in FIG. 6, the first distance when the optical axis K is the optical axis K ₂ is the first distance when the optical axis K is the optical axis K _1. Become bigger. Further, since ΔY ₃ is larger than ΔY ₁ , the first distance when the optical axis K is the optical axis K ₃ is smaller than the first distance when the optical axis K is the optical axis K ₁ .

Map described above, when the optical axis K is an optical axis K _1, is set as the first distance is correct value (a value that matches the second distance). Therefore, when the optical axis K is an optical axis K _2, the first distance is greater than the second distance, when the optical axis K is an optical axis K _3, the first distance, the second distance Smaller. Further, as the optical axis K deviates greatly from the optical axis K _1, the first distance and the difference between the second distance becomes larger.

The vanishing point 31 is an example of a reference position in the vertical direction in the image 27. The coordinate Y _s is an example of the position in the vertical direction of the target 29 in the image 27. Further, the first distance calculation method described above is based on the positional relationship between the position of the target 29 in the image 27 in the vertical direction and the reference position in the vertical direction of the image 27 from the host vehicle to the target 29. It is an example of the method of calculating the 1st distance.

  Returning to FIG. 3, in step 4, the second distance calculation unit 15 uses the millimeter wave sensor 7 to calculate the distance from the host vehicle to the target recognized in step 2 (second distance). . As described above, the second distance calculation method uses the time-of-flight ranging method.

  In step 5, the optical axis deviation detection unit 17 subtracts the second distance calculated in step 4 from the first distance calculated in step 3 to calculate a distance difference. If the first distance is greater than the second distance, the distance difference is a positive value, and if the first distance is less than the second distance, the distance difference is a negative value.

  In step 6, the optical axis deviation detection unit 17 determines whether or not the absolute value of the distance difference calculated in step 5 is larger than a preset threshold value. If the absolute value of the distance difference is greater than the threshold value, the process proceeds to step 7, and if the absolute value of the distance difference is less than the threshold value, the process proceeds to step 8.

As described above, as the optical axis K deviates greatly from the optical axis K _1, since the first distance and the difference between the second distance becomes larger, the optical axis K is deviated from the optical axis K ₁ If the process proceeds to step 7, when the optical axis K is not is largely deviated from the optical axis K ₁ it will proceed to step 8.

In step 7, the optical axis deviation detection unit 17 performs notification using the notification device 23.
In step 8, the correction unit 19 corrects the position of the vanishing point 31 in the direction in which the distance difference calculated in step 5 decreases. This correction will be described with reference to FIGS. 7A and 7B.

Figure 7A (since is downward from the optical axis K ₁₎ the optical axis K is for a K _2, represents an if present above the target 29 is actually in the image 27. In this case, ΔY (ie, ΔY ₂ ) is smaller than ΔY ₁ and the first distance is greater than the second distance. In this case, the correction unit 19 corrects the position of the vanishing point 31 upward. The correction amount is set such that the first distance calculated using the corrected vanishing point 31 is equal to the second distance.

FIG. 7B shows a case where the target 29 is present below the actual position in the image 27 because the optical axis K is K ₃ (because it is upward from the optical axis K ₁ ). In this case, ΔY (that is, ΔY ₃ ) is larger than ΔY ₁ and the first distance is smaller than the second distance. In this case, the correction unit 19 corrects the position of the vanishing point 31 downward. The correction amount is set such that the first distance calculated using the corrected vanishing point 31 is equal to the second distance.

3. Effects produced by the optical axis deviation detection device 1 (1A) The optical axis deviation detection device 1 can easily detect the deviation of the optical axis K in the camera 5. In particular, the shift of the optical axis K can be detected without necessarily recognizing the white line on the road.

(1B) The optical axis deviation detection device 1 can correct the deviation of the optical axis K and increase the calculation accuracy of the first distance.
(1C) The optical axis deviation detection device 1 performs notification when the absolute value of the distance difference is larger than a preset threshold value. As a result, the driver of the host vehicle can know that the deviation of the optical axis K is large. Further, since the notification is performed only when the optical axis K is largely displaced, it is difficult for the driver to be bothered by excessive notification.

(1D) The optical axis deviation detection device 1 acquires the second distance using the millimeter wave sensor 7. As a result, the second distance can be accurately calculated.
<Second Embodiment>
1. Configuration of Optical Axis Deviation Detection Device 1 The configuration of the optical axis deviation detection device 1 of the present embodiment is the same as that of the first embodiment.

2. Processing Performed by Optical Axis Deviation Detection Device 1 Processing executed by the optical axis deviation detection device 1 of the present embodiment will be described with reference to FIG. The processing executed by the optical axis deviation detection device 1 of the present embodiment is basically the same as that of the first embodiment. Below, it demonstrates centering on difference with the said 1st Embodiment.

In step 11, the image acquisition unit 9 captures the front of the host vehicle using the camera 5 and acquires an image.
In step 12, the target recognition unit 11 performs a known image recognition process on the image acquired in step 11 to recognize the target.

  In step 13, the target recognition unit 11 determines whether or not a plurality of targets can be recognized in step 12. If a plurality of targets can be recognized, the process proceeds to step 14, and in other cases (only a single target is recognized or no target is recognized), this processing is terminated.

In step 14, the first distance calculation unit 13 calculates a first distance for each of the plurality of targets. The calculation method of the first distance is the same as that in the first embodiment.
In step 15, the second distance calculation unit 15 calculates a second distance for each of the plurality of targets. The calculation method of the second distance is the same as that of the first embodiment.

In step 16, the optical axis deviation detection unit 17 calculates a distance difference (a value obtained by subtracting the second distance from the first distance) for each of the plurality of targets.
In step 17, the optical axis deviation detection unit 17 determines whether or not the variation between the distance differences is equal to or less than a preset threshold value. Here, the variation between the distance differences has the following meaning. The plurality of targets recognized in the step 12 are 29a, 29b, 29c,. The distance difference in each target is Sa, Sb, Sc. The variation between the distance differences is a standard deviation of Sa, Sb, Sc.

If the variation between the distance differences is equal to or smaller than the threshold value, the process proceeds to step 18, and if the variation between the distance differences is larger than the threshold value, the present process is terminated.
In step 18, first, the optical axis deviation detection unit 17 calculates an average value ave (S) of the distance differences Sa, Sb, Sc... For each target. Next, it is determined whether or not the absolute value of the average value ave (S) is larger than a preset threshold value. If the absolute value of the average value ave (S) is greater than the threshold value, the process proceeds to step 19, and if it is equal to or less than the threshold value, the process proceeds to step 20.

In step 19, the optical axis deviation detection unit 17 performs notification using the notification device 23.
In step 20, the correction unit 19 corrects the position of the vanishing point 31 in the direction in which the average value ave (S) calculated in step 18 decreases. The correction method is the same as that in the first embodiment.

3. Effects exhibited by the optical axis deviation detection device 1 According to the second embodiment described in detail above, the following effects are obtained in addition to the effects (1A) to (1D) of the first embodiment described above.

(2A) The optical axis deviation detection apparatus 1 detects the deviation of the optical axis K on condition that the variation between the distance differences calculated for each of the plurality of targets is equal to or less than a preset threshold value. Therefore, the deviation of the optical axis K can be detected more accurately.
<Third Embodiment>
1. Configuration of Optical Axis Deviation Detection Device 1 The configuration of the optical axis deviation detection device 1 of the present embodiment is the same as that of the first embodiment.

2. Processing Performed by Optical Axis Deviation Detection Device 1 Processing executed by the optical axis deviation detection device 1 of the present embodiment will be described with reference to FIG. The processing executed by the optical axis deviation detection device 1 of the present embodiment is basically the same as that of the first embodiment. Below, it demonstrates centering on difference with the said 1st Embodiment.

In step 21, the image acquisition unit 9 captures the front of the host vehicle using the camera 5 and acquires an image.
In step 22, the target recognition unit 11 performs a known image recognition process on the image acquired in step 21 to recognize the target.

  In step 23, the target recognition unit 11 determines whether or not the target recognized in step 22 is a stopped vehicle. If the vehicle is a stopped vehicle, the process proceeds to step 24. Otherwise (if only a target other than the stopped vehicle has been recognized or no target has been recognized), the process ends.

In step 24, the first distance and the second distance are repeatedly calculated for the recognized stopped vehicle every predetermined time. That is, at each time _{t 1, t 2, t 3} ··· t n, calculating the first distance and the second distance. n is a natural number of 2 or more. The calculation method of the first distance and the second distance is the same as that of the first embodiment. If the host vehicle is traveling, the distance between the host vehicle and the stopped vehicle changes as time elapses.

In step 25, the optical axis deviation detection unit 17 calculates the distance difference at each of the times t ₁ , t ₂ , t ₃ ... T _n . In other words, the first distance calculated at time _{t i,} subtracting the second distance calculated at the same time, calculates the distance difference _{P i} at time _{t i (i} = 1,2,3 _···· n ).

In step 26, the optical axis deviation detection unit 17 determines whether or not the variation between the distance differences P _i is equal to or less than a preset threshold value. Here, the variation of the distance difference P _i, refers to the standard deviation of the distance difference P _i.

When the variation between the distance differences P _i is equal to or smaller than the threshold value, the process proceeds to step 27, and when the variation between the distance differences P _i is larger than the threshold value, this process is terminated.
In step 27, first, the optical axis deviation detection unit 17 calculates the average value ave of the distance difference _{P i (P).} Next, it is determined whether or not the absolute value of the average value ave (P) is larger than a preset threshold value. When the absolute value of the average value ave (P) is larger than the threshold value, the process proceeds to step 28, and when it is equal to or smaller than the threshold value, the process proceeds to step 29.

In step 28, the optical axis deviation detection unit 17 performs notification using the notification device 23.
In step 29, the correction unit 19 corrects the position of the vanishing point 31 in the direction in which the average value ave (P) calculated in step 27 decreases. The correction method is the same as that in the first embodiment.

3. Effects exhibited by the optical axis deviation detection device 1 According to the third embodiment described in detail above, the following effects are obtained in addition to the effects (1A) to (1D) of the first embodiment described above.

(3A) The optical axis misalignment detection apparatus 1 is configured on the condition that the variation between the distance differences P _i calculated repeatedly for the same target (stopped vehicle) that is stopped is equal to or less than a preset threshold value. A shift of K is detected. Therefore, the deviation of the optical axis K can be detected more accurately.
<Other embodiments>
As mentioned above, although embodiment of this invention was described, this invention can take a various form, without being limited to the said embodiment.

(1) In the first to third embodiments, the reference position in the vertical direction in the image 27 may be a position other than the vanishing point.
(2) In the first to third embodiments, other ranging means may be used instead of the millimeter wave sensor 7. For example, means (for example, a radar, a rider, etc.) that can calculate the second distance by time-of-flight ranging using light, radio waves, and sound waves of any wavelength can be appropriately selected and applied.

(3) In the first to third embodiments, when the correction is performed, the position of the vanishing point 31 may be left as it is, and the position of the target 29 may be corrected. Also in this case, ΔY can be set to ΔY ₁ which is a value when there is no deviation of the optical axis K.

(4) In the second and third embodiments, the variation between the distance differences may be other than the standard deviation. For example, the difference between the maximum value and the minimum value among the distance differences may be used. Good.
(5) In the first to third embodiments, the correction may be performed on condition that the absolute value of the distance difference is larger than a predetermined threshold value.

  (6) The functions of one constituent element in the above embodiment may be distributed as a plurality of constituent elements, or the functions of a plurality of constituent elements may be integrated into one constituent element. Further, at least a part of the configuration of the above embodiment may be replaced with a known configuration having the same function. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified only by the wording described in the claim are embodiment of this invention.

  (7) In addition to the optical axis deviation detection device described above, a system including the optical axis deviation detection device as a component, a program for causing a computer to function as a control unit of the optical axis deviation detection device, and a medium on which the program is recorded The present invention can also be realized in various forms such as an optical axis deviation detection method and an optical axis deviation correction method.

DESCRIPTION OF SYMBOLS 1 ... Optical axis deviation detection apparatus, 3 ... Control unit, 5 ... Camera, 7 ... Millimeter wave sensor, 9 ... Image acquisition unit, 11 ... Target recognition unit, 13 ... 1st distance calculation unit, 15 ... 2nd Distance calculation unit, 17 ... optical axis deviation detection unit, 19 ... correction unit, 21 ... own vehicle, 23 ... notification device, 25 ... vehicle control device, 27 ... image, 27A ... lower end, 29 ... target, 31 ... vanishing point

Claims (7)

An image acquisition unit for acquiring an image using a camera;
A target recognition unit for recognizing a target in the image;
A first distance calculation that calculates a first distance that is a distance to the target based on a positional relationship between a vertical position of the target in the image and a reference position in the vertical direction of the image. Unit,
A second distance calculating unit that calculates a second distance that is a distance to the target using time-of-flight ranging;
An optical axis deviation detection unit that detects an optical axis deviation in the camera based on a distance difference between the first distance and the second distance;
An optical axis misalignment detection apparatus comprising: The optical axis deviation detection device according to claim 1,
An optical axis misalignment detection apparatus comprising: a correction unit that corrects the reference position in a direction in which the distance difference decreases. The optical axis deviation detection device according to claim 1 or 2,
The optical axis deviation detection unit is configured to detect the deviation of the optical axis based on the distance difference on condition that variation between the distance differences calculated for each of the plurality of targets is equal to or less than a preset threshold value. An optical axis misalignment detection apparatus characterized by detecting. The optical axis deviation detection device according to claim 1 or 2,
The optical axis deviation detection unit is based on the distance difference based on the distance difference, provided that a variation between the distance differences repeatedly calculated for the same target that is stopped is equal to or less than a preset threshold value. An optical axis misalignment detection apparatus characterized by detecting a misalignment. The optical axis misalignment detection apparatus according to any one of claims 1 to 4,
The optical axis deviation detection unit detects the optical axis deviation when the distance difference is larger than a preset threshold value. The optical axis deviation detection device according to any one of claims 1 to 5,
The optical axis deviation detection device, wherein the target is a vehicle. The optical axis misalignment detection apparatus according to any one of claims 1 to 6,
The second distance calculation unit calculates the second distance using a flight time of millimeter waves.

Images