JP2011180962A - Drive assist device, drive assist method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a function to automatically correct slippage of an image from an initial state. <P>SOLUTION: The drive support device 100 includes a camera 10 mounted on a vehicle; a setting condition storage unit 39 which stores data for setting conditions of the camera 10; a theoretical distance calculation unit 35 which calculates a distance to a predetermined point on a road projected in an image from the position coordinate of the predetermined point within the image by use of the data for setting conditions stored in the setting condition storage unit 39; a stereo ranging unit 34 which determines a distance to the predetermined point; a difference calculation unit 36 which determines a difference between the distance determined by the stereo ranging unit 34 and the distance determined by the theoretical distance calculation unit 35; a projection conversion unit 38 which corrects the camera image based on the difference determined by the difference calculation unit 36; and a guide line superimposition unit 40 which superimposes a guide line for assisting drive to the camera image corrected by the projection conversion unit 38. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a driving support apparatus that superimposes and displays information for supporting driving on an image captured by an in-vehicle camera mounted on a vehicle.

  2. Description of the Related Art In recent years, driving assistance devices that support safe driving of an automobile using an image taken with an in-vehicle camera are known. As a driving support device, the situation around the vehicle is photographed with an in-vehicle camera, the location where the driver becomes a blind spot is presented as a video, and a guide line indicating the traveling direction of the vehicle is superimposed on the video to assist the parking operation. What to do is known.

  The mounting and fine adjustment of the in-vehicle camera are performed at a car body assembly factory in an automobile manufacturer, and after the automobile is delivered to the user, it is not subjected to aftercare in normal use. Therefore, there arises a problem that the camera mounting position is slightly shifted due to a secular change due to running of the automobile for several years to several decades, or a slight contact. When the camera is displaced, especially when a guide line is superimposed, the actual traveling direction of the vehicle is displaced from the superimposed position of the guide line on the image, which causes a driver's judgment to be wrong.

  For example, in Japanese Patent Application Laid-Open No. H10-260707, for the rearrangement of the camera, the position of the bumper for the rear camera and the position of the front turn signal for the side camera are detected from the video and are stored in advance as correct positions. Make a comparison. As a result, if there is a difference between the current video and the template image, it is determined that there is a shift in the camera mounting position.

Japanese Patent No. 4045862

  However, the cause of the displacement of the position of the image and the guide line is not only the displacement of the camera. Another major factor causing the displacement of the image and the guide line is the inclination of the vehicle body caused by the weight of passengers and cargo. In this case, there is no problem with respect to the mounting position of the camera. Therefore, as in Patent Document 1, it is not possible to detect the tilt of the vehicle body only by performing calibration using a part of the vehicle body. There has been a problem that the displacement of the position cannot be corrected sufficiently.

  The present invention has been made to solve the conventional problems, and an object of the present invention is to provide a driving support device having a function of automatically correcting image shift from an initial state.

  The driving support apparatus of the present invention is reflected in the image using a camera attached to a vehicle, a storage unit storing data of installation conditions of the camera, and installation condition data stored in the storage unit. A theoretical distance calculation unit that calculates the distance to a predetermined point on the road from the position coordinates of the predetermined point in the image, a distance measuring unit that calculates a distance to the predetermined point, and a distance that is determined by the distance measuring unit And a difference calculating unit for obtaining a difference between the distance obtained by the theoretical distance calculating unit, a correcting unit for correcting an image photographed by the camera based on the difference obtained by the difference calculating unit, and the correcting unit. And a superimposition unit that superimposes information for supporting driving on the corrected image. Here, the correction unit may correct an image captured by the camera by geometric transformation.

  Thus, based on the difference between the distance obtained by the distance measurement and the distance obtained from the data of the installation conditions (for example, the height from the road surface, the photographing angle, etc.), it can be obtained when photographing in the initial installation state. Deviation from the camera image can be corrected. Thereby, driving assistance information can be appropriately superimposed on a camera image. For example, a white line or a manhole can be used as the predetermined point on the road surface.

  In the driving support device of the present invention, the camera is a stereo camera, and the distance measuring unit has a configuration for obtaining a distance by stereo distance measurement.

  By performing stereo distance measurement in this way, it is possible to correct image shift using only the camera image.

  In the driving support device of the present invention, the theoretical distance calculation unit and the distance measurement unit have a configuration in which a distance is obtained using a part of the vehicle as the predetermined point. With this configuration, it is possible to detect a camera mounting deviation without being affected by the inclination of the vehicle. Here, a part of the vehicle may be a bumper. Since the bumper is photographed as standard, it is not necessary to change the mounting position of the camera for the present invention.

  The driving support device according to the present invention changes the installation condition of the camera from the installation condition stored in the storage unit when the distance calculated by the ranging unit and the distance calculated by the theoretical distance calculation unit do not match. However, the process for obtaining the distance to the predetermined point by the theoretical distance calculation unit and the process for obtaining the difference by the difference calculation unit are repeatedly performed to obtain the difference under a plurality of different installation conditions, and the difference is minimized. A deviation of the shooting angle between the installation condition and the installation condition stored in the storage unit is obtained as a tilt amount, and the correction unit has a configuration for correcting an image shot by the camera using the tilt amount.

  With this configuration, it is possible to obtain the amount of inclination of the photographing angle caused by the camera mounting deviation or the inclination of the vehicle, and to appropriately correct the camera image based on the amount of inclination.

  The in-vehicle camera of the present invention is an in-vehicle camera composed of a plurality of cameras attached to a vehicle, the storage unit storing data of installation conditions of the plurality of cameras, and the installation condition data stored in the storage unit Using a theoretical distance calculation unit for calculating a distance to a predetermined point on the road shown in the image from a position coordinate of the predetermined point in the image, and using the images taken by the plurality of cameras. A distance measuring unit that performs stereo distance measurement to a predetermined point, a difference calculating unit that calculates a difference between the distance obtained by the distance measuring unit and the distance obtained by the theoretical distance calculating unit, and the difference calculating unit And a correction unit that corrects an image captured by the camera based on the difference.

  Thus, based on the difference between the distance obtained by stereo ranging and the distance obtained from the installation condition data, it is possible to correct the deviation from the camera image that would be obtained in the initial installation state. it can.

  The driving support method of the present invention is a method of superimposing and displaying information for driving support on an image taken by a camera attached to a vehicle, and the driving support device takes an image by the camera. A step of reading the installation condition data of the camera from the storage unit, and the driving support apparatus using the installation condition data to a predetermined point on the road shown in the image. Calculating the distance from the position coordinates of the predetermined point in the image, the driving support device measuring the distance to the predetermined point, and the driving support device obtained by distance measurement A step of obtaining a difference between a distance and a distance obtained from position coordinates, a step in which the driving support device corrects an image taken by the camera based on the difference, and the driving support device To corrected image, and has a configuration in which the step of superimposing information for supporting operation.

  The program of the present invention is a program for superimposing and displaying information for supporting driving on an image captured by a camera attached to a vehicle, and is captured by the camera on a computer. A step of acquiring an image, a step of reading out data of installation conditions of the camera from a storage unit, and using the data of the installation conditions, a distance to a predetermined point on a road reflected in the image is determined in the image A step of calculating from a position coordinate of a predetermined point; a step of measuring a distance to the predetermined point; a step of determining a difference between a distance obtained by distance measurement and a distance obtained from the position coordinate; and based on the difference And a step of correcting an image captured by the camera and a step of superimposing information for assisting driving on the corrected image.

  With these configurations, the driving support method and the program according to the present invention appropriately corrects the deviation from the camera image that would be obtained when shooting in the initial installation state, as in the driving support device described above. Driving support information can be appropriately superimposed on an image. Note that various configurations of the above-described driving support device can be applied to the driving support method and program of the present invention.

  The present invention has an effect that it is possible to appropriately correct a deviation from a camera image that would be obtained when shooting in an initial installation state, and to appropriately superimpose driving support information on the camera image.

The block diagram which shows schematic structure of the driving assistance device in embodiment (A) The figure which shows the example of extraction of a bumper area (b) The figure which shows the example of extraction of a road surface area (A) A diagram showing ranging points measured at time T (b) A diagram showing ranging points measured at time T + t (c) A diagram showing integration of ranging points at two times The figure which shows the coordinate value and measuring distance of the ranging point Diagram showing the relationship between the camera mounting position and the road surface when the vehicle is seen from the side Diagram showing the theoretical distance at the center of the optical axis Diagram showing the theoretical distance of an arbitrary point on the vertical optical axis Diagram showing distance measurement points, measurement distance and theoretical distance in the road surface area Diagram showing the relationship between the actual coordinate system and the video coordinate system The figure which shows the picture gap which arises with the gap of the photographing angle of the camera Flow chart for camera mounting correction and theoretical distance measurement (A) Diagram showing an example of video before correction (b) Diagram showing an example of video after correction

Hereinafter, a driving support apparatus according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating a configuration of a driving support apparatus according to an embodiment. The driving support device 100 includes two cameras 10, an original image memory 20, an image correction unit 30, and a guide line superimposing unit 40. The functions of the image correction unit 30 and the guide line superimposing unit 40 may be realized by executing a program. Such a program is also included in the scope of the present invention.

  The two cameras 10 are attached at a predetermined angle at predetermined positions around the rear license plate of the vehicle. The two cameras 10 are mounted apart by a predetermined distance in order to perform distance measurement using a stereo camera. Further, it is assumed that the relationship between the two cameras 10 is completely fixed, and the positional relationship between the two cameras 10 and the relative relationship between the photographing directions are not shifted.

  The camera 10 includes a lens 11, an image sensor 12, an A / D converter 13, and a camera signal processing circuit 14. The camera 10 captures a subject behind the vehicle and generates a video signal. The lens 11 forms an image of light from the subject on the image sensor 12. The image sensor 12 photoelectrically converts the formed image and outputs an analog signal. The A / D converter 13 converts the analog signal output from the image sensor 12 into a digital signal. The camera signal processing circuit 14 performs camera signal processing for generating an image on the A / D converted digital signal. Camera signal processing is mainly luminance signal generation, color signal generation, and aperture signal generation. In addition, general video signal processing such as OB subtraction, white balance adjustment, and noise reduction is performed.

  The original image memory 20 inputs video signals output from the camera signal processing circuit 14 of the camera 10 from the two cameras 10, stores one frame of video from each camera, and a bumper of the image correction unit 30 described later. The video signal is output to the area extraction unit 31, the road surface area extraction unit 32, and the projective conversion unit 38.

  The image correction unit 30 includes a bumper area extraction unit 31, a road surface area extraction unit 32, a selector 33, a stereo distance measurement unit 34, a theoretical distance calculation unit 35, a difference calculation unit 36, and an inclination amount storage unit 37. And a projective transformation unit 38. The image correction unit 30 detects the tilt angle of the shooting angle of the camera 10 based on the images of the two cameras 10 stored in the original image memory 20, and outputs the video signal stored in the original image memory 20. After correcting by projective transformation, the corrected video signal is output to a guide line superimposing unit 40 described later.

  The guide line superimposing unit 40 superimposes a guide line for guiding the traveling direction of the vehicle and the distance from the vehicle on the video signal corrected by the image correcting unit 30, and outputs a video signal on which the guide line is superimposed.

  Next, the image correction unit 30 having the above-described configuration will be described in detail. First, the bumper area extraction unit 31 extracts a bumper area that is an area in which the bumper is shown in the video from the video captured by the two cameras 10 output from the original image memory 20, and extracts the road surface area. The section 32 extracts a road surface area that is an area where the road surface is shown.

  FIG. 2A is a diagram illustrating an example of extraction of a bumper area, and FIG. 2B is a diagram illustrating an example of extraction of a road surface area, and a hatched portion corresponds to an extraction region. The bumper area and road surface area are uniquely determined by the type of vehicle in which the camera 10 is mounted, the mounting position of the camera 10, the mounting angle, lens characteristics, etc., so it is necessary to determine the area by image processing from the video. Absent. Further, the bumper area extracted here is used to detect a mounting deviation of the camera 10, and the road surface area is used to detect a camera image deviation caused by the mounting deviation of the camera 10 and the inclination of the vehicle body.

  Next, the selector 33 selects one of the two areas extracted by the bumper area extraction unit 31 and the road surface area extraction unit 32 and sends the selected area to the subsequent stereo distance measurement unit 34. Which one to select can be switched according to time, for example. The selector 33 is not an essential configuration. If the selector 33 does not exist, two stereo ranging units 34, theoretical distance calculation units 35, and difference calculation units 36 in the subsequent stage may be prepared for the bumper area and the road surface area, respectively. However, in consideration of the fact that the calculation from the stereo distance measurement unit 34 to the difference calculation unit 36 is relatively complicated, in this embodiment, the stereo distance measurement unit 34, the theoretical distance calculation unit 35, the difference calculation unit 36, and the like. A configuration that shares the computing unit is adopted.

  Next, the stereo distance measuring unit 34 measures the distance with respect to either the bumper area or the road surface area selected by the selector 33. The distance is measured at a plurality of points in the area for correction to be performed later. The accuracy of correction is improved as the points for which distance measurement can be performed are widely distributed in the area and the number of points increases. In addition, when the necessary distance measurement points cannot be obtained at once, it is also effective to perform distance measurement at different times and integrate and use the distance measurement results.

  Fig.3 (a)-FIG.3 (c) are figures which show the example which integrates the ranging point of a road surface area. When four ranging points (FIG. 3 (a)) measured at time T and four ranging points (FIG. 3 (b)) measured at time T + t, which is advanced by time t from time T, are integrated, FIG. As shown in 3 (c), a total of eight distance measuring points are obtained. Since the present invention has a feature that the same subject does not have to be used as a distance measuring point at time T and time T + t, processing such as tracking a distance measuring point once measured is not necessary.

  There are already many methods for measuring the distance to a subject in a video based on the video from a stereo camera. In this embodiment, the distance is measured using a known method. The stereo distance measuring unit 34 corresponds to the “ranging unit” of the present invention.

  By performing the stereo distance measurement, the stereo distance measurement unit 34 knows the point at which the distance is obtained in the image and the coordinate value on the image of the point. FIG. 4 is a diagram illustrating an example of a processing result. Here, stereo distance measurement in the road surface area is described as an example, but the same applies to the bumper area. The bumper area is more uniform in color and texture than the road surface area, so it may be difficult to determine the distance measurement point, but the bumper also changes due to the influence of the shadow of the car body. If the distance measuring points are integrated, sufficient distance measuring points can be secured. The stereo distance measurement unit 34 sends the obtained coordinate values to the theoretical distance calculation unit 35 and the difference calculation unit 36, and sends a distance value (this distance is referred to as “measurement distance”) to the difference calculation unit 36.

  When the theoretical distance calculation unit 35 receives the coordinate value on the video of the distance measurement point measured by the stereo distance measurement unit 34, the theoretical distance calculation unit 35 calculates the distance to the coordinate point when it is assumed that there is no deviation in the camera 10 ( This distance is called "theoretical distance"). The theoretical distance calculation unit 35 is connected to a storage unit 39 that stores data of installation conditions of the camera 10. Here, the installation conditions are, for example, the height from the road surface at the mounting position of the camera 10, the mounting angle, the lens characteristics, and the like. A point on the road surface that is horizontal to the vehicle can be theoretically obtained on a one-to-one basis corresponding to the coordinate value on the image when the height of the mounting position of the camera 10, the mounting angle, and the lens characteristics are known. . The lens characteristic is the relationship between the angle of view with respect to the lens and the real image height.

  Here, for the sake of explanation, explanation will be given on a two-dimensional space when the vehicle is viewed from the side. FIG. 5 illustrates the relationship between the height and angle of the mounting position of the camera 10 and the road surface when the vehicle is viewed from the side. FIG. 6 is a diagram for explaining the distance between points photographed at the center of the optical axis on the video. When the height of the camera mounting position is H and the mounting angle is θ from the road surface, the distance to the center point of the image can be expressed by H / cos (θ).

  FIG. 7 is an explanatory diagram of an arbitrary point P on the vertical optical axis. When a point on the road surface that is Y pixels away from the center of the optical axis in the vertical direction is a point on the road surface, the camera light is calculated from the lens characteristics. Since it is understood that the point is away from the axis center by the angle α, the distance to an arbitrary point P is H / cos (θ + α). Similarly, if an arbitrary point on the image is a point on the road surface parallel to the vehicle body, the theoretical distance to the coordinate point can be obtained based on the coordinate value on the image. .

  On the other hand, when the relative positional relationship between the camera 10 and the bumper is known, the point on the bumper can be theoretically obtained on a one-to-one basis corresponding to the coordinate value in the video. The method of calculating the theoretical distance is the same as that on the road surface. The theoretical distance calculation unit 35 sends the theoretical distance of each distance measuring point obtained as described above to the difference calculation unit 36.

  The difference calculation unit 36 receives from the stereo distance measurement unit 34 and the theoretical distance measurement unit 35 the coordinate value of the distance measurement point on the bumper area or the road surface area, the data of the measurement distance, and the data of the theoretical distance. The difference calculation unit 36 calculates the difference between the input measurement distance and the theoretical distance, and feeds back the difference data to the theoretical distance calculation unit 35.

  FIG. 8 is a diagram illustrating an example of the coordinate value of the distance measurement point, the measurement distance data, and the theoretical distance data in the road surface area input to the difference calculation unit 36. If the mounting position of the camera 10 is adjusted to the state when the camera 10 is installed, that is, the same as the installation condition, the measured distance here and the theoretical distance should match. In the example shown in FIG. 8, it can be seen that there is a difference at each coordinate point, and that there is a deviation from the initial installation state. The camera image is corrected based on the difference between the measured distance and the theoretical distance.

  Next, a camera tilt correction method will be described. In this embodiment, a projective transformation coefficient that minimizes the difference between the measured distance and the theoretical distance is obtained. Here, it is defined that the tilt of the camera is caused by rotation with respect to the X axis, rotation with respect to the Y axis, and rotation with respect to the Z axis in an actual coordinate system.

FIG. 9 is a diagram illustrating the relationship between the actual coordinate system and the video coordinate system. An expression for projecting the point P (Xw, Yw, Zw) on the actual coordinate system onto the image coordinates (x, y) is expressed by the following expression (1).
Here, f is the focal length of the camera.

A point P (Xw ′, Yw ′, Zw ′) after rotating the point P on the actual coordinate system by θx around the X axis can be expressed by the following equation (2) using a rotation matrix.
A point (x _x , y _x ) on the image coordinates corresponding to the point P (Xw ′, Yw ′, Zw ′) after rotation in the above equation is expressed by the following equation (3) using the image coordinates (x, y). It is represented by

Similarly, the point (x _y , y _y ) on the image coordinate corresponding to the rotated point obtained by rotating the point P on the actual coordinate system by θy around the Y axis uses the image coordinate (x, y). Is represented by the following equation (4).
Similarly, the point (x _z , y _z ) on the image coordinate corresponding to the rotated point obtained by rotating the point P on the actual coordinate system by θz around the Z axis uses the image coordinate (x, y). Is expressed by the following equation (5).

By using the equations (3), (4), and (5) shown above, the point P on the actual coordinate system is turned to a point after rotation about the X, Y, and Z axes. The corresponding point (x _xyz , y _xyz ) on the video coordinates is expressed by the following equation (6) using the video coordinates (x, y).

The point (x _xyz , y _xyz ) on the image coordinates obtained as described above is shifted by θx around the X axis, θy around the Y axis, and θz around the Z axis with respect to the original attachment position. As shown in FIG. 10, there is a shift in the pixels that are originally imaged due to a shift in the shooting angle. Therefore, the coordinate value for obtaining the theoretical distance is corrected to (x _xyz , y _xyz ). The theoretical distance calculation unit 35 calculates the theoretical distance when it is assumed that the camera mounting position is corrected using the corrected coordinate values (x _xyz , y _xyz ).

  FIG. 11 is a diagram illustrating a flowchart for measuring the theoretical distance while correcting the camera mounting position. First, the camera mounting position is corrected. Specifically, the camera is rotated in the X-axis, Y-axis, and Z-axis directions (S10). The trial for finding the correction amount of the camera mounting position is performed, for example, by changing the deviation θx around the X axis, the deviation θy around the Y axis, and the deviation θz around the Z axis in accordance with the change in the camera attachment deviation amount. Alternatively, a combination of the deviation θx around the X axis, the deviation θy around the Y axis, and the deviation θz around the Z axis in a predetermined angle range may be performed brute force.

  Next, with the camera mounting position corrected, the theoretical distance to the distance measuring point is obtained (S12), and the difference between the obtained theoretical distance and the measured distance is obtained (S14). The process for obtaining the difference is performed for a plurality of distance measuring points, and the sum of the obtained differences is calculated. It is determined whether or not the sum of the differences is the smallest among the previous trials performed by correcting the camera mounting position (S16). As a result, when the difference is minimum (YES in S16), the rotation amount (θx, θy, θz) is recorded (S18).

  Next, it is determined whether or not the number of trials has reached the number of trials of the end condition (S20). The end condition is a predetermined number of trials, and finally the combination of rotation angles (θx, θy, θz) when the sum of the differences between the measured distances and the corrected theoretical distances at all distance measuring points is the smallest. Is the amount of mounting displacement of the camera position. When the number of trials reaches the end condition (YES in S20), the rotation amount finally recorded is output as the tilt amount of the shooting angle, that is, the correction amount for correcting the shooting angle of the camera. The termination condition is not limited to the number of trials, and may be terminated when a combination equal to or less than a predetermined difference sum is found.

  Up to this point, the method for measuring the tilt of the vehicle body has been described using a distance measuring point on the road surface area as an example. However, the same method can be used to measure the amount of camera tilt using the distance measuring point on the bumper area. . The difference calculation unit 36 stores the inclination amount that minimizes the difference in the inclination amount storage unit 37. The inclination amount storage unit 37 stores inclination amount data obtained by using the distance measurement points on the bumper area and inclination amount data obtained by using the distance measurement points on the road area. The tilt amount obtained using the distance measuring point on the bumper indicates the tilt amount of the camera mounting deviation, and is hereinafter referred to as “mounting tilt amount”. Here, it is assumed that the rotation angle representing the attachment tilt amount is (θtx, θty, θtz), and the rotation angle representing the photographing tilt amount is (θsx, θsy, θsz). The amount of tilt obtained using the distance measuring points in the road area indicates the tilt of the camera's shooting angle including camera mounting deviation and vehicle tilt, and is hereinafter referred to as “shooting tilt amount”.

Next, the projective conversion unit 38 performs projective conversion on the input video from the camera using the data of the photographing tilt amount stored in the tilt amount storage unit 37. The projective conversion unit 38 uses the rotation angles (θsx, θsy, θsz) representing the amount of photographing tilt to move the point P on the actual coordinate system around the X axis, the Y axis, and the Z axis (θsx, θsy, A point (x _xyz , y _xyz ) on the image coordinates corresponding to the rotated point rotated by θsz) is obtained. The point (x _xyz , y _xyz ) on the video coordinates is expressed by the following equation (7) using the point (x, y) before conversion.

  The projection-converted video as described above is obtained by correcting the video shift caused by the camera mounting shift and the vehicle body tilt. The projection conversion unit 38 corresponds to the “correction unit” of the present invention. Next, the image corrected by the projection conversion unit 38 is input to the guide line superimposing unit 40, and a guide line indicating the traveling direction of the vehicle is superimposed. The guide line to be superimposed is given in advance.

  FIG. 12A shows a video before correction, and FIG. 12B shows a video example after correction. Since the image deviation is corrected in the image correction unit 30, the deviation of the superimposed position of the image and the guide line is eliminated, and the correct image can be presented to the driver.

  According to the driving support apparatus of such an embodiment, a vehicle is not affected by a camera image shift caused by two factors, a camera attachment shift caused by secular change or the like, and a vehicle body tilt caused by the weight of an occupant or a load. Correction is automatically performed while the vehicle is traveling, and a normal image can always be presented to the driver.

  In addition, according to the driving support device of the above-described embodiment, since the camera image is electrically corrected on the image signal, it is not necessary to physically adjust the mounting position of the camera, and the burden on the user is remarkably reduced. Can do.

  In the embodiment described above, only the rotation about the X axis, the Y axis, and the Z axis has been described as a factor causing the image shift, but the correction target is not limited to this. For example, not only the rotation but also the parallel movement with respect to the X axis, the Y axis, and the Z axis cause the deviation. In this case, Xw, Yw, and Zw in Equation (2) may be replaced with values obtained by subtracting the amount of translation for each axis.

  In the above-described embodiment, as a method of correcting the deviation between the video and the guide line, the guide line is superimposed after correcting the video. However, the guide line is adjusted to the video deviation without changing the video. It is also possible to make the shift between the video and the guide line coincide with each other.

  In the above-described embodiment, when the mounting tilt amount exceeds a predetermined threshold, the user may be notified that the camera mounting angle is tilted. As a result, the driver can get a chance to adjust the mounting angle of the camera.

  As described above, there is an effect that it is possible to appropriately correct the deviation from the camera image that would be obtained when the image was taken in the initial installation state, and the driving support device that superimposes and displays the driving support information on the camera image Useful as.

DESCRIPTION OF SYMBOLS 10 Camera 11 Lens 12 Image pick-up element 13 A / D converter 14 Camera signal processing circuit 20 Original image memory 30 Image correction part 31 Bumper area extraction part 32 Road surface area extraction part 33 Selector 34 Stereo distance measurement part 35 Theoretical distance calculation part 36 Difference Calculation unit 37 Inclination amount storage unit 38 Projection conversion unit 39 Installation condition storage unit 40 Guide line superimposing unit 100 Driving support device

Claims (9)

A camera attached to the vehicle;
A storage unit storing data of installation conditions of the camera;
Using the installation condition data stored in the storage unit, a theoretical distance calculation unit that calculates the distance to a predetermined point on the road reflected in the image from the position coordinates of the predetermined point in the image;
A distance measuring unit for obtaining a distance to the predetermined point;
A difference calculating unit for obtaining a difference between the distance obtained by the distance measuring unit and the distance obtained by the theoretical distance calculating unit;
A correction unit that corrects an image captured by the camera based on the difference obtained by the difference calculation unit;
A superimposing unit that superimposes information for supporting driving on the image corrected by the correcting unit;
A driving support apparatus comprising:   The driving support apparatus according to claim 1, wherein the camera is a stereo camera, and the distance measurement unit obtains a distance by stereo distance measurement.   The driving assistance device according to claim 1, wherein the theoretical distance calculation unit and the distance measurement unit obtain a distance using a part of the vehicle as the predetermined point.   The driving support device according to claim 3, wherein a part of the vehicle is a bumper.   The driving support apparatus according to claim 1, wherein the correction unit corrects an image captured by the camera by geometric transformation. If the distance calculated by the distance measurement unit and the distance calculated by the theoretical distance calculation unit do not match, the theoretical distance calculation unit changes the installation conditions of the camera from the installation conditions stored in the storage unit. The process for obtaining the distance to the predetermined point and the process for obtaining the difference by the difference calculation unit are repeatedly performed to obtain the difference under a plurality of different installation conditions, and the installation condition that minimizes the difference is stored in the storage unit. The deviation of the shooting angle from the set installation conditions is obtained as the amount of tilt,
The driving support device according to claim 1, wherein the correction unit corrects an image captured by the camera using the tilt amount. An in-vehicle camera comprising a plurality of cameras attached to a vehicle,
A storage unit storing installation condition data of the plurality of cameras;
Using the installation condition data stored in the storage unit, a theoretical distance calculation unit that calculates the distance to a predetermined point on the road reflected in the image from the position coordinates of the predetermined point in the image;
A distance measuring unit that performs stereo distance measurement to the predetermined point using images taken by the plurality of cameras;
A difference calculating unit for obtaining a difference between the distance obtained by the distance measuring unit and the distance obtained by the theoretical distance calculating unit;
A correction unit that corrects an image captured by the camera based on the difference obtained by the difference calculation unit;
An in-vehicle camera characterized by comprising: A driving support method for superimposing and displaying information for supporting driving on an image captured by a camera attached to a vehicle,
A step in which the driving support device captures an image with a camera;
The driving support device reads data of installation conditions of the camera from a storage unit;
The driving support device calculates the distance to a predetermined point on the road shown in the image from the position coordinates of the predetermined point in the image, using the installation condition data;
The driving support device measuring the distance to the predetermined point;
The driving support device obtains a difference between a distance obtained by distance measurement and a distance obtained from position coordinates;
The driving support device correcting an image captured by the camera based on the difference;
The driving support device superimposing information for supporting driving on the corrected image;
A driving support method comprising: A program for superimposing and displaying information for supporting driving on an image taken by a camera attached to a vehicle,
Obtaining an image taken by the camera;
Reading data of installation conditions of the camera from a storage unit;
Calculating the distance to a predetermined point on the road shown in the image from the position coordinates of the predetermined point in the image using the data of the installation condition;
Measuring the distance to the predetermined point;
Obtaining a difference between a distance obtained by ranging and a distance obtained from position coordinates;
Correcting an image captured by the camera based on the difference;
Superimposing information for assisting driving on the corrected image;
A program characterized by having executed.