JP6941949B2 - Vehicle image display device - Google Patents

Description

The present invention relates to a vehicle image display device, and more particularly to a vehicle image display device capable of displaying a bird's-eye view image of the own vehicle viewed from above.

Conventionally, a vehicle image display device including an in-vehicle camera that captures an image of the surroundings of the own vehicle has been known. In this vehicle image display device, in recent years, a technique has been developed in which an area directly under the own vehicle, which is a blind spot of the driver, is displayed on a display unit installed in the vehicle interior as a bird's-eye view image together with a transparent image of the vehicle.

For example, in Pat. The device is described. At the same time, a transparent image of the vehicle is displayed on the in-vehicle monitor, and the image is synthesized so that the area directly under the vehicle, which is the blind spot of the driver, can be visually recognized through the transparent portion.

Further, in the vehicle image display device described in Patent Document 2, images taken by a plurality of in-vehicle cameras arranged on the periphery of the vehicle are combined, and a bird's-eye view image including a road surface image of the own vehicle and its surroundings is translucent. It is displayed on the display unit together with the image of the converted vehicle.

In these vehicle image display devices, the time-series captured images captured by the vehicle-mounted camera are subjected to viewpoint conversion processing to generate an image of the region directly under the own vehicle that the vehicle-mounted camera cannot capture in real time, and shaped. It is displayed as an image. As a result, even if there is an obstacle such as a stone in the front portion of each tire which is a blind spot of the driver, the driver can easily grasp the obstacle and perform appropriate driving.

Japanese Unexamined Patent Publication No. 2004-336613 Japanese Unexamined Patent Publication No. 2014-036326

However, in the above-mentioned vehicle image display device, when the vehicle attitude changes significantly during traveling, for example, when the road surface gradient suddenly changes and the vehicle's front-rear inclination changes significantly, the vehicle-mounted camera and the road surface Since the distance of the vehicle and the range of the road surface captured by the in-vehicle camera change, the displayed bird's-eye view image may be distorted. Specifically, the bird's-eye view image continuously displayed on the display unit while the vehicle is running is obtained by imaging a time-series captured image (still image) of the in-vehicle camera that has undergone the viewpoint conversion process. Before and after the change in vehicle attitude, the bird's-eye view image seems to be distorted due to the difference in perspective of the imaged image. As a result, the driver who is watching the image may feel uncomfortable, or the information on the blind spot area required by the driver (for example, the presence or absence of an obstacle in front of the tire) may not be properly obtained. rice field.

The present invention has been made in view of the above problems, and when a bird's-eye view image of the own vehicle is generated based on an image captured by an in-vehicle camera and displayed on a display unit, the bird's-eye view is caused by a change in the vehicle attitude. An object of the present invention is to provide a vehicle image display device capable of reducing distortion appearing in an image and providing appropriate road surface information to a driver.

In order to achieve the above object, the vehicle image display device according to claim 1 performs a viewpoint conversion process based on an in-vehicle camera that photographs the road surface around the own vehicle and image data captured by the in-vehicle camera. It is equipped with a control unit that generates a bird's-eye view image and displays it on a display unit installed in the vehicle interior, and a vehicle attitude sensor that detects the vehicle attitude. The control unit has a platform that defines a reference vehicle attitude. However, when there is a discrepancy between the reference vehicle posture defined by the platform and the vehicle posture detected by the vehicle posture sensor, the deviation amount of the vehicle posture with respect to the image data obtained by the in-vehicle camera is used. The bird's-eye view image is generated by performing correction processing, and the control unit can set the platform based on the detection result of the vehicle posture sensor, and the set reference vehicle posture of the first platform and the vehicle posture. When there is a deviation from the vehicle posture detected by the sensor and the state in which the deviation amount of the vehicle posture is constant continues for a predetermined time or longer, the second vehicle posture detected by the vehicle attitude sensor is used as a reference. It is characterized by migrating to the platform of.

According to this configuration, when there is a discrepancy between the reference vehicle posture of the platform defined by the control unit and the vehicle posture while driving, the vehicle posture with respect to the image data taken by the in-vehicle camera. Since the correction process is performed based on the amount of deviation, it is possible to reduce the distortion of the bird's-eye view image due to the change in the vehicle posture. This makes it possible to reduce the discomfort given to the driver.
Further, according to this configuration, when the road surface having a constant slope continues, a platform suitable for the road surface condition can be newly set, so that the correction process for the set platform can be simplified.

The invention according to claim 2 is characterized in that, in the vehicle image display device according to claim 1, the in-vehicle camera is a front in-vehicle camera that captures an image of the lower front side of the own vehicle.

According to this configuration, it is possible to generate a bird's-eye view image of the area directly under the own vehicle and display it on the display unit while the vehicle is traveling forward based on the image data of the front in-vehicle camera. It's easy.

The invention according to claim 3 is characterized in that, in the vehicle image display device according to claim 1 or 2, the vehicle posture sensor detects at least the inclination of the vehicle in the front-rear direction.

According to this configuration, the inclination of the vehicle in the front-rear direction (that is, the change in the pitch angle of the vehicle) tends to greatly affect the distance between the in-vehicle camera and the road surface and the imaging range of the road surface. By detecting the change in the vehicle attitude sensor and correcting the image data based on the amount of change in the pitch angle, which is the amount of deviation, the control unit can appropriately reduce the discomfort in the bird's-eye view image.

Further, the invention according to claim 4 is the vehicle image display device according to any one of claims 1 to 3, wherein the control unit determines the road surface gradient in the image data based on the deviation amount. It is characterized in that correction processing is performed so as to approximate the road surface gradient in the reference vehicle attitude of the platform.

According to this configuration, when a visual change in the road surface gradient occurs in the image data obtained by the in-vehicle camera due to a change in the vehicle attitude, a correction process is performed to approximate the changed road surface gradient to the road surface gradient in the reference vehicle attitude of the platform. As a result, in the displayed bird's-eye view image, it is possible to make it appear as if the vehicle is traveling on a road surface having a constant slope. As a result, for example, even when the vehicle is traveling on a rough road where the vehicle posture changes, it is possible to display a bird's-eye view image as if the vehicle is traveling on a constant slope road surface (for example, a flat road surface). Therefore, the visibility of the area directly under the vehicle in the bird's-eye view image is improved. As a result, the driver can more accurately grasp the condition of the ground.

According to the vehicle image display device according to the present invention, when the vehicle attitude changes during traveling, correction processing is performed on the captured image data of the in-vehicle camera to display based on the time-series captured image data. It is possible to reduce the visual discomfort that occurs in the imaged portion of the image. As a result, the distortion of the bird's-eye view image displayed on the display unit is reduced, and appropriate road surface information can be provided to the driver.

The block diagram which shows the structure of the image display device for a vehicle which is an embodiment of this invention. The schematic diagram which shows the vehicle provided with the image display device for a vehicle. The flowchart which shows the processing of a control part. The figure explaining the operation of the correction processing in the image display device for a vehicle. (A) is a diagram showing an example of a bird's-eye view image displayed on the display unit of the conventional vehicle image display device, and (b) is a bird's-eye view displayed on the display unit of the vehicle image display device of the present embodiment. The figure which shows the example of the image. The flowchart which shows the processing of a control part. The figure explaining an example of setting of the correction platform.

FIG. 1 is a block diagram showing a configuration of a vehicle image display device 10 according to an embodiment of the present invention. The vehicle image pickup device 10 is mounted on the vehicle (own vehicle) 70, generates a bird's-eye view image of the own vehicle viewed from above based on the image captured by the camera, and displays it on a display unit installed in the vehicle interior. It includes a camera (vehicle-mounted camera) 20, an inclination sensor (vehicle attitude sensor) 30, a control unit 40, and a display unit 50.

The camera 20 includes a fisheye lens or a wide-angle lens capable of wide-angle shooting, and is arranged at the center of the front part of the vehicle 70 in the vehicle width direction as shown in FIG. 2, and has a wide angle including the lower front side of the vehicle 70. Image the area.

The inclination sensor 30 detects the inclination of the vehicle 70, which is the vehicle posture mainly due to the inclination of the road surface. For example, a G sensor, a gyro sensor, a suspension stroke sensor, or the like can be used alone or in combination. When the G sensor is used as the tilt sensor 30, it can be relatively small, simple and inexpensive to configure. In the present embodiment, the tilt sensor 30 detects the tilt angle (pitch angle) of the vehicle 70 in the front-rear direction.

The control unit 40 includes, for example, information processing means such as a CPU, storage means such as RAM and ROM, an input / output interface, and the like, and is connected to the camera 20, the tilt sensor 30, and the display unit 50. The control unit 40 includes a platform setting unit 41, a deviation amount calculation unit 42, a determination unit 43, and a bird's-eye view image generation unit 45.

The platform setting unit 41 is a part for setting a platform as a reference when calculating the amount of deviation of the vehicle posture. Here, the platform defines a reference vehicle posture. In the present embodiment, as shown in FIG. 2, a first platform with the vehicle posture as a reference vehicle posture is set as an initial value based on a state in which the vehicle 70 is placed on a horizontal road surface 80 having a road surface gradient of zero.

The platform setting unit 41 further sets a platform (hereinafter, referred to as a correction PF) as a reference for calculating the deviation amount according to the traveling state to a platform different from the first platform (for example, second and third platforms). Change as appropriate. The platform configuration unit 41, the amount of change for the platform configuration change threshold d _th time and the threshold value T _th is set. Here, the amount of change for which the threshold value _dth is set is the amount of change in the vehicle posture per unit time, and in the present embodiment, the amount of change in the pitch angle per unit time.

The deviation amount calculation unit 42 compares the vehicle posture defined by the correction PF (hereinafter referred to as the reference vehicle posture) with the vehicle posture detected by the inclination sensor 30, and determines the vehicle posture of the vehicle 70 with respect to the reference vehicle posture. Calculate the deviation amount Δ. In the present embodiment, the deviation amount calculation unit 42 compares the pitch angle of the reference vehicle posture with the pitch angle of the vehicle 70 detected by the inclination sensor 30 to calculate the deviation amount Δ of the pitch angle.

The determination unit 43 is a portion that determines whether or not to perform the correction process and determines the requirements for changing the correction PF. More specifically, the determination unit 43 compares the deviation amount threshold delta _th set in advance to be a deviation amount delta calculated by the shift amount calculating part 42, shift amount delta is outside the range of displacement amount threshold delta _th Determine if it is.

The bird's-eye view image generation unit 45 includes a correction processing unit 46, a viewpoint conversion processing unit 47, and an image composition unit 48.

The correction processing unit 46 corrects the image data captured by the camera 20 based on the deviation amount Δ calculated by the deviation amount calculation unit 42. In the present embodiment, coordinate conversion processing is performed according to the amount of deviation Δ so that the road surface gradient of the image captured by the camera 20 is approximated to the road surface gradient in the reference vehicle posture of the correction PF.

The viewpoint conversion processing unit 47 performs a process of performing viewpoint conversion (coordinate conversion) so that the image captured by the camera 20 becomes an image viewed from above the vehicle 70. Further, when the correction processing is performed by the correction processing unit 46, the viewpoint conversion processing unit 47 performs the viewpoint conversion processing on the image after the correction processing.

The image synthesizing unit 48 performs an image synthesizing process for synthesizing (imaging) a time-series bird's-eye view image whose viewpoint has been transformed. The viewpoint conversion process and the image composition process can be performed by, for example, the techniques disclosed in Japanese Patent Application Laid-Open No. 2002-087160.

The bird's-eye view image generation unit 45 further performs a process of synthesizing a transparent outline of the vehicle 70 with the generated bird's-eye view image, and displays this on the display unit 50. As the display unit 50, an in-vehicle monitor installed in the vehicle interior and capable of displaying the image generated by the control unit 40 can be used.

The bird's-eye view image displayed on the screen 55 of the display unit 50 displays the contour shape of the transparent vehicle 70 including the tire contour 52a (see FIG. 5B), and the driver can see the transparent vehicle. Through the image 52, the area R (see FIG. 2) directly below the vehicle 70, which is a blind spot, can be visually recognized.

FIG. 3 is a flowchart showing a bird's-eye view video display process executed by the control unit 40. Hereinafter, the processing of the control unit 40 will be described step by step.

First, when the deviation amount calculation unit 42 of the control unit 40 receives the detection signal from the inclination sensor 30, it is based on the correction PF set by the platform setting unit 41 and the pitch angle of the vehicle 70 detected by the inclination sensor 30. The deviation amount Δ of the pitch angle of the vehicle 70 with respect to the reference vehicle posture of the correction PF is calculated (step S11).

Determining unit 44 determines the calculated amount of deviation delta is whether beyond the scope of the deviation amount threshold delta _th (step S12).

If the deviation amount delta is within the range of displacement amount threshold delta _th (step S12: No), the viewpoint conversion processing unit 46 of the overhead view video generation unit 43, the image data by the camera 20 is captured, the viewpoint conversion processing (Step S14).

Next, the image synthesizing unit 47 synthesizes the time-series images after the viewpoint conversion processing to generate a bird's-eye view image, and also synthesizes a transparent image of the vehicle 70 with the generated bird's-eye view image (step S15). ).

After that, the bird's-eye view image generation unit 43 causes the display unit 50 to display the generated bird's-eye view image (step S16).

In step 12, if the amount of deviation delta is larger than the shift amount threshold delta _th (step S12: Yes), the correction processing unit 46 on the basis of the deviation amount delta, performs correction processing on the image data of the camera 20 ( Step S13).

After that, the viewpoint conversion processing unit 46 performs viewpoint conversion processing on the corrected image data (step S14). Next, the image synthesizing unit 47 synthesizes the time-series images after the viewpoint conversion process to generate a bird's-eye view image, and synthesizes a transparent image of the vehicle 70 with the bird's-eye view image (step S15). After that, the generated bird's-eye view image is displayed on the display unit 50 (step S16).

FIG. 4 is a diagram illustrating the operation of the correction process in the vehicle image display device 10 described above. In the description of FIG. 4, it is assumed that the correction PF of the vehicle 70 is set on the first platform.

When the vehicle 70 travels on an uneven road surface (rough road), the deviation amount Δ of the pitch angle of the vehicle 70 with respect to the reference vehicle posture (vehicle posture shown by the virtual line in FIG. 4) is from the deviation amount threshold value _Δth . When the value becomes large, the correction process according to the deviation amount Δ is executed for the image data captured by the camera 20.

In this correction process, the road surface gradient (road surface gradient 86 shown by the alternate long and short dash line in FIG. 4) of the image captured by the camera 20 is approximated to the road surface gradient (horizontal road surface 80 indicated by the alternate long and short dash line in FIG. 4) in the reference vehicle attitude. As described above, the coordinate conversion process according to the deviation amount Δ is executed. As a result, the image data of the camera 20 is corrected as if the vehicle 70 is traveling on a road surface having a constant slope (here, a horizontal road surface 80).

FIG. 5A is a diagram showing an example of a bird's-eye view image displayed on a display unit of a conventional vehicle image display device, and FIG. 5B is a diagram of the vehicle image display device 10 of the present embodiment. It is a figure which shows the example of the bird's-eye view image displayed on the display part 50. Note that FIG. 5 shows a state in which the characters "ABC" are drawn on the road surface directly below the vehicle 70.

As shown in FIG. 5A, in the conventional vehicle image display device in which the correction process is not performed, the distance between the road surface and the vehicle-mounted camera changes when the pitch angle of the vehicle changes due to driving on a rough road. Due to this, the bird's-eye view image displayed on the screen 155 of the display unit is distorted. Therefore, the characters "ABC" drawn on the road surface, which are visually recognized through the transparent vehicle image 152 on the screen 155, are distorted.

On the other hand, in the vehicle image display device 10 of the present embodiment, the distance between the road surface and the camera 20 is reduced as shown in FIG. 5B by performing supplementary correction processing according to the change in the pitch angle. Even when the vehicle changes, it is possible to reduce the distortion of the characters "ABC" drawn on the road surface that is visually recognized through the vehicle image 52. This makes it possible to provide the driver with accurate road surface information.

Among the changes in vehicle attitude (changes in pitch angle, roll angle, and yaw angle), changes in pitch angle have a great influence on the distance between the camera 20 and the road surface and the imaging range of the road surface by the camera 20. Cheap. In the vehicle image display device 10 of the present embodiment, the pitch angle is detected by the tilt sensor 30, and correction processing according to the deviation amount Δ of the pitch angle with respect to the reference vehicle posture defined by the platform is applied to the image data of the camera 20. By applying this, it is possible to appropriately suppress the distortion of the bird's-eye view image due to the change in the pitch angle.

Further, in the vehicle image display device 10 of the present embodiment, since the bird's-eye view image is generated based on the image captured by one wide-angle camera 20 mounted on the vehicle 70, the images captured by a plurality of cameras are combined. Image processing in the bird's-eye view image generation unit 45 is easier than that for generating a bird's-eye view image.

As described above, the vehicle image display device 10 of the present embodiment can change the correction PF from the initial value of the first platform to another platform according to the traveling state. The setting process of the correction PF will be described below.

FIG. 6 is a flowchart showing a correction PF setting process executed by the control unit 40. Hereinafter, the processing of the control unit 40 will be described step by step.

First, the deviation amount calculation unit 42 of the control unit 40 uses the detection signal from the tilt sensor 30 and the correction PF (for example, the first platform) already set in the platform setting unit 41 as a reference vehicle. The deviation amount Δ of the pitch angle of the vehicle 70 with respect to the posture is calculated (step S21).

Determining unit 44 determines the calculated amount of deviation delta is whether beyond the scope of the deviation amount threshold delta _th (step S22).

If the deviation amount delta is within the range of displacement amount threshold _{delta th} (step S22: No), the control unit 40, the process ends (RETURN).

If the deviation amount delta exceeds the shift amount threshold delta _th (step S22: Yes), the platform setting unit 41, based on the detection signal from the tilt sensor 30, the unit of time Atari vehicle 70 is a variation of the vehicle attitude The change amount d of the pitch angle is calculated, and _{it is determined whether or not the calculated change amount d is within the range of the change amount threshold dth} (step S23), and the change amount d is within the range of the _{change amount threshold dth.} The time T is counted (step S24).

When the change amount d _{does not exceed the change amount threshold value dth} in step S23 (step S23: No), the control unit 40 ends (returns) the process.

The platform setting unit 41 _{repeatedly determines step S23 until the count time T elapses from the time threshold value T th} (step S24: No).

When the counted time T _{exceeds the time threshold T th} (step S24: Yes), that is, a state in which the deviation amount Δ of the pitch angle is substantially constant (change amount d ≦ change amount threshold d _th ) is a predetermined time. When the threshold T _th or more is continued, the average vehicle posture of the vehicle 70 is calculated based on the vehicle posture of the vehicle 70 detected by the inclination sensor 30 from the start of counting the time T to the time threshold T _{th (step).} S25). Here, the average pitch angle of the vehicle 70 _{from the start of counting to the time threshold value Tth} is calculated based on the pitch angle detected by the inclination sensor 30.

After that, the platform setting unit 41 sets a new platform (a second platform having a different reference vehicle posture from the first platform) with the calculated average vehicle posture as the reference vehicle posture as the correction PF (step S26). End (return) the process.

In the present embodiment, deviation amount threshold delta _th used in the platform configuration process has been set to the same value as the deviation amount threshold delta _th used in the overhead view video generation display process them into different values It may be set.

Next, an example of setting the correction PF will be described with reference to FIG. 7. In FIG. 7, the vehicle 70 is moved in order from the vehicle position P ₁ to the vehicle position P _4, the vehicle 70 in the vehicle position P ₁ is traveling on a horizontal road surface, the first platform is set as the correction PF It is assumed that

When the vehicle 70 moves from the vehicle position P ₁ to the vehicle position P _2, the pitch angle changes, the deviation amount delta _P2 relative to the reference vehicle attitude exceeds the shift amount threshold delta _th. Then, the platform setting unit 41 calculates the change amount d of the pitch angle of the vehicle 70, and determines whether or not _{the calculated change amount d is within the range of the change amount threshold value dth.} In the vehicle position _{P 2,} the amount of change d> variation threshold _{d th,} and the first platform is maintained. In the vehicle position P _2, the correction process corresponding to the deviation amount delta _P2 is the overhead image which has been subjected by the correction processing unit 46 is generated.

When the vehicle 70 moves to the vehicle position P ₃ to continue a certain road gradient from the vehicle position P _2, together with the amount of deviation delta exceeds the shift amount threshold delta _th, the pitch angle of the vehicle 70 is a constant of road gradient When the pitch angle is determined to be substantially constant, the change amount d of the pitch angle is within the range of the _{change amount threshold value dth.} When this state continues for the time threshold value T _th or more, the platform setting unit 41 changes the correction PF from the first platform to the second platform. Reference vehicle posture defined by the second platform in this embodiment, the vehicle position shown by the vehicle position P _3.

In the vehicle position P _3, the overhead image generation processing, and calculating the shift amount Δ of the pitch angle relative to the reference vehicle posture of the second platform. In the vehicle position P _3, since the deviation amount delta calculated to be zero (in the range of displacement amount threshold delta _th), correction processing overhead view video is in the production is not executed.

When the vehicle 70 moves from the vehicle position P ₃ to the vehicle position P _4, the pitch angle changes, the deviation amount delta _P4 relative to the reference vehicle posture of the second platform exceeds the shift amount threshold delta _th. Then, the platform setting unit 41 calculates the change amount d of the pitch angle of the vehicle 70, and determines whether or not _{the calculated change amount d is within the range of the change amount threshold value dth.} In the vehicle position _{P 4,} the amount of change d> variation threshold _{d th,} and the second platform is maintained. In the vehicle position P _4, the correction process corresponding to the deviation amount delta _P4 is the overhead image which has been subjected by the correction processing unit 46 is generated.

As described above, when the road surface having a constant slope continues, the correction process for the set platform can be simplified by changing the setting of the correction PF to a platform suitable for the road surface condition. As a result, the load of data processing in the bird's-eye view image generation unit 45 can be reduced.

The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the invention.

For example, the tilt sensor 30 may further detect the roll angle of the vehicle 70 and perform correction processing according to the amount of deviation of the pitch angle Δ and the amount of deviation of the roll angle with respect to the reference vehicle posture. Further, the correction process based on the vehicle posture may be executed after the viewpoint conversion process by the viewpoint conversion processing unit.

Further, the vehicle image display device 10 may be configured to include a plurality of in-vehicle cameras and generate a bird's-eye view image including the vehicle 70 and its peripheral region by synthesizing the bird's-eye view images obtained by these cameras. For the synthesis of such an image, for example, the method shown in JP-A-2010-130413 can be applied. Examples of the plurality of in-vehicle cameras include, in addition to the front camera 20, side cameras arranged on the left and right sides of the vehicle 70 and capable of capturing an image of a region below the side of the vehicle 70, and a rear portion of the vehicle 70. A rear camera arranged in the vehicle 70 and capable of capturing an image of a region below the rear side of the vehicle 70 can be used in combination as appropriate.

10 Vehicle image display device 20 Camera (front in-vehicle camera)
30 Tilt sensor (vehicle attitude sensor)
40 Control unit 45 Bird's-eye view image generation unit 50 Display unit 70 Vehicle

Claims (4)

An on-board camera that shoots the road surface around the vehicle and
A control unit that generates a bird's-eye view image that has undergone viewpoint conversion processing based on the image data captured by the in-vehicle camera and displays it on a display unit installed in the vehicle interior.
Equipped with a vehicle attitude sensor that detects the vehicle attitude,
The control unit has a platform that defines a reference vehicle posture, and when there is a discrepancy between the reference vehicle posture defined by the platform and the vehicle posture detected by the vehicle posture sensor, the in-vehicle camera The image data obtained in the above-mentioned image data is corrected based on the amount of deviation of the vehicle posture to generate the bird's-eye view image .
The control unit can set the platform based on the detection result of the vehicle posture sensor, and there is a deviation between the set reference vehicle posture of the first platform and the vehicle posture detected by the vehicle posture sensor. For vehicles characterized by shifting to a second platform based on the vehicle posture detected by the vehicle posture sensor when the state in which the deviation amount of the vehicle posture is constant continues for a predetermined time or longer. Video display device. The vehicle image display device according to claim 1, wherein the vehicle-mounted camera is a front vehicle-mounted camera that captures an image of the lower front side of the own vehicle. The vehicle image display device according to claim 1 or 2, wherein the vehicle posture sensor detects at least an inclination of the vehicle in the front-rear direction. 3. The vehicle image display device described in.

Images