JP2012208819A - Vehicle outside video display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle outside video display system which is capable of solving the problem that it is difficult to intuitively know how a vehicle moves in accordance with a driving operation.SOLUTION: A vehicle outside video display system is comprised of a vehicle and a smartphone. The smartphone includes terminal attitude detection means which detects an attitude of the smartphone per se and display means which displays information on a screen. The vehicle includes vehicle attitude detection means which detects an attitude of the vehicle per se, vehicle outside video image capturing means which captures a video image outside the vehicle, display parameter determination means which determines a display parameter of the vehicle outside video image to be displayed on the display means from the variation of the terminal attitude detected by the terminal attitude detection means, and image combination means which combines video images captured by the vehicle outside video image capturing means on the basis of the parameter determined by the display parameter determination means.

Description

  The present invention relates to a system capable of easily displaying and operating an application for displaying an image outside a vehicle on a mobile terminal in an automobile.

  There are a wide range of in-vehicle devices that capture images of blind spots invisible to the driver of the car with a camera and display the images on a display such as a navigation device in the car. However, there is a limit to the viewing angle that can be displayed by the camera. is there. For this reason, there exists an apparatus that can change the direction of the field of view using a button or a touch panel operation (for example, Patent Document 1). There is also an apparatus that combines a plurality of camera images and displays a wide-angle range at a time as if the image is seen from above the vehicle, and eliminates the need to specify the camera orientation (for example, Patent Document 2).

JP 7-311052 A JP 2007-292535 A

  However, in an apparatus such as Patent Document 1, it is difficult for the driver to specify the direction of the field of view intuitively, and it is difficult to grasp the angle at which the current field of view is viewed from the driver. It was difficult.

  In addition, in an apparatus such as Patent Document 2, the image is distorted due to the limitations of image synthesis, and even if the driver sees the video from above, it differs greatly from the normal field of view. It is an object of the present invention to provide an outside video display system that can solve the difficulty of intuitively understanding whether a vehicle behaves.

  The vehicle exterior video display system of the present invention is a system that includes a vehicle and a smartphone, and the smartphone has terminal attitude detection means for detecting the attitude of the smartphone itself and display means for displaying information on a screen, and the vehicle The vehicle attitude detection means for detecting the attitude of the vehicle itself, the vehicle exterior video imaging means for capturing the vehicle exterior video, and the vehicle exterior video to be displayed on the display means from the amount of change in the terminal attitude detected by the terminal attitude detection means. Display parameter determining means for determining a display parameter, and image synthesizing means for synthesizing the video captured by the outside-vehicle video capturing means based on the parameter determined by the display parameter determining means.

  The display parameter determination means of the vehicle exterior video display system according to the present invention is characterized in that the display parameter determination means determines from the terminal attitude detected by the terminal attitude detection means and the vehicle attitude detected by the vehicle attitude detection means.

The terminal attitude detection means and the vehicle attitude detection means of the outside-vehicle image display system of the present invention detect a rotation angle in the left-right direction or detect rotation angles in the left-right and up-down directions, and the display parameter determination means When the rotation angle is detected, the relative rotation angle of the terminal with respect to the vehicle in the left-right direction is calculated. When the rotation angle with the left-right and up-down directions is detected, the relative rotation angle of the terminal with respect to the vehicle in the left-right and up-down directions is calculated. The display parameter is determined by calculation.

  The terminal posture detection means of the video display system outside a vehicle of the present invention detects a rotation angle in the left-right direction or detects rotation angles in the left-right and up-down directions, and the display parameter determination means detects a rotation angle in the left-right direction. When calculating the left and right relative rotation angles of the terminal from the initial state, when detecting the left and right and vertical rotation angles, calculate the left and right and vertical relative rotation angles of the terminal from the initial state The display parameter is determined based on this.

  The vehicle and the terminal of the video display system outside a vehicle according to the present invention can communicate by wireless communication means or wired communication means.

  The smartphone of the video display system outside a vehicle according to the present invention further includes an operation unit that detects a user operation, and the display parameter determination unit determines a video display method according to the operation detected by the operation unit. And

  The vehicle of the video display system outside a vehicle according to the present invention further includes a change amount setting unit that determines a coefficient of a change amount of the display parameter according to a rotation angle of the terminal.

  According to the system of the present invention, the direction of the visual field can be adjusted very intuitively in a form similar to a “window frame” or “mirror”. For this reason, the driver can intuitively understand which image is displayed.

The block diagram which shows the system configuration | structure of Embodiment 1 of this invention. (A) Schematic diagram showing when the smartphone screen is viewed in front (b) Schematic diagram showing when the smartphone is tilted (A) Plan view of the vehicle and smartphone viewed from above (b) Elevation view of the vehicle and smartphone viewed from the side Flow chart showing the main operation of the vehicle in the first embodiment. The block diagram which shows the system configuration | structure of Embodiment 2 of this invention. Flow chart showing the main operation of the vehicle in the embodiment B

  Hereinafter, a system according to an embodiment of the present invention will be described. Since various design states can be considered for the system according to the embodiment of the present invention, the system will be individually described below as Embodiments 1 and 2.

(Embodiment 1)
The system configuration of Embodiment A according to the present invention will be described with reference to the block diagram of FIG.
In the first embodiment, it is assumed that respective orientations are obtained using the geomagnetic sensors of the vehicle and the terminal, the absolute positional relationship is calculated, and the display parameters are determined. For this reason, it is not necessary to reset the initial position or adjust the rotation angle, and more accurate display can be performed.
In FIG. 1, a smartphone 10 includes a terminal posture detection unit 1, an operation input unit 2, and a display unit 3.

Here, the terminal posture detection means 1 is comprised by an acceleration sensor, a geomagnetic sensor, and a process part, and is comprised so that the attitude | position of the smart phone 10 can be detected. The detected parameters will be described later.

The operation input unit 2 includes devices such as a touch pad and various switches and peripheral circuits, and is configured to be used when a driver gives a command to a running application. . In almost all smartphones, the display means and the operation means are integrated by a touch panel.
The display means 3 is composed of a liquid crystal display or the like, and is configured to be able to present information to the driver by displaying various information.
The vehicle 20 includes a vehicle attitude detection unit 4, a display parameter determination unit 5, a vehicle outside image photographing unit 6, and an image composition unit 7.
The vehicle attitude detection means 4 is configured by an acceleration sensor, a geomagnetic sensor, and a processing unit, and detects the direction in which the vehicle 20 is facing.

The display parameter determination means 5 is composed of a CPU and a memory, and the direction of the vehicle 20 detected by the vehicle attitude detection means 4 and the attitude of the smartphone 10 detected by the terminal attitude detection means 1, and the driver detected by the operation input means 2. The display parameter of the image outside the vehicle to be displayed on the display means 3 of the smartphone 10 is determined by the operation of. The display parameters to be determined and the detailed determination method will be described later.
The vehicle outside image photographing means 6 is constituted by a single or a plurality of cameras, and photographs images outside the vehicle mainly around the blind spots for the driver.

The image synthesizing means 7 is composed of a CPU and a memory, receives an image from the outside-vehicle image photographing means, and converts the viewpoint based on the parameters determined by the display parameter determining means 5 to synthesize the images. A detailed synthesis method will be described later.
Next, the posture parameters of the smartphone 10 detected by the terminal posture detection unit in FIG. 1 will be described with reference to FIGS. 2a and 2b. 2A is a schematic diagram illustrating the smartphone screen when viewed from the front, and FIG. 2B is a schematic diagram illustrating the smartphone tilted.

  As shown in FIG. 2a, when the screen of the smartphone 10 is viewed in front, the short side direction is defined as + X, -X axis, the long side direction is defined as + Y, -Y axis, and the direction perpendicular to the screen is defined as + Z, -Z axis. .

  The smartphone 10 according to the present embodiment includes a triaxial gyro sensor and detects rotational components φX201, φY202, and φZ203 along the respective axes.

  Furthermore, the smartphone 10 according to the present embodiment includes a biaxial or triaxial acceleration sensor and a geomagnetic sensor. The acceleration sensor can detect the current acceleration vector that the smartphone 10 has like the vector 204 in FIG. 2b (in a stationary state, the component is gravitational acceleration). Further, the geomagnetic sensor can detect a north direction vector indicating a north direction like a vector 205.

  Next, the display parameters and the determination method of the outside video to be displayed on the display unit of the smartphone 10 determined by the display parameter determination unit in FIG. 1 will be described with reference to FIGS. 3a and 3b. 3A is a plan view of the vehicle and the smartphone as viewed from above, and FIG. 3B is an elevational view of the vehicle and the smartphone as viewed from the side.

(Full spec version)
3A is a plan view of the vehicle 20 and the smartphone 10 as viewed from above, and FIG. 3B is an elevational view of the vehicle 20 and the smartphone 10 as viewed from the side. The vehicle 20 is directed in the direction of travel, and + x and -x axes are defined on the left and right, + y and -y axes are defined in the vertical direction, and + z and -z axes are defined in the front and rear direction.

  The vehicle 20 in the present embodiment has a geomagnetic sensor, and can detect a north direction vector indicating a north direction like a vector 301 in FIG. 3A. On the other hand, as described above, the smartphone 10 also detects the north direction vector like the vector 302. By aligning both, it is possible to detect which direction the smartphone 10 is facing the vehicle 20 on the horizontal plane. Specifically, the y-axis direction of the smartphone 10 relative to the vehicle 20 by subtracting the y-axis direction rotation angle θNy of the north direction vector detected by the vehicle 20 from the Y-axis direction rotation angle θNY of the north direction vector detected by the smartphone 10. The rotation angle θy (303) can be calculated.

  Furthermore, the vehicle 20 in this embodiment has an acceleration sensor, and the acceleration of the vehicle 20 can be detected as a vector 304 in FIG. 3b. On the other hand, as described above, the smartphone 10 also detects the acceleration vector like the vector 305. By subtracting the acceleration vector of the vehicle 20 from the acceleration vector of the smartphone 10 and obtaining the rotation angle of the vector with respect to the x-axis, the inclination of the smartphone 10 with respect to the vehicle 20 can be detected as the rotation angle θx (306) in the x-axis direction. it can.

Since the relative positional relationship between the vehicle 20 and the smartphone 10 is obtained by obtaining θx and θy, a foreground-oriented vector 307 and a background-oriented vector 308 in the direction in which the smartphone 10 faces the vehicle 20 are obtained. Can be sought.
(Simple version Ignore car acceleration)

  In the above method, when the vehicle 20 can be regarded as almost stopped, the acceleration vector of the vehicle 20 is not detected, the vector 304 is regarded as zero, and θx can be obtained using only the vector 305. Thereby, the acceleration sensor of the vehicle 20 is not necessary, and a simpler device configuration and method can be taken.

(Use only the simplified version θy)
In addition, in the above method, there is a case where it is only necessary to calculate θy while ignoring the rotation in the x-axis direction shown in FIG. In this case, the foreground vector 307 and the background vector 308 have no y-axis direction component. Although the angle in the vertical direction of the displayed screen is constant, this eliminates the need for the acceleration sensor of the vehicle 20 and the acceleration sensor of the smartphone 10, and allows a simpler device configuration and method.
Next, the image composition method of the image composition means in FIG. 1 will be described. Various known methods can be used for the image composition method.

  As a technique for presenting an image of the periphery of the vehicle to the driver, the image of the rear of the vehicle is synthesized into a single image by three cameras installed at three locations, the left and right rear-view mirrors and the rear center of the vehicle. A technique for displaying the image is known (for example, JP-A-10-257482).

In addition, there is a technique for simulating the situation of observing the rear of the vehicle from one virtual viewpoint using a plurality of cameras as in Japanese Patent Application Laid-Open No. 2007-158427 as much as possible.
Next, the main operation of the vehicle 20 in FIG. 1 will be described using the flowchart of FIG.

When the vehicle 20 starts and finishes the initial operation, the vehicle 20 enters a loop that repeats the flow of FIG. In the first step S400 of the loop, it is determined whether or not there is an instruction to display a video outside the vehicle from the driver. In the case of No, the loop is immediately exited. In the case of Yes, the vehicle 20 is step S4.
In 01, information related to the posture of the smartphone 10 is input from the smartphone 10. The communication method between the smartphone 10 and the vehicle 20 may be via a wired means such as USB or via a wired means such as Wi-Fi. Next, the vehicle 20 inputs the direction of the vehicle 20 in step S402.

  Next, the vehicle 20 determines whether to display the front or the rear. Various methods are conceivable. One enables the driver to select front display or rear display via operation means such as a touch panel. For example, the screen may be switched between forward display and backward display by tapping or double-tapping the screen. For another method, reference is made to the position of the shift lever of the vehicle 20. For example, if it is in the back, it is considered that the driver is going back, so that it is displayed backward, and if it is in any other position, it is displayed forward. For another method, see the tilt of the smartphone 10. That is, the posture of the smartphone 10 input in step S401 includes the inclination of the smartphone 10 as φx, as shown in FIG. If it is directed to an angle looking up above the horizon, it is displayed backward, and if it is negative, that is, the opposite, it is displayed forward. This is because the blind spot of the vehicle is basically below the position of the driver, so that it can be considered that there is no information of the outside image necessary in the direction above the horizon.

  Next, the vehicle 20 determines display parameters in step S404. Next, the vehicle 20 synthesizes an image in step S405. Next, the vehicle 20 outputs an image to the smartphone 10 in step S406, and ends the loop.

(Embodiment 2)
Next, the system configuration of the second embodiment according to the present invention will be described with reference to the block diagram of FIG.

Embodiment 2 assumes a case in which display parameters are determined using only the difference between the initial state of the terminal and the rotation component therefrom. Thereby, the function of the present invention can be realized by a simpler method.
In FIG. 5, the smartphone 10 includes a terminal posture detection unit 1, an operation input unit 2, and a display unit 3.

  The terminal posture detection means 1 is configured by an acceleration sensor, a geomagnetic sensor, and a processing unit, and is configured to detect the posture of the smartphone 10. The detected parameters are the same as those in the embodiment A, and will be omitted.

The operation input means 2 is composed of devices such as a touch pad and various switches and peripheral circuits, and is used when the driver gives an instruction to an application being executed. In almost all smartphones, the display means and the operation means are integrated by a touch panel.
The display means 3 is composed of a liquid crystal display or the like, and presents information to the driver by displaying various information.
The vehicle 20 includes a display parameter determining unit 5, a vehicle outside image photographing unit 6, and an image composition unit 7.

The display parameter determination unit 5 includes a CPU and a memory, and includes a direction of the vehicle 20 detected by the vehicle posture detection unit 4, a posture of the smartphone 10 detected by the terminal posture detection unit 1, and a driver detected by the operation input unit 2. The display parameter of the image outside the vehicle to be displayed on the display means 3 of the smartphone 10 is determined by the operation of. The display parameters to be determined and the detailed determination method will be described later.
The vehicle outside image photographing means 6 is composed of one or a plurality of cameras, and photographs images outside the vehicle mainly around a blind spot for the driver.

  The image synthesizing unit 7 is constituted by a CPU and a memory, inputs video from the outside-vehicle video photographing unit 6, and performs viewpoint conversion of the video based on the parameters determined by the display parameter determining unit 5 to synthesize the video. Since the detailed synthesis method is the same as that in Embodiment 1, the description thereof is omitted.

Next, the display parameters and the determination method of the outside video to be displayed on the display unit 3 of the smartphone 10 which are determined by the display parameter determination unit 5 in FIG. 5 will be described.
The target parameters to be obtained are the rotation angle θy along the y axis and the rotation angle θx along the x axis, as in the first embodiment.

(Full spec version)
According to the rotation angles φY and φX of the smartphone 10, θy and θx are obtained by the following equations. θy is θy = f (φY) = θy0 + kyφY. θx is θx = f (φX) = θx0 + kxφX.

  θy0 and θx0 are rotation angles when φX and φY are zero, respectively, and are referred to as initial rotation angles. Typically, the initial rotation angle is set to zero. θx and θy change linearly by multiplying φX and φY by coefficients ky and kx from the initial rotation angle.

(Determination method of kx, ky)
The coefficients ky and kx are arbitrary constants. In a typical case, the value is 1, but it may be adjusted according to ease of viewing. As an adjustment method, for example, an adjustment screen may be prepared, and the appearance may be changed by increasing or decreasing the value while the driver looks at the screen.

(Use only the simplified version θy)
In addition, in the above method, it may be considered that the rotation in the x-axis direction is ignored and only θy needs to be calculated. In this case, the foreground vector 307 and the background vector 308 in FIG. 3b do not have a component in the y-axis direction. Although the angle in the vertical direction of the displayed screen is constant, this eliminates the need for the acceleration sensor of the vehicle 20 and the acceleration sensor of the smartphone 10, and allows a simpler device configuration and method.
Next, the main operation of the vehicle 20 in FIG. 5 will be described with reference to the flowchart of FIG.

  When the vehicle 20 starts and finishes the initial operation, the vehicle 20 enters a loop that repeats the flow of FIG. In the first step S600 of the loop, it is determined whether or not there is an instruction to display a video outside the vehicle from the driver. In the case of No, the loop is immediately exited. In the case of Yes, the vehicle 20 inputs the information regarding the attitude | position of the smart phone 10 from the smart phone 10 in step S601. The communication method between the smartphone 10 and the vehicle 20 may be via a wired means such as USB or via a wired means such as Wi-Fi. Next, the vehicle 20 inputs the direction of the vehicle 20 in step S602.

Next, the vehicle 20 determines whether to display the front or the rear. Various methods are conceivable. One enables the driver to select front display or rear display via operation means such as a touch panel. For example, the screen may be switched between forward display and backward display by tapping or double-tapping the screen. For another method, reference is made to the position of the shift lever of the vehicle 20. For example, if it is in the back, it is considered that the driver is going back, so that it is displayed backward, and if it is in any other position, it is displayed forward. For another method, see the tilt of the smartphone 10. That is, as shown in FIG. 2, the posture of the smartphone 10 input in step S601 includes the inclination of the smartphone 10 as φx, and this value is positive, that is, the line of sight of the user viewing the screen of the smartphone 10 is If it is directed to an angle looking up above the horizon, it is displayed backward, and if it is negative, that is, the opposite, it is displayed forward. This is because the blind spot of the vehicle is basically below the position of the driver, so that it can be considered that there is no information of the outside image necessary in the direction above the horizon.

  Next, the vehicle 20 determines display parameters in step S604. Next, the vehicle 20 synthesizes an image in step S605. Next, the vehicle 20 outputs an image to the smartphone 10 in step S606, and ends the loop.

  In the first and second embodiments, the communication means between the vehicle 20 and the smartphone 10 may be either wired or wireless. In the case of wireless, since there is nothing that binds the smartphone, the driver can freely rotate the smartphone 10 and view an image in a desired direction. In the case of wired, it is preferable that the smartphone 10 is installed on a terminal installation mechanism called a cradle with a communication connector, and the cradle itself can be rotated up and down or left and right. Thereby, the smart phone 10 can be moved and a video can be seen like a room mirror.

  As described above, according to the present invention, the direction of the visual field can be adjusted very intuitively by changing the posture of the smartphone, and the driver can intuitively understand which image is displayed. This is useful as a video display device outside the vehicle.

DESCRIPTION OF SYMBOLS 1 Terminal attitude | position detection means 2 Operation input means 3 Display means 4 Vehicle attitude | position detection means 5 Display parameter determination means 6 Outside-video imaging | photography means 7 Image composition means 10 Smartphone 20 Vehicle

Claims (7)

A system consisting of a vehicle and a smartphone,
The smartphone
Terminal posture detection means for detecting the posture of the smartphone itself;
Has a display means to display information on the screen,
The vehicle is
Vehicle attitude detection means for detecting the attitude of the vehicle itself;
A vehicle outside video shooting means for shooting a video outside the vehicle;
Display parameter determining means for determining a display parameter of an image outside the vehicle to be displayed on the display means from a change amount of the terminal attitude detected by the terminal attitude detecting means;
An outside-vehicle video display system comprising image composition means for compositing a video photographed by the outside-image video photographing means based on a parameter determined by the display parameter determination means.   2. The vehicle exterior video display system according to claim 1, wherein the display parameter determination unit determines a terminal posture detected by the terminal posture detection unit and a vehicle posture detected by the vehicle posture detection unit.   The terminal attitude detection means and the vehicle attitude detection means detect a rotation angle in the left-right direction or detect rotation angles in the left-right and up-down directions, and the display parameter determination means detects the vehicle when the rotation angle in the left-right direction is detected. When the left and right and up and down rotation angles are detected, the left and right and up and down rotation angles of the terminal with respect to the vehicle are calculated and the display parameters are determined. The outside-vehicle image display system according to claim 1.   The terminal posture detection means detects the rotation angle in the left-right direction or detects the rotation angle in the left-right and up-down directions, and the display parameter determination means detects the rotation angle in the left-right direction when detecting the rotation angle in the left-right direction. Calculate the relative rotation angle in the left and right direction, and when the rotation angle in the left and right and up and down directions is detected, calculate the relative rotation angle in the left and right and up and down directions of the terminal from the initial state, and display parameters based on this The vehicle exterior video display system according to claim 2, wherein:   The vehicle exterior video display system according to any one of claims 1 to 4, wherein the vehicle and the terminal can communicate with each other by a wireless communication unit or a wired communication unit. The smartphone further includes an operation unit that detects a user operation,
The vehicle exterior video display system according to any one of claims 1 to 5, wherein the display parameter determination unit determines a video display method according to an operation detected by the operation unit.   The vehicle exterior image display system according to any one of claims 1 to 6, wherein the vehicle further includes a change amount setting unit that determines a coefficient of a change amount of a display parameter in accordance with a rotation angle of a terminal.

Images