JP5716389B2 - Vehicle display device - Google Patents

Description

  The present invention relates to a vehicular display device that generates and displays an image around a vehicle viewed from a virtual viewpoint above the vehicle by connecting images captured by a plurality of cameras mounted on the vehicle.

  With regard to this type of apparatus, an image generation apparatus that projects an image of a camera onto a bowl-shaped curved surface and generates an image obtained by viewing the curved surface from an arbitrary virtual viewpoint is known (Patent Document 1).

Japanese Patent No. 3286306

  However, there is a problem that it is difficult to grasp the situation around the vehicle in the video generated by the conventional technology.

  The problem to be solved by the present invention is to display an image in which the situation around the vehicle can be easily grasped.

  The present invention generates a deformed coordinate system in which at least a part of the reference curved surface coordinate system is deformed according to the position of the virtual viewpoint with respect to the reference curved surface coordinate system having a predefined curved surface, and performs projection processing on the deformed coordinate system. To solve the above problem.

  According to the present invention, since the video is projected on the deformed coordinate system deformed according to the position of the virtual viewpoint, it is possible to display an image in which the situation around the vehicle can be easily grasped even if the position of the virtual viewpoint is changed. it can.

1 is a configuration diagram of a vehicle display system including an image display device according to an embodiment of the present invention. 2A and 2B are diagrams illustrating an example of the installation position of the vehicle-mounted camera of the present embodiment. It is a figure which shows an example of a reference | standard curved surface coordinate system. It is a flowchart figure which shows the control procedure of the display system for vehicles which concerns on embodiment of this invention. It is a figure which shows an example of the relationship between a reference | standard curved surface coordinate system and a virtual viewpoint. It is a figure for demonstrating the deformation | transformation method of the reference | standard curved surface coordinate system shown in FIG. It is a figure which shows an example of a deformation | transformation coordinate system. 8A schematically shows a cross section of the reference curved surface coordinate system along the line VIII (A) -VIII (A) in FIG. 5, and FIG. 8B shows the reference curved surface coordinates of FIG. 8A. FIG. 8C is a diagram schematically showing a cross section taken along line VIII (C) -VIII (C) of the deformed coordinate system shown in FIG. It is a figure which shows an example of a movement of a deformation | transformation coordinate system. It is a figure which shows the other example of a deformation | transformation coordinate system. It is a figure which shows the condition of a vehicle and a vehicle periphery. It is a figure which shows an example of the image | video projected on the deformation | transformation coordinate system which concerns on this embodiment of this invention. It is a figure which shows the other example of the image | video projected on the deformation | transformation coordinate system which concerns on this embodiment of this invention. It is a figure which shows an example of the image | video concerning a comparison. It is a figure which shows the coordinate system which concerns on a comparison. It is a figure which shows another example of the image | video which concerns on a comparison. It is a figure which shows the other example of the deformation | transformation method of a reference | standard curved surface coordinate system. It is a figure which shows the further another example of the deformation | transformation method of a reference | standard curved surface coordinate system.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a first embodiment of the invention will be described with reference to the drawings. In the present embodiment, a case where the vehicle display device according to the present invention is applied to an in-vehicle display system 1000 will be described as an example. The vehicle display system 1000 displays a monitoring image for grasping the situation of the vehicle and the surroundings of the vehicle on the in-vehicle display. Incidentally, the vehicle display device according to the present invention is a monitoring device that displays a monitoring image of a vehicle on a display of a user's portable terminal, or a monitoring device that displays a monitoring image of each vehicle on a display of a management device of a parking lot. It can also be applied.

  FIG. 1 is a block configuration diagram of a vehicle display system 1000 including an image display device 100 according to the present embodiment. As shown in FIG. 1, a display system 1000 for a vehicle according to this embodiment includes an image display device 100, n vehicle-mounted cameras 1a to 1n fixed outside the host vehicle, an ignition switch 4, and an image display device. An imaging switch 5 that operates 100, a vehicle speed sensor 6, a reverse position switch 7 that outputs a signal when the shift position is reversed, a display changeover switch 8 that switches a display mode of a monitoring image S, which will be described later, and an imaging And a display 3 for displaying images and monitoring images. Each of these devices is connected by a CAN (Controller Area Network) or other in-vehicle LAN, and can exchange information with each other.

  As shown in the figure, the control device 10 of the image display device 100 according to the present embodiment includes a ROM (Read Only Memory) 12 in which a program for displaying a generated monitoring image in a predetermined mode is stored, and the ROM 12 A CPU (Central Processing Unit) 11 as an operation circuit that functions as the image display apparatus 100 by executing a stored program, and a RAM (Random Access Memory) 13 that functions as an accessible storage device are provided. Yes.

  The control device 10 of the image display device 100 according to the present embodiment includes a captured image acquisition function, a viewpoint setting function, a coordinate system deformation function, a projection function, and a display control function. The control apparatus 10 of this embodiment can execute each function by cooperation of the software for implement | achieving the said function, and the hardware mentioned above.

  Below, each function which the control apparatus 10 of the image display apparatus 100 implement | achieves is each demonstrated.

  First, the captured image acquisition function of the control device 10 will be described. The control device 10 of the present embodiment causes the vehicle-mounted cameras 1a to 1n to image the surroundings of the vehicle at a predetermined period in response to a trigger occurrence such as an input operation on the ignition switch 4.

  FIG. 2 is a diagram illustrating an arrangement example when the vehicle-mounted cameras 1 a to 1 n are attached to the vehicle V. In this example, six in-vehicle cameras 1a to 1f are installed at different positions outside the vehicle V, respectively, and take images in six directions around the vehicle. The obtained captured image includes a part of the vehicle and an image around the vehicle. In addition, the arrangement | positioning position of vehicle-mounted camera 1a-1n is not specifically limited, The imaging direction can also be set arbitrarily. Each of the in-vehicle cameras 1a to 1f sends captured images to the image display device 100 at a predetermined cycle.

  Next, the viewpoint setting function of the control device 10 will be described. The control device 10 acquires captured image data transmitted by the in-vehicle cameras 1a to 1n. The control device 10 according to the present embodiment sets a virtual viewpoint of a video (monitoring image) displayed on the display 3. The direction of the virtual viewpoint and the line of sight can be arbitrarily set based on a command from the user input via the display changeover switch 8.

  Moreover, the control apparatus 10 can set a virtual viewpoint according to the driving situation. For example, a virtual viewpoint can be set at the rear of the vehicle to view a video in front of the vehicle while traveling forward, and a virtual viewpoint can be set at the front of the vehicle to view an image behind the vehicle in reverse. Of course, it is also possible to set a virtual viewpoint at the rear of the vehicle in order to view the video behind the vehicle from the rear during reverse. Specifically, the control device 10 can set the position of the virtual viewpoint in response to a signal when the shift position input from the reverse position switch 7 enters reverse. The setting positions of these virtual viewpoints can be set in advance according to the shape of the vehicle. Furthermore, a virtual viewpoint can be set according to the vehicle speed. Specifically, according to the vehicle speed input from the vehicle speed sensor 6, the position of the virtual viewpoint is separated from the vehicle when the vehicle speed is equal to or higher than a predetermined value, and the virtual viewpoint is determined when the vehicle speed is lower than the predetermined value. The position can be made closer to the vehicle.

  Next, the coordinate system deformation function of the control device 10 will be described. The control apparatus 10 according to the present embodiment deforms at least a part of a reference curved surface coordinate system having a predefined curved surface for projecting a captured image in accordance with the position of the virtual viewpoint with respect to the reference curved surface coordinate system.

  The control device 10 of this embodiment defines a reference curved surface coordinate system 131 having a curved surface in advance, and stores this in a storage device such as the RAM 13. FIG. 3 is a diagram illustrating an example of the reference curved surface coordinate system 131. As shown in the figure, the reference curved surface coordinate system 131 of the present embodiment has a bowl shape surrounding the vehicle V. As shown in the figure, the reference curved surface coordinate system 131 of the present embodiment has a center on an xy plane parallel to the mounting surface (traveling surface) of the vehicle V, and has a curvature from the center or near the center. It is formed from a curve or a curved surface having a component in the height direction (z direction) of V.

  A bottom surface parallel to the xy plane can be formed near the center of the reference curved surface coordinate system 131, and the shape thereof can be any shape such as a rectangle, a triangle, an ellipse, or a circle. Further, the curvature of the curved surface of the reference curved surface coordinate system 131 may not be uniform. For example, the curvature of the curved surface near the bottom surface (near the xy plane along the z axis) is relatively large, and the curvature of the curved surface separated from the bottom surface (position separated from the xy plane along the z axis) is relatively May be small (including zero). In addition, the control apparatus 10 can memorize | store in advance the some reference | standard curved surface coordinate system 131 from which a shape differs according to the shape (size, shape) of a vehicle. When a plurality of reference curved surface coordinate systems 131 are stored, a reference curved surface coordinate system 131 used in accordance with vehicle speed and vehicle reverse position on / off is associated with each other, and vehicle vehicle speed and reverse position switch on / off are set. The reference curved surface coordinate system 131 can be selected according to the above.

  As shown in FIG. 3, an image V ′ of the vehicle V prepared in advance may be superimposed on the reference curved surface coordinate system 131 of the present embodiment. The vehicle image V ′ can be created and stored in advance based on the design of the vehicle. In this manner, by superimposing the image V ′ of the vehicle V on the reference curved surface coordinate system 131, it is possible to help understand the relationship between the position and orientation of the vehicle V and surrounding images.

  Then, the control device 10 reads the reference curved surface coordinate system 131 stored in advance, and deforms the shape of the read reference curved surface coordinate system 131. In the present embodiment, at least a part of the reference curved surface coordinate system 131 is deformed. A specific method for deforming the reference curved surface coordinate system 131 will be described later.

  Next, the projection function of the control device 10 will be described. The control device 10 projects the captured image data acquired from the cameras 1a to 1f onto the deformed coordinate system 133 obtained by deforming the reference curved surface coordinate system 131, and sees the vehicle and the surroundings of the vehicle from the set virtual viewpoint. Generate video. The deformed coordinate system 133 corresponds to a so-called world coordinate system (global coordinate system).

  In order to project the captured image data acquired from the cameras 1a to 1f onto the deformed coordinate system after the deformation, the control device 10 coordinates the pixels of the captured image data of the cameras 1a to 1f and the coordinates of the deformed coordinate system 133. Are associated with each other. The mode of the image conversion table 132 is not particularly limited, and the coordinate conversion method known at the time of filing can be used as the image conversion method of the present embodiment.

  Note that the image conversion table 132 may include a plurality of pieces of correspondence information. For example, in order to synthesize each captured image of the cameras 1a to 1f into one image, a first coordinate in which the coordinates of the pixels of the data of the captured images of the cameras 1a to 1f and the coordinates of the pixels of the synthesized image are associated in advance. An image conversion table 132 including correspondence information and second correspondence information that associates the coordinates of the pixels of the composite image and the coordinates of the deformed coordinate system is configured, and the first correspondence is obtained from the captured images of the cameras 1a to 1f. A composite image may be created using the information, and the composite image may be projected onto the deformed coordinate system 133 using the second correspondence information.

  Finally, the display control function of the control device 10 will be described. The control device 10 sends the deformed deformed coordinate system to the display 3 installed inside the vehicle via the in-vehicle LAN, and displays it on the display screen. The control device 10 sends the deformed deformed coordinate system to a portable terminal outside the vehicle or a management device that manages a parked vehicle via a wired or wireless communication line, and displays it on the display screen. You can also.

  Here, based on the flowchart of FIG. 4, operation | movement of the control apparatus 10 of this embodiment is demonstrated. When a specific trigger is generated such as when the ignition switch 4 is turned on and a start command is input to the imaging switch 5, the control device 10 according to the present embodiment starts image display processing.

  As shown in FIG. 4, in step S1, the control device 10 reads a reference curved surface coordinate system 131 prepared in advance. Next, in step S <b> 2, the control device 10 acquires captured images captured at predetermined intervals by the cameras 1 a to 1 f.

  In subsequent step S3, the control device 10 sets a virtual viewpoint of the video to be finally displayed. The setting of the virtual viewpoint is performed based on a command from the display changeover switch 8. The display changeover switch 8 can set a virtual viewpoint of a video to be displayed according to a changeover command input from the user, a vehicle speed acquired from the vehicle speed sensor 6, and an output from the reverse position switch 7.

  Next, in step S4, the control device 10 deforms the reference curved surface coordinate system 131 read in step S1. The control device 10 deforms at least a part of the reference curved surface coordinate system 131 according to the position of the virtual viewpoint with respect to the reference curved surface coordinate system 131 to obtain a deformed coordinate system. This deformation method will be described later.

  Subsequently, in step S5, the control device 10 projects the captured images of the cameras 1a to 1f acquired in step S2 on the deformed coordinate system.

  Finally, the control device 10 causes the display 3 to display an image projected on the deformed coordinate system after the deformation.

  Hereinafter, a deformation method of the reference curved surface coordinate system 131 will be described with reference to FIGS.

  FIG. 5 is a plan view when the read reference curved surface coordinate system 131 is viewed from above the vehicle V (viewed from the plus direction of the z axis in FIG. 3 toward the zero point of the z axis). The reference curved surface coordinate system 131 of the present embodiment includes a substantially rectangular bottom surface B along the planar outer shape of the vehicle V, and a curved surface Q having a component along the z axis with a curvature from the bottom surface. An outer edge Qe is formed at the end of the curved surface Q. Further, the reference curved surface coordinate system 131 has a symmetric shape with respect to the x axis and the y axis in the xy coordinates in plan view.

  FIG. 5 shows a plurality of candidates P1, P2, and P3 (generically also referred to as P) of the virtual viewpoint P. 5, the virtual viewpoints P1 and P3 are located outside the outer edge Qe of the reference curved surface coordinate system 131, and the virtual viewpoint P2 is located inside the outer edge Qe of the reference curved surface coordinate system 131. doing. The distance from the base point o of the reference curved surface coordinate system 131 to the virtual viewpoint P2 is L1, the distance from the base point o of the reference curved surface coordinate system 131 to the virtual viewpoint P3 is L1 + L2, and the base point o of the reference curved surface coordinate system 131 is set. The distance from the virtual viewpoint P1 is L1 + L2 + L3. Note that an image V ′ of the vehicle V is superimposed on FIG. 5 and FIGS.

  FIG. 6 is a diagram illustrating an example of a deformation method of the reference curved surface coordinate system 131. In the present embodiment, as shown in FIG. 5, when the virtual viewpoint P1 is located outside the outer edge Qe of the reference curved surface coordinate system 131, the virtual viewpoint P1 is displayed on the reference curved surface coordinate system 131 as shown in FIG. A part of the reference curved surface coordinate system 131 is deformed so as to be located inside the outer edge Qe.

  To explain from a different point of view, when the distance L1 + L2 + L3 between the virtual viewpoint P and the center o of the reference curved surface coordinate system 131 is not less than a preset first predetermined value, that is, a position separated from the center o of the reference curved surface coordinate system 131 When the virtual viewpoint P is set to the reference curved surface whose distance from the virtual viewpoint P is less than the second predetermined value so that the virtual viewpoint P1 is located inside the outer edge Qe of the reference curved surface coordinate system 131. The reference curved surface coordinate system 131 is deformed so that a part of the outer edge Qe of the coordinate system 131, that is, the portion of the outer edge Qe located near the virtual viewpoint P is separated from the center o of the reference curved surface coordinate system 131. As a result, the entire reference curved surface coordinate system 131 is not enlarged or reduced uniformly, but only the portion close to the virtual viewpoint P1 can be partially enlarged. Incidentally, the reference curved surface coordinate system 131 shown in FIG. 5 is symmetric with respect to the xy axis, but the deformed coordinate system 133 shown in FIG. 6 has a non-target shape with respect to the x axis and / or the y axis. The first predetermined value and the second predetermined value may be the same value or different values.

  The deformation amount of the reference curved surface coordinate system 131 can be defined in advance according to the positional relationship between the virtual viewpoint P1 and the reference curved surface coordinate system 131. Specifically, the virtual viewpoint P1 is referred to by referring to the deformation amount data 134 in which the distance between the virtual viewpoint P1 and the center o of the reference curved surface coordinate system 131 and the deformation amount of a part of the outer edge Qe of the reference curved surface coordinate system 131 are associated in advance. The reference curved surface coordinate system 131 can be deformed according to the deformation amount determined according to the center distance of the reference curved surface coordinate system.

  FIG. 7 is a diagram illustrating an example of the deformed coordinate system R133. As shown in FIG. 7, the virtual viewpoint P1 located at the outer edge Qe of the reference curved surface coordinate system 131 is inside the outer edge Re of the curved surface R of the deformed coordinate system 133, that is, at a position close to the coordinate base point o. is there.

  In the example shown in FIG. 7, a part of the outer edge Qe of the reference curved surface coordinate system 131 is deformed. Although not particularly limited, in this example, the outer edge Qe whose distance from the virtual viewpoint P1 is equal to or smaller than the second predetermined value is deformed from the viewpoint of selectively deforming the area close to the virtual viewpoint P1, but this deformation area is set. The method is not limited. For example, as shown in FIG. 7, the point P1t on the outer edge Qe of the reference curved surface coordinate system 131 is at a distance t from the virtual viewpoint P1, and the outer edge Qe of the reference curved surface coordinate system 131 is also at a distance s from the virtual viewpoint P1. The portion between the upper point P1s can be deformed. That is, the second predetermined value set to define the region close to the virtual viewpoint O1 can be a different value depending on the direction. In this example, the second predetermined value in the longitudinal direction of the vehicle is the distance t, and the second predetermined value in the vehicle width direction is the distance s. The distance t and the distance s can be set arbitrarily, and the distance t and the distance s may be the same value or different values. That is, the region to be deformed is equidistant from the virtual viewpoint P1. The method for setting the region to be deformed is not limited to this, and other examples will be described later.

  8A schematically shows a cross section of the reference curved surface coordinate system 131 along the line VIII (A) -VIII (A) in FIG. 5, and FIG. 8B shows the reference curved surface in FIG. 8A. FIG. 8C is a diagram schematically showing a deformation process of the coordinate system 131, and FIG. 8C is a diagram schematically showing a cross section along the line VIII (C) -VIII (C) of the modified coordinate system 133 shown in FIG.

  As shown in FIG. 8A, the virtual viewpoint P1 is located outside the reference curved surface coordinate system 131. When the vehicle V is viewed from the virtual viewpoint P1 in this state, the outer surface Q1 side of the curved surface Q of the reference curved surface coordinate system 131 is seen. In the present embodiment, a portion of the curved surface Q of the reference curved surface coordinate system 131 where the outer surface Q1 is visible from the virtual viewpoint P1 is deformed. As shown in FIG. 8B, the outer edge Qe of the curved surface Q of the reference curved surface coordinate system 131 is stretched in the direction of the virtual viewpoint P1 by the deformation of the reference curved surface coordinate system 131. Further, due to this deformation, the outer edge Qe of the curved surface Q of the reference curved surface coordinate system 131 is moved to a lower position along the z-axis direction. Thus, by deforming, when the vehicle V is viewed from the virtual viewpoint P1, the inner surface R2 side of the curved surface Q of the reference curved surface coordinate system 131 (deformed coordinate system 133) is seen. That is, as shown in FIG. 8C, the virtual viewpoint P <b> 1 exists inside the curved surface R of the deformed coordinate system 133 with respect to the deformed coordinate system 133. That is, the outer edge Re of the deformed coordinate system 133 is farther from the vehicle V than the virtual viewpoint P1, and the distance from the center o to the outer edge Re is more than the distance LP1 from the center o of the deformed coordinate system 133 to the virtual viewpoint P1. LRe is larger.

  In addition, the control device 10 determines the region to be deformed as a straight line passing through the virtual viewpoint P1 and the point q (not shown) among arbitrary points q on the curved surface Q of the reference curved surface coordinate system 131 and a curved surface passing through the point q. The region including the point q where the angle formed by the tangent plane of Q is less than a predetermined threshold can be deformed. The modified coordinate system 133 after the change is formed from the tangent lines of the reference curved surface coordinate system 131 before the change that passes through the point (the point P1x in FIG. 8B) that moves the virtual viewpoint P1 in the direction opposite to the line-of-sight direction. The A curvature can be given to the tangent, and the deformed portion may be a flat surface or a curved surface. In the deformed deformed coordinate system 133, the angle formed between the tangent plane of the curved surface R passing through the arbitrary point r on the curved surface R and the straight line passing through the point r from the virtual viewpoint P is less than a predetermined angle. It can be.

  Furthermore, the control device 100 refers to the deformation amount data 134 in which the distance between the virtual viewpoint P and the center o of the reference curved surface coordinate system 131 and the deformation amount of a part of the outer edge Qe of the reference curved surface coordinate system 131 are associated in advance. A part of the outer edge Qe of the reference curved surface coordinate system 131 can be deformed by a predetermined amount according to the distance between the virtual viewpoint P and the center o of the reference curved surface coordinate system 131.

  8B and 8C, the control device 100 determines the distance LP1 from the center o of the reference curved surface coordinate system 131 to the virtual viewpoint P1, and the deformation amount (LRe-LQe) of the extension Qe. ) With reference to the deformation amount data 134 in advance, the deformation amount is calculated based on the distance LP1 from the virtual viewpoint P1 and the center o of the reference curved surface coordinate system 131 to the virtual viewpoint P1, and according to the deformation amount. The outer edge Qe of the reference curved surface coordinate system 131 can be deformed. The deformation amount data 134 is not limited as long as the coordinates of the outer edge Qe of the reference curved surface coordinate system 131 to be deformed can be associated with the coordinates of the outer edge Re of the deformed coordinate system 133 after deformation. For example, the deformation amount of the extension Qe in the spatial coordinates defined by the xyz axis after deformation may be configured to correspond to the distance LP1 from the center o of the virtual viewpoint P1.

Further, as shown in FIG. 9, the control device 10 can rotate the virtual viewpoint P and the deformed coordinate system 133 with the center o of the deformed coordinate system 133 as the rotation center. As shown in the figure, when the virtual viewpoint P and the deformed coordinate system 133 are rotated by a predetermined angle α along the trajectory indicated by the broken line with the center o as the rotation center, the virtual viewpoint P moves to the position of P1 ′, The deformed coordinate system 133 is also moved to the position of the deformed coordinate system 133 ′, and the deformed coordinate system 133 ′ shown in FIG. 10 can be obtained. Thus, by rotating the virtual viewpoint P1 and the reference curved surface coordinate system 131 , the direction in which the vehicle V is viewed in the projected image on the deformed coordinate system 133 can be arbitrarily changed.

  Note that the control device 10 can also change the height (position in the z direction) of the virtual viewpoint P when the deformed coordinate system 133 is generated.

  Next, specific examples of images displayed on the display 3 in the vehicle display system 1000 according to the present embodiment will be described with reference to FIGS.

  FIG. 11 is a real image of the vehicle V and the obstacles W1, W2 viewed from a predetermined virtual viewpoint. FIG. 12 is an image obtained by projecting captured image data of the vehicle V and surrounding obstacles W1 and W2 in this state on the deformed coordinate system 133 of FIG. 7 with reference to the virtual viewpoint P1 shown in FIG. Similarly, FIG. 13 is an image obtained by projecting captured image data on the deformed coordinate system 133 ′ of FIG. 10 based on the virtual viewpoint P1 ′ shown in FIG.

  As shown in FIG.12 and FIG.13, the image | video which the display system 1000 for vehicles of this embodiment displays has a small distortion amount of an image. As a result, it is possible to display an image in which the user can easily understand the situation around the vehicle.

By the way, when the virtual viewpoint P is set outside the reference curved surface coordinate system 131 and the height of the outer edge Qe of the reference curved surface coordinate system 131 is higher than the virtual line of sight (the line of sight of viewing the vehicle V from the virtual viewpoint P), As shown in FIG. 8A, the reference curved surface coordinate system 131 is viewed from the outer surface Q1.

  FIG. 14 shows an image projected on the reference curved surface coordinate system 131 in such a state. As shown in FIG. 14, since the shapes of the obstacles W1x and W2x near the vehicle V are distorted, it is difficult for the user to grasp the situation around the vehicle. In addition, although the display surface of the display 3 is rectangular, the state of the vehicle V ′ and the surroundings of the vehicle cannot be expressed as a rectangular image. For this reason, as shown in FIG. 14, a region where no video is displayed (shaded line Sh) occurs, and a part of the display screen of the display 3 cannot be used.

  In addition, as shown in FIG. 15, when a circular or elliptical (undeformed) coordinate system having a large radius is used for projection so that the virtual viewpoint P1 is located inside the outer edge Qe ′ of the coordinate system. The video is as shown in FIG. In the image shown in FIG. 16, the obstacles W1y and W2y having a height are actually expressed as if they are falling on the ground. For this reason, since the obstacles W1y and W2y look like objects with no height (for example, signs on the road), it is difficult for the user to grasp the situation around the vehicle.

  On the other hand, according to the image display device 100 of the present embodiment described above, as shown in FIGS. 12 and 13, the shape of the obstacles W1 and W2 having height is small, and the image is missing (FIG. 14). In this case, a rectangular image suitable for the display screen of the display 3 can be provided. As a result, it is possible to display an image in which the user can easily understand the situation around the vehicle.

Hereinafter, another method for obtaining the deformed coordinate system 133 from the reference coordinate system 131 will be described with reference to FIGS. 17 and 18.
When the virtual viewpoint P is located inside the outer edge Qe of the reference curved surface coordinate system 131, that is, when the distance from the center o of the reference curved surface coordinate system 131 to the virtual viewpoint P is less than a predetermined value. The reference curved surface coordinate system 131 is deformed so that the outer edge Qe of the reference curved surface coordinate system 131 approaches the center o of the reference curved surface coordinate system 131. That is, the area of the reference curved surface coordinate system 131 when viewed from above is reduced. In this process, the control device 10 deforms the outer edge Qe of the reference curved surface coordinate system 131 according to the position of the virtual viewpoint P1 with respect to the reference curved surface coordinate system 131. The deformation amount of the outer edge Qe can be determined according to the relationship (for example, distance) between the position of the virtual viewpoint P1 and the position of the outer edge Qe. In the example shown in FIG. 17, the deformation amount of the outer edge Qe having a small distance from the virtual viewpoint P1 is relatively small, and the deformation amount of the outer edge Qe having a large distance from the virtual viewpoint P1 is relatively large. The amount of deformation corresponding to the distance from the virtual viewpoint P1 can be defined in advance. Thereby, the shape of the deformation coordinate 133 can be changed according to the position of the virtual viewpoint P with respect to the vehicle V (reference coordinate system 131).

  Note that the control device 10 sets the entire outer edge Qe of the reference curved surface coordinate system 131 by a predetermined distance to the center o of the reference curved surface coordinate system 131 so that the virtual viewpoint P1 is positioned outside the outer edge Qe of the reference curved surface coordinate system 131. The reference curved surface coordinate system 131 may be partially deformed so that the virtual viewpoint P1 is positioned closer to the inner edge than the outer edge Qe of the reference curved surface coordinate system 131.

  Further, when the control apparatus 10 of the present embodiment deforms the reference curved surface coordinate system 131 to obtain the deformed coordinate system 133, the control device 10 changes the position of the virtual viewpoint P1 and deforms the position of the outer edge Re of the shape coordinate system 133. can do. For example, as shown in FIG. 18, the virtual viewpoint P1 is changed to a point P2 close to the center o, and the outer edge Re1 of the deformed coordinate system 133 is changed to the outer edge Re2 close to the center o. In this deformation process, the deformation amount data in which the distance between the virtual viewpoint P1 and the center o and the deformation amount of the outer edge Re of the deformation coordinate system 133 are associated in advance is referred to, and the virtual viewpoint P1 and the center o of the deformation coordinate system 133 are A part of the outer edge Re1 of the deformed coordinate system 133 is deformed by a predetermined amount according to the distance. Thereby, the shape of the deformation coordinate 133 can be changed according to the position of the virtual viewpoint P with respect to the vehicle V (reference coordinate system 131).

  Since this invention is comprised as mentioned above and acts as mentioned above, there exist the following effects.

  The image display device 100 according to the present embodiment of the present invention projects the captured images of the cameras 1a to 1f onto a deformed coordinate system 133 in which a part of the reference curved surface coordinate system 131 is deformed according to the position of the virtual viewpoint P. Therefore, even if the position of the virtual viewpoint P changes, it is possible to provide an image in which the surrounding situation can be easily grasped.

  In the image display device 100 according to the present embodiment of the present invention, when the virtual viewpoint P is located outside the outer edge Qe of the reference curved surface coordinate system 131, the virtual viewpoint P is inside the outer edge Qe of the reference curved surface coordinate system 131. Since the deformed coordinates 133 are obtained by deforming a part of the reference curved surface coordinate system Qe so as to be located in the image, the image presented to the user is not included in the image of the outer surface of the projected coordinates, and the image is missing ( A rectangular image suitable for the display screen of the display 3 can be provided without the portion corresponding to the hatched line Sh in FIG. In addition, since a part of the reference curved surface coordinate system Qe is deformed so that the virtual viewpoint P is located on the inner side of the outer edge Qe of the reference curved surface coordinate system 131, there is little distortion of the shape of the obstacle having a height. Furthermore, by performing deformation processing according to the positional relationship between the virtual viewpoint P and the reference curved surface coordinate system 131, appropriate deformation coordinates can be obtained according to the situation of the vehicle being driven, and an image corresponding to the situation can be obtained. Can be displayed. As a result, it is possible to display an image in which the user can easily understand the situation around the vehicle.

  Similarly, when the distance between the virtual viewpoint P and the center o of the reference curved surface coordinate system 131 is equal to or greater than the first predetermined value, the virtual viewpoint P is positioned inside the outer edge Qe of the reference curved surface coordinate system 131. The reference curved surface coordinate system 131 is modified so that a part of the outer edge Qe of the reference curved surface coordinate system 131 whose distance from the virtual viewpoint P is less than the second predetermined value is separated from the center o of the reference curved surface coordinate system 131. Thus, since the deformed coordinates 133 are obtained, it is possible to provide a rectangular image with no missing image and to reduce the distortion of the image of a three-dimensional object existing around the vehicle.

  Further, in the image display device 100 of the present embodiment, when the virtual viewpoint P is located inside the outer edge Qe of the reference curved surface coordinate system 131, the outer edge Qe of the reference curved surface coordinate system 131 is the center o of the reference curved surface coordinate system 131. Therefore, when the reference curved surface coordinate system 131 is too large with respect to the position of the virtual viewpoint P, the reference curved surface coordinate system 131 can be deformed small. At this time, a portion that does not adversely affect the visibility, for example, a region separated from the virtual viewpoint P (a region having a distance greater than or equal to a predetermined value from the virtual viewpoint P) is reduced to be close to the center o, thereby affecting the visibility. For example, an area close to the virtual viewpoint P (an area having a distance less than a predetermined value from the virtual viewpoint P) can ensure the distance from the center o and widen the display area to ensure visibility. Thereby, it is possible to display an image in which the user can easily understand the situation around the vehicle.

  In addition, the image display apparatus 100 according to the present embodiment has deformation amount data in which a distance between the virtual viewpoint P and the center o of the reference curved surface coordinate system 131 and a partial deformation amount of the outer edge Qe of the reference curved surface coordinate system 131 are associated in advance. 134, a part of the outer edge Qe of the reference curved surface coordinate system 131 is deformed by a predetermined amount according to the distance between the virtual viewpoint P and the center o of the reference curved surface coordinate system 131, so the distance between the virtual viewpoint P and the vehicle V Accordingly, it is possible to quickly obtain the deformed coordinate system 133 that has been subjected to quantitative deformation.

Further, the image display apparatus 100 according to the present embodiment changes according to the situation of the vehicle being driven by moving the virtual viewpoint P and the reference curved surface coordinate system 131 with the center o of the deformed coordinate system 131 as the rotation center. According to the position of the virtual viewpoint P, the deformed coordinates 131 for viewing the vehicle V from an appropriate direction can be obtained, and an image corresponding to the situation can be displayed.

  In addition, the image display apparatus 100 according to the present embodiment refers to the deformation amount data in which the distance between the virtual viewpoint P and the center o of the deformation coordinate system 131 and the deformation amount of the outer edge Re of the deformation coordinate system 133 are correlated in advance. Since a part of the outer edge Re of the deformed coordinate system 133 is deformed by a predetermined amount according to the distance between the viewpoint P and the center o of the deformed coordinate system 133, the position of the virtual viewpoint P that changes according to the situation of the vehicle being driven is set. Accordingly, the deformed coordinates 131 viewed from the virtual viewpoint P provided at an appropriate distance can be obtained, and an image corresponding to the situation can be displayed.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  That is, in the present specification, the image display device 100 will be described as an example of the vehicle display device according to the present invention, but the present invention is not limited to this.

  Further, in this specification, as an example of the vehicle display device according to the present invention, the image display device 100 including the control device 10 including the CPU 11, the ROM 12, and the RAM 13 will be described as an example. However, the present invention is not limited to this. Absent.

  Further, in the present specification, as one aspect of a vehicle display device having captured image acquisition means, viewpoint setting means, coordinate system deformation means, projection means, and display control means according to the present invention, captured image acquisition is performed. The image display device 100 including the control device 10 that executes the function, the viewpoint setting function, the coordinate system deformation function, the projection function, and the display control function will be described as an example. However, the present invention is not limited to this. It is not something.

1000 ... display systems for vehicles 1a to 1n ... vehicle-mounted camera 100 ... image display device V ... vehicle 10 ... control device 11 ... CPU
12 ... ROM
13 ... RAM
131 ... Reference curved surface coordinate system 132 ... Image conversion table 133 ... Deformed coordinate system 134 ... Deformation amount data 3 ... Display 4 ... Ignition switch 5 ... Imaging switch 6 ... Vehicle speed sensor 7 ... Reverse position switch 8 ... Display changeover switch

Claims (7)

Captured image acquisition means for acquiring data of a captured image captured by a camera mounted on the vehicle;
Viewpoint setting means for setting a virtual viewpoint of the video to be displayed;
Reference curved surface coordinates having a bottom surface along the planar outer shape of the vehicle and a curved surface having a curvature from the bottom surface and having a component in the height direction of the vehicle, which are defined in advance for projecting the captured image. Coordinate system deformation means for deforming a part of the system according to the position of the virtual viewpoint with respect to the reference curved surface coordinate system to obtain a deformed coordinate system;
Projecting means for projecting the captured image data onto the deformed deformed coordinate system and generating an image of the vehicle and the surroundings of the vehicle from the set virtual viewpoint;
Display control means for displaying the generated video on a display screen;
When the virtual viewpoint is located outside the outer edge of the reference curved surface coordinate system, the coordinate system deforming unit is configured so that the virtual viewpoint is located inside the outer edge of the reference curved surface coordinate system. A vehicle display device that deforms a portion of a coordinate system whose distance from the virtual viewpoint is less than a predetermined value . The vehicle display device according to claim 1,
When the distance between the virtual viewpoint and the center of the reference curved surface coordinate system is equal to or greater than a first predetermined value, the coordinate system deforming means is positioned inside the outer edge of the reference curved surface coordinate system. As described above, the reference curved surface coordinate system is deformed so that a part of the outer edge of the reference curved surface coordinate system whose distance from the virtual viewpoint is less than a second predetermined value is separated from the center of the reference curved surface coordinate system. A display device for a vehicle. The vehicle display device according to claim 1 or 2 ,
When the virtual viewpoint is located inside the outer edge of the reference curved surface coordinate system, the coordinate system deforming unit is configured so that the outer edge of the reference curved surface coordinate system approaches the center of the reference curved surface coordinate system. A display device for a vehicle, wherein a part of a coordinate system is deformed. In the display apparatus for vehicles as described in any one of Claims 1-3 ,
The coordinate system deforming means refers to deformation amount data in which the virtual viewpoint, the center distance of the reference curved surface coordinate system, and a deformation amount of a part of the outer edge of the reference curved surface coordinate system are associated in advance, and the virtual viewpoint And a part of the outer edge of the reference curved surface coordinate system is deformed by a predetermined amount in accordance with the distance between the centers of the reference curved surface coordinate system and the reference curved surface coordinate system. In the display apparatus for vehicles as described in any one of Claims 1-4 ,
The vehicle display device, wherein the coordinate system deforming means moves the virtual viewpoint and the reference curved surface coordinate system with the center of the deformed coordinate system as a rotation center. In the vehicle display device according to any one of claims 1 to 5 ,
The coordinate system deforming means refers to deformation amount data in which a distance between the virtual viewpoint and the center of the deformed coordinate system and a deformation amount of an outer edge of the deformed coordinate system are associated in advance, and the virtual viewpoint and the deformed coordinate system A vehicle display device characterized in that a part of the outer edge of the deformed coordinate system is deformed by a predetermined amount according to the distance of the center of the vehicle. Captured image acquisition means for acquiring data of a captured image captured by a camera mounted on the vehicle;
Viewpoint setting means for setting a virtual viewpoint of the video to be displayed;
Reference curved surface coordinates having a bottom surface along the planar outer shape of the vehicle and a curved surface having a curvature from the bottom surface and having a component in the height direction of the vehicle, which are defined in advance for projecting the captured image. Coordinate system deformation means for deforming a part of the system according to the position of the virtual viewpoint with respect to the reference curved surface coordinate system to obtain a deformed coordinate system;
Projecting means for projecting the captured image data onto the deformed deformed coordinate system and generating an image of the vehicle and the surroundings of the vehicle from the set virtual viewpoint;
Display control means for displaying the generated video on a display screen;
When the virtual viewpoint is located inside the outer edge of the reference curved surface coordinate system, the coordinate system deforming unit is configured so that the outer edge of the reference curved surface coordinate system approaches the center of the reference curved surface coordinate system. A display device for a vehicle, wherein a part of a coordinate system is deformed .

Images