JP2016042704A - Image display system, image processing device, and image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate judgment of a distance between a vehicle body and an obstacle when confirming the clearance between the vehicle body and the obstacle on the basis of images captured by vehicle-mounted cameras.SOLUTION: An image display system 120 comprises: composite image generation means 32 which generates a composite image viewed from a virtual view point on the basis of the images captured with cameras 51 to 53 arranged on a vehicle 9; display means 20 which displays the composite image on a display screen; acceptance means 21 which accepts first designation input designating a partial area of an overhead image surrounding the vehicle 9 generated by the composite image generation means 32 and displayed on the display screen; and display control means 12 which displays an expanded image of the designated area on the display screen as full-screen display when the first designation input is accepted.SELECTED DRAWING: Figure 1

Description

  Embodiments discussed herein relate to techniques for displaying images in a vehicle.

  2. Description of the Related Art Conventionally, there is known an image display system that is mounted on a vehicle such as an automobile and displays an image obtained by photographing the periphery of the vehicle with an in-vehicle camera on a display in the vehicle interior. By using this image display system, a user (typically a driver) can grasp the situation around the vehicle almost in real time.

  For example, the outer region of the front fender that is on the opposite side of the driver's seat is likely to be a blind spot from the driver's seat, and it is difficult for the user to grasp the clearance between the vehicle body and the obstacle. On the other hand, if the image display system is used, an image showing the outer region of the front fender is acquired by photographing with the in-vehicle camera, and the image is displayed on the display in the vehicle interior. As a result, the user can easily confirm the clearance between the vehicle body on the opposite side of the driver's seat and the obstacle, for example, when the vehicle is brought close.

  An overhead video display means for displaying an overhead video around the vehicle on the display based on each video taken by a plurality of cameras, and the surrounding of the vehicle in the overhead video is divided into a plurality of areas and each is defined as an imaging area An imaging area defining means for performing detection, a selection area setting means for detecting that one of the imaging areas is selected by the user on the display, and setting the detected imaging area as a selection area, and a target video corresponding to the selection area There has been proposed a vehicle periphery display device provided with an enlarged display means for enlarging and displaying the image on a display.

  Further, in the vehicle periphery display device, it has been proposed to enlarge and display the selected images of the front camera and the rear camera on the entire display screen of the monitor.

JP 2009-239664 A JP 2010-147516 A

  When the clearance between the vehicle body and the obstacle is grasped, the location to be watched by the captured image of the in-vehicle camera differs depending on the user. Therefore, the user's convenience is higher in the bird's-eye view image synthesized from the images of the plurality of in-vehicle cameras and the enlarged image than in the image taken from the fixed viewpoint.

  However, the conventional image display system displays an overhead image of the entire periphery of the vehicle and an enlarged image thereof in a small display area on a part of the display screen of the display device. For this reason, it has been difficult for the user to grasp an accurate sense of distance between the vehicle and the obstacle.

  The apparatus and method according to the embodiment provides a sense of distance between the obstacle and the obstacle when the clearance between the vehicle body and the obstacle is confirmed based on an image obtained by photographing the periphery of the vehicle with the in-vehicle camera. The purpose is to facilitate judgment.

  According to one embodiment, an image display system mounted on a vehicle is provided. The image display system generates a composite image generated from a virtual viewpoint based on a captured image obtained by a camera disposed on a vehicle, and displays the composite image generated by the composite image generation unit on a display screen. Receiving means for accepting the first designation input for designating a partial area of the bird's-eye view image around the vehicle generated by the composite image generation means and displayed on the display screen, and accepting the first designation input In such a case, a display control means for displaying an enlarged image of the designated area on the display screen in a full screen is provided.

  An image display system according to another embodiment includes a composite image generation unit that generates a composite image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in a vehicle, and a composite image generated by the composite image generation unit And a display means for displaying the photographed image on a display screen; a receiving means for receiving a designation input for designating a partial area of the overhead image around the vehicle generated by the composite image generation means and displayed on the display screen; When a designation input is received, display control means is provided for displaying an enlarged image of the designated area on the display screen. The one or more display modes of the display screen controlled by the display control means include a state in which a photographed image obtained by the camera is displayed on the display screen. When a designation input is received during display of the photographed image, the display control means Displays a first animation that is generated by the composite image generation means and changes from the captured image to the overhead image and a second animation that changes from the overhead image to the enlarged image.

  According to the device or method of the present disclosure, it is possible to display a bird's-eye view image that enlarges an arbitrary portion around the vehicle on a full screen. For this reason, when the user confirms the clearance between the vehicle body and the obstacle, it is easy to grasp the sense of distance between the obstacle.

  When the display on the display is directly switched to a part of the enlarged image of the overhead view image, the user loses sight of the enlarged display and feels anxious about whether or not the operation is accepted as intended.

  If animations that change from a camera image to a bird's-eye view image and from a bird's-eye view image to an enlarged image are displayed, the user can easily confirm that the display is performed as intended.

It is a block diagram which shows the structural example of an image display system. It is a figure which shows the example of the position where a vehicle-mounted camera is arrange | positioned at a vehicle. It is explanatory drawing of an example of the method of producing | generating a synthesized image. (A)-(C) are explanatory drawings of the example of the enlarged image of the bird's-eye view image produced | generated by the synthesized image production | generation part. It is explanatory drawing of the 1st example of the state transition diagram of a display mode. It is a figure which shows the example of the display content in navigation mode. It is a figure which shows the example of the display content in the 2 screen display state for back modes. (A) And (B) is a figure which shows the example of the display content in a 1st back camera image display state and a 2nd back camera image display state. It is a figure which shows the example of the display content in the 2 screen display state for front modes. (A) And (B) is a figure which shows the example of the display content in a front camera image display state and a side camera image display state. It is a figure which shows the example of the display content in surrounding confirmation mode. It is a flowchart of the 1st example of the display switching process in a 2 screen display state. It is explanatory drawing of the 1st example of the display switching process in a 2 screen display state. It is FIG. (1) which shows the display state of an enlarged bird's-eye view image. (A)-(C) is explanatory drawing of the example of a change of a display of 2nd area | region R2. (A) is a flowchart of an animation display process, and (B) is a flowchart of an intermediate image display process in animation display. It is a flowchart of the 2nd example of the display switching process in a 2 screen display state. (A) And (B) is explanatory drawing of the 2nd example of the display switching process in a 2 screen display state. (A)-(C) is explanatory drawing of the 1st example of the method of changing the position of the viewpoint of an enlarged bird's-eye view image. (A)-(C) are explanatory drawings of the 2nd example of the method of changing the position of the viewpoint of an enlarged bird's-eye view image. It is explanatory drawing of the 2nd example of the state transition diagram of a display mode. It is a figure which shows the example of the display content in a full-screen bird's-eye view display state. It is a flowchart of the display switching process in a full screen overhead view display state. (A) And (B) is explanatory drawing of the display switching process in a full-screen bird's-eye view display state.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<1. System configuration>
FIG. 1 is a block diagram illustrating a configuration example of the image display system 120. This image display system 120 is mounted on a vehicle (in this embodiment, an automobile), and has a function of photographing the periphery of the vehicle, generating an image, and displaying the image in the passenger compartment. By using this image display system 120, a user (typically a driver) of the image display system 120 can grasp the state around the vehicle in almost real time. The term “image” used in the following description includes both the meanings of still images and moving images.

  As shown in FIG. 1, the image display system 120 mainly includes an image processing device 100 that generates an image showing a state around a vehicle, and a navigation device 20 that displays various types of information to a user who gets on the vehicle. I have. An image generated by the image processing apparatus 100 is displayed on the navigation apparatus 20.

  The navigation device 20 provides navigation guidance to the user, and controls the display 21 such as a liquid crystal provided with a touch panel function, an operation unit 22 including a hard switch that is operated by the user, and the entire device. And a control unit 23. The navigation device 20 is installed on an instrument panel or the like of the vehicle so that the screen of the display 21 is visible from the user. Various user instructions are received by the operation unit 22 and the display 21 as a touch panel. The control unit 23 is configured by a computer including a CPU, a RAM, a ROM, and the like, and various functions including a navigation function are realized by the CPU performing arithmetic processing according to a predetermined program.

  The navigation apparatus 20 is communicably connected to the image processing apparatus 100, and can transmit and receive various control signals to and from the image processing apparatus 100 and receive images generated by the image processing apparatus 100. An image based on the function of the navigation device 20 alone is normally displayed on the display 21 under the control of the control unit 23, but the state around the vehicle generated by the image processing device 100 by changing the operation mode is displayed. The image shown is displayed. Thereby, the navigation device 20 also functions as a display device that receives and displays the image generated by the image processing device 100.

  The image processing apparatus 100 is configured as an ECU (Electronic Control Unit) whose main body 10 has a function of generating an image, and is arranged at a predetermined position of the vehicle. The image processing apparatus 100 includes a photographing unit 5 that photographs the periphery of the vehicle, and an image that generates a composite image viewed from a virtual viewpoint based on a photographed image obtained by photographing the periphery of the vehicle with the photographing unit 5. Functions as a generator.

  The photographing unit 5 is electrically connected to the main body unit 10 and operates based on a signal from the main body unit 10. The photographing unit 5 includes a front camera 51, a back camera 52, and a side camera 53 that are in-vehicle cameras. Each of the in-vehicle cameras 51 to 53 includes a lens and an image sensor and electronically acquires an image.

  The plurality of in-vehicle cameras 51 to 53 are respectively arranged at different positions of the vehicle. FIG. 2 is a diagram illustrating an example of positions where the in-vehicle cameras 51 to 53 are disposed on the vehicle 9.

  As shown in FIG. 2, the front camera 51 is provided in the vicinity of the license plate mounting position at the front end of the vehicle 9, and its optical axis 51 a is directed in the straight traveling direction of the vehicle 9. The back camera 52 is provided in the vicinity of the license plate mounting position at the rear end of the vehicle 9, and its optical axis 52 a is directed in the direction opposite to the straight traveling direction of the vehicle 9. The mounting positions of the front camera 51 and the back camera 52 are preferably substantially at the center of the left and right, but may be slightly shifted from the center of the left and right in the left and right direction. The side cameras 53 are provided on the left and right door mirrors 93, respectively, and the optical axis 53a thereof is directed to the outside of the vehicle 9 along the left-right direction of the vehicle 9 (direction orthogonal to the straight traveling direction).

  A fish-eye lens or the like is adopted as the lens of these in-vehicle cameras 51 to 53, and the in-vehicle cameras 51 to 53 have an angle of view θ of 180 degrees or more. For this reason, the surroundings of the vehicle 9 can be image | photographed by utilizing the four vehicle-mounted cameras 51-53.

  Returning to FIG. 1, the main body unit 10 of the image processing apparatus 100 includes a control unit 1 that controls the entire apparatus, an image generation unit 3 that processes a captured image acquired by the imaging unit 5 and generates an image for display. The navigation communication unit 42 that communicates with the navigation device 20 is mainly provided.

  Various instructions from the user received by the operation unit 22 or the display 21 of the navigation device 20 are received by the navigation communication unit 42 as control signals and input to the control unit 1. As a result, the image processing apparatus 100 can operate in response to a user operation on the navigation apparatus 20.

  The image generation unit 3 is configured as a hardware circuit capable of various image processing, and includes a captured image adjustment unit 31 and a composite image generation unit 32 as main functions.

  The captured image adjustment unit 31 performs adjustment for the captured image acquired by the imaging unit 5. Specifically, the captured image adjustment unit 31 performs image processing such as brightness correction on the captured image.

  The composite image generation unit 32 is a vehicle viewed from an arbitrary virtual viewpoint around the vehicle based on the plurality of captured images acquired by the plurality of on-board cameras 51 to 53 of the imaging unit 5 and adjusted by the captured image adjustment unit 31. A composite image showing the state of the surrounding area is generated. A method by which the composite image generation unit 32 generates a composite image will be described later.

  The composite image generated by the composite image generation unit 32 is output to the navigation device 20 by the navigation communication unit 42. Thereby, a composite image showing at least a part of the area around the vehicle is displayed on the display 21 of the navigation device 20.

  The control unit 1 is configured as a computer including a CPU, a RAM, a ROM, and the like, and various control functions are realized by the CPU performing arithmetic processing according to a predetermined program. An image control unit 11 and a display control unit 12 shown in the drawing show some of the functions of the control unit 1 realized in this way.

  The image control unit 11 performs control related to image processing executed by the image generation unit 3. For example, the image control unit 11 instructs various parameters necessary for generating a composite image generated by the composite image generation unit 32. The display control unit 12 performs control related to display contents displayed on the display 21 of the navigation device 20. Specific processing contents of the display control unit 12 will be described later.

  The main body 10 of the image processing apparatus 100 further includes a nonvolatile memory 40 and a card reading unit 44, which are connected to the control unit 1.

  The nonvolatile memory 40 is configured by a flash memory or the like that can maintain stored contents even when the power is turned off. The nonvolatile memory 40 stores viewpoint data 4a. The viewpoint data 4a is used when determining the virtual viewpoint of the composite image.

  The card reading unit 44 reads a memory card MC that is a portable recording medium. The card reading unit 44 includes a card slot in which the memory card MC can be attached and detached, and reads data stored in the memory card MC installed in the card slot. Data read by the card reading unit 44 is input to the control unit 1.

  The memory card MC is configured by a flash memory or the like capable of storing various data, and the image processing apparatus 100 can use various data stored in the memory card MC. For example, a program (firmware) that realizes the function of the control unit 1 can be updated by storing a program in the memory card MC and reading the program.

<2. Image composition processing>
Next, a method in which the composite image generation unit 32 of the image generation unit 3 generates a composite image that shows a state in which the periphery of the vehicle 9 is viewed from an arbitrary virtual viewpoint based on a plurality of captured images obtained by the imaging unit 5. explain. FIG. 3 is an explanatory diagram of an example of a technique for generating a composite image.

  When the front camera 51, the back camera 52, and the side camera 53 of the photographing unit 5 are simultaneously photographed, four photographed images P1 to P4 that respectively indicate the front, rear, left side, and right side of the vehicle 9 are acquired. The That is, the four captured images P1 to P4 acquired by the capturing unit 5 include information indicating the entire periphery of the vehicle at the time of capturing.

  Next, each pixel of the four captured images P1 to P4 is projected onto a three-dimensional curved surface SP in a virtual three-dimensional space. The three-dimensional curved surface SP has, for example, a substantially hemispherical shape (a bowl shape), and a center portion (a bottom portion of the bowl) is determined as a position where the vehicle 9 exists. The correspondence between the positions of the pixels included in the captured images P1 to P4 and the positions of the pixels of the solid curved surface SP is determined in advance. For this reason, the value of each pixel of the three-dimensional curved surface SP can be determined based on this correspondence and the value of each pixel included in the captured images P1 to P4.

  The correspondence between the positions of the pixels of the captured images P1 to P4 and the positions of the pixels of the three-dimensional curved surface SP is the arrangement of the four in-vehicle cameras 51 to 53 in the vehicle 9 (inter-distance, ground height, optical axis angle, etc. ). Table data indicating the correspondence is stored in advance in the nonvolatile memory 40.

  Further, data indicating the shape and size of the vehicle body stored in advance in the nonvolatile memory 40 is used, and a polygon model indicating the three-dimensional shape of the vehicle 9 is virtually configured. The configured model of the vehicle 9 is arranged in a substantially hemispherical central portion determined as the position of the vehicle 9 in the three-dimensional space in which the three-dimensional curved surface SP is set.

  Further, the virtual viewpoint VP is set by the control unit 1 for the three-dimensional space where the solid curved surface SP exists. The virtual viewpoint VP is defined by the viewpoint position and the visual field direction, and is set to an arbitrary visual field position corresponding to the periphery of the vehicle 9 in this three-dimensional space toward an arbitrary visual field direction.

  Then, according to the set virtual viewpoint VP, a necessary area on the three-dimensional curved surface SP is cut out as an image. The relationship between the virtual viewpoint VP and a necessary area in the three-dimensional curved surface SP is determined in advance, and is stored in advance in the nonvolatile memory 40 or the like as table data. On the other hand, rendering is performed on a polygon model according to the set virtual viewpoint VP, and the resulting two-dimensional vehicle image is superimposed on the cut-out image. Thereby, the composite image which shows a mode that the vehicle 9 and the periphery of the vehicle 9 were seen from arbitrary virtual viewpoints VP will be produced | generated.

  For example, when the virtual viewpoint VPa is set such that the viewpoint position is a position just above the center of the vehicle 9 and the viewing direction is a direction immediately below, the vehicle 9 and the vehicle 9 A composite image CPa of the bird's-eye view showing the area around the vehicle 9 is generated. Also, as shown in the figure, when a virtual viewpoint VPb in which the viewpoint position is the left rear of the position of the vehicle 9 and the visual field direction is substantially in front of the vehicle 9 is set, the entire periphery from the left rear of the vehicle 9 is set. As seen, a composite image CPb indicating the vehicle 9 and the area around the vehicle 9 is generated. Furthermore, an enlarged composite image is generated when the position of the virtual viewpoint VPa is close to the vehicle, and a reduced composite image that reflects the surroundings of the vehicle is generated when the position of the virtual viewpoint VPa is moved away from the vehicle.

  The composite image generation unit 32 can also generate a partially enlarged image in which a part of the bird's-eye view image showing the vehicle 9 and the area around the vehicle 9 is enlarged. A composite image CP1 of the bird's-eye view image in FIG. 4A shows the vehicle 9 and a region around the vehicle 9 in the same manner as the composite image CPa in FIG.

  For example, if the viewpoint position is set to the upper left position of the position of the vehicle 9 and close to the vehicle 9, and the virtual viewpoint is set so that the visual field direction is substantially directly below, the composite image CP2 of the enlarged image in FIG. Is generated. The enlarged image CP2 is an enlarged image in which the portion of the region 60 of the overhead image CP1 in FIG. Further, for example, when a virtual viewpoint is set in which the viewpoint position is a position on the upper left of the position of the vehicle 9 and close to the vehicle 9 and the visual field direction is substantially directly below, the composition of the enlarged image in FIG. An image CP3 is generated. The enlarged image CP3 is an enlarged image in which a portion of the region 61 of the overhead image CP1 in FIG.

  In the case of actually generating a composite image, it is not necessary to determine the values of all the pixels of the three-dimensional curved surface SP, and only the values of the pixels in the area necessary corresponding to the set virtual viewpoint VP are photographed. By determining based on the images P1 to P4, the processing speed can be improved.

  In the image display system 120, by using such a function of the composite image generation unit 32, only the composite image showing the vehicle 9 and the entire periphery of the vehicle 9 as an overhead view, or only a part of the peripheral region around the vehicle 9. Can be generated and displayed on the display 21.

<3. Display mode>
The image display system 120 has a plurality of display modes for displaying a navigation mode, a captured image acquired by the imaging unit 5 and a composite image synthesized by the composite image generation unit 32. These display modes are switched by the control of the display control unit 12 based on the traveling state of the vehicle 9, the position of the shift lever, the user's operation via the operation unit 22, the touch panel of the display 21, and the like. Hereinafter, each embodiment of the display mode switching process by the display control unit 12 will be described.

<4. First Example>
FIG. 5 is an explanatory diagram of a first example of a state transition diagram of the display mode switched by the display control unit 12. The navigation mode ST <b> 10 is an operation mode in which a map image for navigation guidance is displayed on the display 21 by the function of the navigation device 20. In the navigation mode ST10, a navigation display Dn for route guidance as shown in FIG.

  Please refer to FIG. For example, when the position of the shift lever reaches the “R” position, the display control unit 12 changes the display mode to the back mode ST20. When the position of the shift lever is other than the “R” position, the display control unit 12 returns the display mode to the navigation mode ST10.

  For example, when the position of the shift lever is a position other than the “R” position and the speed of the vehicle 9 is less than the predetermined speed V1, the display control unit 12 changes the display mode to the front mode ST30. When the speed of the vehicle 9 is equal to or higher than the predetermined speed V1, the display control unit 12 returns the display mode to the navigation mode ST10.

  When the position of the shift lever becomes the “R” position during the front mode ST30, the display control unit 12 may directly transition the display mode to the back mode ST20. Further, when the position of the shift lever is a position other than the “R” position during the back mode ST20 and the speed of the vehicle 9 is less than the predetermined speed V1, the display control unit 12 directly changes the display mode to the front mode ST30. You may make a transition.

  For example, when the position of the shift lever is a position other than the “R” position and the operation unit 22 receives a predetermined operation from the user, the display control unit 12 transitions the display mode to the surrounding confirmation mode ST40. The predetermined operation may be a long press of a hard switch, for example. In the following description and drawings, the hard switch may be referred to as “hard SW”.

  When the operation unit 22 receives a predetermined return operation from the user, the display control unit 12 returns the display mode to the navigation mode ST10. For example, the predetermined return operation may also be a long press of the hard SW.

  When the operation unit 22 receives a predetermined operation from the user during the back mode ST20 or the front mode ST30, the display control unit 12 may change the display mode to the surrounding confirmation mode ST40. At this time, the display control unit 12 stores the display mode of the transition source. When the operation unit 22 receives a predetermined return operation from the user, the display control unit 12 returns the display mode to the original display mode.

  Next, display contents in each mode ST20, ST30, and ST40 will be described. The back mode ST20 has display modes of a back mode two-screen display state ST21, a first back camera image display state ST22, and a second back camera image display state ST23.

  When a predetermined camera switch provided as a hard switch of the operation unit 22 or a soft switch set on the touch panel of the display 21 is operated, the display control unit 12 changes the display mode between the states ST21 to ST23. In the following description and drawings, the camera switch may be referred to as “camera SW”.

  FIG. 7 is a diagram illustrating an example of display contents in the back-mode two-screen display state ST21. Reference numeral V0 indicates a display range when a predetermined full screen display is performed on the display 21. In the following description, the display range V0 when performing a predetermined full screen display may be expressed as “full screen V0”. The full screen V0 may be the entire display screen of the display 21. In addition, the full screen V0 is slightly more than the entire display screen of the display 21 as in the case of displaying the full screen V0 with a predetermined margin inside the entire display screen of the display 21 in the predetermined full screen display. It may be a small area. The back mode two-screen display Dtb has a first region R1 and a second region R2 in the entire screen V0. The display control unit 12 displays a composite image of the overhead image CPw of the entire periphery of the vehicle 9 in the first region R1.

  In addition, the display control unit 12 displays a captured image of the back camera 52 or a partial image of the bird's-eye view image of the entire periphery of the vehicle 9 in the second region R2. In the display example shown in FIG. 7, the captured image Pb1 of the back camera 52 is displayed in the second region R2. The display content switching process in the second region R2 executed by the display control unit 12 will be described later.

  FIGS. 8A and 8B are diagrams illustrating examples of display contents in the first back camera image display state ST22 and the second back camera image display state ST23, respectively. A screen display (hereinafter sometimes referred to as “first back camera image display”) Db1 in the first back camera image display state ST22 is a captured image Pb1 of the back camera 52 displayed on the full screen V0 on the display 21. Including full screen display.

  The screen display (hereinafter sometimes referred to as “second back camera image display”) Db2 in the second back camera image display state ST23 is a full screen display of the captured image Pb2 of the back camera 52 displayed on the full screen V0. including. The angle of view is different between the image Pb1 and the image Pb2.

  Please refer to FIG. The front mode ST30 has display modes of a front mode two-screen display state ST31, a front camera image display state ST32, and a side camera image display state ST33. When the camera SW is operated, the display control unit 12 changes the display mode between the states ST31 to ST33.

  FIG. 9 is a diagram showing an example of display contents in the front mode two-screen display state ST31. The front mode two-screen display Dtf has a first region R1 and a second region R2 in the entire screen V0. The display control unit 12 displays a composite image of the overhead image CPw of the entire periphery of the vehicle 9 in the first region R1.

  In addition, the display control unit 12 displays a captured image of the front camera 51 or a partial image of an overhead image of the entire periphery of the vehicle 9 in the second region R2. In the display example shown in FIG. 9, a blind corner monitor (BCM: Blind Corner Monitor) image Pf1 photographed by the front camera 51 is displayed in the second region R2. The display content switching process in the second region R2 executed by the display control unit 12 will be described later.

  FIGS. 10A and 10B are diagrams illustrating examples of display contents in the front camera image display state ST32 and the side camera video display state ST33. The screen display (hereinafter sometimes referred to as “front camera image display”) Df in the front camera image display state ST32 overlooks the entire periphery of the vehicle 9 from the BCM image Pf1 and an arbitrary virtual viewpoint behind the vehicle 9. Including the display of the composite image Pf2.

  The screen display (hereinafter sometimes referred to as “side camera image display”) Ds in the side camera image display state ST33 includes displays of images Ps1 and Ps2 captured by the left and right side cameras 53, respectively.

  Next, display contents in the surrounding confirmation mode ST40 will be described. FIG. 11 shows an example of display contents in the surrounding confirmation mode ST40. The screen display (hereinafter sometimes referred to as “all-around confirmation display”) Dc in the surrounding confirmation mode ST <b> 40 includes display of a stereoscopic bird's-eye view image including the scenery around the vehicle 9.

  Next, switching of the display of the second region R2 in the back mode two-screen display state ST21 and the front mode two-screen display state ST31 will be described. FIG. 12 is a flowchart of a first example of the display switching process in the back-mode two-screen display state ST21.

  In other embodiments, the following operations AA to AG may be steps. In operation AA, the display control unit 12 displays the back mode two-screen display Dtb on the display 21. The displayed back mode two-screen display Dtb is shown in FIG.

  The overhead image CPw displayed in the first region R1 is divided into a plurality of partitioned regions A0 to A8. Note that the boundary lines indicating the regions indicated by dotted lines in FIG. 13 may or may not be actually displayed on the display 21. The same applies to the other embodiments.

  Please refer to FIG. In operation AB, the display control unit 12 determines whether or not there is a user input by touching the camera SW. When the camera SW is touched (operation AB: Y), the process proceeds to operation AG. In operation AG, the display control unit 12 changes the display mode to the first back camera image display state ST22 and ends the process.

  If the camera SW has not been touched (operation AB: N), the process proceeds to operation AC. In operation AC, the display control unit 12 determines whether or not there is an input from the user who touches the first region R1 by operating the touch panel of the display 21.

  When the first region R1 is touched (operation AC: Y), the process proceeds to operation AD. If the first region R1 is not touched (operation AC: N), the processing returns to operation AB.

  The display control unit 12 switches the display image in the second region R2 according to which of the regions A0 to A8 in the first region R1 illustrated in FIG. 13 is touched by the user. The correspondence relationship between the area touched by the user and the display image in the second area R2 is shown below.

Area A0: an image designated to be displayed by default at the time of transition to the back mode two-screen display state ST21 (for example, it may be a captured image Pb1 of the back camera 52)
Vehicle left front area A1: Composite image of enlarged overhead view seen from right above the vehicle left front area Vehicle front area A2: Composite image of enlarged bird's eye view seen from right above the vehicle right Front area A3: a composite image of an enlarged overhead image viewed from right above the vehicle right front area A4: a composite image of an enlarged overhead image viewed from the vehicle left side A4: right side of the vehicle right side of the vehicle Area A5: a composite image of an enlarged overhead image viewed from right above the vehicle right side area A6: a composite image of an enlarged overhead image viewed from right above the vehicle left rear area A6: vehicle rear Region A7: Composite image of an enlarged overhead view seen from right above the vehicle rear region A8: Composite image of an enlarged overhead image seen right above the vehicle right rear region A8:

  Please refer to FIG. In operation AD, the display control unit 12 determines whether the touched area is an area for designating a default display image. In this example, the display control unit 12 determines whether or not the touched area is the area A0. If the touched area is not an area for designating a default display image (operation AD: N), the process proceeds to operation AE. Otherwise (operation AD: Y), the process proceeds to operation AF.

  In operation AE, the composite image generation unit 32 generates an enlarged overhead image determined based on the correspondence relationship according to the touched area. The display control unit 12 displays the enlarged overhead image generated by the composite image generation unit 32 in the second region R2. Thereafter, the processing returns to operation AB.

  FIG. 14 shows a state where the enlarged overhead image is displayed in the second region R2. In this example, in response to the user touching the area A6 on the left rear side of the vehicle, an enlarged bird's-eye view image CP6 as seen from directly above the left rear side of the vehicle is displayed in the second area R2.

  Note that the composite image generation unit 32 may insert a dimension line 70 indicating the actual size of the enlarged overhead image into the enlarged overhead image CP6. By displaying the dimension line 70, the user can easily grasp the sense of distance between the vehicle body and the obstacle. In other examples described later, dimension lines may be inserted in the enlarged overhead view image.

  Please refer to FIG. In operation AF, the display control unit 12 displays, in the second region R2, an image designated to be displayed by default at the time of transition to the back mode two-screen display state ST21. In this example, the display control unit 12 displays the captured image Pb1 of the back camera 52 in the second region R2. Thereafter, the processing returns to operation AB.

  Next, animation display when switching the display in the second region R2 in the process shown in FIG. 12 will be described. For example, consider a case where the display image of the second region R2 is switched from the captured image Pb1 of the back camera 52 that has been displayed by default to the enlarged overhead image CP6 on the left rear side of the vehicle.

  If the display is switched directly from the captured image Pb1 to the enlarged overhead image CP6, the user may lose sight of what is enlarged and displayed around the vehicle 9. In this case, the user is anxious about whether or not the operation is accepted as intended.

  Therefore, in this embodiment, when the display image in the second region R2 is switched, an overhead image CPw is displayed after displaying an animation in which the viewpoint position continuously changes from the image before the switching to the overhead image CPw of the entire vehicle periphery. The animation in which the viewpoint position continuously changes from the image to the image after switching is displayed.

  As described above, once the viewpoint is continuously changed to the bird's-eye view image CPw of the entire vehicle periphery and then continuously changed to the image after switching, the user can easily understand where the image is enlarged. . For this reason, it becomes easy to grasp that the display process is executed as intended.

  FIG. 15A to FIG. 15C are explanatory diagrams of a display change example of the second region R2. Now, it is assumed that the captured image Pb1 of the back camera 52 as shown in FIG. 15A is displayed in the second region R2 in the state before the display switching of the second region R2.

  When the display control unit 12 switches the display image of the second region R2 to the enlarged overhead view image CP6, the display 21 displays the surroundings of the vehicle shown in FIG. 15B from the captured image Pb1 shown in FIG. An animation that continuously changes to the entire overhead view image CPw is displayed. After that, the display 21 displays an animation that continuously changes from the overhead image CPw to the enlarged overhead image CP6 shown in FIG.

  FIG. 16A is a flowchart of an animation display process. In other embodiments, the following operations BA and BB may be steps.

  In operation BA, the composite image generation unit 32 continuously displays each intermediate image of the animation that continuously changes from the image viewed from the viewpoint of the captured image Pb1 of the back camera 52 to the image viewed from the virtual viewpoint of the overhead image CPw of the entire vehicle periphery. Is generated. The display control unit 12 sequentially displays the intermediate images generated by the composite image generation unit 32.

  In operation BB, the composite image generation unit 32 generates each intermediate image of an animation that continuously changes from the image viewed from the virtual viewpoint of the overhead image CPw of the entire vehicle periphery to the image viewed from the virtual viewpoint of the enlarged overhead image CP6. . The display control unit 12 sequentially displays the intermediate images generated by the composite image generation unit 32.

  FIG. 16B is a flowchart of processing for displaying an intermediate image of the animation of the composite image when the virtual viewpoint is moved from a certain starting point to an ending point. In other embodiments, the operations CA to CE described below may be steps.

  In operation CA, the image control unit 11 sets the viewpoint position Vpp and the visual field direction Op of the virtual viewpoint at the movement start point in the composite image generation unit 32. In operation CB, the image control unit 11 sets the viewpoint position Vfp and the visual field direction Of of the virtual viewpoint at the end point of the movement in the composite image generation unit 32.

  In operation CA, the composite image generating unit 32 determines the viewpoint position Vpm and the visual field direction Om of the virtual viewpoint at the midpoint of movement. For example, the determination of the virtual viewpoint at the midpoint may be as follows.

  In this embodiment, the coordinates of the viewpoint Vpm are expressed by a three-dimensional orthogonal coordinate system that is relatively fixed to the vehicle 9. The coordinates of the start position Vpp are (Xp, Yp, Zp), and the coordinates of the end position Vfp are (Xf, Yf, Zf). Also, the time taken to move the viewpoint position is T (seconds), and the frame rate is F (fps). The composite image generation unit 32 determines the coordinates of the virtual viewpoint at the intermediate point in each frame by interpolation between the start point and the end point as follows.

  The composite image generation unit 32 calculates the movement distances Dx, Dy, and Dz of the virtual viewpoint in the X-axis direction, the Y-axis direction, and the Z-axis direction by the following equations.

Dx = Xf-Xp
Dy = Yf−Yp
Dz = Zf-Zp

  The composite image generation unit 32 calculates the average movement amounts mx, my, and mz per frame in the X-axis direction, the Y-axis direction, and the Z-axis direction by the following equations.

mx = Dx / (F × T)
my = Dy / (F × T)
mz = Dz / (F × T)

  The composite image generation unit 32 calculates the X coordinate Xn, the Y coordinate Yn, and the Z coordinate Zn of the viewpoint position Vpm during the movement by the following expression.

Xn = X _n-1 + α × mx
Yn = Y _n-1 + α × my
Zn = Z _n-1 + α × mz

X _n−1 , Y _n−1 and Z _n−1 in the above expression are the X coordinate, Y coordinate and Z coordinate of the viewpoint position Vpm one frame before, respectively. Α is a coefficient of the moving speed of the virtual viewpoint.

  The viewing direction Om of the virtual viewpoint at the midpoint of movement can also be determined by the same interpolation process.

  In operation CD, the composite image generation unit 32 generates a composite image viewed from the virtual viewpoint determined in operation CC. The display control unit 12 sequentially displays the images generated by the composite image generation unit 32.

  In operation CE, the composite image generation unit 32 determines whether the midpoint calculated in operation CC has reached the end point. When the midpoint reaches the end point (operation CE: Y), the process ends. When the midpoint does not reach the end point (operation CE: N), the processing returns to operation CC.

  In addition, when switching the display image in the second region R2 from the enlarged bird's-eye view image to the captured image Pb1 of the back camera 52, an animation that changes via the bird's-eye view image CPw of the entire vehicle periphery is displayed as described above. May be. Furthermore, when switching the display image in the second region R2 from one enlarged overhead image to another enlarged overhead image, an animation that changes via the overhead image CPw may be displayed in the same manner as described above.

  The switching of the display of the second region R2 in the back mode two-screen display state ST21 has been described above. The switching of the display of the second region R2 in the front mode two-screen display state ST31 may be the same as described above.

<5. Second Embodiment>
Next, another example of the display process when the overhead image CPw displayed in the first region R1 is touched in the back mode two-screen display state ST21 will be described. In this embodiment, when the bird's-eye view image CPw is touched, the enlarged bird's-eye view image is displayed on the full screen V0 on the full screen.

  FIG. 17 is a flowchart of a second example of the display switching process in the back-mode two-screen display state ST21. The process in the front mode two-screen display state ST31 may be the same. In other embodiments, the following operations DA to DH may be steps.

  In operation DA, the display control unit 12 displays the back mode two-screen display Dtb on the display 21. The displayed back mode two-screen display Dtb is shown in FIG. The display content of FIG. 18A is the same as the back-mode two-screen display Dtb described with reference to FIG. In this example, immediately after the transition to the state ST21, the photographed image Pb1 of the back camera 52 designated to be displayed by default is displayed in the second region R2.

  Refer to FIG. In the operation DB, the display control unit 12 determines whether or not there is a user input by touching the camera SW. When the camera SW is touched (operation DB: Y), the process proceeds to operation DH. In operation DH, the display control unit 12 changes the display mode to the first back camera image display state ST22 and ends the process.

  If the camera SW is not touched (operation DB: N), the process proceeds to operation DC. In operation DC, the display control unit 12 determines whether or not there is an input from the user who touches the first region R1 by operating the touch panel of the display 21.

  When the first region R1 is touched (operation DC: Y), the process proceeds to operation DD. If the first region R1 is not touched (operation DC: N), the process returns to the operation DB.

  In operation DD, the composite image generation unit 32 generates an enlarged overhead image obtained by enlarging an area touched by the user from among the divided areas A0 to A8 in the first area R1. The display control unit 12 displays the enlarged overhead image generated by the composite image generation unit 32 in full screen.

  FIG. 18B shows a state where the enlarged overhead image is displayed on the full screen V0 of the display 21. In this example, in response to the user touching the area A6 on the left rear side of the vehicle, an enlarged overhead image CP6 obtained by viewing the left rear side of the vehicle from directly above is displayed in full screen. A “return button” 71 described later is also displayed in the full screen V0.

  Refer to FIG. In operation DE, the display control unit 12 determines whether or not there is an input from the user who touches the touch panel. When the touch panel is touched (operation DE: Y), the process proceeds to operation DF. If the touch panel is not touched (operation DE: N), the process returns to operation DE.

  In operation DF, the display control unit 12 determines whether or not the touched portion is the return button 71. When the return button 71 is touched (operation DF: Y), the processing returns to operation DA. If the touched location is not the return button 71 (operation DF: N), the process proceeds to operation DG.

  In operation DG, the composite image generation unit 32 changes the position of the viewpoint of the enlarged overhead image displayed on the full screen according to the touched position on the enlarged overhead image. Thereafter, the processing returns to operation DE.

  According to the present embodiment, it is possible to display a bird's-eye view image showing an arbitrary portion around the vehicle in full screen on the display screen. For this reason, when the user confirms the clearance between the vehicle body and the obstacle, it is easy to grasp the sense of distance between the obstacle.

  Next, a process of changing the viewpoint position of the enlarged overhead image in the operation DG in the display process shown in FIG. 17 will be described. FIGS. 19A to 19C are explanatory diagrams of a first example of a method for changing the position of the viewpoint of the enlarged overhead image.

  In the state of FIG. 19A, an enlarged overhead view image CP6 showing a left rear region A6 of the vehicle 9 is displayed on the full screen V0. The enlarged overhead view image CP6 displayed on the full screen includes a plurality of partial areas ALU, ARU, ALD, and ARD arranged around the center C of the full screen V0.

  When it is detected that the partial areas ALU, ARU, ALD, and ARD are touched by the user, the display control unit 12 changes the position according to the touched area and the position of the viewpoint of the enlarged overhead image currently displayed. The position of the viewpoint of the enlarged overhead image is determined.

  For example, FIG. 19B shows an enlarged overhead image that is displayed when the upper left area ALU is touched during the display of the enlarged overhead image CP6. When the upper left area ALU is touched during the display of the enlarged overhead image CP6, the viewpoint moves forward, and the enlarged overhead image CP4 of the area A4 on the left side of the vehicle is displayed in full screen.

  When the upper left area ALU is touched during the display of the enlarged overhead image CP4, the viewpoint moves forward, and the enlarged overhead image of the area A1 on the left front side of the vehicle is displayed in full screen. Further, when the lower left area ALD is touched during the display of the enlarged overhead image CP4, the viewpoint moves downward, and the enlarged overhead image CP6 of the area A6 on the left rear side of the vehicle is displayed on the full screen.

  For example, FIG. 19C shows an enlarged overhead view image that is displayed when the lower right area ARD is touched while the enlarged overhead view image CP6 is displayed. When the lower right area ARD is touched during the display of the enlarged overhead image CP6, the viewpoint moves to the right, and the enlarged overhead image CP7 of the area A7 behind the vehicle is displayed on the full screen.

  If the upper left area ALU is touched during the display of the enlarged overhead image CP7, the viewpoint moves to the left front, and the enlarged overhead image CP4 of the area A4 on the left side of the vehicle is displayed on the full screen. When the upper right area ARU is touched during the display of the enlarged overhead image CP7, the viewpoint moves rightward and the enlarged overhead image of the area A5 on the right side of the vehicle is displayed in full screen.

  If the lower left area ALD is touched during display of the enlarged overhead image CP7, the viewpoint moves to the left, and the enlarged overhead image CP6 of the area A6 on the left rear side of the vehicle is displayed on the full screen. If the lower right area ARD is touched during the display of the enlarged overhead image CP7, the viewpoint moves to the right, and the enlarged overhead image of the area A8 on the right rear side of the vehicle is displayed in full screen.

  In addition, the example of the arrangement method of the partial area and the movement direction of the viewpoint is merely an example for description, and various arrangement methods and movement directions can be employed.

  The composite image generation unit 32 generates an enlarged overhead image viewed from the viewpoint position determined by the display control unit 12. The display control unit 12 displays the generated enlarged overhead image. Thereby, the display range of the enlarged overhead image slides.

  When changing the viewpoint of the enlarged overhead image, the composite image generating unit 32 generates an animation in which the enlarged overhead image continuously slides to the changed viewpoint position while keeping the height of the viewpoint in the same state. May be.

  When the enlarged overhead view image is displayed on the full screen, there is no space for displaying the overhead view image of the entire vehicle periphery. For this reason, when designating where to enlarge with the enlarged overhead view image, it is impossible to use the overhead view image of the entire vehicle periphery. Therefore, it is necessary to provide a means for the user to instruct a change of viewpoint.

  According to the present embodiment, means for changing the viewpoint of the enlarged overhead image when the enlarged overhead image is displayed on the full screen is provided. Furthermore, according to the present embodiment, since the viewpoint of the enlarged overhead image can be changed by an intuitive operation using the touch panel, the convenience for the user is improved.

  According to the present embodiment, it is possible to move from the current enlarged overhead image to the adjacent enlarged overhead image with one touch. In particular, when an image around the vehicle is displayed as in the present embodiment, the relative positional relationship between the obstacle watched by the user and the vehicle often changes continuously as the vehicle moves.

  In the present embodiment, since the viewpoint can be moved to the adjacent enlarged bird's-eye view image with one touch, it becomes easy to track an obstacle that relatively continuously moves around the vehicle.

  Subsequently, an example of another method for changing the position of the viewpoint of the enlarged overhead image will be described. 20A to 20C are explanatory diagrams of a second example of a method for changing the position of the viewpoint of the enlarged overhead image. In the state of FIG. 20A, an enlarged overhead view image CP <b> 6 showing the left rear region of the vehicle 9 is displayed on the full screen V <b> 0 of the display 21.

  When the enlarged bird's-eye view image is displayed on the full screen, the composite image generation unit 32 inserts an arrow display 80 for receiving an instruction to change the viewpoint by the user into the image. When the arrow display 80 is touched by the user, the composite image generation unit 32 generates an enlarged overhead view image that exists in the direction of the touched arrow. The display control unit 12 displays the enlarged overhead view image on the display and slides the position of the viewpoint. When changing the viewpoint of the enlarged overhead image, the composite image generation unit 32 may generate an animation in which the enlarged overhead image slides continuously.

  The composite image generation unit 32 may insert a fixed arrow display that always indicates eight directions including four directions (up, down, left, and right) and an oblique direction into the image. Instead of this, only an arrow display indicating the direction in which another magnified overhead image that can be displayed is present may be inserted into the image.

  In this case, the display control unit 12 determines the direction in which another magnified overhead image that can be displayed exists, as viewed from the position of the magnified overhead image currently being displayed. In the case of the enlarged overhead image CP6, an enlarged overhead image showing the area A4 on the left side of the vehicle 9 and an enlarged overhead image showing the area A7 behind the vehicle 9 can be displayed.

  When the display control unit 12 determines a direction in which another magnified overhead image that can be displayed is present, the composite image generation unit 32 inserts an arrow display 80 indicating this direction into the currently displayed image. In the state of FIG. 20A, an arrow 81 indicating “front”, which is a direction in which an enlarged bird's-eye image showing a region A4 on the left side of the vehicle 9 exists, and an enlarged bird's-eye image showing a region A7 behind the vehicle 9. An arrow 82 indicating “right”, which is a direction in which “” exists, is displayed.

  When either of the arrows 81 and 82 is touched by the user, the composite image generation unit 32 generates an enlarged overhead view image that exists in the direction of the touched arrow. For example, FIG. 20B shows an enlarged overhead view image that is displayed when the arrow 81 indicating the front is touched during the display of the enlarged overhead view image CP6. When the arrow 81 is touched during the display of the enlarged overhead image CP6, the viewpoint moves forward, and the enlarged overhead image CP4 of the region A4 on the left side of the vehicle is displayed on the full screen.

  In the state in which the enlarged overhead image CP4 is displayed, an arrow 81 indicating “front”, which is a direction in which the enlarged overhead image indicating the left front area A1 of the vehicle 9 exists, and an area A5 on the right side of the vehicle 9 are displayed. An arrow 82 indicating “right”, which is the direction in which the enlarged bird's-eye view image is present, and an arrow 83 indicating “rear”, which is the direction in which the enlarged bird's-eye image indicating the left rear region A6 of the vehicle 9 is present, are displayed.

  Further, for example, FIG. 20C shows an enlarged bird's-eye view image that is displayed when the arrow 82 indicating “right” is touched during the display of the enlarged bird's-eye view image CP6. When the arrow 82 is touched during the display of the enlarged overhead image CP6, the viewpoint moves to the right, and the enlarged overhead image CP7 of the area A7 behind the vehicle is displayed on the full screen.

  In the state where the enlarged bird's-eye view image CP7 is displayed, an arrow 81 indicating “front”, which is a direction in which the enlarged bird's-eye image showing the area A2 in front of the vehicle 9 exists, and a region A8 on the right rear of the vehicle 9 are shown. An arrow 82 indicating “right”, which is the direction in which the enlarged overhead image exists, and an arrow 84 indicating “left”, which is the direction in which the enlarged overhead image indicating the left rear area A6 of the vehicle 9 exists, are displayed.

  This embodiment also provides a means for changing the viewpoint of the enlarged overhead image when the enlarged overhead image is displayed on the full screen. Furthermore, according to the present embodiment, the viewpoint of the enlarged overhead image can be changed by an intuitive operation using the touch panel, and the convenience for the user is improved.

  Also according to this embodiment, it is possible to move from the current enlarged overhead image to the adjacent enlarged overhead image with one touch.

<6. Third Example>
Next, an example of another display mode switching process by the display control unit 12 will be described. FIG. 21 is an explanatory diagram of a second example of the state transition diagram of the display mode. The same parts as those in the state transition diagram shown in FIG.

  The transition method between the navigation mode ST10, the back mode ST20, the front mode ST30, and the surrounding confirmation mode ST40 may be the same as the method described above with reference to FIG.

  In this embodiment, the back mode ST20 includes a full-screen overhead display state ST24. When the camera switch is operated in the back mode ST20, the display control unit 12 changes the display mode between the states ST24 and ST21 to ST23.

  FIG. 22 is a diagram illustrating an example of display contents in the full-screen overhead display state ST24. The screen display (hereinafter, may be referred to as “full screen overhead view display”) Dww in the full screen overhead view state ST24 is a full screen overhead image CPww of the entire periphery of the vehicle 9 displayed on the full screen V0 on the display 21. The composite image of is displayed.

  Compared with the overhead view image CPw displayed in the above-described back mode two-screen display Dtb, the full-screen overhead view CPww has a wider angle of view by increasing the position of the viewpoint, and a wider range around the vehicle 9. It is a bird's-eye view image to be displayed.

  Refer to FIG. The front mode ST30 also includes a full screen overhead view display state ST34. When the camera switch is operated in the front mode ST20, the display control unit 12 changes the display mode between the states ST34 and ST31 to ST33. The display content in the full-screen overhead display state ST34 may be the same as the display content in the full-screen overhead display state ST24.

  FIG. 23 is a flowchart of the display switching process in the full-screen overhead display state ST24. The processing in the full-screen overhead display state ST34 may be the same. In another embodiment, each of the following operations EA to EH may be a step.

  In operation EA, the display control unit 12 displays the full screen overhead view Dww on the display 21. The displayed full screen overhead view Dww is shown in FIG. The full-screen overhead image CPwww displayed on the full screen is divided into a plurality of sections of areas A11 to A18. Note that the boundary lines indicating the regions indicated by dotted lines in FIG. 24 may or may not be actually displayed on the display 21.

  Refer to FIG. In operation EB, the display control unit 12 determines whether or not there is a user input by touching the camera SW. When the camera SW is touched (operation EB: Y), the process proceeds to operation EH. In operation EH, the display control unit 12 changes the display mode to the back-screen two-screen display state ST21 and ends the process.

  If the camera SW is not touched (operation EB: N), the process proceeds to operation EC. In operation EC, the display control unit 12 determines whether or not there is an input from the user who touches the touch panel. When the touch panel is touched (operation EC: Y), the process proceeds to operation ED. If the touch panel is not touched (operation EC: N), the processing returns to operation EB.

  In operation ED, the display control unit 12 selects an enlarged overhead image to be displayed on the full screen V0 in accordance with which of the areas A11 to A18 of the full screen overhead image CPww is touched by the user. The correspondence between the area touched by the user and the display image is shown below.

Area A11 in front of the left side of the vehicle: Composite image of an enlarged bird's-eye view obtained by magnifying the left front of the vehicle from directly above. Area A12 in front of the vehicle: Composite image of an enlarged bird's-eye view in which the front of the vehicle is enlarged from directly above. Front area A13: Composite image of enlarged overhead view seen from right above vehicle right front area A14: Composite image of enlarged overhead view seen from right side of vehicle A14: Right side of vehicle right side of vehicle Area A15: a composite image of an enlarged overhead image viewed from right above the vehicle right side area A16: a composite image of an enlarged overhead image viewed from right above the vehicle left rear area A16: vehicle rear Region A17: Composite image of an enlarged overhead view seen from right above the vehicle rear region A18: Composite image of an enlarged overhead view seen from right above the vehicle right rear region A18

  The composite image generation unit 32 generates an enlarged overhead image that is determined based on the correspondence relationship according to the touched region. The display control unit 12 displays the enlarged overhead image generated by the composite image generation unit 32 in full screen.

  FIG. 24B shows a state where the enlarged overhead image is displayed on the full screen. In this example, in response to the user touching the area A16 on the left rear side of the vehicle, an enlarged overhead image CP16 in which the left rear side of the vehicle is viewed from directly above is displayed on the full screen.

  Refer to FIG. In operation EE, the display control unit 12 determines whether or not there is an input from the user who touches the touch panel. When the touch panel is touched (operation EE: Y), the process proceeds to operation EF. If the touch panel is not touched (operation EE: N), the process returns to operation EE.

  In operation EF, the display control unit 12 determines whether or not the touched portion is the return button 71. When the return button 71 is touched (operation EF: Y), the processing returns to operation EA. If the touched location is not the return button 71 (operation EF: N), the process proceeds to operation EG.

  In operation EG, the composite image generation unit 32 changes the position of the viewpoint of the enlarged overhead image displayed on the full screen in accordance with the touched position on the enlarged overhead image. Thereafter, the processing returns to operation EE. The process of changing the viewpoint position of the enlarged overhead image is the same as the process described with reference to FIGS. 19A to 19C and FIGS. 20A to 20C. It's okay.

  According to the present embodiment, it is possible to easily switch to the full-screen display of the enlarged bird's-eye view image by one-touching a desired position in the full-screen bird's-eye view image CPww representing a relatively wide range around the vehicle 9. Further, the user can easily switch to the full-screen overhead image CPwww by touching the return button 71 inserted in the enlarged overhead view image.

  As described above, since it is possible to switch between the full-view bird's-eye view image CPww and the enlarged bird's-eye view image with one touch, it is possible to improve convenience when the user confirms the clearance between the vehicle body and the obstacle.

For example, since the full-screen overhead image CPww represents a relatively wide range, for example, a white line of a parking space can be confirmed from a relatively long distance in a parking lot or the like. For this reason, the direction to the parking space can be determined from a distance, and when approached, it can be switched to an enlarged bird's-eye view image with a single touch to accurately grasp the relative distance from the obstacle.
In the above embodiment, the overhead image is divided into a plurality of regions, and an enlarged overhead image of the touched region is generated and displayed. In other words, the viewpoint position exists in the vicinity immediately above the center of each region. In this case, when there is an obstacle at the boundary between adjacent regions, it may be difficult to check the obstacle. Therefore, as a modification, instead of dividing the overhead image into a plurality of regions, an enlarged overhead image centered on a point touched on the overhead image may be generated and displayed. That is, an enlarged overhead image is generated with the point touched on the overhead image as a viewpoint position. In this way, since it is possible to display an enlarged bird's-eye view image centered on a location to be confirmed, it is easy to confirm an obstacle.

DESCRIPTION OF SYMBOLS 20 Navigation apparatus 21 Display 22 Operation part 11 Image control part 12 Display control part 31 Composite image generation part 51 Front camera 52 Back camera 53 Side camera 120 Image display system

Claims (14)

An image display system mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means on a display screen;
Display control means for generating an enlarged overhead view image centered on the touched point and displaying it on the display screen when the overhead view image displayed on the display screen is touched;
With
One or more display modes of the display screen controlled by the display control means include at least a state in which the overhead image is displayed on a full screen,
When the overhead image displayed on the full screen is touched, the display control unit generates an enlarged overhead image centered on the touched point and displays the entire screen on the display screen. Image display system. An image display system mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means on a display screen;
Display control means for generating an enlarged overhead view image centered on the touched point and displaying it on the display screen when the overhead view image displayed on the display screen is touched;
With
One or more display modes of the display screen controlled by the display control means include a state of displaying a captured image obtained by the camera on the display screen,
The display control means includes a first animation that changes from the captured image to the overhead image when the overhead image displayed on the display screen is touched during display of the captured image, and the overhead image. An image display system for displaying a second animation that changes to the enlarged overhead view image. An image display system mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means on a display screen;
Display control means for generating an enlarged overhead view image centered on the touched point and displaying it on the display screen when the overhead view image displayed on the display screen is touched;
With
The display control means, when the enlarged overhead view image is displayed, when any of a plurality of areas obtained by dividing the enlarged overhead image is designated, the field of view of the enlarged overhead image according to the designated area An image display system characterized by changing the position of the screen. An image display system mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means on a display screen;
Display control means for generating an enlarged overhead view image centered on the touched point and displaying it on the display screen when the overhead view image displayed on the display screen is touched;
With
The display control means receives the designation input designating the arrow display displayed on the display screen when the enlarged overhead view image is displayed, and displays the enlarged overhead view image according to the designated arrow. An image display system characterized by changing a position of a visual field. The one or more display modes of the display screen controlled by the display control means include a two-screen display state having at least a first area and a second area,
5. The image display system according to claim 2, wherein the display control unit receives a touch of the overhead image displayed in the first area in the two-screen display state.   The image display system according to claim 1, wherein the composite image generation unit inserts a dimension line into the enlarged overhead view image. An image display system mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means and the captured image on a display screen;
Display control means for generating an enlarged overhead image centered on the touched point and displaying it on the display screen when the overhead image displayed on the display screen is touched;
With
The display control means includes a first animation that changes from the captured image to the overhead image when the overhead image displayed on the display screen is touched during display of the captured image, and the overhead image. An image display system for displaying a second animation that changes to the enlarged overhead view image. An image processing device for generating an image to be displayed on a display device mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means on a display screen of the display device;
Display control means for generating an enlarged overhead view image centered on the touched point and displaying it on the display screen when the overhead view image displayed on the display screen is touched;
With
One or more display modes of the display screen controlled by the display control means include a state of displaying a captured image obtained by the camera on the display screen,
The display control means includes a first animation that changes from the captured image to the overhead image when the overhead image displayed on the display screen is touched during display of the captured image, and the overhead image. An image processing apparatus that displays a second animation that changes to the enlarged overhead view image. An image processing device for generating an image to be displayed on a display device mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means and the captured image on a display screen;
Display control means for generating an enlarged overhead image centered on the touched point and displaying it on the display screen when the overhead image displayed on the display screen is touched;
With
The display control means includes a first animation that changes from the captured image to the overhead image when the overhead image displayed on the display screen is touched during display of the captured image, and the overhead image. An image processing apparatus that displays a second animation that changes to the enlarged overhead view image. An image display method for displaying an image in a vehicle,
Generating an overhead image around the vehicle viewed from a virtual viewpoint based on a captured image obtained by a camera arranged in the vehicle;
Displaying the overhead image on a display screen;
When the overhead view image displayed on the display screen is touched, an enlarged overhead view image centered on the touched point is generated,
The enlarged overhead view image is displayed in full screen on the display screen,
When the overhead image displayed on the display screen is touched while the captured image is displayed, a first animation that changes from the captured image to the overhead image and a change from the overhead image to the enlarged overhead image Generating a second animation, and
An image display method for displaying the first and second animations on the display screen. An image display method for displaying an image in a vehicle,
Generating an overhead image around the vehicle viewed from a virtual viewpoint based on a captured image obtained by a camera arranged in the vehicle;
Displaying the overhead image and the captured image on a display screen;
When the overhead image displayed on the display screen is touched, a first animation that changes from the captured image to the overhead image and an enlarged overhead image centered on the touched point from the overhead image A second animation that changes, and
An image display method for displaying the first and second animations on the display screen. An image display system mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means on a display screen;
Display control means for generating an enlarged overhead image centered on the touched point and displaying it on the display screen when the overhead image displayed on the display screen is touched;
An image display system comprising: An image display system mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means on a display screen;
A first animation that changes from the captured image to the overhead image when the overhead image displayed on the display screen is touched during display of the captured image, and the touched point from the overhead image Display control means for displaying a second animation that changes to an enlarged overhead view image centered on
An image display system comprising: An image display system mounted on a vehicle,
Synthetic image generation means for generating an overhead image viewed from a virtual viewpoint based on a captured image obtained by a camera disposed in the vehicle;
Display means for displaying the overhead image generated by the composite image generation means on a display screen;
Display control means for generating an enlarged overhead image centered on the touched point and displaying it on the display screen when the overhead image displayed on the display screen is touched;
With
The display control means changes the position of the field of view of the magnified overhead image when receiving an instruction to change the position of the field of view of the magnified overhead image while the magnified overhead image is displayed. Image display system.

Images