JP2013078151A - Display device for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device for vehicle that allows an occupant of the vehicle to easy grasp the kind of an obstacle and the distance between the obstacle and the vehicle.SOLUTION: The display device for vehicle includes photography means of acquiring a circumference photographed image of the vehicle; image conversion means of generating a mirror conversion image by converting coordinates of the circumference photographed image; bird's-eye image generation means of generating a bird's-eye image including an image of the whole vehicle; and display means of displaying the mirror-converted image and bird's-eye image on the same display screen. The display means displays the bird's-eye image in a rectangular first display area preset on the display screen with the vehicle front of the image of the whole vehicle up, and also displays the mirror-converted image in a second display area set on a side of the rectangular first display area and biased to below a side edge of the rectangle.

Description

  The present invention relates to a vehicular display device that is mounted on a vehicle and allows an occupant of the vehicle to visually recognize an image of the surroundings of the vehicle.

  Conventionally, for driving assistance when parking a vehicle, the front, rear, left side, and right side of the vehicle are photographed by a plurality of (for example, four) in-vehicle cameras, and further, the captured images captured by the in-vehicle cameras are captured. Then, by converting to a bird's-eye view image looking down from above the vehicle and connecting them together, a single bird's-eye view image (hereinafter referred to as a whole bird's-eye view image) overlooking the vehicle and its surroundings is generated and displayed. A vehicle display device is known.

  In such a vehicular display device, when there is an object (hereinafter referred to as an obstacle) that hinders the traveling of the vehicle in an imaging overlap area that is imaged with adjacent in-vehicle cameras, the obstacle is entirely The problem that it is not displayed on a bird's-eye view image or not displayed clearly has been pointed out.

  In order to solve such a problem, there is proposed a vehicle display device that switches to a state in which a captured image captured by each in-vehicle camera is displayed when an obstacle is detected in a state where the whole overhead view image is displayed. (For example, see Patent Document 1).

JP 2007-235529 A

  However, in the vehicular display device described in Patent Document 1, when a captured image captured by each in-vehicle camera is displayed, the vehicle occupant can detect obstacles behind the vehicle only by visually recognizing the captured image. There was a problem that it was difficult to grasp the type and the distance between the obstacle and the vehicle. The in-vehicle camera that is photographed to generate the whole overhead view image needs to photograph the entire periphery of the vehicle with a small number of cameras, and the photographed image by the in-vehicle camera is an image when the vehicle occupant directly sees the obstacle. Because it is very different.

  The present invention has been made in view of these problems, and provides a vehicle display device that makes it easy for a vehicle occupant to know the type of obstacle and the distance between the obstacle and the vehicle. For the purpose.

  The vehicle display device according to claim 1, which has been made to achieve the above object, is a vehicle display device installed in a vehicle, and repeatedly captures a predetermined imaging region set in advance around the vehicle, Mirror conversion image imitating an image of the rear of the vehicle seen through the mirror from the viewpoint of the occupant sitting on a predetermined seat in the vehicle by converting the surrounding photographing image with the photographing means for acquiring the surrounding photographing image An image conversion unit that generates a bird's-eye view image that is an image of the bird's-eye view of the vehicle from the viewpoint above the vehicle and includes the entire vehicle image, and a mirror-converted image, Display means for displaying the bird's-eye view image on the same display screen, the display means bird's-eye view so that the front of the vehicle of the entire image of the vehicle faces upward in a rectangular first display area preset on the display screen Picture Displays, there is laterally rectangular first display area, and the second display area set biased to the lower sides of the rectangle, and displaying the mirror converted image.

  According to the vehicle display device configured as described above, it becomes easy for the vehicle occupant to know the type of the obstacle and the distance between the obstacle and the vehicle.

  Further, in the vehicle display device according to claim 1, as described in claim 2, the display unit displays a mirror frame image showing a frame of the in-vehicle mirror and is inside the frame indicated by the mirror frame image. You may make it display a mirror conversion image.

  In the vehicular display device configured as described above, since the frame of the in-vehicle mirror is displayed around the mirror-converted image, the mirror-converted image corresponds to an image reflected in the in-vehicle mirror. It can be easily grasped by a vehicle occupant.

  Further, in the vehicle display device according to claim 1 or 2, as described in claim 3, the mirror frame image may be displayed so that the mirror frame image is transmitted.

  According to the vehicular display device configured as described above, the image of the boundary portion between the driver mirror image and the image indicating the surrounding area is not hidden behind the frame indicated by the mirror frame image and cannot be seen. .

  Further, in the vehicle display device according to any one of claims 1 to 3, as described in claim 4, the display unit may display the left and right mirror-converted images on the same display screen. .

  According to the vehicular display device configured as described above, a vehicle occupant can simultaneously check the left and right mirror-converted images when viewing one display screen. Therefore, the vehicle occupant does not need to move his / her line of sight, and can easily perform the confirmation work behind the vehicle.

1 is a block diagram showing a configuration of a vehicle display system 1. FIG. It is a flowchart which shows the first half part of a periphery image display process. It is a flowchart which shows the second half part of a periphery image display process. It is a figure which shows the 1st, 2nd, 3rd reference display mode. It is a figure which shows the 4th, 5th, 6th reference display mode. It is a figure which shows the 7th, 8th, 9th reference display mode. It is a figure which shows the display mode of a display apparatus. It is a figure which shows the 10th reference display aspect. It is a block diagram which shows the structure of the display system 1 for vehicles of a 1st reference example. It is a flowchart which shows the periphery image display process of a 1st reference example. It is a figure which shows the approach determination method. It is a figure which shows the display mode of a 1st reference example. It is a block diagram which shows the structure of the display system 1 for vehicles of the 2nd reference example. It is a flowchart which shows the periphery image display process of the 2nd reference example. It is a figure which shows the start condition from parallel parking. It is a figure which shows the display mode of another reference example. It is a figure which shows the display mode of another reference example. It is a figure which shows the moving range MR of a vehicle. It is a figure which shows the condition where a mobile body approaches from the vehicle back at the time of parking. It is a figure which shows the display mode of another reference example.

(First embodiment)
A first embodiment of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a vehicle display system 1 of the present embodiment.

  A vehicle display system 1 is mounted on a vehicle and, as shown in FIG. 1, an in-vehicle camera 2 that continuously captures the periphery of the vehicle, a sensor group 3 that includes a plurality of sensors that detect the state of the vehicle, An image processing device 4 that performs image processing based on an image captured by the camera 2 and a detection result by the sensor group 3 and a display device 5 that displays an image processed by the image processing device 4 are provided.

  The in-vehicle camera 2 includes a front camera 11, a rear camera 12, a left side camera 13, and a right side camera 14. The front camera 11 is installed at the front end of the vehicle (in this embodiment, a front bumper), and photographs the situation in front of the vehicle. The rear camera 12 is installed at the rear end of the vehicle (in this embodiment, the rear bumper) and photographs the situation behind the vehicle. The left side camera 13 is installed at the left end of the vehicle (in this embodiment, the side mirror on the left side of the vehicle), and photographs the situation on the left side of the vehicle. The right side camera 14 is installed at the right end of the vehicle (in this embodiment, the side mirror on the right side of the vehicle), and captures the situation on the right side of the vehicle.

  Furthermore, the sensor group 3 includes a shift lever position sensor 21 and a mirror angle sensor 22. The shift lever position sensor 21 detects the position of the shift lever of the vehicle. The mirror angle sensor 22 detects the inclination angle of the side mirror on the left side of the vehicle (hereinafter referred to as the left side mirror), the side mirror on the right side of the vehicle (hereinafter referred to as the right side mirror), and the rearview mirror in the vehicle interior.

  The image processing apparatus 4 includes a camera interface (hereinafter referred to as camera I / F) 31, a vehicle interface (hereinafter referred to as vehicle I / F) 32, a display interface (hereinafter referred to as display I / F) 33, and a storage unit. 34, the operation part 35, the control part 36, and the bus | bath 37 which connects these.

  The camera I / F 31 is an interface for inputting captured images from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14.

  The vehicle I / F 32 is an interface for inputting signals from the various sensors 21 and 22. The signals from the sensors 21 to 23 may be directly input, or may be input via various ECUs (electronic control devices) mounted on the vehicle.

  The display I / F 33 is an interface for outputting image data processed by the image processing device 4 to the display device 5.

  The storage unit 34 stores various types of information in a nonvolatile storage medium (a hard disk in the present embodiment). The storage unit 34 stores a viewpoint conversion map database 41.

  The viewpoint conversion map database 41 stores a plurality of types of viewpoint conversion maps. Here, the viewpoint conversion map is a map for generating a converted image viewed from a viewpoint different from the viewpoint at the time of shooting by performing coordinate conversion of the shot image of the in-vehicle camera 2. In the present embodiment, the viewpoint conversion map M1 for generating a conversion image viewed from the viewpoint above the vehicle, the left side mirror viewed from the viewpoint of a passenger seated in the driver's seat (hereinafter referred to as the driver). A viewpoint conversion map M2 for generating a conversion image, a viewpoint conversion map M3 for generating a conversion image in which the right side mirror is viewed from the viewpoint of the driver, and a conversion image in which the rearview mirror is viewed from the viewpoint of the driver , A viewpoint conversion map M4 for generating an image, a viewpoint conversion map M5 for generating a converted image obtained by performing coordinate conversion of the captured image of the rear camera 12 in the left-right direction, and a conversion in which the vehicle is viewed from an obliquely upper viewpoint in front of the vehicle A viewpoint conversion map M6 for generating an image is stored.

  The operation unit 35 is used for inputting a command by an external operation from a vehicle occupant, and includes a plurality of keys for external operation.

  The control unit 36 is mainly configured of a microcomputer including a CPU 51, a ROM 52, a RAM 53, and the like. Here, the ROM 52 stores a program for causing the CPU 51 to execute a peripheral image display process described later.

  In the vehicle display system 1 configured as described above, the CPU 51 of the control unit 36 executes a peripheral image display process for displaying an image around the vehicle.

  Here, the procedure of the peripheral image display process executed by the CPU 51 of the vehicle display system 1 will be described with reference to FIGS. FIG. 2 is a flowchart showing the first half of the peripheral image display process, and FIG. 3 is a flowchart showing the second half of the peripheral image display process. This peripheral image display process is a process that is repeatedly executed while the CPU 51 of the control unit 36 is activated (powered on).

  When this peripheral image display process is executed, the CPU 51 first performs an operation for setting the display mode of the display device 5 (hereinafter referred to as a display mode setting operation) via the operation unit 35 in S10. It is determined whether or not.

  In the present embodiment, a total of eleven types of display modes are provided, that is, the display mode illustrated in FIG. 7 and the reference display modes illustrated in FIGS. 4 to 6 and 8. A display mode can be selected by performing a display mode setting operation.

  First, as shown in FIG. 4A, the first reference display mode is an image of the vehicle and its surroundings (hereinafter referred to as a top view image) TV01 from the viewpoint above the vehicle, and a viewpoint from the driver. This is an aspect in which an image (hereinafter referred to as a left side mirror image) LM01 viewed from the left side mirror and an image (hereinafter referred to as a right side mirror image) RM01 viewed from the viewpoint from the driver are displayed. In the first display mode, the top view image TV01 is arranged at the center of the display screen, the left side mirror image LM01 is arranged on the left side of the display screen, and the right side mirror image RM01 is arranged on the right side of the display screen. Further, the display areas of the left side mirror image LM01 and the right side mirror image RM01 are formed in a substantially rectangular shape.

  The second reference display mode is a mode in which a top view image TV02, a left side mirror image LM02, and a right side mirror image RM02 are displayed as shown in FIG. In the second display mode, the top view image TV02 is arranged at the center of the display screen, the left side mirror image LM02 is arranged on the left side of the display screen, and the right side mirror image RM02 is arranged on the right side of the display screen. Further, the display areas of the left side mirror image LM02 and the right side mirror image RM02 are formed in a substantially rectangular shape. Furthermore, on the top view image TV02, a display RDL (RDR) indicating the range of the vehicle peripheral area shown in the left side mirror image LM02 (right side mirror image RM02) is added (enclosed by a dotted line in the figure). See area).

  The third reference display mode is a mode in which a top view image TV03, a left side mirror image LM03, and a right side mirror image RM03 are displayed as shown in FIG. In the third display mode, the top view image TV03 is arranged at the center of the display screen, the left side mirror image LM03 is arranged on the left side of the display screen, and the right side mirror image RM03 is arranged on the right side of the display screen. The left side mirror image LM03 (right side mirror image RM03) includes a mirror frame image FL03 (FR03) showing a frame having the same shape as the side mirror of the vehicle on which the vehicle display system 1 is mounted, and a mirror frame image FL03. (FR03) and a converted image TL03 (TR03) displayed inside.

  In addition, as shown in FIG. 5A, the fourth reference display mode is a vehicle whole image CA04 showing a whole image of a vehicle on which the vehicle display system 1 is mounted, a left side mirror image LM04, and a right side mirror. In this mode, the image RM04 is displayed. In the fourth display mode, the entire vehicle image CA04 is arranged at the center of the display screen, the left side mirror image LM04 is arranged on the left side of the display screen, and the right side mirror image RM04 is arranged on the right side of the display screen. The left side mirror image LM04 (right side mirror image RM04) includes a mirror frame image FL04 (FR04) showing a frame having the same shape as the side mirror of the vehicle on which the vehicle display system 1 is mounted, and a mirror frame image FL04. (FR04) and a converted image TL04 (TR04) displayed inside.

  Further, as shown in FIG. 5B, the fifth reference display mode is that the entire vehicle image CA05, the left side mirror image LM05, the right side mirror image RM05, and the rearview mirror in the vehicle interior are viewed from the viewpoint of the driver. This is a mode of displaying an image (hereinafter referred to as a rearview mirror image) BM05. In the fifth display mode, the entire vehicle image CA05 is displayed at the center of the display screen, the left side mirror image LM05 is displayed on the left side of the display screen, the right side mirror image RM05 is displayed on the right side of the display screen, and the rear view mirror image BM05 is displayed on the upper side of the display screen. Is placed. The left side mirror image LM05 (right side mirror image RM05) includes a mirror frame image FL05 (FR05) showing a frame having the same shape as the side mirror of the vehicle on which the vehicle display system 1 is mounted, and a mirror frame image FL05. (FR05) and a converted image TL05 (TR05) displayed inside. Further, the rearview mirror image BM05 includes a mirror frame image FB05 showing a frame having the same shape as the rearview mirror of the vehicle on which the vehicle display system 1 is mounted, and a converted image TB05 displayed inside the mirror frame image FB05. Is done.

  The sixth reference display mode is a mode in which a top view image TV06, a left side mirror image LM06, a right side mirror image RM06, and a rearview mirror image BM06 are displayed as shown in FIG. In this sixth display mode, the top view image TV06 is in the center of the display screen, the left side mirror image LM06 is on the left side of the display screen, the right side mirror image RM06 is on the right side of the display screen, and the rearview image BM06 is on the upper side of the display screen. Is placed. Further, the display areas of the left side mirror image LM06 and the right side mirror image RM06 are formed in a substantially rectangular shape. Further, the rearview mirror image BM06 includes a mirror frame image FB06 showing a frame having the same shape as the rearview mirror of the vehicle on which the vehicle display system 1 is mounted, and a converted image TB06 displayed inside the mirror frame image FB06. Is done.

  The seventh reference display mode is a mode in which a top view image TV07, a left side mirror image LM07, a right side mirror image RM07, and a rearview mirror image BM07 are displayed as shown in FIG. In the seventh display mode, the top view image TV07 is in the center of the display screen, the left side mirror image LM07 is on the left side of the display screen, the right side mirror image RM07 is on the right side of the display screen, and the rearview mirror image is on the lower side of the display screen. BM07 is arranged. Further, the display areas of the left side mirror image LM07, the right side mirror image RM07, and the rearview mirror image BM07 are formed in a substantially rectangular shape.

  In addition, as shown in FIG. 6B, the eighth reference display mode includes an image RV08 obtained by coordinate-converting the image captured by the rear camera 12 in the left-right reverse direction (hereinafter referred to as a rear left-right reverse-converted image), and a left side mirror image. This is a mode in which the LM08 and the right side mirror image RM08 are displayed. In the eighth display mode, a rear left / right reverse conversion image RV08 is arranged at the center of the display screen, a left side mirror image LM08 is arranged on the left side of the display screen, and a right side mirror image RM08 is arranged on the right side of the display screen. Further, the display areas of the left side mirror image LM08 and the right side mirror image RM08 are formed in a substantially rectangular shape.

  In addition, as shown in FIG. 6C, the ninth reference display mode includes an image (hereinafter referred to as a top rear view image) TR09 viewed from an obliquely upper viewpoint in front of the vehicle, a left side mirror image LM09, This is a mode in which the right side mirror image RM09 is displayed. In the ninth display mode, the top rear view image TR09 is arranged at the center of the display screen, the left side mirror image LM09 is arranged on the left side of the display screen, and the right side mirror image RM09 is arranged on the right side of the display screen. In addition, the display areas of the left side mirror image LM09 and the right side mirror image RM09 are formed in a substantially rectangular shape.

  Here, the display mode in the present embodiment is a mode in which a top view image TV10, a left side mirror image LM10, and a right side mirror image RM10 are displayed as shown in FIG. In the tenth display mode, the top view image TV10 is arranged at the center of the display screen, the left side mirror image LM10 is arranged on the left side of the display screen, and the right side mirror image RM10 is arranged on the right side of the display screen. Further, the left side mirror image LM10 (right side mirror image RM10) is a conversion obtained by performing coordinate conversion on a captured image of the left side camera 13 (right side camera 14) from a viewpoint viewed by the driver through the left side mirror (right side mirror). An image (hereinafter, referred to as a side mirror viewpoint conversion image) TL10 (TR10) and a frame having the same shape as the side mirror of the vehicle on which the vehicle display system 1 is mounted are shown as transmitting through the conversion image. It consists of a mirror frame image FL10 (FR10). That is, in the side mirror viewpoint conversion image TL10 (TR10), the image in the frame of the mirror frame image FL10 (FR10) corresponds to the scenery that the driver visually recognizes through the side mirror.

  In addition, the tenth reference display mode is a mode in which the display is changed in three stages as shown in FIGS. That is, as shown in FIG. 8 (a), only the top view image TV11 is displayed at the center of the display screen. When a preset display start condition is satisfied, as shown in FIG. 8B, the top view image TV11, the left side mirror image LM11, and the right side mirror image RM11 are displayed. Thereafter, when a preset display start time (for example, 5 seconds) elapses, as shown in FIG. 8C, instead of the left side mirror image LM11 and the right side mirror image RM11, the side mirror viewpoint converted image TL11 is displayed. And TR11 are displayed.

  As shown in FIG. 2, when it is determined in S10 that the display mode setting operation has not been performed (S10: NO), the process proceeds to S30. On the other hand, if it is determined that the display mode setting operation has been performed (S10: YES), display setting is performed according to the display mode instructed by the display mode setting operation in S20, and the process proceeds to S30. For example, when the display mode setting operation indicates the first display mode, the display mode is set to display in the first display mode.

  In S30, based on the signal from the shift lever position sensor 21, it is determined whether or not the position of the shift lever is reverse “R”. Here, when the position of the shift lever is not reverse “R” (S30: NO), the process proceeds to S160. On the other hand, when the position of the shift lever is reverse “R” (S30: YES), in S40, it is determined whether or not the display mode set in S20 displays a top view image. That is, it is determined whether or not the display mode of FIG. 7 and the first, second, third, sixth, seventh and tenth reference display modes are set.

  Here, when the display mode is to display a top view image (S40: YES), in S50, captured images are taken from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. get.

  In S60, a top view image is generated using the captured image acquired in S50. Specifically, first, the viewpoint conversion map M1 stored in the viewpoint conversion map database 41 is stored in each of the four captured images acquired from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. Use to convert coordinates. Thereafter, for the four converted images that have been subjected to the coordinate conversion process, a combining process for joining the overlapping portions of the four converted images is performed. Thereby, the top view image of 1 sheet which looked down at the vehicle and the whole circumference | surroundings is produced | generated. In S70, the top view image generated in S60 is displayed on the display device 5 in the display mode set in S20, and the process proceeds to S160.

  In S40, if the display mode does not display the top view image (S40: NO), it is determined in S80 whether or not the display mode is to display the rear left / right reverse conversion image. That is, it is determined whether or not the eighth display mode is set. Here, when the display mode is to display a backward left / right reversely converted image (S80: YES), a captured image is acquired from the rear camera 12 in S90.

  In S100, a backward left / right inversely transformed image is generated using the captured image acquired in S90. Specifically, the captured image acquired from the rear camera 12 is coordinate-converted using the viewpoint conversion map M5 stored in the viewpoint conversion map database 41. Thereby, one back left-right reverse conversion image is produced | generated. Further, in S110, the backward left / right inversely converted image generated in S100 is displayed on the display device 5 in the display mode set in S20, and the process proceeds to S160.

  In S80, if the display mode does not display the backward left / right reverse conversion image (S80: NO), it is determined in S120 whether or not the display mode displays a top rear view image. That is, it is determined whether or not the ninth display mode is set. If the display mode is to display the top rear view image (S120: YES), the captured image is acquired from the rear camera 12, the left side camera 13, and the right side camera 14 in S130.

  In S140, a top rear view image is generated using the captured image acquired in S130. Specifically, first, coordinate conversion is performed on the three captured images acquired from the rear camera 12, the left camera 13, and the right camera 14 using the viewpoint conversion map M 6 stored in the viewpoint conversion map database 41. To do. Thereafter, for the three converted images that have been subjected to the coordinate conversion process, a combining process for connecting overlapping portions of the three converted images is performed. Thereby, a top rear view image is generated. Further, in S150, the top rear view image generated in S140 is displayed on the display device 5 in the display mode set in S20, and the process proceeds to S160.

  In S120, when the display mode does not display the top rear view image (S120: NO), the process proceeds to S160.

  Then, when the process proceeds to S160, as shown in FIG. 3, captured images are acquired from the rear camera 12, the left side camera 13, and the right side camera 14. Thereafter, in S170, using the captured image acquired in S160, a converted image obtained by viewing the left side mirror from the viewpoint of the driver (hereinafter referred to as a left side mirror converted image), and the right side mirror from the viewpoint of the driver. A converted image that has been viewed (hereinafter, referred to as a right side mirror converted image) and a converted image in which the rearview mirror is viewed from the viewpoint of the driver (hereinafter referred to as a rearview mirror converted image) are generated. Specifically, the three captured images acquired from the left side camera 13, the right side camera 14, and the rear camera 12 are respectively converted into a viewpoint conversion map M2, a viewpoint conversion map M3, and a viewpoint conversion map database 41. Coordinate conversion is performed based on the viewpoint conversion map M4 and the detection result of the mirror angle sensor 22. Accordingly, the left side mirror converted image, the right side mirror converted image, and the rearview mirror converted image change in conjunction with changes in the tilt angles of the left side mirror, the right side mirror, and the rearview mirror.

  Thereafter, in S180, the left side mirror converted image, right side mirror converted image, and rearview mirror converted image generated in S170 are processed in accordance with the display mode set in S20. For example, when the fifth display mode is set, the mirror frame image FL05, the mirror frame image FR05, and the mirror are respectively applied to the left side mirror converted image, the right side mirror converted image, and the rearview mirror converted image. Processing for adding the frame image FB05 is performed to generate a left side mirror image LM05, a right side mirror image RM05, and a rearview mirror image BM05.

  In S190, the image generated in S180 is displayed on the display device 5, and the peripheral image display process is temporarily terminated. In S190, in the case of the fourth and fifth display modes, the entire vehicle images CA04 and CA05 are additionally displayed.

  In the vehicle display system 1 configured as described above, first, the rear, left side, and right side of the vehicle are repeatedly photographed by the rear camera 12, the left side camera 13, and the right side camera 14, and the captured images (hereinafter, referred to as the following) ) (Referred to as a surrounding image) (S160). Then, from the viewpoint of the occupant (driver) sitting in the driver's seat, the left side mirror, the right side mirror, and the back that are visually recognized by the driver when viewing the left side mirror, the right side mirror, and the rear view mirror. A left side mirror converted image, a right side mirror converted image, and a rearview mirror converted image showing a mirror image are generated by coordinate-converting the acquired surrounding captured image (S170). Thereafter, the left side mirror converted image, the right side mirror converted image, and the rearview mirror converted image are displayed (S190). Hereinafter, the left side mirror, the right side mirror, and the rearview mirror are collectively referred to as an in-vehicle mirror. Further, the left side mirror converted image, the right side mirror converted image, and the rearview mirror converted image are collectively referred to as a mirror converted image.

  According to the vehicle display system 1 configured as described above, it is possible to display an image of the in-vehicle mirror when the in-vehicle mirror is viewed from the driver's viewpoint. In addition, since the driver usually confirms the obstacle behind the vehicle by the image reflected in the in-vehicle mirror, the driver views the type of the obstacle behind the vehicle and the obstacle by looking at the image reflected in the in-vehicle mirror. Know the distance between the vehicle and the vehicle. That is, the vehicle display system 1 makes it easy for the driver to grasp the type of obstacle behind the vehicle and the distance between the obstacle and the vehicle without visually recognizing the in-vehicle mirror. it can.

  In the display mode of FIG. 7 and the third, fourth, fifth, and tenth reference display modes, an image (mirror frame image) showing a frame of an in-vehicle mirror is displayed and a mirror conversion image is displayed inside the mirror frame image. To do. In other words, since the frame of the in-vehicle mirror is displayed around the mirror-converted image, it is possible for the vehicle occupant to easily grasp that the mirror-converted image corresponds to an image reflected in the in-vehicle mirror. it can.

  In the display mode of FIG. 7 and the third, fourth, and fifth reference display modes, the mirror frame image is an image showing the same frame as the frame of the in-vehicle mirror. For this reason, it is possible to make it easier for the vehicle occupant to understand that the mirror-converted image corresponds to an image reflected on the in-vehicle mirror.

  In the display mode of FIG. 7, from the viewpoint of the driver, an image of the left side mirror (right side mirror) visually recognized by the driver when the left side mirror (right side mirror) is viewed (hereinafter referred to as driver mirror image). ) And an image showing an area around the imaging area arranged around the image and corresponding to the driver mirror image. Among the left side mirror converted image (right side mirror converted image), The part corresponding to the driver mirror image is displayed inside the frame indicated by the mirror frame image, and the frame indicated by the mirror frame image is transmitted through the left side mirror converted image (right side mirror converted image). Display a mirror frame image. For this reason, since the image of the in-vehicle mirror (driver mirror image) when viewing the in-vehicle mirror from the driver's viewpoint and the image of the surrounding area are displayed, the driver can see the in-vehicle mirror. You can see a wider area than you do. Further, the frame indicated by the mirror frame image is transmitted through the mirror converted image. For this reason, the image of the boundary portion between the driver mirror image and the image indicating the surrounding area is not hidden behind the frame indicated by the mirror frame image.

  In addition, a left side mirror, a right side mirror, and a rearview mirror are attached to the vehicle, and a plurality of driver mirror images corresponding to each of the left side mirror, the right side mirror, and the rearview mirror are provided. A plurality of mirror-converted images showing the respective mirror images are displayed on the same display screen. For this reason, if the passenger | crew of a vehicle sees one display screen, he can confirm simultaneously the image corresponding to each of a left side mirror, a right side mirror, and a rear-view mirror. Therefore, it is not necessary for the vehicle occupant to move the line of sight from one in-vehicle mirror to another in-vehicle mirror in order to confirm the images reflected on the plurality of in-vehicle mirrors, so that the confirmation work behind the vehicle can be easily performed. Can do.

  Further, in the fifth and sixth reference display modes, on the same display screen, the left side mirror converted image, the right side mirror converted image, and the left side mirror, the right side mirror, and the rearview mirror are matched to each other. And a rearview mirror conversion image is arranged. For this reason, it is easy for the vehicle occupant to determine which of the plurality of mirror-converted images displayed on the same display screen corresponds to each of the plurality of in-vehicle mirrors attached to the vehicle. It can be grasped.

  Further, in the second reference display mode, display RDL and RDR indicating where the areas shown in the left side mirror converted image and the right side mirror converted image correspond to the area around the vehicle are performed. For this reason, it is possible for the vehicle occupant to easily grasp the regions shown in the left side mirror converted image and the right side mirror converted image.

  Further, the in-vehicle mirror is configured to be able to adjust the tilt angle, and the mirror angle sensor 22 detects the tilt angle of the in-vehicle mirror. Then, by performing coordinate conversion according to the angle detected by the mirror angle sensor 22, a mirror converted image is generated (S170). For this reason, the image which looked at the in-vehicle mirror in the inclination angle from the viewpoint of the driver can be displayed in conjunction with the change in the inclination angle of the in-vehicle mirror.

  In the eighth reference display mode, the image captured by the rear camera 12 is displayed on the same display screen as the mirror-converted image (S90, S110). Therefore, the vehicle occupant can simultaneously check the image corresponding to the in-vehicle mirror and the image taken by the rear camera 12 by looking at one display screen.

  In the eighth reference display mode, an image that is reversed from the image captured by the rear camera 12 is generated (S100), and this image is displayed on the same display screen as the mirror-converted image (S110). . For this reason, when the image image | photographed with the rear camera 12 is displayed, the displayed image turns into an image in which right and left corresponded with the passenger | crew of the vehicle.

  In the display mode of FIG. 7 and the first, second, third, sixth, seventh, and tenth reference display modes, a top view image that overlooks the vehicle and the entire surroundings using the captured image captured by the in-vehicle camera 2 is displayed. The top view image is generated and displayed on the same display screen as the mirror converted image. For this reason, if the passenger | crew of a vehicle sees one display screen, he can confirm the image corresponding to a vehicle-mounted mirror, and a top view image simultaneously.

  In the embodiment described above, the vehicle display system 1 is the vehicle display device according to the present invention, the in-vehicle camera 2 and the processes of S50, S90, and S130 are the photographing means according to the present invention, and the process of S170 is the image converting means according to the present invention. The left side mirror, the right side mirror, and the rearview mirror are in-vehicle mirrors in the present invention, the processes in S70, S110, and S190 are display means in the present invention, the mirror angle sensor 22 is the mirror angle detection means in the present invention, and the processes in S100 are The left and right reverse image generating means in the present invention, the process of S60, is the overhead image generating means in the present invention.

  Further, the front, rear, left side, and right side of the vehicle are predetermined shooting areas in the present invention, and the driver is a predetermined seat occupant in the present invention.

  The present invention is not limited to the above-described embodiments, and can take various forms as long as they belong to the technical scope of the present invention.

  For example, in the above-described embodiment, the display of the image viewed from the mirror from the viewpoint of the driver is shown, but the image viewed from the mirror from the viewpoint of the passenger sitting in a seat other than the driver's seat is displayed. Also good.

(First Reference Example)
A first reference example will be described below. In the first reference example, only the parts different from the above embodiment will be described.

  The vehicle display system 1 of the first reference example is the same as that of the first embodiment except that the configuration of the vehicle display system 1 and the peripheral image display process are changed.

  FIG. 9 is a block diagram showing the configuration of the vehicle display system 1 of the first reference example.

  As shown in FIG. 9, the vehicle display system 1 of the first reference example includes a clearance sonar 23 that detects the presence and distance of an object by transmitting an ultrasonic wave as a detection wave and receiving the reflected wave. Except for the point added to 3, it is the same as the first embodiment.

  Next, the procedure of the peripheral image display process of the first reference example will be described with reference to FIG. FIG. 10 is a flowchart showing the peripheral image display process of the first reference example.

  When this peripheral image display processing is executed, the CPU 51 first performs an operation for setting the display range of the side mirror image (hereinafter referred to as display range setting operation) via the operation unit 35 in S310. It is determined whether or not. If it is determined that the display range setting operation has not been performed (S310: NO), the process proceeds to S330. On the other hand, when it is determined that the display range setting operation has been performed (S10: YES), display setting is performed according to the display range instructed by the display range setting operation in S320, and the process proceeds to S330.

  In S330, it is determined whether or not an operation for setting the tilt angles of the left side mirror and the right side mirror (hereinafter referred to as tilt angle setting operation) has been performed via the operation unit 35. If it is determined that the tilt angle setting operation has not been performed (S330: NO), the process proceeds to S350. On the other hand, if it is determined that the tilt angle setting operation has been performed (S330: YES), display setting is performed according to the tilt angle instructed by the tilt angle setting operation in S340, and the process proceeds to S350.

  In S350, based on the signal from the shift lever position sensor 21, it is determined whether or not the position of the shift lever is reverse “R”. Here, when the position of the shift lever is not reverse “R” (S350: NO), the peripheral image display process is temporarily ended. On the other hand, if the position of the shift lever is reverse “R” (S350: YES), the captured image is acquired from the front camera 11, the rear camera 12, the left camera 13, and the right camera 14 in S360. .

  In S370, a top view image is generated using the captured image acquired in S360. Specifically, first, the viewpoint conversion map M1 stored in the viewpoint conversion map database 41 is stored in each of the four captured images acquired from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. Use to convert coordinates. Thereafter, for the four converted images that have been subjected to the coordinate conversion process, a combining process for joining the overlapping portions of the four converted images is performed. Thereby, the top view image of 1 sheet which looked down at the vehicle and the whole circumference | surroundings is produced | generated. In step S380, the top view image generated in step S370 is displayed on the display device 5.

  Thereafter, in S390, based on the detection result of the clearance sonar 23, it is determined whether or not the object is approaching the vehicle and the risk of the approach.

  Specifically, when the object is a moving body, as shown in FIG. 11A, based on the speed Vm at which the moving body approaches the vehicle and the distance Lm between the vehicle and the moving body. The time required for the moving object to reach the vehicle (hereinafter referred to as arrival time) is calculated, and four types of "emergency", "danger", "caution", and "safety" are calculated according to this calculation time. The risk level is set. That is, when the arrival time is less than the first determination time T1 (for example, 1 second), it is determined as “emergency”, and the arrival time is less than the second determination time T2 (for example, 3 seconds) that is longer than the first determination time T1. In some cases, it is determined as “dangerous”, and when the arrival time is less than a third determination time T3 (for example, 5 seconds) longer than the second determination time T2, it is determined as “caution”. Further, when the arrival time is equal to or longer than the third determination time T3 or the moving body is moving away, it is determined as “safe”.

  When the object is a fixed object, as shown in FIG. 11B, “emergency”, “danger”, “caution”, and “depending on the distance Lm between the vehicle and the fixed object” There are four types of danger, “safe”. That is, when a fixed object is approaching and the distance Lm is less than the first determination distance L1 (for example, 1 m), it is determined as “emergency”, and the second determination distance L2 is longer than the first determination distance L1. When the distance is less than (for example, 3 m), it is determined as “danger”, and when the distance Lm is less than the third determination distance L3 (for example, 5 m) that is longer than the second determination distance L2, it is determined as “caution”. Further, when the distance Lm is equal to or greater than the third determination distance L3 or the fixed object is moving away, it is determined as “safe”.

  In S400, it is determined whether or not “safe” is determined in S390. Here, when it is determined as “safe” (S400: YES), the peripheral image display process is temporarily ended.

  On the other hand, when it is not determined to be “safe” (S400: NO), a captured image is acquired from the left side camera 13 and the right side camera 14 in S410. Thereafter, in S420, using the captured image acquired in S410, a converted image in which the left side mirror is viewed from the viewpoint from the driver (hereinafter referred to as a left side mirror converted image), and a right side mirror from the viewpoint from the driver. A converted image (hereinafter referred to as a right side mirror converted image) is generated. Specifically, the two captured images acquired from the left side camera 13 and the right side camera 14 are set in the viewpoint conversion map M2, the viewpoint conversion map M3, and the S340 stored in the viewpoint conversion map database 41, respectively. Coordinate conversion is performed based on the tilt angle. Therefore, the left side mirror converted image and the right side mirror converted image change in conjunction with the change in the tilt angle set in S340.

  Thereafter, in S430, the left side mirror converted image and the right side mirror converted image generated in S420 are displayed according to the display range set in S320 and the risk determined in S390, as shown in FIG. Process.

  Specifically, first, in accordance with the display range set in S320, the left side mirror converted image and the right side mirror converted image generated in S420 are cut out and fitted into the mirror frame image. That is, depending on the display range set in S340, the display range of the mirror-converted image is different from the range actually reflected on the side mirror. Thereafter, when it is determined as “attention” in S390, as shown in FIG. 12A, the side mirror image SMG01 is processed to be displayed next to the top view image TVG (in the drawing, the right side mirror is displayed). Only images are shown). This side mirror image SMG01 approaches a mirror frame image MFG01 showing the frame of the side mirror, a mirror conversion image MTG01 displayed inside the mirror frame image MFG01, and an object indication image OPG01 showing an approaching object. And a direction indication image DPG01 showing the moving direction of the moving object. The object instruction image OPG01 and the direction instruction image DPG01 are displayed on the mirror conversion image MTG01.

  If it is determined as “attention” in S390, the side mirror image SMG02 is processed to be displayed beside the top view image TVG as shown in FIG. The side mirror image SMG02 includes a mirror frame image MFG02, a mirror conversion image MTG02, an object instruction image OPG02, and a direction instruction image DPG02. The side mirror image SMG02 is displayed in a darker color and larger than the mirror frame image MFG01. Further, the object instruction image OPG02 and the direction instruction image DPG02 are displayed larger on the mirror conversion image MTG02 than the object instruction image OPG01 and the direction instruction image DPG01.

  Further, when it is determined as “emergency” in S390, the side mirror image SMG03 is processed to be displayed beside the top view image TVG as shown in FIG. The side mirror image SMG03 includes a mirror frame image MFG03, a mirror conversion image MTG03, an object instruction image OPG03, and a direction instruction image DPG03. The side mirror image SMG03 is displayed larger than the mirror frame image MFG02. Further, the object instruction image OPG03 and the direction instruction image DPG03 are displayed larger on the mirror conversion image MTG03 than the object instruction image OPG02 and the direction instruction image DPG02. Further, the object instruction image OPG01 and 02 and the direction instruction image DPG01 and 02 are planar display, whereas the object instruction image OPG03 and the direction instruction image DPG03 are stereoscopic display.

  When the process of S430 is completed, the image generated in S430 is displayed on the display device 5 in S440, and the peripheral image display process is temporarily ended.

  In the vehicular display system 1 configured as described above, when an operation for setting the display range of the side mirror image (display range setting operation) is performed, display corresponding to the display range instructed by the display range setting operation is performed. Settings are made (S310, S320). For this reason, it becomes possible to change the display range of a mirror conversion image by operation of the passenger | crew of a vehicle.

  When an operation for setting the tilt angles of the left side mirror and the right side mirror (tilt angle setting operation) is performed, display setting is performed according to the tilt angle instructed by the tilt angle setting operation (S330, S340). ). Then, when it is assumed that the left side mirror and the right side mirror have the set inclination angles, the left side mirror 13 and the right side camera are images of the left side mirror and the right side mirror that are visually recognized by the driver. 14 is generated by coordinate-transforming the captured image acquired from 14 (S420). For this reason, when it is assumed that the inclination angle of the left side mirror and the right side mirror is the set inclination angle, the above-mentioned inclination angle is set to the vehicle occupant for the image of the in-vehicle mirror visually recognized by the driver. It can be changed by the operation of.

  Further, an object approaching the vehicle is detected by the clearance sonar 23 (S390), and a mirror conversion image is displayed when an object approaching the vehicle is detected (S400, S440). For this reason, when an object approaches the vehicle, a mirror conversion image is automatically displayed. Therefore, by displaying the mirror conversion image, the vehicle occupant can be informed that the object is approaching the vehicle. .

  In the reference example described above, the process of S420 is the image conversion means in this reference example, the process of S440 is the display means in this reference example, the processes of S310 and S320 are the first setting means in this reference example, and the processes of S330 and S340. Is the second setting means in this reference example, the clearance sonar 23 and processing of S390 are the approach detection means in this reference example, the display range setting operation is the first external operation in this reference example, and the tilt angle setting operation is the first setting in this reference example. 2 External operation.

(Second reference example)
A second reference example will be described below. In the second reference example, only the parts different from the first reference example will be described.

  The vehicle display system 1 of the second reference example is the same as that of the second embodiment except that the configuration of the vehicle display system 1 and the peripheral image display process are changed.

  FIG. 13 is a block diagram illustrating a configuration of the vehicle display system 1 of the second reference example.

  As shown in FIG. 13, the vehicle display system 1 according to the second reference example includes a vehicle speed sensor 24 that detects the traveling speed of the vehicle and a winker sensor 25 that detects on / off of the winker of the vehicle. Except for these points, the second reference example is the same as the first reference example.

  Next, the procedure of the peripheral image display process of the second reference example will be described with reference to FIG. FIG. 14 is a flowchart showing the peripheral image display process of the second reference example.

  When the peripheral image display process is executed, first, in S510, the CPU 51 determines whether or not the vehicle on which the vehicle display system 1 is mounted is stopped based on the detection result of the vehicle speed sensor 24. . Here, if the vehicle is not stopped (S510: NO), the peripheral image display process is temporarily ended. On the other hand, when the vehicle is stopped (S510: YES), it is determined in S520 whether or not the position of the shift lever is the drive “D” based on the signal from the shift lever position sensor 21.

  Here, when the position of the shift lever is not the drive “D” (S520: NO), the peripheral image display process is temporarily ended. On the other hand, when the position of the shift lever is the drive “D” (S520: YES), a captured image is acquired from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 in S530. .

  In step S540, a top view image is generated using the captured image acquired in step S530. Further, in S550, the top view image generated in S540 is displayed on the display device 5.

  Thereafter, in S560, similarly to S390, based on the detection result of the clearance sonar 23, it is determined whether or not the object is approaching the vehicle and the risk of the approach. In S570, it is determined whether or not “safe” is determined in S560. Here, if it is determined as “safe” (S570: YES), it is determined in S580 whether or not the winker is on based on the detection result of the winker sensor 25.

  If the turn signal is off (S580: NO), the peripheral image display process is temporarily terminated. On the other hand, when the turn signal is on (S580: YES), the process proceeds to S590. In S570, when it is not determined as “safe” in S560 (S570: NO), the process proceeds to S590.

  In S590, the captured images are acquired from the left side camera 13 and the right side camera 14. Thereafter, in S600, a left side mirror converted image and a right side mirror converted image are generated using the captured image acquired in S590.

  Further, in S610, similarly to S430, the left side mirror converted image and the right side mirror converted image generated in S600 are processed according to the risk determined in S560. In step S620, the image generated in step S610 is displayed on the display device 5, and the peripheral image display process is temporarily terminated.

  In the vehicle display system 1 configured as described above, the vehicle speed sensor 24 detects the traveling speed of the vehicle, the shift lever position sensor 21 detects the position of the shift lever of the vehicle, and the turn signal sensor 25 turns on the turn signal of the vehicle.・ Detects OFF. Then, based on the detection results of the vehicle speed sensor 24, the shift lever position sensor 21, and the winker sensor 25, when the vehicle stops, the position of the shift lever is the drive “D”, and the winker is on, the mirror The converted image is displayed (S510, S520, S580, S620). For this reason, for example, as shown in FIG. 15, in a situation where the vehicle needs to be confirmed from behind while entering the roadway from the vehicle stop state, an image when the vehicle-mounted mirror is viewed is displayed as an operation or instruction of the vehicle occupant. It can be displayed automatically without depending on.

  In the reference example described above, the process of S600 is the image conversion means in this reference example, the process of S620 is the display means in this reference example, the vehicle speed sensor 24 is the vehicle speed detection means in this reference example, and the shift lever position sensor 21 is this reference. The shift position detecting means and the winker sensor 25 in the example are the winker detecting means in the present reference example.

  16 to 20 are diagrams for explaining another reference example.

  As shown by the side mirror image SMG11 of FIG. 16, by displaying up to the vanishing point VP of the road, a distant obstacle may be reliably displayed. Further, the highlight color may be changed according to the approaching speed of the moving body, or a display that emphasizes the moving speed, such as a motion blur display, may be performed.

  Further, the background of the side mirror may be fixed and displayed even when the vehicle moves. Thereby, the user can intuitively grasp the moving speed of the object. Also, auxiliary information such as distance may be added to the side mirror image SMG11 in order to make it easier for the driver to know the position of the object.

  Further, as shown by the rearview mirror image BMG11 in FIG. 16, the display range may be expanded to the area where the approaching moving body is shown. In the rearview mirror image BMG11, since the moving body is approaching from the right side of the display area (see arrow Y11), the right display area of the rearview mirror image BMG11 is extended.

  In addition, as shown by the rearview mirror image BMG11 in FIG. 16, when there is an object that is an indication of parking, in order to make it easier for the driver to recognize this object, a three-dimensional highlight display or a rearview mirror image is performed. You may make it protrude from BMG11 (refer display object OB11). For the above three-dimensional highlighting, the shape is taken as a three-dimensional coordinate from the usual photographed image, the surface material and the like are pasted as a texture, recorded as a virtual reality (VR) image, and recorded on the vehicle. Depending on the moving position, it may be displayed so that the driver can reliably use it as an indication of parking. As a result, an object that is a guide during fine weather is clearly displayed even at night or in the rain, so that the driver can park the vehicle safely.

  Further, as shown by a top view image TVG11 in FIG. 16, when a fixed object such as a pole approaches within a set distance, the fixed object may be displayed in an easy-to-see manner on the top view image (pole display PD11 is displayed). reference).

  Further, in the above reference example, when the position of the shift lever is reverse “R” and the object is approaching, for example, as shown in FIG. 17A, the top view image TVG21, the left side mirror image SMG21, and the right side The display of the mirror image SMG22 is shown. However, when an object approaches from the left rear side (right rear side) of the vehicle, as shown in FIG. 17B, the top view image TVG22 and the left side mirror image SMG23 are displayed and the left side mirror image SMG23 is displayed. You may make it display in an enlarged manner. Further, when an object is approaching from the right rear side of the vehicle, as shown in FIG. 17C, the top view image TVG23 and the right side mirror image SMG24 are displayed and the right side mirror image SMG24 is displayed in an enlarged manner. It may be.

  In the above example, as shown in FIG. 11, the risk is set based on the time required for the moving body to reach the vehicle (arrival time) and the distance between the vehicle and the fixed object. showed that. However, the risk may be set by changing the distance according to the direction of the approaching object. Further, the degree of danger may be set according to the relative speed and the moving direction of the vehicle. Further, the degree of danger may be set according to the angle of the object with respect to the traveling direction of the vehicle.

  In addition, you may make it set said risk based on the following references | standards. That is, when an object is already in the moving range MR (described later) of the vehicle, it is “emergency”. When an object is about to enter the moving range MR, it is “dangerous”. “Caution” may be set when the object is present, and “safe” may be set when the object is not approaching the moving range MR. Here, as shown in FIG. 18, the moving range MR is, when moving from the stop position SP to the designated parking position PP, the parking moving speed is set to, for example, 0.5 m / sec. Calculate whether or not to do.

  Note that the moving range MR may be obtained by changing the parking moving speed in accordance with the driving skill of the driver. For example, a skilled driver may increase the parking movement speed and obtain the movement range MR because the vehicle movement speed during parking may be large and the vehicle can be parked in a short time. Further, if the warning / risk notification timing by displaying the side mirror image can be changed according to the parking moving speed, the delay of the notification timing is reduced and the driver's troublesomeness due to the notification is reduced.

  In addition, regarding the approach determination of the fixed object, an object having a set size or more may be targeted. In addition, when there is a fixed object that the driver normally uses as a guide for parking, the fixed object may be displayed even if the fixed object does not exist within the movement range MR. For this reason, in a parking lot where the number of times of use is high, it is preferable that the user can designate an object that is a guide for parking. For example, you may enable it to set using the image data on which the parking frame, the wheel stopper, the side wall, etc. were displayed.

  In addition, when it is detected that an object is likely to contact the vehicle when the vehicle is parked, a side mirror image of the side that is likely to contact (for example, left if the pedestrian is likely to contact the left side of the vehicle) An enlarged display and a reduced display of the shape of the mirror frame in the side mirror image) and voice notification (for example, a sound that causes a crack) may be performed to notify the occupant that the situation is an emergency. Simultaneously with the enlarged display and reduced display of the shape of the mirror frame, the engine control or brake control device performs processing for emergency stop so that when the object comes into contact with the vehicle, the vehicle is brought into a state close to the stop state. You may make it do.

  Further, when an object comes into contact with the side mirror, a moving image that closes the side mirror may be displayed to notify contact with a side obstacle.

  In addition, when the door is opened after completion of parking, there is a possibility that the side mirror will display an image that changes to the door when there is a possibility that it will come into contact with a fixed object or moving body on the side, so that attention of contact may be urged. . Similarly, when the trunk door is likely to come into contact with the rear of the vehicle, a display that calls attention to the trunk door contact may be added to the side mirror image instead of the rearview mirror image.

  In addition, as shown in FIG. 19, when an automobile, a bicycle, a pedestrian, etc. approach from the rear of the vehicle when the vehicle is moved backward by 90 degrees and parked in the parking area, these approaching objects are It is difficult to see from the field of view and is not visible with the side mirror and rearview mirror For this reason, when the object located in the range which cannot be seen with a vehicle-mounted mirror approaches, a side mirror image and a rearview mirror image may be produced | generated using the image | photographed image by the vehicle-mounted camera 2, and you may make it display on the display apparatus 5. FIG. . As a result, the driver can be informed of the danger.

  For example, when the vehicle is located at the position P1 for stopping before parking in the parking section PKA, it is detected that the moving body is approaching from the left side of the roadway and sidewalk behind the vehicle. The left side mirror image may be displayed, and the target object may be highlighted on the left side mirror image according to the detected position and speed of the object.

  Further, when the vehicle is positioned at a position P2 where the vehicle is turned to park in the parking area PKA, a side mirror image is displayed so that the road (background) on which the moving body moves is almost fixed. It may be. That is, an image that is not affected by the turning of the vehicle is displayed. This facilitates the approach position determination of which roads and sidewalks have obstacles and pedestrians and approaches or moves away from them. For this purpose, the image taken by the in-vehicle camera 2 is rotationally corrected, or if it is a previously parked position, the road image recorded at that time is used as a fixed image, and the detected obstacle, pedestrian, and You may make it display a mobile body superimposed. Further, it is determined by image, sonar, etc. in which range the vehicle is in the parking area PKA, and when a driving correction is required, a side mirror image is displayed (for example, a side mirror in a contacting direction is displayed). You may make it perform a caution display (like a human ear).

  In addition, when the vehicle is moved backward to enter the parking area PKA and crosses a sidewalk or the like, if a pedestrian or bicycle is likely to pass or is passing through the sidewalk, a display showing the approach of the moving body is displayed as a side mirror image. You may make it add to. In addition, when a pedestrian or the like is about to pass through the parking area PKA, walk to a side mirror image in a direction in which the pedestrian approaches (for example, a left side mirror image when the pedestrian approaches from the left side of the vehicle). You may make it add the display which shows a person's position and distance. Further, when there is a danger of collision, an alarm for notifying the collision may be issued.

  In addition, when the vehicle is located at the position P3 where the vehicle is accommodated in the parking section PKA and there is a wheel stop behind the vehicle, a notification is made that the wheel contact is made. Good. In addition, when there is a possibility of coming into contact with a vehicle in the rear section even if there is a wheel stopper, the distance to the vehicle in the rear may be guided.

  In addition, when the vehicle stops in the parking section PKA, the distance from the vehicles in the left and right parking sections of the own parking section is measured according to the opening / closing mechanism of the vehicle door (whether it is a sliding door). The warning display when opening the door may be added to the side mirror image. Moreover, you may make it audio | voice guide to the passenger | crew of a vehicle the warning when opening a door.

  In the above example, the clearance sonar 23 is used to detect the presence and distance of an object, but it may be detected using an image or a laser.

  Also, as shown in FIG. 20, it is preferable that a part of the vehicle body is reflected in the side mirror image (see regions BR01 and BR02 surrounded by a dotted line in the figure). Thereby, it is possible to make the vehicle occupant intuitively understand that the side mirror image shows an image shown in the side mirror.

  In addition, since the vehicle occupant usually confirms the obstacle behind the vehicle by the image reflected on the in-vehicle mirror, the vehicle occupant looks at the image reflected on the in-vehicle mirror, Know the distance between the obstacle and the vehicle. That is, the vehicle display device described above makes it easy for the vehicle occupant to grasp the type of obstacle behind the vehicle and the distance between the obstacle and the vehicle without visually recognizing the in-vehicle mirror. .

  Further, in the above vehicle display device, the mirror conversion image is an image showing a predetermined occupant mirror image and an image arranged around the image and showing an area around the imaging area corresponding to the predetermined occupant mirror image. The display means displays a portion corresponding to the predetermined occupant mirror image in the mirror converted image within the frame indicated by the mirror frame image, and the frame indicated by the mirror frame image is transparent to the mirror converted image. As shown, the mirror frame image may be displayed. In the vehicle display device configured as described above, an image of the in-vehicle mirror (predetermined occupant mirror image) when viewing the in-vehicle mirror from the viewpoint of the predetermined seat occupant and an image of the surrounding area are displayed. The vehicle occupant can confirm a wider area than when viewing the in-vehicle mirror. Further, the frame indicated by the mirror frame image is transmitted through the mirror converted image. For this reason, the image of the boundary part between the predetermined occupant mirror image and the image indicating the surrounding area is not hidden behind the frame indicated by the mirror frame image.

  Further, the first setting means may set the display range of the mirror converted image based on a preset first external operation. According to the vehicle display device configured as described above, the display range of the mirror-converted image can be changed by the operation of the vehicle occupant.

  In addition, the image conversion means is a surrounding photographed image obtained by the photographing means with the image of the in-vehicle mirror visually recognized by a predetermined seat occupant assuming that the in-vehicle mirror tilt angle is a preset predetermined tilt angle. And the second setting means may set the predetermined inclination angle based on a second external operation set in advance. According to the vehicular display device configured as described above, when the inclination angle of the in-vehicle mirror is assumed to be the predetermined inclination angle, the predetermined inclination angle is set for the image of the in-vehicle mirror visually recognized by the predetermined seat occupant. It can be changed by the operation of the vehicle occupant.

  The in-vehicle mirror is configured so that the tilt angle thereof can be adjusted. The mirror angle detection unit detects the mirror angle that is the tilt angle of the in-vehicle mirror, and the image conversion unit is detected by the mirror angle detection unit. A mirror conversion image may be generated by performing coordinate conversion according to the mirror angle. According to the vehicular display device configured in this way, in conjunction with a change in the tilt angle (mirror angle) of the in-vehicle mirror, an image obtained by viewing the in-vehicle mirror at the tilt angle from the viewpoint of the predetermined seat occupant Can be displayed.

  The display means may display the surrounding photographed image acquired by the photographing means on the same display screen as the mirror converted image. According to the vehicle display device configured in this manner, a vehicle occupant can simultaneously confirm an image corresponding to the in-vehicle mirror and a surrounding captured image by looking at one display screen.

  By the way, when the surrounding photographed image is a photograph of the rear of the vehicle, when the surrounding photographed image acquired by the photographing means is displayed as it is, the displayed image is a left-right reversed image with respect to the vehicle occupant. It becomes. This is because, since the photographing means is photographing the rear of the vehicle, the one located on the right (left) of the vehicle occupant facing the front of the vehicle is arranged on the left (right) in the surrounding photographed image. This is because that. Therefore, when the surrounding photographed image is a photograph of the rear of the vehicle, as described in claim 12, the left-right reversed image generating means is reversed left and right with respect to the surrounding photographed image acquired by the photographing means. It is preferable that the left-right reverse image, which is the obtained image, is generated, and the display means displays the left-right reverse image generated by the left-right reverse image generation means on the same display screen as the mirror-converted image. According to the vehicular display device configured as described above, when the image acquired by the photographing unit is displayed, the displayed image is an image in which the left and right sides of the occupant of the vehicle coincide with each other.

  Further, the overhead image generation means generates an overhead image in which the vehicle is seen from the viewpoint above the vehicle, using the surrounding captured image acquired by the imaging means, and the display means generates the overhead image generated by the overhead image generation means. May be displayed on the same display screen as the mirror-converted image. According to the display device for a vehicle configured in this manner, a vehicle occupant can simultaneously check an image corresponding to the in-vehicle mirror and an overhead image by looking at one display screen.

  The approach detection means may detect a vehicle approaching object that is an object approaching the vehicle, and the display means may display a mirror conversion image when the approach detection means detects a vehicle approaching object. In the vehicular display device configured as described above, when an object approaches the vehicle, a mirror conversion image is automatically displayed. Therefore, when the mirror conversion image is displayed, the object approaches the vehicle occupant. You can tell that you are doing.

  The vehicle speed detecting means detects the moving speed of the vehicle, the shift position detecting means detects the shift position of the transmission of the vehicle, the winker detecting means detects lighting of the winker of the vehicle, the display means is the vehicle speed detecting means, Based on the detection results of the shift position detection means and the blinker detection means, a mirror conversion image is displayed when the vehicle is stopped, the shift position is a position where the vehicle can advance, and the blinker is lit. It may be. According to the vehicular display device configured as described above, in a situation where the vehicle needs to be confirmed from the rear while entering the roadway from a vehicle stop state, an image when the in-vehicle mirror is viewed is displayed on the vehicle occupant. It can be displayed automatically regardless of operation or instruction.

DESCRIPTION OF SYMBOLS 1 ... Display system for vehicles, 2 ... Car-mounted camera, 3 ... Sensor group, 4 ... Image processing apparatus, 5 ... Display apparatus, 11 ... Front camera, 12 ... Rear camera, 13 ... Left side camera, 14 ... Right side camera, DESCRIPTION OF SYMBOLS 21 ... Shift lever position sensor, 22 ... Mirror angle sensor, 23 ... Clearance sonar, 24 ... Vehicle speed sensor, 25 ... Winker sensor, 31 ... Camera I / F, 32 ... Vehicle I / F, 33 ... Display I / F, 34 ... Storage unit, 35 ... Operation unit, 36 ... Control unit, 37 ... Bus, 41 ... Viewpoint conversion map database, 51 ... CPU, 52 ... ROM, 53 ... RAM

Claims (4)

A vehicle display device installed in a vehicle,
Photographing means for repeatedly photographing a predetermined photographing area around the vehicle and obtaining a surrounding photographing image;
Image conversion means for generating a mirror-converted image imitating an image of the rear of the vehicle seen from the viewpoint of a passenger sitting on a predetermined seat in the vehicle by converting the coordinates of the surrounding captured image; ,
An overhead image generation means for generating an overhead image including the entire image of the vehicle, which is an image of the vehicle viewed from above the vehicle using the surrounding captured image;
Display means for displaying the mirror-converted image and the overhead image on the same display screen;
The display means includes
Displaying the overhead image so that the vehicle front of the entire image of the vehicle faces upward in a rectangular first display area set in advance on the display screen;
The vehicle display, wherein the mirror-converted image is displayed in a second display area that is set laterally of the rectangular first display area and below the side of the rectangular shape. apparatus.   2. The vehicle according to claim 1, wherein the display unit displays a mirror frame image indicating a frame of a vehicle-mounted mirror and displays the mirror converted image inside a frame indicated by the mirror frame image. Display device.   The vehicle display device according to claim 1, wherein the mirror frame image is displayed so that the mirror frame image is transmitted.   4. The vehicle display device according to claim 1, wherein the display means displays the left and right mirror-converted images on the same display screen.

Images