WO2017086057A1

WO2017086057A1 - Display device for vehicles and display method for vehicles

Info

Publication number: WO2017086057A1
Application number: PCT/JP2016/080091
Authority: WO
Inventors: 昇勝俣
Original assignee: 株式会社Ｊｖｃケンウッド
Priority date: 2015-11-17
Filing date: 2016-10-11
Publication date: 2017-05-26

Abstract

A first acquisition unit 30 acquires a type 1 image in which the periphery of a vehicle is imaged. A first image generation unit 32 generates a type 1 bird's-eye image by converting the viewpoints of the type 1 image so as to be seen from above the vehicle. A display control unit 34 makes the type 1 bird's-eye image displayed. A second acquisition unit 36 acquires a type 2 image in which a range more apart from the vehicle than in the type 1 image is imaged from a position higher than for the type 1 image. A second image generation unit 38 generates a type 2 bird's-eye image by converting the viewpoints of the type 2 image so as to be seen from above the vehicle. An object detection unit 40 detects an object in the periphery of the vehicle. When an object is detected, the display control unit 34 makes a type 2 bird's-eye image that corresponds to the direction of the detected image displayed in addition to the type 1 bird's-eye image.

Description

VEHICLE DISPLAY DEVICE AND VEHICLE DISPLAY METHOD

The present invention relates to vehicle display technology, and more particularly to a vehicle display device and a vehicle display method for displaying an image.

A technology is widely used in which cameras are provided at a plurality of locations on a vehicle, and the captured images are synthesized by converting the viewpoints to obtain a bird's-eye view image. Such a display is displayed at the time of entering a garage, for example, and is used for the purpose of confirming the periphery of the vehicle and grasping the position. For example, when there is an obstacle in the vicinity of the vehicle when entering the garage, the obstacle is also displayed in the bird's-eye view image, but since the viewpoint conversion processing is performed, it is difficult to grasp the distance to the obstacle. Therefore, when an obstacle is detected, an original image obtained by imaging the obstacle detection area is displayed (for example, see Patent Document 1).

JP 2007-235529 A

The imaging range of the original image displayed when an obstacle is detected is the same as the imaging range of the overhead image. Therefore, an obstacle existing in a range that is not displayed in the overhead image is not displayed in the original image. On the other hand, when displaying a bird's-eye view image over a wide range so that a distant obstacle is displayed, it becomes difficult for the driver to grasp the situation in the vicinity of the vehicle.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique that makes it easier for the driver to grasp the situation in the vicinity of the vehicle, particularly the presence of an object that becomes an obstacle.

In order to solve the above problems, a vehicle display device according to an aspect of the present embodiment includes a first acquisition unit that acquires a first type image obtained by imaging the periphery of a vehicle, and a first type acquired by the first acquisition unit. A first image generation unit that generates a first type overhead image and a first type overhead image generated by the first image generation unit by converting the viewpoint as viewed from above the vehicle. And a second acquisition unit that acquires a second type image obtained by imaging a range separated from the vehicle from the first type image from a position higher than the first type image acquired by the first acquisition unit, A second image generation unit that generates a second type overhead image by converting the viewpoint as viewed from above the vehicle with respect to the second type image acquired by the second acquisition unit, and objects around the vehicle And an object detection unit for detecting. When the object detection unit detects an object, the display control unit is a second type overhead image generated by the second image generation unit in addition to the first type overhead image, and corresponds to the direction of the detected object. A second type overhead image is also displayed.

Another aspect of the present embodiment is a vehicle display method. In this method, a first type overhead image is obtained by acquiring a first type image obtained by imaging the periphery of the vehicle and converting the viewpoint of the acquired first type image as viewed from above the vehicle. A step of displaying the generated first type overhead image, and a step of acquiring a second type image obtained by imaging a range separated from the vehicle from the first type image from a position higher than the first type image. Generating a second type overhead image by converting the viewpoint as seen from above the vehicle with respect to the acquired second type image, detecting an object around the vehicle, A second type overhead image generated corresponding to the direction of the detected object is displayed in addition to the first type overhead image.

It should be noted that any combination of the above-described constituent elements and a representation of the present embodiment converted between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present embodiment.

According to this embodiment, it is possible to make it easier for the driver to grasp the situation in the vicinity of the vehicle, particularly the presence of an object that becomes an obstacle.

FIGS. 1A to 1B are views showing the appearance of a vehicle according to Embodiment 1 of the present invention. It is a figure which shows the structure of the display apparatus for vehicles which concerns on Example 1 of this invention. FIG. 2 is a perspective view showing an imaging range formed around the vehicle in FIGS. It is a figure which shows the 1st kind overhead view image produced | generated in the 1st image generation part of FIG. FIG. 2 is a perspective view showing another imaging range formed around the vehicle in FIGS. It is a figure which shows the bird's-eye view image produced | generated in the display control part of FIG. It is a figure which shows another bird's-eye view image produced | generated in the display control part of FIG. It is a flowchart which shows the display procedure by the display apparatus for vehicles of FIG. It is a flowchart which shows the display procedure by the display apparatus for vehicles which concerns on Example 2 of this invention. It is a figure which shows the structure of the display apparatus for vehicles which concerns on Example 3 of this invention. It is a figure which shows the bird's-eye view image produced | generated in the display control part of FIG.

Example 1
Before describing the present invention specifically, the premise will be described first. Embodiment 1 of the present invention relates to a vehicle display device that generates a bird's-eye view image by viewpoint conversion with respect to images taken by a plurality of image pickup units installed in a vehicle, and displays the generated bird's-eye view image. In the overhead image, an object that exists in the vicinity of the vehicle is displayed, but an object that exists away from the vehicle, such as an obstacle that exists at a position 1 m or more away from the vehicle, is not displayed. The vehicle display device according to the present embodiment executes the following process in order to display an object that exists away from the vehicle. The vehicle includes a plurality of first imaging units at a lower portion of the vehicle, and the vehicle display device generates a first type overhead image from a first type image captured by each of the plurality of first imaging units. The vehicle is also provided with a plurality of second imaging units at a position higher than the first imaging unit, and the vehicular display device receives the second type image captured by each of the plurality of second imaging units from the second type image. A seed bird's-eye view image is generated.

Furthermore, the vehicle is also provided with a sensor, and the vehicle display device detects the presence of an object based on the detection result of the sensor. Here, the vehicular display device displays the first type overhead image when the presence of the object is not detected. However, when the presence of the object is detected, the vehicular display device displays the first type overhead image in addition to the first type overhead image. Two types of overhead images are displayed. The height at which the second imaging unit is installed is a position higher than the height at which the first imaging unit is installed, and is installed at a position where it is possible to photograph far away from the first imaging unit. For this reason, the second type overhead image can display a farther object than the first type overhead image. On the other hand, when there is no object, the display of the second overhead image is unnecessary. Therefore, the display is switched depending on whether or not an object exists.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Specific numerical values and the like shown in the examples are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. .

FIGS. 1A to 1B show the appearance of the vehicle 100 according to the first embodiment of the present invention. FIG. 1A shows a top view of the vehicle 100, and FIG. 1B shows a side view. A first front imaging unit 10 is installed in a front portion of the vehicle 100, for example, a bumper, a bonnet, or the like. A first rear imaging unit 12 is installed in a rear portion of the vehicle 100, such as a bumper or a trunk. A first left side imaging unit 14 is installed on a left side portion of the vehicle, for example, a lower portion of the left door mirror. A first right side imaging unit 16 is installed on the right side of the vehicle so as to be symmetrical with the first left side imaging unit 14. Here, the first front imaging unit 10 to the first right imaging unit 16 are collectively referred to as a first imaging unit.

The second front imaging unit 18 is arranged in the vehicle near the roof front of the vehicle 100, and the second rear imaging unit 20 is arranged in the vehicle near the roof rear of the vehicle 100. The 2nd front image pick-up part 18 and the 2nd back image pick-up part 20 are arranged in the direction which can be photoed in a position higher than a 1st image pick-up part, and a distant place. Therefore, the second front imaging unit 18 and the second rear imaging unit 20 can image an object farther away than the first imaging unit. The second front imaging unit 18 and the second rear imaging unit 20 are installed in the passenger compartment near the roof of the vehicle 100, but are higher than the first front imaging unit 10 and the first rear imaging unit 12. As long as it is installed at a position, the front and rear bumpers and front and rear bodies of the vehicle 100 may be used. Here, the second front imaging unit 18 and the second rear imaging unit 20 are collectively referred to as a second imaging unit. Further, in FIGS. 1A to 1B, the second imaging units are disposed only in the front and rear of the vehicle 100, but may be disposed in the front and rear and right and left.

The front sensor 22 is disposed in front of the vehicle 100 as in the first front imaging unit 10, and the rear sensor 24 is disposed in the rear of the vehicle 100 as in the first rear imaging unit 12. For example, the front sensor 22 is disposed in the vicinity of the first front imaging unit 10, and the rear sensor 24 is disposed in the vicinity of the first rear imaging unit 12.

FIG. 2 shows a configuration of the vehicle display device 50 according to the first embodiment of the present invention. The vehicle display device 50 includes a first front imaging unit (first front camera) 10, a first rear imaging unit (first rear camera) 12, a first left imaging unit (first left camera) 14, a first 1 right side imaging unit (first right side camera) 16, second front imaging unit (second front camera) 18, second rear imaging unit (second rear camera) 20, front sensor 22, rear sensor 24, display panel 52 is connected. The vehicle display device 50 includes a first acquisition unit 30, a first image generation unit 32, a display control unit 34, a second acquisition unit 36, a second image generation unit 38, and an object detection unit 40.

The first front imaging unit 10, the first rear imaging unit 12, the first left side imaging unit 14, and the first right side imaging unit 16 are arranged as shown in FIG. FIG. 3 is a perspective view showing an imaging range formed around the vehicle 100. The first front imaging unit 10 forms a front imaging region 60 so as to go forward from the first front imaging unit 10, and captures an image in the front imaging region 60. The first rear imaging unit 12 forms a rear imaging region 62 so as to go rearward from the first rear imaging unit 12, and captures an image in the rear imaging region 62. The first left side imaging unit 14 forms a left side imaging region 64 so as to go to the left from the first left side imaging unit 14, and captures an image in the left side imaging region 64. The first right side imaging unit 16 forms a right side imaging region 66 so as to go right from the first right side imaging unit 16, and images an image in the right side imaging region 66.

The front imaging area 60 indicated by hatching in FIG. 3 is a range from the range indicated by hatching to the position immediately below the installation position of the first front imaging part 10 in the vehicle 100 in the range where the first front imaging unit 10 can shoot. , And a range that is cut out by the first image generation unit 32 and subjected to viewpoint conversion processing. Similarly, the rear imaging area 62 is a range in which the first rear imaging unit 12 can shoot, and a range from a range indicated by hatching to a position immediately below the installation position of the first rear imaging unit 12 in the vehicle 100 is the first image generation. This indicates that the range is cut out by the unit 32 and subjected to viewpoint conversion processing. Similarly, the left-side imaging area 64 has a range from the range indicated by hatching to the position immediately below the installation position of the first left-side imaging unit 14 in the vehicle 100 in the range in which the first left-side imaging unit 14 can shoot. This indicates that the range is cut out by one image generation unit 32 and subjected to viewpoint conversion processing. Similarly, the right side imaging region 66 has a range from the range indicated by oblique lines to the position immediately below the installation position of the first right side imaging unit 16 in the vehicle 100 in the range in which the first right side imaging unit 16 can shoot. This indicates that the range is cut out by one image generation unit 32 and subjected to viewpoint conversion processing. The periphery of the vehicle 100 is imaged by images captured by the first imaging unit. Returning to FIG.

The first front imaging unit 10, the first rear imaging unit 12, the first left side imaging unit 14, and the first right side imaging unit 16 capture an image as described above. The image is a moving image, but may be a still image taken continuously. The first front imaging unit 10, the first rear imaging unit 12, the first left side imaging unit 14, and the first right side imaging unit 16 output the captured images to the first acquisition unit 30. The first acquisition unit 30 includes images (hereinafter referred to as “first type images”) from each of the first front imaging unit 10, the first rear imaging unit 12, the first left side imaging unit 14, and the first right side imaging unit 16. ) To get. That is, the first acquisition unit 30 acquires a first type image obtained by imaging the periphery of the vehicle 100. The first type image acquired by the first acquisition unit 30 is processed by the first image generation unit 32.

The first image generation unit 32 executes processing of the first type image acquired by the first acquisition unit 30. The first image generation unit 32 performs processing for converting the viewpoint of the first type image as viewed from above the vehicle 100 and generating the first type overhead image. A known technique may be used for the conversion and the overhead image generation processing. For example, each pixel of the image is projected onto a three-dimensional curved surface in a virtual three-dimensional space, and a virtual viewpoint from above the vehicle 100 is used. Depending on, a necessary area of the three-dimensional curved surface is cut out. The cut out region corresponds to an image whose viewpoint has been converted. An example of the generated overhead image is shown in FIG.

FIG. 4 shows a first type overhead image 80 generated by the first image generation unit 32. In the center part of the first type bird's-eye view image 80 in FIG. The own vehicle icon 78 is an image showing the upper surface of the vehicle 100. A front image 70 is arranged in front of the host vehicle icon 78, a rear image 72 is arranged behind the host vehicle icon 78, and a left side image 74 is arranged on the left side of the host vehicle icon 78. A right side image 76 is arranged on the right side. Returning to FIG. That is, the first image generation unit 32 generates the first type overhead image 80 by converting the viewpoint as seen from above the vehicle 100 with respect to the first type image acquired by the first acquisition unit 30. To do. The first type overhead image 80 generated by the first image generation unit 32 is processed by the display control unit 34.

The display control unit 34 executes a process of displaying the first type overhead image 80 generated by the first image generation unit 32. The display control unit 34 displays the first type overhead image 80 on the display panel 52. Note that the timing at which the first-type bird's-eye view image 80 is displayed on the display panel 52 is an arbitrary timing that requires confirmation of the surroundings of the vehicle. For example, the reverse gear of the vehicle 100 may be selected when entering the garage. . A first type bird's-eye view image 80 as shown in FIG. 4 is displayed on the display panel 52.

The second front imaging unit 18 and the second rear imaging unit 20 are arranged as shown in FIG. FIG. 5 is a perspective view showing another imaging range formed around the vehicle 100. The second front imaging unit 18 forms a front imaging region 63 so as to go forward from the second front imaging unit 18, and images an image in the front imaging region 63. The front imaging area 63 extends in front of the vehicle 100 more than the front imaging area 60. The second rear imaging unit 20 forms a rear imaging region 65 so as to extend rearward from the second rear imaging unit 20 and captures an image in the rear imaging region 65. The rear imaging area 65 extends to the rear of the vehicle 100 more than the rear imaging area 62.

A front imaging area 63 indicated by hatching in FIG. 5 has a range from the range indicated by hatching to the position immediately below the installation position of the second front imaging part 18 in the vehicle 100 in the range where the second front imaging unit 18 can shoot. , And the range that is cut out by the second image generation unit 38 and subjected to the viewpoint conversion process. Similarly, the rear imaging area 65 is a range in which the second rear imaging unit 20 can shoot, and a range from a range indicated by hatching to a position immediately below the installation position of the second rear imaging unit 20 in the vehicle 100 is the second image generation. This indicates that the range is cut out by the unit 38 and the viewpoint conversion process is performed. Note that a comparison between an image captured by the second front imaging unit 18 and the second rear imaging unit 20 and an image captured by the first front imaging unit 10 and the first rear imaging unit 12 will be described later. The periphery of the vehicle 100 is also imaged by images captured by these second imaging units. Returning to FIG.

The second front imaging unit 18 captures an image forward from a position higher than the first front imaging unit 10. Therefore, the image captured by the second front imaging unit 18 includes a location farther from the vehicle 100 than the image captured by the first front imaging unit 10. That is, the second front imaging unit 18 can image farther than the first front imaging unit 10. The imaging range of the second front imaging unit 18 may partially overlap the imaging range of the first front imaging unit 10 or may be a range that does not overlap with the imaging range of the first front imaging unit 10.

The second rear imaging unit 20 captures an image backward from a position higher than the first rear imaging unit 12. Therefore, the image captured by the second rear imaging unit 20 includes a place farther from the vehicle 100 than the image captured by the first rear imaging unit 12. That is, the second rear imaging unit 20 can image farther than the first rear imaging unit 12. The imaging range of the second rear imaging unit 20 may partially overlap the imaging range of the first rear imaging unit 12 or may be a range that does not overlap with the imaging range of the first rear imaging unit 12. In this case as well, the image is a moving image, but may be a still image taken continuously. The second front imaging unit 18 and the second rear imaging unit 20 output the captured image to the second acquisition unit 36.

The second acquisition unit 36 acquires images (hereinafter referred to as “second type images”) from each of the second front imaging unit 18 and the second rear imaging unit 20. That is, the second acquisition unit 36 acquires the second type image captured from a position higher than the first type image. The second type image acquired by the second acquisition unit 36 is processed by the second image generation unit 38.

The second image generation unit 38 executes processing of the second type image acquired by the second acquisition unit 36. The 2nd image generation part 38 performs a process which converts a viewpoint with respect to a 2nd type image as it saw from the upper direction of the vehicle 100, and produces | generates a 2nd type bird's-eye view image. The processing in the second image generation unit 38 is the same as the processing in the first image generation unit 32. That is, the second image generation unit 38 generates the second type overhead image 82 by converting the viewpoint as viewed from above the vehicle with respect to the second type image acquired by the second acquisition unit 36. . Such a second type overhead image 82 is an overhead image in a range farther from the vehicle 100 than the first type overhead image 80. In the first type bird's-eye view image 80, images in the four directions of the front image 70 and the right side image 76 are synthesized, but the second type bird's-eye view image 82 is generated from the second type image in one direction. . The second type overhead image 82 generated by the second image generation unit 38 is processed by the display control unit 34.

The front sensor 22 and the rear sensor 24 are arranged as shown in FIG. The front sensor 22 and the rear sensor 24 are, for example, a millimeter wave sensor or an infrared sensor. Moreover, a 2nd imaging part may be sufficient. When the second imaging unit is used for the front sensor 22 and the rear sensor 24, the object detection unit 40 performs edge detection processing or the like on the image captured by the second imaging unit, and detects an obstacle. An identification number for identification is assigned to each of the front sensor 22 and the rear sensor 24.

The object detection unit 40 is connected to the front sensor 22 and the rear sensor 24 and detects an object around the vehicle 100. Since a known technique may be used for the detection of the object, a specific description is omitted here. For example, when an infrared laser is used for the front sensor 22 or the rear sensor 24, based on the time difference when the infrared laser is irradiated in a range that is the detection direction of the vehicle 100 and the infrared laser reflected on the object is received. , Detect objects. Note that the detection range of the object detection unit 40 is set to be farther than the imaging range of the first type image acquired by the first acquisition unit 30. The object detection unit 40 detects an object in either the front sensor 22 or the rear sensor 24, and notifies the display control unit 34 of the detection of the object when the distance to the detected object is larger than the threshold value. At that time, the identification number of the front sensor 22 or the rear sensor 24 that detected the object is also notified. Here, the threshold value is set to be a value on the far side of the imaging range of the first type image.

The display control unit 34 executes display processing of the first type overhead image 80 generated by the first image generation unit 32. The display control unit 34 also executes display processing of the second type overhead image 82 generated by the second image generation unit 38. Further, when an object is detected by the object detection unit 40, the display control unit 34 acquires a notification and an identification number from the object detection unit 40. When the notification from the object detection unit 40 is not acquired, the display control unit 34 displays the first type overhead image 80 on the display panel 52 as before. On the other hand, when acquiring the notification from the object detection unit 40, the display control unit 34 causes the display panel 52 to display the second type overhead image 82 in addition to the first type overhead image 80. The timing at which the second type overhead image 82 is displayed on the display panel 52 in addition to the first type overhead image 80 is the timing at which the notification from the object detection unit 40 is acquired after the first type overhead image 80 is displayed. .

FIG. 6 shows a bird's-eye view image generated by the display control unit 34. This corresponds to the case where the notification from the object detection unit 40 is acquired and the acquired identification number indicates the rear sensor 24. When the acquired identification number indicates the rear sensor 24, an object existing behind the vehicle 100 is detected by the rear sensor 24. As illustrated, the display control unit 34 arranges the first type overhead image 80, particularly the second type overhead image 82 in which the obstacle 84 that is an object is displayed below the rear image 72. In this way, the display control unit 34 displays the second type overhead image 82 corresponding to the direction of the detected object in the direction in which the object detection unit 40 detects the object in the first type overhead image 80. When the display as shown in FIG. 6 is performed based on the object detected by the rear sensor 24, it is appropriate that the vehicle 100 is moving backward.

FIG. 7 shows another overhead image generated by the display control unit 34. This corresponds to the case where the notification from the object detection unit 40 is acquired and the acquired identification number indicates the front sensor 22. When the acquired identification number indicates the front sensor 22, an object that exists in front of the vehicle 100 is detected by the front sensor 22. As shown in the figure, the display control unit 34 arranges the first type bird's-eye view image 80, particularly the second type bird's-eye view image 82 on which the obstacle 84 is displayed above the front image 70. Also in this case, the display control unit 34 displays the second type overhead image 82 corresponding to the direction of the detected object in the direction in which the object detection unit 40 detects the object in the first type overhead image 80. When the display as shown in FIG. 7 is performed based on the object detected by the front sensor 22, it is appropriate that the vehicle 100 is moving forward.

Here, the processing of the display control unit 34 for generating such a bird's-eye view image will be described in more detail. When the acquired identification number indicates the rear sensor 24, the display control unit 34 selects the second type overhead image 82 generated from the second type image captured by the second rear imaging unit 20. The display control unit 34 arranges the selected second type overhead image 82 below the rear image 72 corresponding to the first rear imaging unit 12 facing rearward in the same manner as the second rear imaging unit 20. At that time, the display control unit 34 may display the angle of view of the second type overhead image 82 wider than the angle of view of the first type overhead image 80 in the direction in which the object detection unit 40 detects the object. Further, the display control unit 34 may display the second type overhead image 82 in an enlarged manner than the angle of view of the first type overhead image 80 in the direction in which the object detection unit 40 detects the object. It should be noted that a known technique may be used to display a wide angle of view and display an enlarged view, and thus description thereof is omitted here. Further, as shown in FIG. 6, the display control unit 34 may move the first type overhead image 80 upward and display the second type overhead image 82 below the first type overhead image 80.

On the other hand, when the acquired identification number indicates the front sensor 22, the display control unit 34 selects the second type overhead image 82 generated from the second type image captured by the second front imaging unit 18. The display control unit 34 arranges the selected second type overhead image 82 on the upper side of the front image 70 corresponding to the first front imaging unit 10 facing forward like the second front imaging unit 18. Also in this case, the display control unit 34 may display the angle of view of the second type overhead image 82 wider than the angle of view of the first type overhead image 80 in the direction in which the object detection unit 40 detects the object. Further, the display control unit 34 may display the second type overhead image 82 in an enlarged manner than the angle of view of the first type overhead image 80 in the direction in which the object detection unit 40 detects the object. Further, as shown in FIG. 7, the display control unit 34 may move the first type overhead image 80 downward and display the second type overhead image 82 above the first type overhead image 80.

By performing the display as shown in FIG. 6 and FIG. 7, when the obstacle 84 exists far from the range displayed as the first type overhead image 80, the display range of the overhead image is substantially expanded. The second type overhead image 82 is displayed in the direction in which the obstacle 84 is detected. For this reason, the driver | operator can grasp | ascertain the presence and positional relationship of an obstacle more appropriately by confirming the display of a bird's-eye view image in addition to his own visual observation.

Further, the second imaging unit from which the second acquisition unit 36 acquires the second type image is arranged at a higher position than the first imaging unit from which the first acquisition unit 30 acquires the first type image. When the second imaging unit is disposed near the roof of the vehicle 100 as illustrated in FIG. 1, the second imaging unit is disposed at a position higher than the position of the driver, and thus the obstacle 84 is viewed from above from its own viewpoint. Thus, the stereoscopic effect of the obstacle 84 can be grasped more appropriately.

This configuration can be realized in terms of hardware by a CPU, memory, or other LSI of any computer, and in terms of software, it can be realized by a program loaded in the memory, but here it is realized by their cooperation. Draw functional blocks. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The operation of the vehicular display device 50 having the above configuration will be described. FIG. 8 is a flowchart showing a display procedure by the vehicle display device 50. When the display condition of the first type overhead image 80 is satisfied (Y in S10), the display control unit 34 causes the display panel 52 to display the first type overhead image 80 (S12). If the front sensor 22 or the rear sensor 24 does not detect the obstacle 84 (N in S14), the process returns to step 10. When the front sensor 22 or the rear sensor 24 detects the obstacle 84 (Y in S14), if the obstacle 84 does not exist farther than the range of the first type bird's-eye image 80 (N in S16), the process goes to step 10. Return. If the obstacle 84 exists farther than the range of the first type overhead image 80 (Y in S16), the display control unit 34 performs the second type overhead view in the detection direction of the obstacle 84 in the first type overhead image 80. The image 82 is superimposed and displayed (S18), and the process returns to step 16. If the display conditions for the first type overhead image 80 are not satisfied (N in S10), the process is terminated.

According to the present embodiment, when the presence of an object is not detected, only the first type bird's-eye view image is displayed, so that the driver can easily understand the situation in the vicinity of the vehicle. Further, when the presence of an object is detected, the second type bird's-eye image captured from a position higher than the first type bird's-eye view image is also displayed, so that the driver can easily recognize the presence of a distant object. In addition, since the second type overhead image corresponding to the detected direction of the object is displayed, reduction in the size of the first type overhead image can be suppressed. In addition, since it is possible to detect the presence of an object farther than the imaging range of the first type image, it is possible to detect the presence of an object that is not included in the first type overhead image.

In addition, since the second type overhead image is displayed in the direction in which the object is detected in the first type overhead image, the positional relationship between the first type overhead image and the second type overhead image can be easily grasped. . Further, since the angle of view of the second type overhead image is displayed wider than the angle of view of the first type overhead image, it is possible to easily grasp the position where the object is present. In addition, since the second type overhead image is displayed in an enlarged manner than the angle of view of the first type overhead image, the presence of the object can be easily grasped.

In general, the virtual viewpoint is set above the center of the vehicle 100, but the first-type overhead image 80 and the second-type overhead image 82 may have different virtual viewpoint positions. For example, the first type bird's-eye view image 80 is a bird's-eye view image with a virtual viewpoint above the center of the vehicle 100, and the second type bird's-eye view image 82 is a virtual viewpoint shifted to the front of the vehicle 100 from the first type bird's-eye view image 80. It may be. The second type overhead image 82 displays a wider range in front of the vehicle 100 than the first type overhead image 80. For this reason, by setting the virtual viewpoint position of the second type overhead image 82 in front of the vehicle 100, in the case of the display form as shown in FIG. 7, the first type overhead image 80 and the second type overhead image 82 Discomfort is reduced.

(Example 2)
Next, Example 2 will be described. As in the first embodiment, the second embodiment relates to a vehicular display device that generates a bird's-eye view image by viewpoint conversion with respect to images captured by a plurality of imaging units installed in the vehicle, and displays the generated bird's-eye view image. . The vehicle display device according to the first embodiment displays the second type overhead image in addition to the first type overhead image when the presence of an object not included in the first type overhead image is detected. On the other hand, when the vehicle display device according to the second embodiment detects an object that is not included in the first type overhead image and is included in the second type overhead image, In addition, display of the second type overhead image is started. The vehicle 100 and the vehicle display device 50 according to the second embodiment are the same type as those in FIGS. Here, it demonstrates centering on the difference with Example 1. FIG.

The object detection unit 40 is connected to the front sensor 22 and the rear sensor 24 as before, and detects objects around the vehicle 100. The object detection unit 40 detects an object in either the front sensor 22 or the rear sensor 24, and the distance to the detected object is farther than the imaging range of the first type image, and the second type image When it is included in the imaging range, the display control unit 34 is notified of object detection.

When the object detection unit 40 detects an object, the display control unit 34 obtains a notification from the object detection unit 40 and an identification number. When the notification from the object detection unit 40 is acquired, the display control unit 34 causes the display panel 52 to display the second type overhead image 82 in addition to the first type overhead image 80. Here, the timing at which the second type overhead image 82 is displayed on the display panel 52 in addition to the first type overhead image 80 is the timing at which the notification from the object detection unit 40 is acquired after the first type overhead image 80 is displayed. It is. That is, when the object detected by the object detection unit 40 is outside the range of the first type overhead image 80 and within the range of the second type overhead image 82, the display control unit 34 adds to the first type overhead image 80. Then, the display of the second type overhead image 82 corresponding to the detected direction of the object is started.

The operation of the vehicular display device 50 having the above configuration will be described. FIG. 9 is a flowchart showing a display procedure by the vehicle display device 50 according to the second embodiment of the present invention. When the display condition of the first type overhead image 80 is satisfied (Y in S100), the display control unit 34 causes the display panel 52 to display the first type overhead image 80 (S102). If the front sensor 22 or the rear sensor 24 does not detect the obstacle 84 (N in S104), the process returns to step 100. When the front sensor 22 or the rear sensor 24 detects the obstacle 84 (Y in S104), when the obstacle 84 does not exist farther than the range of the first type overhead image 80, or the range of the second type overhead image 82 If not included (N in S106), the process returns to Step 100. When the obstacle 84 exists farther than the range of the first type overhead image 80 and is included in the range of the second type overhead image 82 (Y in S106), the display control unit 34 displays the first type overhead image. The second type bird's-eye view image 82 is superimposed and displayed in the detection direction of the obstacle 84 at 80 (S108), and the process returns to step 106. If the display condition of the first type overhead image 80 is not satisfied (N in S100), the process is terminated.

According to the present embodiment, when the object is outside the range of the first type overhead image and within the range of the second type overhead image, the second type corresponding to the direction of the detected object in addition to the first type overhead image. Since the bird's-eye view image is displayed, it is possible to reliably display the object that is not included in the first type bird's-eye image and is included in the second type bird's-eye view image. In addition, since the second type overhead image is displayed in the direction in which the object is detected in the first type overhead image, the positional relationship between the first type overhead image and the second type overhead image can be easily grasped. .

(Example 3)
Next, Example 3 will be described. Example 3 relates to a vehicle display device that generates an overhead image by viewpoint conversion for images captured by a plurality of imaging units installed in a vehicle, and displays the generated overhead image, as in the past. Until now, when the first type overhead image includes an object, the second type overhead image is not displayed. On the other hand, in Example 3, even if the first type overhead image includes an object, the second type overhead image is displayed in addition to the first type overhead image. At that time, a display is made so that it can be determined that the object included in the first type overhead image and the object included in the second type overhead image are the same. The vehicle 100 according to the third embodiment is the same type as that shown in FIG. Here, it demonstrates centering on the difference from before.

FIG. 10 shows a configuration of the vehicle display device 50 according to the third embodiment of the present invention. The vehicle display device 50 further includes an identity determination unit 42 in the vehicle display device 50 shown in FIG. The object detection unit 40 is connected to the front sensor 22 and the rear sensor 24 as before, and detects objects around the vehicle 100. When the object detection unit 40 detects an object in either the front sensor 22 or the rear sensor 24, the object detection unit 40 notifies the display control unit 34 and the identity determination unit 42 of the detection of the object.

In the identity determination unit 42, the object detection unit 40 notifies the detection of the object. At that time, the position of the detected object is also notified. The identity determination unit 42 receives the first type overhead image 80 from the first image generation unit 32 and the second type overhead image 82 from the second image generation unit 38. Furthermore, the identity determination unit 42 acquires the position information and the traveling direction of the vehicle 100. In addition, since a well-known technique should just be used for acquisition of the positional information and the advancing direction of the vehicle 100, description is abbreviate | omitted here. Since the identity determination unit 42 recognizes in advance the angle of view of the first type overhead image 80, the first determination unit 42 determines the first angle based on the position information and traveling direction of the vehicle 100 and the angle of view of the first type overhead image 80. The coordinates for each of the plurality of pixels included in the seed bird's-eye view image 80 are acquired. In addition, the identity determination unit 42 performs the same process on the second type bird's-eye image 82, thereby acquiring the coordinates for each of the plurality of pixels included in the second type bird's-eye image 82.

The identity determination unit 42 includes the same object when the position (coordinates) at which the object detection unit 40 detects the object is included in the first type overhead image 80 and the second type overhead image 82. Is determined. The identity determination unit 42 performs image recognition processing on the first-type overhead image 80 and the second-type overhead image 82, compares the shapes of the objects acquired in the image recognition processing, and the same object is found. It may be determined that it is included. As described above, the identity determination unit 42 determines the identity of the first type overhead image 80 and the second type overhead image 82 of the object detected by the object detection unit 40. The identity determination unit 42 outputs a determination result on whether or not the same object is included to the display control unit 34.

When the object detection unit 40 detects an object, the display control unit 34 displays the second type overhead image 82 in addition to the first type overhead image 80. Further, when displaying the second type overhead image 82 in addition to the first type overhead image 80, the display control unit 34, based on the determination result by the identity determination unit 42, the first type overhead image 80 and the second type. The display which can determine the identity of the object currently displayed on each of the bird's-eye view images 82 is performed. When the detected object is displayed only in the second type bird's-eye view image 82, a display capable of determining the identity is unnecessary. However, when the vehicle 100 comes close to the detected object and the object is displayed in both the second type overhead image 82 and the first type overhead image 80, a display capable of determining the identity is made. .

FIG. 11 shows an overhead image generated by the display control unit 34. Similar to FIG. 6, this corresponds to the case where the notification from the object detection unit 40 is acquired and the acquired identification number indicates the rear sensor 24. As illustrated, the display control unit 34 arranges the first type overhead image 80, particularly the second type overhead image 82 in which the obstacle 84 that is an object is displayed below the rear image 72. The obstacle 84 is also displayed on the first type overhead image 80. Here, the obstacle 84 included in the first type bird's-eye image 80 and the obstacle 84 included in the second type bird's-eye image 82 are determined to be the same by the identity determination unit 42. Therefore, the same object marker 86 is shown on the obstacle 84 included in the first type overhead image 80 and the obstacle 84 included in the second type overhead image 82. The same object marker 86 is a display that can determine the identity of an object, and is a display that surrounds each with a frame of the same shape or the same color.

According to the present embodiment, when the second type overhead image is displayed in addition to the first type overhead image, the identity of the objects displayed in each of the first type overhead image and the second type overhead image is determined. Since possible display is performed, the same object displayed on each of the first type overhead image and the second type overhead image can be easily recognized. In addition, since the same object displayed in each of the first type overhead image and the second type overhead image is easily recognized, the position of the object can be easily recognized in a situation where the vehicle approaches the object. .

The present invention has been described above based on the embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

10 First front imaging unit, 12 First rear imaging unit, 14 First left side imaging unit, 16 First right side imaging unit, 18 Second front imaging unit, 20 Second rear imaging unit, 22 Front sensor, 24 Rear Sensor, 30 first acquisition unit, 32 first image generation unit, 34 display control unit, 36 second acquisition unit, 38 second image generation unit, 40 object detection unit, 50 vehicle display device, 52 display panel, 100 vehicle .

According to the present invention, it is possible to make it easier for the driver to grasp the situation in the vicinity of the vehicle, particularly the presence of an object that becomes an obstacle.

Claims

A first acquisition unit that acquires a first type image obtained by imaging the periphery of the vehicle;
A first image generation unit that generates a first type overhead image by converting a viewpoint as seen from above the first type image acquired in the first acquisition unit;
A display control unit for displaying the first type overhead image generated in the first image generation unit;
A second acquisition unit that acquires a second type image obtained by imaging a range separated from the vehicle from the first type image from a position higher than the first type image acquired by the first acquisition unit;
A second image generation unit that generates a second type overhead image by converting the viewpoint as seen from above the second type image acquired by the second acquisition unit;
An object detection unit for detecting objects around the vehicle,
When the object detection unit detects an object, the display control unit is a second type overhead image generated by the second image generation unit in addition to the first type overhead image, and the detected object A vehicular display device that also displays a second type overhead image corresponding to a direction.
2. The vehicular display device according to claim 1, wherein the object detection unit has a detection range farther than an imaging range of the first type image acquired by the first acquisition unit.
The display control unit generates a second type overhead image generated by the second image generation unit in a direction in which the object detection unit detects an object among the first type overhead images generated by the first image generation unit. The vehicle display device according to claim 1, wherein the display device is displayed.
The display control unit displays the angle of view of the second type overhead image wider than the angle of view of the first type overhead image in the direction in which the object detection unit detects the object. The vehicle display device according to any one of the above.
4. The display control unit according to claim 1, wherein the display control unit displays the second type overhead image in an enlarged manner than the angle of view of the first type overhead image in the direction in which the object detection unit detects the object. 5. The vehicle display device according to any one of the above.
The second image generation unit generates a second type overhead image in a range farther from the vehicle than the first type overhead image generated by the first image generation unit,
When the object detected by the object detection unit is outside the first type overhead image and within the second type overhead image, the display control unit adds the first type overhead image to the detected object. 6. The vehicular display device according to claim 1, wherein display of the second type overhead image corresponding to the direction is started.
An identity determination unit for determining the identity of the first type overhead image and the second type overhead image of the object detected by the object detection unit;
When the display control unit displays the second type overhead image in addition to the first type overhead image, based on the determination result by the identity determination unit, each of the first type overhead image and the second type overhead image The vehicular display device according to any one of claims 1 to 6, wherein a display capable of determining the identity of the displayed object is performed.
Acquiring a first type image obtained by imaging the periphery of the vehicle;
A step of generating a first type overhead image by converting the viewpoint as seen from above the vehicle with respect to the acquired first type image;
Displaying the generated first-type overhead image,
Obtaining a second type image obtained by imaging a range separated from the vehicle from the first type image from a position higher than the first type image;
Generating a second type overhead image by converting the viewpoint as seen from above the vehicle with respect to the acquired second type image;
Detecting objects around the vehicle;
A step of displaying a second type overhead image corresponding to the direction of the detected object in addition to the first type overhead image when the object is detected;
A vehicle display method comprising the steps of: