JP6593258B2 - VEHICLE DISPLAY DEVICE, VEHICLE DISPLAY METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to a display technology for a vehicle, and more particularly to a display device for a vehicle that displays an image, a display method for a vehicle, and a program.

  When a display image for calling attention to other vehicles existing around the vehicle is displayed on the windshield of the vehicle, it becomes easier for the driver of the vehicle to recognize objects existing around the vehicle. If the display image displayed on the windshield overlaps with the white line of the pedestrian crossing, the brightness of the pedestrian crossing displayed with the white line is high due to the reflection of light. Is difficult to see the displayed image. For this reason, when the display area of the display image overlaps the viewing area of the high-luminance area, the display area is changed (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 2006-21116

  The viewpoint image is processed so that the image captured by the imaging unit installed on the four sides of the vehicle is looked down from above the vehicle, thereby generating an overhead image. When displaying a bird's-eye view image, it is desirable to alert the driver of a portion with a poor road surface condition.

  This invention is made | formed in view of such a condition, The objective is to provide the technique which grasps | ascertains a road surface state also in a bird's-eye view image | video.

  In order to solve the above-described problem, a vehicle display device according to an aspect of the present invention includes a video acquisition unit that acquires a video obtained by capturing an image of the periphery of the vehicle, and a video acquired from the video acquisition unit from above the vehicle. Based on the road surface state acquired by the generation unit that generates a bird's-eye view image by converting the viewpoint as seen, the road surface state acquisition unit that acquires the road surface state around the vehicle, and the road surface state acquisition unit A road surface state determination unit that determines whether there is a road surface state that requires a vehicle, a stop determination unit that determines that the vehicle is stopped or a vehicle speed is less than a predetermined speed when the vehicle is stopped, When the stop determination unit determines that the vehicle is in a stopped state and the road surface state determination unit determines that there is a road surface state that requires attention, an image indicating the road surface state that requires attention is superimposed on the overhead view image. Comprising a processing unit, a display control unit for displaying on the display unit an overhead video image is superimposed in the video processing unit.

  Another aspect of the present invention is a vehicle display method. The method includes a step of acquiring a video obtained by imaging the periphery of the vehicle, a step of generating a bird's-eye view video by converting a viewpoint as viewed from above the vehicle, and a periphery of the vehicle A step of acquiring a road surface state in the vehicle, a step of determining whether or not a road surface state requiring attention exists based on the acquired road surface state, and a case where the vehicle stops or the vehicle speed is less than a predetermined speed. When it is determined that the vehicle has stopped, and when the vehicle stopped state is determined and it is determined that there is a road surface state that requires attention, an image indicating the road surface state that requires attention is superimposed on the overhead view video. And a step of causing the display unit to display a bird's-eye view image on which an image is superimposed.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to grasp the road surface state even in a bird's-eye view video.

It is a perspective view which shows the imaging range formed in the periphery of the vehicle which concerns on Example 1 of this invention. It is a figure which shows the sensor mounted in the vehicle of FIG. It is a figure which shows the structure of the display apparatus for vehicles mounted in the vehicle of FIG. It is a figure which shows the bird's-eye view image | video produced | generated in the production | generation part of FIG. It is a figure which shows arrangement | positioning of the left front sensor of FIG. It is a figure which shows arrangement | positioning of the left side sensor of FIG. It is a figure which shows the bird's-eye view video produced | generated in the video processing part of FIG. It is a figure which shows another bird's-eye view image | video produced | generated in the image | video process part of FIG. It is a figure which shows another another bird's-eye view image | video produced | generated in the video processing part of FIG. It is a figure which shows the image | video produced | generated in the image | video process part of FIG. It is a flowchart which shows the display procedure by the display apparatus for vehicles of FIG. It is a figure which shows the structure of the display apparatus for vehicles which concerns on Example 2 of this invention.

Example 1
Before describing the present invention specifically, the premise will be described first. Embodiment 1 of the present invention relates to a vehicular display device that generates a bird's-eye view video by viewpoint conversion for images taken by a plurality of imaging units installed in a vehicle, and displays the generated bird's-eye view video. By using the bird's-eye view video, the driver can easily grasp the situation around the vehicle. Therefore, when the vehicle is stopped, contact with surrounding obstacles is easily prevented. On the other hand, the situation around the vehicle includes road surface conditions in addition to the presence of surrounding obstacles. The road surface state indicates, for example, whether the road surface is dry or wet. Here, when there is a puddle at the position where the door of the vehicle is opened, there is a possibility that the shoes and clothes of the driver who gets off get wet. Therefore, it is desired that the driver or the like be alerted about the road surface condition.

  In order to cope with this, the vehicular display device according to the present embodiment executes the following processing. A plurality of ultrasonic sensors are installed in the vehicle. In the ultrasonic sensor, the transmitter transmits an ultrasonic wave toward the road surface, and the receiver receives the reflected wave. Here, the intensity of the reflected wave when the road surface is wet is smaller than the intensity of the reflected wave when the road surface is dry. Therefore, the vehicular display device determines whether the road surface is dry or wet by measuring the intensity of the reflected wave, and displays an image corresponding to the determination result superimposed on the overhead video.

  Embodiments of the present invention will be described below with reference to the drawings. Specific numerical values and the like shown in the examples are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. .

  FIG. 1 is a perspective view showing an imaging range formed around the vehicle 100 according to the first embodiment of the present invention. A front imaging unit 12 is installed in a front portion of the vehicle 100, for example, a bumper, a bonnet, or the like. The front imaging unit 12 forms a front imaging region 20 so as to go forward from the front imaging unit 12, and captures an image in the front imaging region 20. A left side imaging unit 14 is installed in a left side portion of the vehicle 100, for example, a lower portion of the left door mirror. The left side imaging unit 14 forms a left side imaging region 22 so as to go to the left from the left side imaging unit 14, and captures an image in the left side imaging region 22.

  A rear imaging unit 16 is installed in a rear portion of the vehicle 100, such as a bumper or a trunk. The rear imaging unit 16 forms a rear imaging region 24 so as to extend rearward from the rear imaging unit 16, and captures an image in the rear imaging region 24. A right side imaging unit 18 (not shown) is installed on the right side portion of the vehicle 100 so as to be symmetrical with the left side imaging unit 14. The right side imaging unit 18 forms a right side imaging region 26 so as to go right from the right side imaging unit 18, and captures an image in the right side imaging region 26. The front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18 are used as the imaging unit 10, and the periphery of the vehicle 100 is imaged by an image captured by the imaging unit 10.

  FIG. 2 shows a sensor mounted on the vehicle 100. The vehicle 100 includes a left side sensor 114, a right side sensor 118, a left front sensor 120, a right front sensor 122, a left rear sensor 124, and a right rear sensor 126. The left front sensor 120 and the right front sensor 122 are installed in a front portion of the vehicle 100. Here, the left front sensor 120 is disposed on the left side, and the right front sensor 122 is disposed on the right side. Further, the left side sensor 114 is disposed on the left side portion of the vehicle 100, and the right side sensor 118 is disposed on the right side portion of the vehicle 100. The left rear sensor 124 and the right rear sensor 126 are installed in the rear portion of the vehicle 100. Here, the left rear sensor 124 is disposed on the left side, and the right rear sensor 126 is disposed on the right side.

  The left front sensor 120, the right front sensor 122, the left rear sensor 124, and the right rear sensor 126 are arranged to detect whether there is a puddle at a position where the tire of the vehicle 100 moves. The left side sensor 114 and the right side sensor 118 are disposed at the door position of the vehicle 100. When vehicle 100 is a four-door vehicle, a sensor may be arranged to correspond to each door.

  FIG. 3 shows a configuration of the vehicle display device 30 mounted on the vehicle 100. The vehicle display device 30 is connected to the imaging unit 10, the display unit 50, and the sensor 110. The imaging unit 10 includes a front imaging unit 12, a left side imaging unit 14, a rear imaging unit 16, and a right side imaging unit 18, and the sensor 110 includes a left side sensor 114, a right side sensor 118, a left front sensor 120, and a right front. A sensor 122, a left rear sensor 124, and a right rear sensor 126 are included. The vehicle display device 30 includes a video acquisition unit 32, a generation unit 34, a video processing unit 40, a display control unit 42, a road surface state acquisition unit 60, a road surface state determination unit 62, and a stop determination unit 64. Further, the road surface state determination unit 62 includes a positional relationship determination unit 66.

  The front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18 capture an image as described above. The front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18 output a video to the video acquisition unit 32. The video acquisition unit 32 acquires video from each of the front imaging unit 12, the left side imaging unit 14, the rear imaging unit 16, and the right side imaging unit 18. That is, the video acquisition unit 32 acquires a video obtained by imaging the periphery of the vehicle 100. The video acquisition unit 32 outputs the acquired video to the generation unit 34.

  The generation unit 34 inputs the video from the video acquisition unit 32. The production | generation part 34 produces | generates a bird's-eye view image | video by performing the process which converts a viewpoint as it saw from the upper direction of the vehicle 100 with respect to the image | video. A known technique may be used for the conversion. For example, each pixel of the image is projected onto a three-dimensional curved surface in a virtual three-dimensional space, and a three-dimensional image is displayed according to a virtual viewpoint from above the vehicle 100. Cut out the necessary area of the curved surface. The cut-out area corresponds to a video whose viewpoint has been converted.

  FIG. 4 shows an overhead view image 78 generated by the generation unit 34. In the central portion of the bird's-eye view video 78, a vehicle icon 80 that is an image showing the vehicle 100 is arranged. The own vehicle icon 80 is an image of the vehicle 100 as viewed from above, and has the size of the vehicle 100 in the overhead view image 78. A front image 70 is arranged in front of the host vehicle icon 80, a left side image 72 is arranged on the left side of the host vehicle icon 80, a rear image 74 is arranged behind the vehicle 100, and a right side of the vehicle 100 is displayed. A right side image 76 is arranged. Returning to FIG. The generation unit 34 outputs the overhead view video 78 to the video processing unit 40.

  The left side sensor 114, the right side sensor 118, the left front sensor 120, the right front sensor 122, the left rear sensor 124, and the right rear sensor 126 are combined as a sensor 110. For example, an existing technique such as a sensor using ultrasonic waves can be applied to the sensor 110. When ultrasonic waves are used for the sensor 110, the transmitter of the sensor 110 transmits the ultrasonic waves to a predetermined range, and the reflected wave from the predetermined range is received by the receiver of the sensor 110. The sensor 110 measures the intensity of the received reflected wave. Since a well-known technique should just be used for a measurement of intensity | strength, description is abbreviate | omitted here.

  FIG. 5 shows the arrangement of the left front sensor 120. The left front sensor 120 is used to detect whether there is a puddle or the like in the traveling direction. The left front sensor 120 is disposed at a height of “A1” from the road surface 90 and is disposed so as to mainly detect a point away from the vehicle 100 by “B1”. For example, the left front sensor 120 is disposed at a position near the bonnet on the left side portion of the front end of the vehicle 100. “B1” is set to 2 m. The right front sensor 122, the left rear sensor 124, and the right rear sensor 126 in FIG. 2 are similarly arranged.

  FIG. 6 shows the arrangement of the left side sensor 114. The left side sensor 114 is used to detect whether there is a puddle or the like at a position where an occupant of the vehicle 100 gets off from the left side of the vehicle 100. The left side sensor 114 is disposed at a height of “A2” from the road surface 90, and is disposed so as to mainly detect a point away from the vehicle 100 by “B2”. For example, the left side sensor 114 is disposed on a door or on a door mirror. “B2” is set to 10 cm. Returning to FIG. The sensor 110 outputs the measured intensity to the road surface state acquisition unit 60.

  The road surface state acquisition unit 60 inputs strength information from the sensor 110. The intensity information corresponds to the ultrasonic reflection state on the road surface 90. Further, as described above, when the strength is high, the road surface 90 is dry, and when the strength is low, the road surface 90 is wet. Therefore, it can be said that the strength information is a road surface condition around the vehicle 100. The road surface state acquisition unit 60 outputs the strength information from the sensor 110 to the road surface state determination unit 62.

  The road surface state determination unit 62 inputs strength information from the road surface state acquisition unit 60. The road surface state determination unit 62 determines whether or not the road surface 90 is wet based on the strength information. More specifically, the road surface state determination unit 62 holds a threshold value in advance, and compares the intensity with the threshold value. When the strength is larger than the threshold value, the road surface state determination unit 62 determines that the road surface 90 is dry. On the other hand, when the strength is equal to or less than the threshold value, the road surface state determination unit 62 determines that the road surface 90 is wet. This is a process based on the fact that when an ultrasonic sensor is used as the sensor 110, the reflection on the wet road surface 90 is weaker than the reflection on the dry road surface 90. The wet road surface 90 includes puddles and freezing. Here, the wetness of the road surface 90 corresponds to the presence of a road surface state requiring attention. Therefore, it can be said that the road surface state determination unit 62 determines whether or not there is a road surface state requiring attention based on the road surface state, that is, the ultrasonic reflection state.

  When the road surface state determination unit 62 determines that the wet road surface 90 exists, the positional relationship determination unit 66 determines the wet road surface 90 based on the positional relationship between the position of the wet road surface 90 and the door of the vehicle 100. Is determined in the vicinity of the door of the vehicle 100. The left side sensor 114 and the right side sensor 118 are disposed in the vicinity of the door, and the left front sensor 120, the right front sensor 122, the left rear sensor 124, and the right rear sensor 126 are not disposed in the vicinity of the door. Therefore, when it is determined that the wet road surface 90 exists for the strength of the left side sensor 114 or the right side sensor 118, the positional relationship determination unit 66 determines that the position of the wet road surface 90 is near the door of the vehicle 100. Is determined to exist. On the other hand, when it is determined that the wet road surface 90 exists with respect to the strength of either the left front sensor 120 or the right rear sensor 126, the positional relationship determination unit 66 determines that the position of the wet road surface 90 is that of the vehicle 100. It is determined that it does not exist in the vicinity of the door. The road surface state determination unit 62 and the positional relationship determination unit 66 output the determination result to the video processing unit 40.

  The stop determination part 64 acquires the information regarding a driving | running state from ECU (Electronic Control Unit) via CAN (Controller Area Network). The information regarding the traveling state is information on the traveling speed of the vehicle 100, for example. The stop determination unit 64 determines that the vehicle is in a stopped state when the vehicle is stopped or when the vehicle speed is less than a predetermined speed based on the information related to the traveling state. Here, the predetermined speed is set to 10 km / h, for example. The stop determination unit 64 outputs the determination result to the video processing unit 40.

  The video processing unit 40 inputs determination results from the road surface state determination unit 62, the positional relationship determination unit 66, and the stop determination unit 64. When the stop determination unit 64 determines the stop state of the vehicle 100 and the road surface state determination unit 62 determines that the wet road surface 90 exists, the video processing unit 40 displays an image showing a road surface state requiring attention as a bird's-eye view video. 78 is superimposed. In particular, when the positional relationship determination unit 66 determines that the position of the wet road surface 90 exists in the vicinity of the door of the vehicle 100, the video processing unit 40 determines that the vehicle in the vehicle icon 80 in the overhead image 78. An image indicating a road surface state requiring attention is superimposed on the position of 100 doors.

  FIG. 7 shows an overhead video 78 generated in the video processing unit 40. This is an overhead view when it is determined that the position of the wet road surface 90 does not exist in the vicinity of the door of the vehicle 100, and the wet road surface 90 is determined to exist with respect to the strength of the left front sensor 120. This is an image 78. In the bird's-eye view video 78, a left front notification image 150, a right front notification image 154, a left rear notification image 156, and a right rear notification image 160 are shown. Here, the left front notification image 150 and the right front notification image 154 are arranged in front of the host vehicle icon 80, and the left rear notification image 156 and the right rear notification image 160 are arranged behind the host vehicle icon 80.

  The left front notification image 150 is an image for indicating “the wet road surface 90 is present” or “the dry road surface 90 is present” with respect to the intensity of the left front sensor 120. When “wet road surface 90 is present”, the left front notification image 150 is indicated by “white solid color”, and when “wet road surface 90 is present”, the left front notification image 150 is indicated by “shaded pattern”. It is. Here, the former is used as an example. Of these, the left front notification image 150 in the case of “white solid color” corresponds to the image indicating the road surface state requiring attention as described above.

  The right front notification image 154, the left rear notification image 156, and the right rear notification image 160 are similarly shown to the right front sensor 122, the left rear sensor 124, and the right rear sensor 126. As an example, since these are indicated by “shaded pattern”, this corresponds to the case where “the dry road surface 90 exists”. When the vehicle 100 is moving forward, there is no need to display the left rear notification image 156 and the right rear notification image 160. Therefore, the left rear notification image 156 and the right rear notification image 160 are “network” regardless of the road surface state. It may be indicated by “hanging pattern”. On the other hand, when the vehicle 100 is moving backward, there is no need to display the left front notification image 150 and the right front notification image 154. Therefore, the left front notification image 150 and the right front notification image 154 are “network” regardless of the road surface condition. It may be indicated by “hanging pattern”.

  FIG. 8 shows another overhead view video 78 generated in the video processing unit 40. This is an overhead image 78 when it is determined that the position of the wet road surface 90 exists in the vicinity of the door of the vehicle 100. In the bird's-eye view video 78, a left side notification image 162 and a right side notification image 164 are arranged instead of the left front notification image 150 and the like in FIG. Here, the left side notification image 162 is arranged on the left side of the own vehicle icon 80, and the right side notification image 164 is arranged on the right side of the own vehicle icon 80. The right side notification image 164 is an image for indicating “the wet road surface 90 is present” or “the dry road surface 90 is present” with respect to the intensity of the right side sensor 118. When “wet road surface 90 is present”, the right side notification image 164 is indicated by “white solid color”, and when “wet road surface 90 is present”, the right side notification image 164 is indicated by “shaded pattern”. It is.

  Here, the former is used as an example. Of these, the right side notification image 164 in the case of “white solid color” corresponds to the image indicating the road surface condition requiring attention as described above. The left side notification image 162 is similarly shown to the left side sensor 114. As an example, the left-side notification image 162 is indicated by “shaded pattern”, which corresponds to the case where “the dry road surface 90 exists”. In addition, since the display of the left side notification image 162 and the right side notification image 164 is necessary when getting off the vehicle, the display of the left side notification image 162 and the right side notification image 164 is started when the speed of the vehicle 100 is reduced. May be.

  FIG. 9 shows still another overhead view video 78 generated by the video processing unit 40. This corresponds to a case where the left rear notification image 160, the left side notification image 162, and the right side notification image 164 are displayed from the left front notification image 150. Further, when the speed of the vehicle 100 becomes higher than the slow speed, the tilt angle of the left side sensor 114 and the right side sensor 118 is changed to be the same as the tilt angle of the left front sensor 120 or the like.

  Up to now, the video processing unit 40 has superimposed an image showing a road surface state requiring attention on the overhead view video 78. On the other hand, an image indicating a road surface state requiring attention may be superimposed on the video imaged by the imaging unit 10. FIG. 10 shows a video generated by the video processing unit 40. Here, a left front notification image 150 and a right front notification image 154 are arranged in the front video 170 captured by the front imaging unit 12. The left front notification image 150 and the right front notification image 154 are the same as before. Returning to FIG.

  The display control unit 42 inputs the overhead view video 78 from the video processing unit 40. The display control unit 42 displays the overhead view video 78 on the display unit 50 by executing a process of displaying the overhead view video 78. The display unit 50 is, for example, a display panel. A bird's-eye view image 78 as shown in any of FIGS. 7 to 10 is displayed on the display unit 50.

  This configuration can be realized in terms of hardware by a CPU, memory, or other LSI of any computer, and in terms of software, it can be realized by a program loaded in the memory, but here it is realized by their cooperation. Draw functional blocks. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The operation of the vehicle display device 30 configured as described above will be described. FIG. 11 is a flowchart showing a display procedure by the vehicle display device 30. The generation unit 34 generates the overhead view video 78 (S10). The road surface state acquisition unit 60 acquires the road surface state (S12). If there is a road surface condition requiring attention (Y in S14), if the stop determination unit 64 determines a stop state (Y in S16), the video processing unit 40 superimposes the image on the overhead view video 78 (S18). If there is no road surface condition requiring attention (N in S14), or if the stop determination unit 64 does not determine a stop state (N in S16), step 18 is skipped. The display unit 50 displays the overhead view video 78 (S20).

  According to the present embodiment, when it is determined that there is a road surface state requiring attention, an image indicating the road surface state requiring attention is superimposed on the overhead image, so that the road surface state requiring attention can be notified. Further, since the road surface state requiring attention is informed, it is possible to grasp whether or not the road surface is wet even in the overhead view image. In addition, when it is determined that a stop state is determined and there is a road surface condition that requires attention, an image showing the road surface condition that requires attention is superimposed on the overhead view image, so that the driver who is about to get off the vehicle has an overhead view. The road surface condition can be grasped also in the video.

  In addition, when it is determined that the position of the road surface state requiring attention exists in the vicinity of the door, an image is superimposed on the position of the door in the own vehicle icon in the overhead view image, so that the driver who is about to get off the vehicle On the other hand, the road surface state can be grasped even in the overhead view video. Further, since it is determined whether or not the road surface is wet based on the ultrasonic reflection state on the road surface, the road surface state can be specified. Moreover, since the intensity of the reflected wave with respect to the transmitted ultrasonic wave is compared with a threshold value, it can be determined whether or not the road surface is wet.

(Example 2)
Next, Example 2 will be described. Example 2 is related with the display apparatus for vehicles which displays the bird's-eye view image by which the road surface state was shown similarly to Example 1. FIG. In the second embodiment, a case where there is a road surface state that requires attention when the door of the vehicle is opened will be described. The vehicle 100 according to the second embodiment is the same type as that shown in FIGS. Here, it demonstrates centering on the difference from before.

  FIG. 12 shows a configuration of a vehicle display device 30 according to the second embodiment of the present invention. In the vehicle display device 30, a seating information acquisition unit 52 is added to the vehicle display device 30 of FIG. The seating information acquisition unit 52 is connected to seating sensors (not shown) disposed in each of a plurality of seats in the vehicle 100 and receives detection results from the seating sensors. A known technique may be used for the seating sensor. The detection result from the seating sensor indicates whether or not an occupant is sitting on the seat and can be said to be seating presence / absence information. Based on such seating presence / absence information, the seating information acquisition unit 52 identifies the seat on which the occupant is seated. The seating information acquisition unit 52 outputs information related to the seat on which the occupant is seated to the video processing unit 40.

  The video processing unit 40 inputs information regarding the seat on which the occupant is seated from the seating information acquisition unit 52. The video processing unit 40 identifies a door corresponding to a seated seat among the doors of the vehicle 100 based on the information. In addition, the video processing unit 40 superimposes an image indicating a road surface state requiring attention on the identified door position. For example, when the specified door position is the right door of the vehicle 100, the video processing unit 40 displays only the right side notification image 164 and does not display the left side notification image 162. Further, as described above, when the right side notification image 164 is indicated by “white solid color”, it can be said that an image indicating a road surface state requiring attention is superimposed. On the other hand, when the specified door position is the left door of the vehicle 100, the same processing is performed.

  According to the present embodiment, since the image is superimposed on the position of the door corresponding to the seat with the seat, the image can be displayed on the door that is likely to be used for the latter. Moreover, since an image is displayed with respect to the door with high possibility of being used for the latter, it can call attention.

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

  In the first and second embodiments, the road surface state acquisition unit 60 uses the sensor 110 to acquire the ultrasonic reflection state on the road surface 90 as the road surface state. However, the present invention is not limited to this. For example, the road surface state acquisition unit 60 may acquire the shade change state in the overhead image 78 as the road surface state using the overhead image 78 generated by the generation unit 34. At that time, the road surface state determination unit 62 determines whether or not the road surface 90 is wet based on the shade change state acquired by the road surface state acquisition unit 60. For example, the road surface state determination unit 62 derives a change in shade for each adjacent pixel, and determines that the portion is a boundary such as a puddle when the change in shade is greater than a threshold value. According to this modification, a shape such as a puddle can be specified.

  The road surface state acquisition unit 60 uses the sensor 110 to acquire the ultrasonic reflection state on the road surface 90 as the road surface state, and uses the overhead view video 78 generated by the generation unit 34 as the road surface state. You may acquire the change state of the light and shade in the image | video 78. In that case, the road surface state determination unit 62 detects a portion where the road surface 90 is wet based on the ultrasonic reflection state, and then, based on the change state of shading around the detected portion, a puddle or the like. Detect boundary of. According to this modification, the shape of a puddle etc. can be specified efficiently.

  In the first and second embodiments, the display control unit 42 causes the display unit 50 to display a bird's-eye view video 78 on which an image indicating a road surface state requiring attention is superimposed. However, the present invention is not limited thereto. For example, an LED (Light Emitting Diode) may be provided near the doorknob, and the display control unit 42 may display the LED in the same color as the screen. When a road surface condition requiring attention is detected, the door cannot be unlocked or the door may be forcibly locked. You may tell people who are getting on the road that there is a bad road near the door. According to this modification, it is possible to notify a road surface state requiring attention.

  DESCRIPTION OF SYMBOLS 10 Imaging part, 12 Front imaging part, 14 Left side imaging part, 16 Rear imaging part, 18 Right side imaging part, 20 Front imaging area, 22 Left side imaging area, 24 Rear imaging area, 26 Right side imaging area, 30 Vehicle Display device, 32 video acquisition unit, 34 generation unit, 40 video processing unit, 42 display control unit, 50 display unit, 52 seating information acquisition unit, 60 road surface state acquisition unit, 62 road surface state determination unit, 64 stop determination unit, 66 Position relationship determination unit, 100 vehicle, 110 sensor, 114 left side sensor, 118 right side sensor, 120 left front sensor, 122 right front sensor, 124 left rear sensor, 126 right rear sensor.

Claims (7)

An image acquisition unit that acquires an image of the periphery of the vehicle;
A generation unit that generates a bird's-eye view video by converting the viewpoint as viewed from above the vehicle with respect to the video acquired in the video acquisition unit;
A road surface state acquisition unit for acquiring a road surface state around the vehicle;
Based on the road surface state acquired by the road surface state acquisition unit, a road surface state determination unit that determines whether or not there is a road surface state requiring attention;
A stop determination unit that determines that the vehicle is in a stopped state when the vehicle is stopped or when the speed of the vehicle is less than a predetermined speed;
A video processing unit that superimposes an image indicating a road surface state requiring attention on a bird's-eye view image when the stop determination unit determines that the vehicle is stopped and the road surface state determination unit determines that a road surface state requiring attention exists. When,
A display control unit that causes the display unit to display a bird's-eye view image on which an image is superimposed in the video processing unit;
A vehicle display device comprising: When the road surface state determining unit determines that there is a road surface state requiring attention, the position of the road surface state requiring attention is determined based on the positional relationship between the position of the road surface state requiring attention and the door of the vehicle. A positional relationship determination unit that determines whether or not the vehicle exists in the vicinity of the door,
The generation unit adds an own vehicle icon indicating the position of the vehicle to the overhead view video,
When the positional relationship determination unit determines that the position of the road surface state requiring attention exists in the vicinity of the door of the vehicle, the video processing unit determines the vehicle in the vehicle icon in the overhead view video. The vehicle display device according to claim 1, wherein an image indicating a road surface state requiring attention is superimposed on the position of the door. The road surface state acquisition unit acquires an ultrasonic reflection state on the road surface as a road surface state,
The vehicle according to claim 1 or 2, wherein the road surface state determination unit determines whether or not the road surface is wet based on an ultrasonic reflection state of the road surface acquired by the road surface state acquisition unit. Display device. The road surface state acquisition unit acquires a change state of shading in an overhead image as a road surface state,
The said road surface state determination part determines whether the road surface is wet based on the change state of the shade which the said road surface state acquisition part acquired, The Claim 1 characterized by the above-mentioned. Vehicle display device. A seating information acquisition unit for acquiring seating presence / absence information for a plurality of seats in the vehicle;
5. The image processing unit according to claim 1, wherein an image indicating a road surface state requiring attention is superimposed on a position of a door corresponding to a seated seat among the doors of the vehicle. Vehicle display device. Obtaining an image of the surroundings of the vehicle;
Generating a bird's-eye view image by converting the viewpoint as seen from above the vehicle with respect to the acquired image; and
Obtaining a road surface condition around the vehicle;
Determining whether there is a road surface condition requiring attention based on the acquired road surface state;
Determining that the vehicle is in a stopped state when the vehicle is stopped, or when the speed of the vehicle is less than a predetermined speed;
A step of superimposing an image indicating a road surface state requiring attention on the overhead view image when it is determined that a stop state of the vehicle is determined and a road surface state requiring attention exists;
Displaying a bird's-eye view image on which an image is superimposed on a display unit;
A vehicle display method comprising the steps of: Obtaining an image of the surroundings of the vehicle;
Generating a bird's-eye view image by converting the viewpoint as seen from above the vehicle with respect to the acquired image; and
Obtaining a road surface condition around the vehicle;
Determining whether there is a road surface condition requiring attention based on the acquired road surface state;
Determining that the vehicle is in a stopped state when the vehicle is stopped, or when the speed of the vehicle is less than a predetermined speed;
A step of superimposing an image indicating a road surface state requiring attention on the overhead view image when it is determined that a stop state of the vehicle is determined and a road surface state requiring attention exists;
The program for making a computer perform the step which displays the bird's-eye view image with which the image was superimposed on a display part.

Images