WO2018055873A1 - Dispositif de génération d'image vidéo de vue aérienne, système de génération d'image vidéo de vue aérienne, procédé de génération d'image vidéo de vue aérienne, et programme - Google Patents

Dispositif de génération d'image vidéo de vue aérienne, système de génération d'image vidéo de vue aérienne, procédé de génération d'image vidéo de vue aérienne, et programme Download PDF

Info

Publication number
WO2018055873A1
WO2018055873A1 PCT/JP2017/024558 JP2017024558W WO2018055873A1 WO 2018055873 A1 WO2018055873 A1 WO 2018055873A1 JP 2017024558 W JP2017024558 W JP 2017024558W WO 2018055873 A1 WO2018055873 A1 WO 2018055873A1
Authority
WO
WIPO (PCT)
Prior art keywords
obstacle
video
overhead
image
vehicle
Prior art date
Application number
PCT/JP2017/024558
Other languages
English (en)
Japanese (ja)
Inventor
恒夫 里見
浅山　学
俊夫 森
Original Assignee
株式会社Ｊｖｃケンウッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2016184461A external-priority patent/JP6644264B2/ja
Priority claimed from JP2017073368A external-priority patent/JP6730617B2/ja
Application filed by 株式会社Ｊｖｃケンウッド filed Critical 株式会社Ｊｖｃケンウッド
Priority to EP17852654.7A priority Critical patent/EP3462733B1/fr
Priority to CN201780023741.3A priority patent/CN109076195B/zh
Publication of WO2018055873A1 publication Critical patent/WO2018055873A1/fr
Priority to US16/208,650 priority patent/US10587801B2/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/301—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8093—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for obstacle warning

Definitions

  • the present invention relates to an overhead video generation device, an overhead video generation system, an overhead video generation method, and a program.
  • Patent Document 1 A technique related to a vehicle periphery display device that displays a vehicle overhead view image together with a vehicle image is known (see, for example, Patent Document 1).
  • the display panel of an in-vehicle device such as a navigation device that displays a bird's-eye view image of the vehicle is often horizontally long. For this reason, a vertically long overhead image is not displayed on the entire surface of the display panel. Therefore, a technique is known in which, when an obstacle close to the host vehicle is detected, a bird's-eye view image of the vehicle and a video image of the direction of the obstacle are displayed side by side (for example, see Patent Document 2).
  • the present invention has been made in view of the above, and an object thereof is to enable appropriate confirmation of obstacles around a vehicle.
  • an overhead image generation apparatus includes a video acquisition unit that acquires a peripheral video obtained by photographing the periphery of a vehicle, and obstacles detected in the vicinity of the vehicle.
  • An obstacle information acquisition unit that acquires obstacle information, and an overhead view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit,
  • a video generation unit that generates a bird's-eye view image obtained by synthesizing information indicating the obstacle acquired by the obstacle information acquisition unit, and a display control unit that causes the display unit to display the bird's-eye view image generated by the video generation unit. It is characterized by.
  • An overhead image generation system includes the above-described overhead image generation device, a camera that captures the periphery of the vehicle and supplies a periphery image to the image acquisition unit, detects an obstacle around the vehicle, and detects the obstacle An obstacle detection unit that supplies obstacle information to the object information acquisition unit, and at least one of the display unit.
  • the bird's-eye view video generation method includes a video acquisition step of acquiring a peripheral video obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring obstacle information of an obstacle detected in the periphery of the vehicle, A bird's-eye view image is generated from the surrounding image acquired in the image acquisition step so that the vehicle is viewed from above, and the obstacle acquired in the obstacle information acquisition step is shown in the center of the bird's-eye view image.
  • the program according to the present invention includes a video acquisition step of acquiring a peripheral video obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring obstacle information of an obstacle detected in the vicinity of the vehicle, and the video acquisition step. Generates an overhead view image obtained by performing viewpoint conversion processing so that the vehicle is looked down from above, and synthesizes information indicating the obstacle acquired in the obstacle information acquisition step in the center of the overhead view image
  • the computer operating as the overhead video generation device executes a video generation step for generating the overhead video and a display control step for displaying the overhead video generated in the video generation step on the display unit.
  • FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment.
  • FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment.
  • FIG. 3 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the first embodiment.
  • FIG. 4 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment.
  • FIG. 5 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the first embodiment.
  • FIG. 6 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the second embodiment.
  • FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment.
  • FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment.
  • FIG. 3 is
  • FIG. 7 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the second embodiment.
  • FIG. 8 is a flowchart showing the flow of processing in the overhead video generation device of the overhead video generation system according to the third embodiment.
  • FIG. 9 is a diagram showing an example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 10 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 11 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 12 is a graph showing an example of the relationship between the distance between the arcs and the distance to the obstacle in the obstacle notification icon.
  • FIG. 13 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 14 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 15 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 16 is a block diagram illustrating a configuration example of the overhead view video generation system according to the seventh embodiment.
  • FIG. 17 is a diagram illustrating a display panel that displays an overhead video generated by the overhead video generation system according to the seventh embodiment.
  • FIG. 18 is a diagram showing a display panel that displays a horizontally long overhead image generated by the overhead image generation system according to the seventh embodiment.
  • FIG. 19 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the seventh embodiment.
  • FIG. 20 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 21 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the seventh embodiment.
  • FIG. 22 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eighth embodiment.
  • FIG. 23 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 24 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the ninth embodiment.
  • FIG. 20 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 21 is a flowchart showing a flow of processing in the overhead video generation device of the overhead
  • FIG. 25 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 26 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the tenth embodiment.
  • FIG. 27 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 28 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eleventh embodiment.
  • FIG. 29 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 30 is a diagram illustrating another example of a method for generating a horizontally long overhead image in the overhead image generation system.
  • FIG. 31 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 30.
  • FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment.
  • the bird's-eye view video generation system 1 generates a bird's-eye view video 100 (see FIG. 2).
  • the overhead image generation device 40 and the overhead image generation system 1 are mounted on a vehicle. In addition to what is mounted on the vehicle, the overhead image generation device 40 and the overhead image generation system 1 may be a portable device that can be used in the vehicle.
  • the overhead view video generation system 1 includes a front camera (camera) 11, a rear camera (camera) 12, a left side camera (camera) 13, a right side camera (camera) 14, and a front left sensor (obstacle detection unit).
  • 21A front center sensor (obstacle detection unit) 21B, front right sensor (obstacle detection unit) 21C, rear left sensor (obstacle detection unit) 22A, rear center sensor (obstacle detection unit) 22B , A rear right sensor (obstacle detection unit) 22C, a display panel 31, and an overhead image generation device 40.
  • the front camera 11 is arranged in front of the vehicle and photographs the periphery around the front of the vehicle.
  • the front camera 11 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.
  • the rear camera 12 is arranged at the rear of the vehicle and photographs the periphery around the rear of the vehicle.
  • the rear camera 12 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.
  • the left side camera 13 is arranged on the left side of the vehicle, and takes a picture around the left side of the vehicle.
  • the left side camera 13 outputs the captured video to the video acquisition unit 42 of the overhead view video generation device 40.
  • the right-side camera 14 is arranged on the right side of the vehicle and photographs the periphery around the right side of the vehicle.
  • the right side camera 14 outputs the captured video to the video acquisition unit 42 of the overhead view video generation device 40.
  • the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 photograph all directions of the vehicle.
  • the front left sensor 21A is disposed on the left front side of the vehicle and detects an obstacle on the left front side of the vehicle.
  • the front left sensor 21A is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the front left sensor 21 ⁇ / b> A detects an object having a height from the ground that may come into contact with the vehicle. For example, the front left sensor 21A detects an obstacle at a distance of about 5 m from the vehicle.
  • the front left sensor 21 ⁇ / b> A detects an obstacle in a range of, for example, about 40 ° with the center of the sensor as the center when viewed in the vertical direction.
  • the detection range of the front left sensor 21A may overlap with a part of the detection range of the front center sensor 21B.
  • the front left sensor 21 ⁇ / b> A outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front left sensor 21A, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the front center sensor 21B is arranged at the front center of the vehicle and detects an obstacle at the front center of the vehicle.
  • the front center sensor 21B is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the front center sensor 21B detects a sensor having a height from the ground that may come into contact with the vehicle.
  • the front center sensor 21B detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the front center sensor 21B detects an obstacle in a range of, for example, about 40 ° with the center portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the front center sensor 21B may overlap with a part of the detection range of the front left sensor 21A and the front right sensor 21C.
  • the front center sensor 21B outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front center sensor 21B, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the front right sensor 21C is arranged on the front right side of the vehicle and detects an obstacle on the front right side of the vehicle.
  • the front right sensor 21C is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the front right sensor 21 ⁇ / b> C detects a thing having a height from the ground that may come into contact with the vehicle.
  • the front right sensor 21C detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the front right sensor 21C detects an obstacle in a range of, for example, about 40 ° with the center portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the front right sensor 21C may overlap with a part of the detection range of the front center sensor 21B.
  • the front right sensor 21 ⁇ / b> C outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front right sensor 21C, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the front left sensor 21A, the front center sensor 21B, and the front right sensor 21C detect an obstacle ahead of the vehicle.
  • the rear left sensor 22A is disposed on the rear left side of the vehicle and detects an obstacle on the rear left side of the vehicle.
  • the rear left sensor 22A is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the rear left sensor 22A detects an object having a height from the ground that may come into contact with the vehicle.
  • the rear left sensor 22A detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the rear left sensor 22A detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the rear left sensor 22A may overlap with a part of the detection range of the rear center sensor 22B.
  • the rear left sensor 22A outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear left sensor 22A, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the rear center sensor 22B is disposed at the rear center of the vehicle and detects an obstacle at the rear center of the vehicle.
  • the rear center sensor 22B is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • Back center sensor 22B detects what has a height from the ground which may be in contact with a vehicle.
  • the rear center sensor 22B detects an obstacle at a distance of about 5 m from the vehicle.
  • the rear center sensor 22B detects an obstacle in a range of, for example, about 40 ° with the center of the sensor as the center when viewed in the vertical direction.
  • the detection range of the rear center sensor 22B may overlap with a part of the detection range of the rear left sensor 22A and the rear right sensor 22C.
  • the rear center sensor 22B outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead image generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear center sensor 22B, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the rear right sensor 22C is disposed on the rear right side of the vehicle and detects an obstacle on the rear right side of the vehicle.
  • the rear right sensor 22C is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the rear right sensor 22C detects an object having a height from the ground that may come into contact with the vehicle.
  • the rear right sensor 22C detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the rear right sensor 22C detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the rear right sensor 22C may overlap with a part of the detection range of the rear center sensor 22B.
  • the rear right sensor 22C outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear right sensor 22C, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C detect an obstacle behind the vehicle.
  • the display panel 31 is a display including, for example, a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminescence) display.
  • the display panel 31 displays the overhead video 100 based on the video signal output from the overhead video generation device 40 of the overhead video generation system 1.
  • the display panel 31 may be dedicated to the overhead view video generation system 1 or may be used jointly with other systems including a navigation system, for example.
  • the display panel 31 is disposed at a position that is easily visible to the driver.
  • the overhead view video generation device 40 includes a control unit 41 and a storage unit 49.
  • the control unit 41 is an arithmetic processing unit configured with, for example, a CPU (Central Processing Unit).
  • the control unit 41 loads the program stored in the storage unit 49 into the memory and executes instructions included in the program.
  • the control unit 41 includes a video acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, an overhead video generation unit (video generation unit) 46, a superimposed video generation unit (video generation unit) 47, Display control unit 48.
  • the video acquisition unit 42 acquires a peripheral video obtained by photographing the periphery of the vehicle. More specifically, the video acquisition unit 42 acquires video output from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The video acquisition unit 42 outputs the acquired video to the overhead view video generation unit 46.
  • the obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in the vicinity of the vehicle.
  • the obstacle information acquisition unit 43 acquires obstacle information including the distance to the detected obstacle. More specifically, the obstacle information acquisition unit 43 acquires the obstacle information output by the front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C. To do.
  • the obstacle information acquisition unit 43 outputs the acquired obstacle information to the superimposed video generation unit 47.
  • the vehicle information acquisition unit 44 acquires vehicle information serving as a trigger for displaying a bird's-eye view video, such as vehicle gear operation information, from a CAN (Controller Area Network) or various sensors that sense the state of the vehicle.
  • vehicle information acquisition unit 44 outputs the acquired vehicle information to the overhead view video generation unit 46.
  • the bird's-eye view video generation unit 46 generates the bird's-eye view video 100 that has been subjected to the viewpoint conversion process so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. More specifically, the bird's-eye view image generation unit 46 generates the bird's-eye view image 100 based on images taken by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the bird's-eye view image 100 may be any known method and is not limited.
  • the overhead image generation unit 46 outputs the generated overhead image 100 to the display control unit 47.
  • the overhead view image 100 will be described with reference to FIG.
  • FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment.
  • the bird's-eye view image 100 has a vertically long rectangular shape.
  • the bird's-eye view image 100 is located at the center surrounded by the front image 101, the rear image 102, the left side image 103, the right side image 104, the front image 101, the rear image 102, the left side image 103, and the right side image 104.
  • the front video 101, the rear video 102, the left side video 103, the right side video 104, and the central video 105 may be separated by a frame-like boundary line.
  • the central video 105 is generated in a vertically long rectangular shape. In the central video 105, the boundaries of the front video 101, the rear video 102, the left video 103, and the right video 104 are separated by lines.
  • the center image 105 shows the position of the vehicle in the overhead view image 100.
  • FIG. 2 diagonal broken lines indicating boundaries between the front video 101, the rear video 102, the left side video 103, and the right side video 104 are illustrated for explanation, but are actually displayed on the display panel 31.
  • the broken line is not displayed in the overhead view video 100. The same applies to the other figures.
  • the superimposed video generation unit 47 generates an overhead video 100 in which information indicating an obstacle is superimposed on the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43.
  • the superimposed video generation unit 47 superimposes information indicating the direction in which the obstacle is detected on the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. 100 is generated.
  • the superimposed video generation unit 47 indicates information indicating the direction in which the obstacle is detected as an obstacle notification icon (information indicating the obstacle) 120.
  • the obstacle notification icon 120 schematically shows the detection direction in the horizontal direction of each sensor that detects an obstacle, and the arrangement direction of the plurality of arcs is the detection direction starting from the sensor mounting position, or the vehicle. For example, a radial arrangement with the center.
  • the width of the arc constituting the obstacle notification icon 120 may indicate the detection range of each sensor that detects the obstacle, or may be a fixed width corresponding to the detection direction regardless of the detection range of each sensor. .
  • the obstacle notification icon 120 is an icon for notifying an obstacle.
  • the obstacle notification icon 120 indicates the distance and direction to the obstacle.
  • the obstacle notification icon 120 includes a front left icon (information indicating an obstacle) 121, a front center icon (information indicating an obstacle) 122, a front right icon (information indicating an obstacle) 123, and a rear left icon (an obstacle is displayed).
  • the front left icon 121 is an icon for notifying an obstacle on the left front side of the vehicle. More specifically, the front left icon 121 is an icon that notifies that an obstacle has been detected by the front left sensor 21A. The front left icon 121 is superimposed on the upper left of the central image 105 of the overhead view image 100 in FIG.
  • the front left icon 121 is composed of a triple arc-shaped curve.
  • the arc-shaped curve bulges toward the outside of the overhead view image 100.
  • the triple arc-shaped curve has a radius that decreases from the outside of the bird's-eye view image 100 toward the center.
  • the triple arc-shaped curve is shortened from the outside of the overhead view image 100 toward the center side.
  • the front left icon 121 may change the color according to the distance to the obstacle. For example, the front left icon 121 is displayed in green when the distance to the obstacle is greater than or equal to the first predetermined distance. The front left icon 121 is displayed in yellow when the distance to the obstacle is equal to or greater than a second predetermined distance that is less than the first predetermined distance and smaller than the first predetermined distance. The front left icon 121 is displayed in red when the distance to the obstacle is less than the second predetermined distance.
  • the front left icon 121 may change the number of arcuate curves according to the distance to the obstacle. For example, the front left icon 121 displays the outermost arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is equal to or greater than the first predetermined distance. The front left icon 121 displays the outermost arc-shaped curve and the center arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is less than the first predetermined distance and greater than or equal to the second predetermined distance. To do. The front left icon 121 displays all triple arcuate curves when the distance to the obstacle is less than the second predetermined distance.
  • the front center icon 122 is an icon for notifying an obstacle at the front center of the vehicle. More specifically, the front center icon 122 is an icon that notifies that an obstacle has been detected by the front center sensor 21B. The front center icon 122 is superimposed on the upper center of the center image 105 of the overhead image 100 in FIG. The front center icon 122 is configured in the same manner as the front left icon 121.
  • the front right icon 123 is an icon for notifying an obstacle on the right front side of the vehicle. More specifically, the front right icon 123 is an icon that notifies that an obstacle has been detected by the front right sensor 21C. The front right icon 123 is superimposed on the upper right of the central image 105 of the overhead image 100 in FIG. The front right icon 123 is configured in the same manner as the front left icon 121.
  • the rear left icon 124 is an icon for notifying an obstacle on the left rear side of the vehicle. More specifically, the rear left icon 124 is an icon for notifying that an obstacle has been detected by the rear left sensor 22A.
  • the rear left icon 124 is superimposed on the lower left of the central image 105 of the overhead image 100 in FIG.
  • the rear left icon 124 is configured in the same manner as the front left icon 121.
  • the rear center icon 125 is an icon that notifies an obstacle at the rear center of the vehicle. More specifically, the rear center icon 125 is an icon for notifying that an obstacle has been detected by the rear center sensor 22B. The rear center icon 125 is superimposed on the lower center of the center image 105 of the overhead image 100 in FIG. The rear center icon 125 is configured in the same manner as the front left icon 121.
  • the rear right icon 126 is an icon for notifying an obstacle on the rear right side of the vehicle. More specifically, the rear right icon 126 is an icon for notifying that an obstacle has been detected by the rear right sensor 22C. The rear right icon 126 is superimposed on the lower right of the central image 105 of the overhead image 100 in FIG. The rear right icon 126 is configured in the same manner as the front left icon 121.
  • the display control unit 48 causes the display panel 31 to display the overhead video 100 generated by the superimposed video generation unit 47.
  • the storage unit 49 stores data required for various processes in the overhead view video generation device 40 and various processing results.
  • the storage unit 49 is, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.
  • FIG. 3 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the first embodiment.
  • the control unit 41 determines whether or not to start the overhead view video display (step S11). As an example of the determination to start the overhead view video display, the control unit 41 determines whether or not to disclose the overhead view video display based on the presence or absence of a reverse trigger.
  • the reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle is rearward in the front-rear direction of the vehicle.
  • the control unit 41 determines not to start the overhead view video display (No in step S11), and executes the process of step S11 again.
  • the control unit 41 determines to start the overhead view video display (Yes in Step S11), and proceeds to Step S12.
  • the control unit 41 generates and displays the bird's-eye view video 100 (step S12). More specifically, the control unit 41 causes the overhead view video generation unit 46 to generate the overhead view video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the control unit 41 determines whether an obstacle has been detected (step S13). More specifically, the control unit 41 determines whether or not obstacle information has been acquired by the obstacle information acquisition unit 43. When it is determined that the obstacle information acquisition unit 43 has acquired the obstacle information (Yes in step S13), the control unit 41 proceeds to step S14. When it is determined that the obstacle information acquisition unit 43 has not acquired the obstacle information (No in step S13), the control unit 41 proceeds to step S15.
  • the control unit 41 superimposes and displays an obstacle notification icon 120 indicating an obstacle at the center of the overhead view video 100 (step S14). More specifically, the control unit 41 causes the superimposed video generation unit 47 to indicate the direction in which the obstacle is detected in the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. The overhead view video 100 in which the obstacle notification icon 120 is superimposed on is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the generation unit 47 When the control unit 41 acquires obstacle information from a plurality of sensors including the front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C,
  • the generation unit 47 generates the overhead image 100 in which a plurality of obstacle notification icons 120 are superimposed on the central image 105 of the overhead image 100.
  • the control unit 41 determines whether or not to end the overhead view video display (step S15). More specifically, the control unit 41 determines whether or not to end the bird's-eye view video display based on the presence or absence of the reverse trigger. When there is no backward trigger, in other words, when the backward trigger is released, the control unit 41 determines to end the overhead view video display (Yes in step S15), and ends the process. When there is a reverse trigger, the control unit 41 determines not to end the overhead view video display (No in step S15), returns to step S13, and continues the process.
  • the overhead image generation system 1 superimposes the obstacle notification icon 120 on the center of the overhead image 100 so as to indicate the direction in which the obstacle is detected, and displays the display panel. 31 outputs a video signal.
  • the display panel 31 displays the bird's-eye view video 100 together with the navigation screen, for example, based on the video signal output from the bird's-eye view video generation system 1.
  • FIG. 4 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment.
  • the rear image 102 of the overhead view image 100 includes an obstacle image 130 of the obstacle on the left rear side.
  • Control unit 41 determines that an obstacle has been detected in step S13. Then, in step S14, the control unit 41 detects the obstacle in the central image 105 of the overhead view video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43 in the superimposed image generation unit 47, and then left An overhead view image 100 in which the obstacle notification icon 120 is superimposed so as to indicate the direction is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • FIG. 5 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the first embodiment.
  • a vehicle icon 140 that is a vehicle icon indicating a vehicle when the vehicle is viewed from above is synthesized with the central image 105 of the overhead view image 100.
  • the control unit 41 causes the superimposed video generation unit 47 to generate an overhead video 100 in which the obstacle notification icon 120 is superimposed on the vehicle icon 140 so as to indicate the left direction after the obstacle is detected.
  • the overhead notification icon 120 when an obstacle is detected around the vehicle, the overhead notification icon 120 is superimposed on the central image 105 of the overhead image 100 so as to indicate the direction in which the obstacle is detected.
  • the video 100 is displayed on the display panel 31.
  • the obstacle notification icon 120 since the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100, the obstacle notification icon 120 and the obstacle reflected in the overhead image 100 are not superimposed.
  • the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
  • this embodiment can make it possible to appropriately check obstacles around the vehicle.
  • the obstacle notification icon 120 can notify the direction and distance in which the obstacle is detected.
  • this embodiment can further improve the visibility of the obstacle notification icon 120 when the vehicle icon 140 is not displayed on the central image 105 of the overhead view image 100.
  • this embodiment can make it possible to more appropriately check obstacles around the vehicle.
  • the vehicle icon 140 may be displayed on the center image 105 of the overhead view image 100.
  • this embodiment can display the bird's-eye view image 100 which can recognize the direction of the obstacle with respect to a vehicle more easily.
  • this embodiment can make it possible to appropriately check obstacles around the vehicle.
  • FIG. 6 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the second embodiment.
  • FIG. 7 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the second embodiment.
  • the overhead image generation system 1 has the same basic configuration as the overhead image generation system 1 of the first embodiment. In the following description, components similar to those in the overhead view video generation system 1 are denoted by the same reference numerals or corresponding reference numerals, and detailed description thereof is omitted.
  • the overhead image generation system 1 of the present embodiment is different from the overhead image generation system 1 of the first embodiment in the processing of the obstacle information acquisition unit 43, the superimposed image generation unit 47, and the control unit 41 of the overhead image generation device 40. .
  • the obstacle notification icon 120 includes a second front left icon 121, a front center icon 122, a front right icon 123, a rear left icon 124, a rear center icon 125, and a rear right icon 126 in the first embodiment.
  • Icon information indicating obstacle
  • second front center icon information indicating obstacle
  • second front right icon information indicating obstacle
  • second rear left icon information indicating obstacle
  • second A rear center icon information indicating an obstacle
  • a second rear right icon information indicating an obstacle
  • the front left icon 121 is an icon for notifying an obstacle on the left front side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front left icon 121 is an icon for notifying that the front left sensor 21A has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the front center icon 122 is an icon for notifying an obstacle at the front center of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front center icon 122 is an icon for notifying that the front center sensor 21B has detected an obstacle whose distance to the obstacle is less than a threshold value.
  • the front right icon 123 is an icon for notifying an obstacle on the right front side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front right icon 123 is an icon for informing that the front right sensor 21C has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the rear left icon 124 is an icon for notifying an obstacle on the left rear side of the vehicle whose distance to the obstacle is less than the threshold. More specifically, the rear left icon 124 is an icon that notifies that the rear left sensor 22A has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the rear center icon 125 is an icon for notifying an obstacle at the rear center of the vehicle whose distance to the obstacle is less than a threshold value. More specifically, the rear center icon 125 is an icon for informing that the rear center sensor 22B has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the rear right icon 126 is an icon for notifying an obstacle on the right rear side of the vehicle whose distance to the obstacle is less than a threshold value. More specifically, the rear right icon 126 is an icon for notifying that the rear right sensor 22C has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the second front left icon is an icon for notifying an obstacle on the left front side of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front left icon is an icon for notifying that the front left sensor 21A has detected an obstacle whose distance to the obstacle is not less than a threshold value.
  • the second front left icon is superimposed on the upper left outside the central image 105 of the overhead image 100.
  • the second front left icon is configured in the same manner as the front left icon 121.
  • the second front center icon is an icon for notifying an obstacle at the front center of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front center icon is an icon for notifying that the front center sensor 21B has detected an obstacle whose distance to the obstacle is a threshold value or more.
  • the second front center icon is superimposed on the upper center outside the center image 105 of the overhead view image 100.
  • the second front center icon is configured in the same manner as the front left icon 121.
  • the second front right icon is an icon for notifying an obstacle on the right front side of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front right icon is an icon for informing that the front right sensor 21C has detected an obstacle whose distance to the obstacle is not less than a threshold value.
  • the second front right icon is superimposed on the upper right outside the center image 105 of the overhead view image 100.
  • the second front right icon is configured in the same manner as the front left icon 121.
  • the second rear left icon is an icon for notifying an obstacle on the left rear side of the vehicle whose distance to the obstacle is equal to or greater than a threshold value. More specifically, the second rear left icon is an icon for notifying that the rear left sensor 22A has detected an obstacle whose distance to the obstacle is a threshold value or more.
  • the second rear left icon is superimposed on the lower left outside the center image 105 of the overhead view image 100.
  • the second rear left icon is configured in the same manner as the front left icon 121.
  • the second rear center icon is an icon for notifying an obstacle at the rear center of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second rear center icon is an icon for informing that the rear center sensor 22B has detected an obstacle whose distance to the obstacle is a threshold value or more. The second rear center icon is superimposed on the lower center outside the center image 105 of the overhead image 100. The second rear center icon is configured in the same manner as the front left icon 121.
  • the second rear right icon is an icon for notifying an obstacle on the right rear side of the vehicle whose distance to the obstacle is a threshold value or more. More specifically, the second rear right icon is an icon for notifying that the rear right sensor 22C has detected an obstacle whose distance to the obstacle is not less than a threshold value.
  • the second rear right icon is superimposed on the lower right outside the center image 105 of the overhead view image 100.
  • the second rear right icon is configured in the same manner as the front left icon 121.
  • the obstacle information acquisition unit 43 acquires information including the distance to the detected obstacle.
  • the obstacle information acquisition unit 43 outputs information including the acquired distance to the obstacle to the superimposed video generation unit 47.
  • the superimposed image generation unit 47 displays the vehicle icon 140 in the central image 105 of the overhead image 100. Is displayed, and an overhead video 100 is generated by superimposing an obstacle notification icon 120, which is information indicating an obstacle, on the outside of the central portion of the overhead video 100.
  • the superimposed video generation unit 47 displays the obstacle notification icon 120 in the center of the overhead view video 100.
  • the overhead view video 100 superimposed on the video 105 is generated.
  • step S21, step S23, step S26, and step S27 in the flowchart shown in FIG. 6 is the same as the processing in step S11, step S13, step S14, and step S15 in the flowchart shown in FIG.
  • the control unit 41 generates and displays the bird's-eye view image 100 on which the host vehicle icon 140 is superimposed (step S22). More specifically, the control unit 41 causes the overhead view video generation unit 46 to generate the overhead view video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. The control unit 41 synthesizes the vehicle icon 140 in the center of the generated bird's-eye view image 100 with the display control unit 48. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the control unit 41 determines whether or not the distance to the obstacle is greater than or equal to a threshold value (step S24). More specifically, when the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold (Yes in step S24), the control unit 41 proceeds to step S25. When the distance to the detected obstacle acquired by the obstacle information acquiring unit 43 is less than the predetermined threshold (No in step S24), the control unit 41 proceeds to step S26.
  • the predetermined threshold indicates that the position at which the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon are displayed is an overhead image 100. It is set to a value that does not overlap with the obstacle video. More specifically, the predetermined threshold is the outermost arc shape of the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon. Any value that is equal to or greater than the distance from the vehicle corresponding to the position of the curve is acceptable. For example, the predetermined threshold may be about 2 m.
  • the control unit 41 superimposes and displays the obstacle notification icon 120 on the bird's-eye view image 100 outside the host vehicle icon 140 (step S25). More specifically, the control unit 41 uses the superimposed image generation unit 47 to display the second front left icon and the overhead image 100 on the outside of the vehicle icon 140 based on the obstacle information acquired by the obstacle information acquisition unit 43. An overhead view image 100 in which at least one of a second front center icon, a second front right icon, a second rear left icon, a second rear center icon, and a second rear right icon is superimposed is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the overhead view video generation system 1 displays the second forward left icon, the second forward center icon, and the first forward icon 100 on the overhead view video 100 outside the vehicle icon 140.
  • a bird's-eye view image 100 in which at least one of a second front right icon, a second rear left icon, a second rear center icon, and a second rear right icon is superimposed is generated, and a video signal is output to the display panel 31.
  • the overhead view video generation system 1 includes a front left icon 121, a front center icon 122, a front right icon 123, a rear left icon 124, and a rear side.
  • An overhead view image 100 in which at least one of the center icon 125 and the rear right icon 126 is superimposed is generated, and a video signal is output to the display panel 31.
  • Control unit 41 determines that an obstacle has been detected in step S23.
  • step S24 the control unit 41 determines that the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold (Yes in step S24).
  • step S25 the control unit 41 superimposes the second rear left icon 127 indicating the left direction after detecting the obstacle on the overhead view video 100 outside the host vehicle icon 140, as shown in FIG. Is generated.
  • the control unit 41 displays the generated overhead image 100 on the display panel 31.
  • the present embodiment when the distance to the obstacle is equal to or greater than the predetermined threshold, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the overhead image 100 outside the host vehicle icon 140 is generated and displayed. Display on the panel 31.
  • the obstacle notification icon 120 when the distance to the obstacle is equal to or greater than a predetermined threshold, the obstacle notification icon 120 is superimposed on the overhead view image 100 outside the host vehicle icon 140, so that the obstacle notification icon 120 and the overhead view image 100 are reflected. Obstacles do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.
  • the present embodiment when the distance to the obstacle is less than the predetermined threshold, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100 is generated and displayed on the display panel 31.
  • the obstacle notification icon 120 when the distance to the obstacle is less than the predetermined threshold, the obstacle notification icon 120 is superimposed on the central image 105 of the overhead view video 100, so that the obstacle reflected in the obstacle notification icon 120 and the overhead view image 100 is displayed. And do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
  • the display position of the obstacle notification icon 120 in the overhead view video 100 changes according to the distance to the obstacle. For this reason, this embodiment can make it easier to grasp the distance to the obstacle.
  • FIG. 8 is a flowchart showing the flow of processing in the overhead video generation device of the overhead video generation system according to the third embodiment.
  • the overhead image generation system 1 of the present embodiment is different from the overhead image generation system 1 of the second embodiment in the processing of the obstacle information acquisition unit 43, the superimposed image generation unit 47, and the control unit 41 of the overhead image generation device 40. .
  • the front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C have a detection range that is farther from the range displayed as the overhead image 100.
  • the obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in a range far from the range displayed as the overhead view video 100.
  • the superimposed image generation unit 47 is a central portion of the overhead image 100 if the distance to the detected obstacle is far from the range displayed as the overhead image 100.
  • the vehicle icon 140 is displayed on the vehicle, and the overhead image 100 is generated by superimposing the obstacle notification icon 120 on the bird's-eye image 100 outside the vehicle icon 140.
  • the superimposed video generation unit 47 is based on the obstacle information acquired by the obstacle information acquisition unit 43. If the distance to the detected obstacle is within the range displayed as the overhead image 100, the obstacle notification icon The overhead view image 100 is generated by superimposing 120 on the central image 105 of the overhead view image 100 so as to indicate the direction in which the obstacle is detected.
  • steps S31 to S33 and steps S35 to S37 in the flowchart shown in FIG. 8 are the same as the processes in steps S21 to S23 and steps S25 to S27 in the flowchart shown in FIG.
  • the control unit 41 determines whether or not the distance to the obstacle is greater than or equal to the display range of the bird's-eye view image 100 (step S34).
  • the control unit 41 proceeds to step S35.
  • the control unit 41 proceeds to step S36.
  • the overhead view video generation system 1 displays the obstacle notification icon 120 on the overhead view video 100 outside the vehicle icon 140 when the distance to the obstacle is a range farther than the display range of the overhead view video 100.
  • the superimposed bird's-eye view image 100 is generated and a video signal is output to the display panel 31.
  • the overhead image generation system 1 When the distance to the obstacle is included in the display range of the overhead image 100, the overhead image generation system 1 generates the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100, and the display panel 31 outputs a video signal.
  • the obstacle notification icon 120 is superimposed on the bird's-eye view image 100 outside the host vehicle icon 140 when the distance to the obstacle is a range farther than the display range of the bird's-eye view image 100.
  • An overhead image 100 is generated and displayed on the display panel 31.
  • the obstacle notification icon 120 is superimposed on the outside of the vehicle icon 140, so the obstacle notification icon 120 and the overhead view video are displayed.
  • the obstacle reflected in 100 does not overlap.
  • the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
  • this embodiment can make it possible to appropriately check obstacles around the vehicle.
  • the present embodiment when the distance to the obstacle is included in the display range of the overhead image 100, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100 is generated and displayed on the display panel 31.
  • the obstacle notification icon 120 when the distance to the obstacle is included in the display range of the overhead image 100, the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100.
  • the obstacles reflected in the do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
  • This embodiment can display the obstacle notification icon 120 even when there is an obstacle in a range far from the display range of the overhead view video 100. For this reason, this embodiment can notify the obstacle of the range far from the display range of the bird's-eye view image 100.
  • the overhead view video generation system 1 according to the present embodiment will be described.
  • the overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in processing in the superimposed video generation unit 47 and the control unit 41 of the overhead video generation device 40. More specifically, it differs from the bird's-eye view image generation system 1 of the first embodiment in that an obstacle located in the traveling direction of the vehicle is detected in determining whether an obstacle is detected in the control unit 41.
  • the superimposed image generation unit 47 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 47 superimposes the obstacle notification icon 120 on the central image 105 of the overhead view video 100 for the obstacle located in the traveling direction of the vehicle.
  • the overhead view video 100 is generated.
  • step S13 the control unit 41 determines whether an obstacle is detected in the traveling direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43. When it is determined that the obstacle is detected in the traveling direction of the vehicle (Yes in Step S13), the control unit 41 proceeds to Step S14. When it is determined that no obstacle is detected in the traveling direction of the vehicle (No in step S13), the control unit 41 proceeds to step S15.
  • control unit 41 displays the obstacle notification icon 120 for obstacles approaching the vehicle, and hides the obstacle notification icon 120 for obstacles moving away from the vehicle.
  • this embodiment can make it possible to more appropriately check obstacles around the vehicle.
  • FIG. 9 is a diagram showing an example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 10 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 11 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 12 is a graph showing an example of the relationship between the distance between the arcs and the distance to the obstacle in the obstacle notification icon.
  • the overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in the processing in the superimposed video generation unit 47 of the overhead video generation device 40.
  • the obstacle notification icon 120 has a plurality of components facing in the direction in which the obstacle is detected, and indicates the distance to the obstacle by the interval between the plurality of components.
  • the obstacle notification icon 120 includes a triple arc as a constituent element.
  • the obstacle notification icon 120 indicates the distance to the obstacle at intervals of arcs.
  • the obstacle notification icon 120 indicates that the arc interval is wider as the distance to the obstacle is larger, and the arc interval is narrower as the distance to the obstacle is smaller.
  • the interval between the arcs of the obstacle notification icon 120 may be set in a straight line so as to increase as the distance to the obstacle increases as shown by a solid line in FIG.
  • the interval between the arcs of the obstacle notification icon 120 may be set in a staircase shape so as to increase as the distance to the obstacle increases as indicated by a broken line in FIG. More specifically, when the distance to the obstacle is not less than the first threshold, for example, not less than 2 m, the arc interval is set as the first interval d1.
  • the first threshold may be a distance of a boundary included in a range to be displayed as the overhead view video 100. When the distance to the obstacle is equal to or greater than the first threshold, the outermost arc-shaped curve may be displayed outside the central image 105 of the overhead image 100.
  • the arc interval is set to a second interval d2 that is smaller than the first interval d1.
  • the arc interval is set to a third interval d3 that is smaller than the second interval d2.
  • the arcs may be overlapped with the interval between the arcs set to zero.
  • the distance to the obstacle is the distance from each sensor that detects the obstacle to the obstacle.
  • the distance to the obstacle substantially coincides with the distance from the end of the vehicle in the obstacle direction to the obstacle.
  • the superimposed video generation unit 47 displays the obstacle notification icon 120 in which the arc interval is changed based on the detected distance to the obstacle acquired by the obstacle information acquisition unit 43 in the central video 105 of the overhead view video 100.
  • the superimposed bird's-eye view image 100 is generated.
  • step S14 the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc.
  • the overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated.
  • the arc interval is the first interval d1.
  • the outermost arc-shaped curve is located outside the central image 105.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 9 on the display panel 31. Since the arc interval of the obstacle notification icon 120 and the outermost arc-shaped curve are located outside the central image 105, the obstacle is located near the boundary of the range to be displayed as the overhead image 100. I understand that.
  • step S14 the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc.
  • the overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated.
  • the arc interval is the second interval d2.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 10 on the display panel 31. It can be seen from the arc interval of the obstacle notification icon 120 that the obstacle is located within the range displayed as the bird's-eye view video 100.
  • step S14 the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc.
  • the overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated.
  • the arc interval is the third interval d3.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 11 on the display panel 31. Since the arc interval of the obstacle notification icon 120 is narrow, it can be seen that the obstacle is located in the vicinity of the rear end of the vehicle.
  • control unit 41 generates the overhead image 100 in which the obstacle notification icon 120 with the arc interval changed is superimposed on the central image 105 of the overhead image 100 based on the detected distance to the obstacle. .
  • the center image 105 of the overhead view image 100 indicates the direction in which the obstacle is detected, and the distance to the obstacle is indicated by the arc interval.
  • the overhead image 100 on which the obstacle notification icon 120 is superimposed is displayed on the display panel 31.
  • the obstacle notification icon 120 can display the direction of the obstacle and the distance to the obstacle.
  • FIG. 13 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 14 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 15 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • the overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in the processing in the superimposed video generation unit 47 of the overhead video generation device 40.
  • the obstacle notification icon 120 has a horizontal detection range of the sensor detecting the obstacle and a horizontal direction of the detected obstacle depending on the horizontal width of the icon itself. Indicates the width.
  • the obstacle notification icon 120 has a larger arc length as the detection range in the horizontal direction of the sensor detecting the obstacle is larger, and the detection range in the horizontal direction of the sensor detecting the obstacle. The narrower is the smaller the arc length.
  • the obstacle notification icon 120 indicates that the arc length increases as the obstacle's horizontal width increases, and the arc length decreases as the obstacle's horizontal width decreases.
  • the obstacle notification icon 120 may have a projected width from the center of the overhead image 100 to both ends of the obstacle in the horizontal direction as the length of the arc.
  • the superimposed image generation unit 47 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 47 sets the width of the information indicating the obstacle as a width corresponding to the horizontal width of the detected obstacle. An overhead image 100 superimposed on the central image 105 is generated.
  • the control unit 41 includes a detection range in the horizontal direction of the sensor that detects the obstacle, based on the horizontal width of the icon itself, in the central video 105 of the overhead video 100 in the superimposed video generation unit 47.
  • An overhead view image 100 on which an obstacle notification icon 120 indicating the horizontal width of the detected obstacle is superimposed is generated.
  • the horizontal width of the obstacle here is the apparent width detected by the sensor, that is, the projected width.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead image 100 illustrated in FIG. 13 on the display panel 31. It can be seen from the horizontal width of the obstacle notification icon 120 that the detection range in the horizontal direction of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrow.
  • step S ⁇ b> 14 the control unit 41 causes the superimposed video generation unit 47 to generate the overhead video 100 in which the obstacle notification icon 120 is superimposed on the central video 105 of the overhead video 100. Since the projected width from the center of the overhead view image 100 to both ends in the horizontal direction of the obstacle is wide, the horizontal width of the obstacle notification icon 120 is the horizontal width of the obstacle notification icon 120 of FIG. Wider.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 14 on the display panel 31. Since the horizontal width of the obstacle notification icon 120 is wider than that in FIG. 13, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are widened. That is, it can be seen that the obstacle is approaching the rear end of the vehicle.
  • the superimposed video generation unit 47 displays the horizontal width of the obstacle notification icon 120 as shown in FIGS. 13 to 14 in accordance with the proximity state of the obstacle.
  • a change in the width of the obstacle notification icon 120 in the horizontal direction indicates whether the obstacle is approaching or moving away from the vehicle.
  • step S ⁇ b> 14 the control unit 41 causes the superimposed video generation unit 47 to generate the overhead video 100 in which the obstacle notification icon 120 is superimposed on the central video 105 of the overhead video 100.
  • the obstacle notification icon 120 has a horizontal width that is narrower than the horizontal width of the obstacle notification icon 120 of FIG.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead image 100 illustrated in FIG. 15 on the display panel 31. Since the horizontal width of the obstacle notification icon 120 is narrower than that in FIG. 14, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrowed. That is, it can be seen that the horizontal width of the obstacle is narrow.
  • the center image 105 of the bird's-eye view image 100 indicates the direction in which the obstacle is detected, and the obstacle width is indicated by the horizontal width of the arc.
  • An overhead video 100 on which the obstacle notification icon 120 is superimposed is displayed on the display panel 31 so as to indicate the width in the horizontal direction.
  • the obstacle notification icon 120 can display the direction of the obstacle and the horizontal width of the obstacle.
  • the horizontal width of the obstacle notification icon 120 is changed and displayed as shown in FIGS. 13 to 14, for example.
  • FIG. 16 is a block diagram illustrating a configuration example of the overhead view video generation system according to the seventh embodiment.
  • the overhead image generation system 1A has the same basic configuration as the overhead image generation system 1 of the first embodiment.
  • the overhead view video generation system 1A will be described with reference to FIG.
  • the overhead image generation system 1A includes a front camera 11, a rear camera 12, a left side camera 13, a right side camera 14, a front sensor (obstacle detection unit) 21, and a rear sensor (obstacle detection unit) 22.
  • a left side sensor (obstacle detection unit) 23 a right side sensor (obstacle detection unit) 24, a display panel 31, and an overhead image generation device 40A.
  • the front sensor 21 is disposed in front of the vehicle and detects an obstacle in front of the vehicle.
  • the front sensor 21 is, for example, an infrared sensor or an ultrasonic sensor.
  • the front sensor 21 detects an object having a height from the ground that may come into contact with the vehicle.
  • the front sensor 21 detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the front sensor 21 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the front sensor 21 may overlap with a part of the detection range of the left side sensor 23 and the right side sensor 24.
  • the front sensor 21 may be configured by a combination of a plurality of sensors.
  • the front sensor 21 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
  • the rear sensor 22 is disposed behind the vehicle and detects an obstacle behind the vehicle.
  • the rear sensor 22 is, for example, an infrared sensor or an ultrasonic sensor.
  • the rear sensor 22 detects an object having a height from the ground that may come into contact with the vehicle.
  • the rear sensor 22 detects an obstacle at a distance of about 5 m from the vehicle.
  • the rear sensor 22 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the rear sensor 22 may overlap with a part of the detection range of the left side sensor 23 and the right side sensor 24.
  • the rear sensor 22 may be configured by a combination of a plurality of sensors.
  • the rear sensor 22 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
  • the left side sensor 23 is arranged on the left side of the vehicle and detects an obstacle on the left side of the vehicle.
  • the left side sensor 23 is, for example, an infrared sensor or an ultrasonic sensor.
  • the left side sensor 23 detects an object having a height from the ground that may come into contact with the vehicle.
  • the left side sensor 23 detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the left side sensor 23 detects an obstacle in a range of, for example, about 30 ° with the center of the sensor as the center when viewed in the vertical direction.
  • the detection range of the left side sensor 23 may overlap with a part of the detection range of the front sensor 21 and the rear sensor 22.
  • the left side sensor 23 may be configured by a combination of a plurality of sensors.
  • the left side sensor 23 outputs the obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead image generation device 40A.
  • the right side sensor 24 is arranged on the right side of the vehicle and detects an obstacle on the right side of the vehicle.
  • the right side sensor 24 is, for example, an infrared sensor or an ultrasonic sensor.
  • the right side sensor 24 detects an object having a height from the ground that may come into contact with the vehicle.
  • the right side sensor 24 detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the right side sensor 24 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the right side sensor 24 may overlap with a part of the detection range of the front sensor 21 and the rear sensor 22.
  • the right side sensor 24 may be configured by a combination of a plurality of sensors.
  • the right side sensor 24 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
  • the display panel 31 has a horizontally long rectangular shape. On the display panel 31, a vertically long overhead view image 100 as shown in FIG. 17 or a horizontally long overhead view image 200 as shown in FIG. 18 is displayed.
  • the overhead view image 200 having a horizontally long shape is a horizontally long overhead view image that matches the aspect ratio of the display surface of the display panel 31.
  • the control unit 41A includes a video acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, a video generation unit 45A, and a display control unit 48.
  • the obstacle information acquisition unit 43 acquires the obstacle information output by the front sensor 21, the rear sensor 22, the left side sensor 23, and the right side sensor 24.
  • the video generation unit 45A generates an overhead video 100 and an overhead video 200.
  • the video generation unit 45A includes an overhead video generation unit 46A and a video synthesis unit 47A.
  • the overhead video generation unit 46A generates the overhead video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. More specifically, the overhead view video generation unit 46A generates the overhead view video 100 based on the peripheral video captured by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the bird's-eye view image 100 may be any known method and is not limited. The overhead view video generation unit 46A outputs the generated overhead view video 100 to the display control unit 48.
  • FIG. 17 is a diagram illustrating a display panel that displays an overhead video generated by the overhead video generation system according to the seventh embodiment.
  • the bird's-eye view image 100 has a vertically long rectangular shape.
  • the bird's-eye view image 100 includes a front image 101, a rear image 102, a left side image 103, and a right side image 104.
  • a range surrounded by the front video 101, the rear video 102, the left side video 103, and the right side video 104 is referred to as a central portion of the overhead view video 100.
  • the central part of the overhead image 100 is the position of the vehicle in the overhead image 100.
  • the front video 101, the rear video 102, the left video 103, the right video 104, and the central portion of the overhead video 100 may be separated by a frame-like boundary line 110.
  • FIG. 17 diagonal broken lines indicating boundaries between the front video 101, the rear video 102, the left side video 103, and the right side video 104 are illustrated for explanation, but are actually displayed on the display panel 31.
  • the broken line may or may not be displayed in the overhead view video 100. The same applies to the other figures.
  • the overhead image generation unit 46A Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead image generation unit 46A does not change the display direction of the generated overhead image 100 when the direction in which the obstacle is detected is the front or the rear of the vehicle.
  • the overhead view image 200 having a horizontally long shape is generated.
  • the display direction is not changed means that the vertical direction of the bird's-eye view image 100 and the vertical direction of the bird's-eye view image 200 are kept aligned with the longitudinal direction of the vehicle.
  • that the display orientation is not changed is that, in the overhead view video 200 having a horizontally long shape, the front video 201 is arranged on the upper side, the rear video 202 is arranged on the lower side, the left side video 203 is arranged on the left side, and the right side In other words, it means that the right side image 204 is maintained.
  • the vertical direction of the bird's-eye view image 200 matches the longitudinal direction of the vehicle, so that the recognizability of the horizontally-viewed bird's-eye view image 200 can be maintained.
  • FIG. 18 is a diagram showing a display panel that displays a horizontally long overhead image generated by the overhead image generation system according to the seventh embodiment.
  • the bird's-eye view image 200 includes a front image 201, a rear image 202, a left side image 203, and a right side image 204.
  • a range surrounded by the front video 201, the rear video 202, the left side video 203, and the right side video 204 is referred to as a central portion of the overhead view video 200.
  • the central part of the overhead view image 200 is the position of the vehicle in the overhead view image 200.
  • the front video 201, the rear video 202, the left video 203, the right video 204, and the central portion of the overhead video 200 may be separated by a frame-like boundary line 210.
  • FIG. 19 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the seventh embodiment.
  • FIG. 20 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the bird's-eye view image generation unit 46A generates a bird's-eye view image 200 having a horizontally long shape by expanding a part of the surrounding image from the front camera 11 or the rear camera 12 with a fixed aspect ratio. More specifically, the overhead view video generation unit 46A expands a part of the front image 101 and a part of the rear image 102 from the generated vertically long overhead image 100 with a fixed aspect ratio to obtain a horizontally long shape. A video 200 is generated.
  • the overhead view video generation unit 46A generates a horizontally long overhead view video 200 from the cutout range A1, the cutout range A2, the cutout range A3, and the cutout range A4 of the overhead view video 100. More specifically, the overhead view video generation unit 46A expands the aspect ratio of the cutout range A1 and the cutout range A2 of the overhead view video 100 while fixing the aspect ratio to the aspect ratio of the display surface of the display panel 31. The bird's-eye view video 200 is generated.
  • the cutout range A1 is a rectangular range including a range in front of the vehicle that the driver needs to check.
  • the cut-out range A1 includes a part of the front image 101 of the overhead image 100, a part of the left side image 103, and a part of the right side image 104.
  • the cut-out range A1 includes a boundary between the front video 101 and the left side video 103 and a boundary between the front video 101 and the right side video 104.
  • the cut-out range A1 includes from the front end of the vehicle to about a few tens of centimeters forward.
  • the cutout range A1 includes from the left end of the vehicle to the left side of about several tens of cm.
  • the cutout range A1 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the cut-out range A2 is a rectangular range including a range behind the vehicle that the driver needs to check.
  • the cutout range A ⁇ b> 2 includes a part of the rear image 102 of the overhead image 100, a part of the left side image 103, and a part of the right side image 104.
  • the cutout range A ⁇ b> 2 includes a boundary between the rear image 102 and the left side image 103 and a boundary between the rear image 102 and the right side image 104.
  • the cut-out range A2 includes from the rear end of the vehicle to the rear of about several tens of centimeters.
  • the cutout range A2 includes from the left end of the vehicle to the left side of about several tens of cm.
  • the cutout range A2 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the cutout range A2 has the same shape and the same area as the cutout range A1.
  • the cutout area A3 is a rectangular area located between the cutout area A1 and the cutout area A2 of the overhead view image 100.
  • the cutout range A3 includes from the left end of the vehicle to the left side of about several tens of cm.
  • the left end portion of the cutout range A3 is positioned on the same straight line as the left end portion of the cutout range A1 and the left end portion of the cutout range A2.
  • the cutout range A4 is a rectangular range located between the cutout range A1 and the cutout range A2 of the overhead view image 100.
  • the cutout range A4 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the right end of the cutout range A4 is located on the same straight line as the right end of the cutout range A1 and the right end of the cutout range A2.
  • the cutout range A4 has the same shape and the same area as the cutout range A3.
  • a horizontally long overhead image 200 generated from such a bird's-eye view image 100 includes a cut-out range A1 image 211, a cut-out range A2 image 212, a cut-out range A3 image 213, and a cut-out range.
  • A4 video 214 The horizontally long overhead image 200 generated from such a bird's-eye view image 100 includes a cut-out range A1 image 211, a cut-out range A2 image 212, a cut-out range A3 image 213, and a cut-out range.
  • the image 211 is an image obtained by enlarging the image of the cutout range A1 of the overhead view image 100 until the aspect ratio is fixed and the width is the same as the width of the display panel 31.
  • the image of the tree displayed in the image 211 is an image obtained by fixing and expanding the image of the tree displayed in the cutout area A1 of the overhead image 100 shown in FIG.
  • the video 212 is a video obtained by enlarging the video of the cutout range A2 of the overhead video 100 until the aspect ratio is fixed and the horizontal width becomes the same as the horizontal width of the display panel 31.
  • the image of the tree displayed in the image 212 is an image obtained by expanding and fixing the aspect ratio of the image of the tree displayed in the cutout range A2 of the overhead image 100 shown in FIG.
  • the video 213 is a video in which the size and shape of the video in the cutout range A3 of the overhead video 100 are changed in accordance with the display panel 31 so that the boundary between the video 211 and the video 212 is smoothly continuous.
  • the video 213 may be a video obtained by compressing the video in the cutout range A3 of the overhead video 100 in the vertical direction and expanding in the horizontal direction.
  • the video 214 is a video in which the size and shape of the video in the cutout range A4 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 211 and the video 212 is smoothly continuous.
  • the video 214 may be a video obtained by compressing the video in the cutout range A4 of the overhead video 100 and vertically expanding the video.
  • the bird's-eye view image 200 includes a front image 201, a rear image 202, a left side image 203, and a right side image 204.
  • the front image 201 is a part of the image 211.
  • the rear video 202 is a part of the video 212.
  • the left side image 203 includes a lower left portion of the image 211, an upper left portion of the image 212, and an image 213.
  • the right side video 204 includes a lower right portion of the video 211, an upper right portion of the video 212, and a video 214.
  • the video synthesizing unit 47A is located at the center of the overhead view video 100 and the overhead view video 200 generated by the overhead view video generation unit 46A, and corresponds to the direction in which the obstacle information acquisition unit 43 detects the obstacle, the front camera 11 or the rear camera 12.
  • the bird's-eye view video 100 and the bird's-eye view image 200 are generated by synthesizing the peripheral images from the left side camera 13 or the right side camera 14.
  • the peripheral video from the front camera 11, the rear camera 12, the left side camera 13, or the right side camera 14 to be synthesized is a video before viewpoint conversion.
  • the video composition unit 47A outputs the generated overhead view video 100 and the overhead view video 200 to the display control unit 48.
  • the video composition unit 47A detects the obstacle in the central portion of the overhead image 100.
  • the overhead view image 100 is generated by combining the vertically long peripheral images from the left side camera 13 or the right side camera 14 corresponding to.
  • the peripheral video synthesized at the center of the overhead view video 100 is a peripheral video that has not been subjected to viewpoint conversion processing.
  • the video compositing unit 47A may display the boundary line 110 of the overhead view video 100 by thickening or changing the color of the direction in which the obstacle is detected.
  • the video synthesis unit 47A synthesizes the vertically long peripheral video 103A from the left side camera 13 in the center of the overhead view video 100 as shown in FIG. To do.
  • the rear image 102 and the surrounding image 103A of the overhead image 100 include an obstacle image 300 of a rear obstacle.
  • a boundary line 110A on the left side of the overhead view image 100 is a thick line.
  • the video composition unit 47A detects an obstacle at the center of the horizontally-viewed overhead image 200 when the direction in which the obstacle information acquisition unit 43 detects the obstacle is the front or the rear of the vehicle.
  • An overhead view image 200 is generated by combining horizontally long peripheral images from the front camera 11 or the rear camera 12 corresponding to the determined direction.
  • the peripheral video synthesized in the central portion of the overhead video 200 is a peripheral video that has not been subjected to the viewpoint conversion process.
  • the video compositing unit 47A may display the boundary line 210 of the overhead view video 200 by thickening or changing the color of the direction in which the obstacle is detected.
  • the video composition unit 47A synthesizes the horizontally long peripheral image 202A from the rear camera 12 in the center of the overhead view image 200 as shown in FIG.
  • the rear image 202 and the surrounding image 202A of the overhead image 200 include an obstacle image 300 of a rear obstacle.
  • a lower boundary line 210A of the bird's-eye view image 200 is a thick line.
  • FIG. 21 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the seventh embodiment.
  • Control unit 41A determines whether or not to start bird's-eye view video display (step ST11). In the present embodiment, the control unit 41A determines whether to start the overhead view video display based on the presence / absence of the backward trigger.
  • the reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle is the rear of the vehicle. If there is no reverse trigger, control unit 41A determines not to start the overhead view video display (No in step ST11), and executes the process of step ST11 again. If there is a reverse trigger, control unit 41A determines to start the overhead view video display (Yes in step ST11), and proceeds to step ST12.
  • the control unit 41A generates and displays the overhead view video 100 (step ST12). More specifically, the control unit 41A causes the overhead video generation unit 46A to generate the overhead video 100 that has been subjected to the viewpoint conversion processing so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. Then, the control unit 41A causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • Control unit 41A determines whether an obstacle has been detected (step ST13). More specifically, the control unit 41A determines whether or not obstacle information has been acquired by the obstacle information acquisition unit 43. If the control unit 41A determines that the obstacle information acquisition unit 43 has acquired the obstacle information (Yes in step ST13), the control unit 41A proceeds to step ST14. If the controller 41A determines that the obstacle information acquisition unit 43 has not acquired the obstacle information (No in step ST13), the control unit 41A proceeds to step ST17.
  • the control unit 41A determines whether an obstacle located in the traveling direction of the vehicle is detected based on the obstacle information acquired by the obstacle information acquisition unit 43.
  • Positioning in the traveling direction of the vehicle means positioning in a direction in which the distance to the vehicle decreases as the vehicle travels, and includes a range in which an obstacle interferes with the vehicle as the vehicle travels.
  • the control unit 41A determines whether or not an obstacle within a range centered on the traveling direction of the vehicle is detected, and the vehicle steers.
  • it is determined whether or not obstacles are detected in a range including the range in which the obstacle interferes with the vehicle by the traveling direction and steering of the vehicle.
  • the control unit 41A determines whether the obstacle detection direction is the left direction or the right direction (step ST14). In the present embodiment, the control unit 41A determines whether the detection direction of the obstacle with the shortest distance to the vehicle is the left direction or the right direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43. Determine whether. More specifically, the control unit 41A determines whether the obstacle information of the obstacle with the minimum distance to the vehicle acquired by the obstacle information acquisition unit 43 is the obstacle information from the left side sensor 23 or the right side sensor 24. Determine whether or not.
  • step ST14 When it is determined that the obstacle information of the obstacle with the shortest distance to the vehicle is the obstacle information from the left side sensor 23 or the right side sensor 24 (Yes in step ST14), the control unit 41A proceeds to step ST15. move on. If the controller 41A determines that the obstacle information of the obstacle with the shortest distance to the vehicle is not the obstacle information from the left side sensor 23 or the right side sensor 24 (No in step ST14), the control unit 41A goes to step ST16. move on.
  • the control unit 41A generates the bird's-eye view image 100 in which the peripheral image in the obstacle detection direction is displayed at the center of the bird's-eye view image 100 (step ST15). More specifically, the control unit 41 corresponds to the direction in which the video composition unit 47A detects the obstacle with the shortest distance to the vehicle at the center of the overhead image 100 generated by the overhead image generation unit 46A.
  • An overhead view image 100 is generated by synthesizing a vertically long peripheral image from the camera 13 or the right side camera 14. Then, the control unit 41A causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the control unit 41A generates a bird's-eye view image 200 in which a peripheral image in the obstacle detection direction is displayed at the center of the horizontally long bird's-eye view image 200 (step ST16). More specifically, the control unit 41A causes the overhead view video generation unit 46A to generate the overhead view video 200 having a horizontally long shape without changing the display direction of the overhead view video 100 generated in step ST12. Then, the control unit 41A receives from the front camera 11 or the rear camera 12 corresponding to the direction in which the obstacle with the shortest distance to the vehicle is detected at the center of the horizontally long overhead image 200 in the video composition unit 47A. A bird's-eye view image 200 obtained by synthesizing a horizontally long peripheral image is generated. Then, the control unit 41 ⁇ / b> A causes the display control unit 48 to cause the display panel 31 to display the generated overhead image 200 having a horizontally long shape.
  • Control unit 41A determines whether or not to continue the overhead video display (step ST17). More specifically, the control unit 41A determines whether or not to continue the overhead view video display based on the presence / absence of the backward trigger. If there is a reverse trigger, control unit 41A determines to continue the overhead view video display (Yes in step ST17), and returns to step ST13 to continue the process. When there is no reverse trigger, in other words, when the reverse trigger is released, control unit 41A determines that the overhead view video display is not continued (No in step ST17), and ends the process.
  • the overhead view video generation system 1A has a left-side camera 13 corresponding to the direction in which the obstacle is detected in the center of the overhead view video 100 or A bird's-eye view image 100 is generated by synthesizing a vertically long peripheral image from the right-side camera 14.
  • the overhead view video generation system 1A When an obstacle is detected in the front or rear direction of the vehicle, the overhead view video generation system 1A has a front camera 11 or a rear camera corresponding to the direction in which the obstacle is detected at the center of the horizontally overhead view 200. 12 generates a bird's-eye view image 200 obtained by synthesizing a horizontally long peripheral image from 12.
  • the bird's-eye view video 100 and the bird's-eye view image 200 are generated by synthesizing the surrounding images corresponding to the direction in which the obstacle is detected in the center of the bird's-eye view image 100 and the bird's-eye view image 200.
  • this embodiment can also confirm the surrounding image of the direction which detected the obstacle with the bird's-eye view image 100 and the bird's-eye view image 200, without moving a big eyes
  • the present embodiment can suppress the movement of the driver's line of sight and appropriately check obstacles around the vehicle.
  • a vertically long image from the left side camera 13 or the right side camera 14 corresponding to the direction in which the obstacle is detected is displayed at the center of the overhead view image 100.
  • a bird's-eye view image 100 is generated by synthesizing the surrounding image of the shape.
  • the front camera 11 or the rear camera 12 corresponding to the direction in which the obstacle is detected is displayed at the center of the horizontally-viewed overhead view image 200.
  • a bird's-eye view video 200 is generated by synthesizing the surrounding video.
  • this embodiment can maintain the state in which the vertical direction of the bird's-eye view image 100 and the vertical direction of the bird's-eye view image 200 are combined with the longitudinal direction of the vehicle, regardless of the detection direction of the obstacle. For this reason, according to this embodiment, when the driver confirms the bird's-eye view image 100 or the bird's-eye view image 200, the periphery of the vehicle can be easily confirmed.
  • the overhead image 200 is obtained by synthesizing a peripheral image corresponding to the direction in which the obstacle is detected in the center of the horizontally overhead bird's eye image 200. Is generated and displayed on the display panel 31.
  • the bird's-eye view image 200 having a horizontally long shape can be displayed on the entire display surface of the display panel 31. Thereby, this embodiment can display a surrounding image largely in the center part of the bird's-eye view image 200, when an obstacle is detected in the front direction or the rear direction of the vehicle.
  • the video 211 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A1 of the overhead view video 100.
  • the front image 201 of the overhead image 200 has the same aspect ratio as the image of the cutout range A1 of the overhead image 100.
  • the video 212 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A2 of the overhead view video 100.
  • the rear video 202 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A2 of the overhead view video 100.
  • the front video 201 and the rear video 202 of the overhead view video 200 have the same aspect ratio and are not deformed. For this reason, according to the present embodiment, the driver can check the front video 201 and the rear video 202 of the bird's-eye view video 200 without a sense of incongruity.
  • FIG. 22 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eighth embodiment.
  • FIG. 23 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the overhead image generation system 1A has the same basic configuration as the overhead image generation system 1A of the seventh embodiment.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the seventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46A Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead view video generation unit 46A, when the direction in which the obstacle is detected is the front or the rear of the vehicle, the peripheral video from the front camera 11 or the rear camera 12 Is vertically compressed to generate a bird's-eye view image 200 having a horizontally long shape. More specifically, the overhead view video generation unit 46A compresses the front video 101 and the rear video 102 in the vertical direction from the vertically elongated overhead view video 100 that has been generated, and generates the overhead view video 200 having a horizontally long shape.
  • the cut-out range A1 includes a boundary between the front video 101 and the left side video 103 and a boundary between the front video 101 and the right side video 104.
  • the cut-out range A1 includes from the left end of the vehicle to the left side of about several tens of centimeters.
  • the cut-out range A1 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the cutout range A2 has the same vertical width as the vertical width of the rear video 102 of the overhead view video 100.
  • the cutout range A ⁇ b> 2 includes a boundary between the rear image 102 and the left side image 103 and a boundary between the rear image 102 and the right side image 104.
  • the cut-out range A2 includes from the left end of the vehicle to the left side of about several tens of centimeters.
  • the cutout range A2 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the cutout area A2 has the same shape and the same area as the cutout area A1.
  • the cutout range A3 and the cutout range A4 are the same as in the seventh embodiment.
  • the overhead view video generation unit 46A compresses the cutout range A1 and the cutout range A2 of the overhead view video 100 in the vertical direction, and generates the overhead view video 200 having a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31. .
  • the video 211 is a video obtained by compressing the video of the cutout range A1 of the overhead video 100 in the vertical direction.
  • the image of the tree displayed in the image 211 is an image obtained by compressing the image of the tree displayed in the cutout range A1 of the overhead image 100 illustrated in FIG. 22 in the vertical direction.
  • the tree image displayed in the image 211 includes the entire tree image displayed in the front image 101 of the overhead image 100.
  • the video 212 is a video obtained by compressing the video of the cutout range A2 of the overhead video 100 in the vertical direction.
  • the image of the tree displayed in the image 212 is an image obtained by compressing the image of the tree displayed in the cutout range A2 of the overhead image 100 shown in FIG. 22 in the vertical direction.
  • the tree image displayed in the image 212 includes the entire tree image displayed in the rear image 102 of the overhead image 100.
  • the video 213 and the video 214 are the same as in the seventh embodiment.
  • the video 211 of the overhead video 200 is a video obtained by compressing the video of the cutout range A1 of the overhead video 100 in the vertical direction.
  • the video 211 of the overhead video 200 includes the entire vertical direction of the front video 101.
  • the video 212 of the bird's-eye view video 200 is a video obtained by compressing the video of the cutout range A2 of the bird's-eye view video 100 in the vertical direction.
  • the video 212 of the overhead video 200 includes the entire vertical direction of the rear video 102.
  • the driver can confirm the same range in the overhead image 200 as the vertical direction of the front image 101 of the overhead image 100 and the vertical direction of the rear image 102.
  • FIG. 24 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the ninth embodiment.
  • FIG. 25 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the seventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46 ⁇ / b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle.
  • a vertically long peripheral image is compressed in the vertical direction to generate a bird's-eye view image 200 having a horizontally long shape.
  • the overhead view video generation unit 46A generates the overhead view video 200 by compressing the cutout range A3 and the cutout range A4 of the generated overhead view video 100 in the vertical direction.
  • the cutout range A3 includes a boundary between the left side video 103 and the front video 101 and a boundary between the left side video 103 and the rear video 102.
  • the cut-out range A3 includes from the front end of the vehicle to about a few meters ahead.
  • the cut-out range A3 includes from the rear end of the vehicle to the rear about several meters.
  • the cutout range A4 has the same lateral width as the lateral width of the right-side video 104 of the overhead view video 100.
  • the cutout range A4 includes a boundary between the right side video 104 and the front video 101 and a boundary between the right side video 104 and the rear video 102.
  • the cut-out range A4 includes a distance of about several meters from the front end of the vehicle.
  • the cutout range A4 includes from the rear end of the vehicle to the rear about several meters.
  • the cutout area A4 has the same shape and the same area as the cutout area A3.
  • the cutout range A1 is a rectangular range located between the cutout range A3 and the cutout range A4.
  • the cut-out range A1 includes from the front end of the vehicle to about a few meters ahead.
  • the front end portion of the cutout range A1 is located on the same straight line as the front end portion of the cutout range A3 and the front end portion of the cutout range A4.
  • the cutout range A2 is a rectangular range located between the cutout range A3 and the cutout range A4.
  • the cut-out range A2 includes from the front end of the vehicle to about a few meters ahead.
  • the front end portion of the cutout range A2 is located on the same straight line as the front end portion of the cutout range A3 and the front end portion of the cutout range A4.
  • the overhead view video generation unit 46A compresses the cut-out range A1 and the cut-out range A2 of the generated overhead view video 100 in the vertical direction so as to have a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31. Is generated.
  • the video 213 is a video obtained by compressing the video in the cutout range A3 of the overhead view video 100 in the vertical direction.
  • the image of the tree displayed in the image 213 is an image obtained by compressing the image of the tree displayed in the cutout range A3 of the overhead image 100 in the vertical direction.
  • the tree image displayed in the image 213 includes the entire tree image displayed in the left-side image 103 of the overhead image 100.
  • the video 214 is a video obtained by compressing the video of the cutout range A4 of the overhead view video 100 in the vertical direction.
  • the image of the tree displayed in the image 214 is an image obtained by compressing the image of the tree displayed in the cutout range A4 of the overhead image 100 in the vertical direction.
  • the tree image displayed in the image 214 includes the entire tree image displayed in the right-side image 104 of the overhead image 100.
  • the video 211 is a video in which the size and shape of the video in the cutout range A1 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous.
  • the video 211 may be a video obtained by enlarging the video in the cutout range A1 of the overhead view video 100 in the horizontal direction.
  • the video 212 is a video in which the size and shape of the video in the cutout range A2 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous.
  • the video 212 may be a video obtained by enlarging the video in the cutout range A2 of the overhead view video 100 in the horizontal direction.
  • the video 213 of the overhead video 200 is a video obtained by compressing the video of the cutout range A3 of the overhead video 100 in the vertical direction.
  • the video 213 of the overhead video 200 includes the entire horizontal direction of the left side video 103.
  • the video 214 of the bird's-eye view video 200 is a video obtained by compressing the video of the cutout range A4 of the bird's-eye view video 100 in the vertical direction.
  • the image 214 of the bird's-eye view image 200 includes the entire horizontal direction of the right-side image 104.
  • the driver can confirm the same range in the overhead view video 200 as the horizontal direction of the left side video 103 and the horizontal direction of the right side video 104 of the overhead view video 100.
  • FIG. 26 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the tenth embodiment.
  • FIG. 27 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the ninth embodiment in the method of generating the overhead view video 200 having a horizontally long shape by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46 ⁇ / b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle.
  • the top and bottom portions are synthesized except for the center portion of the vertically long peripheral image, and the overhead view image 200 having a horizontally long shape is generated.
  • the 26 includes an upper part A31, a central part A32, and a lower part A33.
  • the upper part A31 includes a boundary between the left side image 103 and the front image 101.
  • the lower part A33 includes a boundary between the left side video 103 and the rear video 102.
  • the cutout range A4 includes an upper part A41, a central part A42, and a lower part A43.
  • the upper part A41 includes a boundary between the right side image 104 and the front image 101.
  • the lower part A43 includes a boundary between the right side image 104 and the rear image 102.
  • the cutout area A4 has the same shape and the same area as the cutout area A3.
  • the cutout range A1 and the cutout range A2 are the same as in the ninth embodiment.
  • the overhead image generation unit 46A combines the upper portion A31 and the lower portion A33 of the cutout range A3 of the generated overhead view image 100, and combines the upper portion A41 and the lower portion A43 of the cutout range A4 to display the display surface of the display panel 31.
  • a bird's-eye view image 200 having a horizontally long shape matching the aspect ratio is generated.
  • the video 213 is a video obtained by combining the upper part A31 and the lower part A33 excluding the central part A32 of the cutout range A3 of the overhead view video 100.
  • the video 213 is a video that does not include the video of the tree displayed in the central portion A32 of the cutout range A3 of the overhead view video 100 shown in FIG.
  • a line may be displayed at the boundary between the upper part A31 and the lower part A33 in order to show that the central part A32 of the cutout range A3 of the overhead view video 100 is omitted.
  • the video 214 is a video obtained by synthesizing the upper part A41 and the lower part A43 excluding the central part A42 of the cutout range A4 of the overhead view video 100.
  • the image 214 is an image that does not include the image of the tree displayed in the central portion A42 of the cutout range A4 of the overhead image 100 shown in FIG.
  • a line may be displayed at the boundary between the upper part A41 and the lower part A43 in order to show that the central part A42 of the cutout range A4 of the overhead view video 100 is omitted.
  • the video 211 and the video 212 are the same as in the ninth embodiment.
  • the video 213 of the bird's-eye view video 200 is composed of the upper part A31 and the lower part A33 excluding the central part A32 of the cutout range A3 of the bird's-eye view picture 100.
  • the video 213 of the bird's-eye view video 200 can largely display the front and rear of the left side video 103 of the bird's-eye view video 100.
  • the image 214 of the bird's-eye view image 200 combines the upper part A41 and the lower part A43 excluding the central part A42 of the cutout range A4 of the bird's-eye view image 100.
  • the video 214 of the bird's-eye view video 200 can largely display the front and rear of the right-side video 104 of the bird's-eye view video 100.
  • FIG. 28 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eleventh embodiment.
  • FIG. 29 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the ninth embodiment in the method of generating the overhead view video 200 having a horizontally long shape by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46 ⁇ / b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle.
  • the vertically long peripheral image is biased in the direction in which the obstacle is detected, and the overhead image 200 having a horizontally long shape is generated.
  • the direction in which the obstacle is detected will be described as the rear.
  • the cutout range A3 shown in FIG. 28 is a range that is biased backward in the obstacle detection direction in the left side image 103 of the overhead view image 100.
  • the cut-out range A3 includes the lower side from the center in the vertical direction of the overhead view image 100.
  • the cut-out range A4 is a range that is biased toward the rear in the obstacle detection direction in the right-side video 104 of the overhead view video 100.
  • the cut-out range A4 includes the lower side from the central portion in the vertical direction.
  • the cutout area A4 has the same shape and the same area as the cutout area A3.
  • the cutout range A5 is a rectangular range located between the cutout range A3 and the cutout range A4 of the overhead view image 100.
  • the upper end portion of the cutout range A5 is located on the same straight line as the upper end portion of the cutout range A1 and the upper end portion of the cutout range A2.
  • the lower end portion of the cutout range A5 is located on the same straight line as the lower end portion of the cutout range A1 and the lower end portion of the cutout range A2.
  • the overhead view video generation unit 46A generates an overhead view video 200 having a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31 from the cutout range A3, the cutout range A4, and the cutout range A5 of the generated overhead view video 100. Generate.
  • the video 213 is a video of the cutout range A3 of the overhead video 100.
  • the image of the tree displayed in the image 213 includes the entire image of the tree displayed in the cutout area A3 of the overhead image 100 shown in FIG.
  • the image 214 is an image of the cutout range A4 of the overhead image 100.
  • the tree image displayed in the image 214 includes the entire tree image displayed in the cutout range A4 of the overhead view image 100 shown in FIG.
  • the video 215 is a video in which the size and shape of the video in the cutout range A5 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous.
  • the video 215 may be a video obtained by enlarging the video in the cutout range A5 of the overhead view video 100 in the horizontal direction.
  • the left-side image 103 and the right-side image 104 of the overhead image 100 are biased in the obstacle detection direction to have a horizontally long shape.
  • An overhead image 200 is generated.
  • the bird's-eye view image 200 can display a larger direction in which the obstacle is detected in the bird's-eye view image 100.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the eleventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46 ⁇ / b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle.
  • An overhead image 200 having a horizontally long shape is generated by deviating a vertically long peripheral image in the traveling direction of the vehicle.
  • the left side image 103 and the right side image 104 of the overhead view image 100 are biased in the traveling direction of the vehicle so as to have a horizontally long shape.
  • a video 200 is generated.
  • the bird's-eye view image 200 can display the traveling direction of the vehicle larger in the bird's-eye view image 100.
  • the constituent elements of the overhead view video generation system 1 shown in the figure are functionally conceptual, and need not be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or a part of them is functionally or physically distributed or integrated in arbitrary units according to the processing load or usage status of each device. May be.
  • the configuration of the overhead view video generation system 1 is realized by, for example, a program loaded in a memory as software.
  • the above embodiment has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.
  • the superimposed video generation unit 47 may display the distance to the obstacle numerically together with the obstacle notification icon 120 on the central video 105 of the overhead video 100. Good.
  • the superimposed image generation unit 47 displays the shape or the shape according to the distance to the obstacle together with the obstacle notification icon 120 on the central image 105 of the overhead view image 100. A figure whose color changes may be displayed.
  • the overhead view video generation system 1 can notify the distance to the obstacle in addition to the change of the obstacle notification icon 120 according to the distance to the obstacle. Thereby, the bird's-eye view video generation system 1 can make it possible to more appropriately check obstacles around the vehicle.
  • the control unit 41 may display the obstacle notification icon 120 for the obstacles with high priority.
  • the priority order may be the highest priority order with the smallest distance to the obstacle.
  • the priority may be the highest priority when the obstacle is a moving object.
  • the priority may be the highest priority when the obstacle is moving and is approaching the vehicle.
  • an icon indicating that the obstacle is a moving object may be displayed in the central image 105 of the overhead view image 100.
  • the icon indicating a moving object may be, for example, an icon indicating a pedestrian or an icon indicating a vehicle.
  • the obstacle notification icon 120 has been described as being composed of a triple arc-shaped curve, it is not limited to this.
  • the obstacle notification icon 120 may be, for example, an arrow-shaped figure indicating the direction of the obstacle.
  • the obstacle notification icon 120 may be an arrow-shaped figure whose thickness and size are changed according to the distance to the obstacle.
  • the sensor serving as the obstacle detection unit includes six sensors including a front left sensor 21A, a front center sensor 21B, a front right sensor 21C, a rear left sensor 22A, a rear center sensor 22B, and a rear right sensor 26.
  • the number of sensors is not limited. If it is desired to detect the direction of an obstacle more finely, the number of sensors may be increased.
  • the obstacle notification icon 120 corresponds to the number of sensors, the front left icon 121, the front center icon 122, the front right icon 123, the rear left icon 124, the rear center icon 125, and the rear right icon 126.
  • the present invention is not limited to this. In other words, the obstacle notification icon 120 only needs to associate the detection result of each sensor with the icon to be displayed.
  • the bird's-eye view image generation unit 46 may generate a bird's-eye view image 200 having a horizontally long shape by compressing the central portion of the peripheral image from the front camera 11 or the rear camera 12 in the vertical direction. This will be described in more detail with reference to FIGS.
  • FIG. 30 is a diagram illustrating another example of a method for generating a horizontally long overhead image in the overhead image generation system.
  • FIG. 31 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 30.
  • the overhead image generation unit 46 compresses the cutout range A6 at the center of the left side video 103 of the overhead view video 100 and the cutout range A7 at the center of the right side video 104 in the vertical direction, and displays the display panel 31 on the display panel 31.
  • a bird's-eye view image 200 having a horizontally long shape matching the aspect ratio of the surface is generated.
  • the control unit 41A synthesizes the peripheral video corresponding to the direction in which the obstacle with the shortest distance to the vehicle is detected at the center of the overhead video 100 and the overhead video 200.
  • the peripheral video may be selectable by the user.
  • peripheral images of the overhead view video 100 and the overhead view video 200 at the position touched by the user may be synthesized on the display surface of the display panel 31.
  • the control unit 41A may reduce the color information of the left-side image 203 and the right-side image 204 of the overhead image 200, for example, when the obstacle detection direction is forward or backward. Alternatively, when the obstacle detection direction is forward or backward, the control unit 41A may display the left side video 203 and the right side video 204 of the overhead view video 200, for example, in a single color display. Alternatively, the control unit 41A may reduce the color information of the left side video 203 and the right side video 204 of the overhead view video 200, for example, when the obstacle detection direction is front or rear. Alternatively, the control unit 41A may reduce the luminance of the left side video 203 and the right side video 204 of the overhead view video 200, for example, when the obstacle detection direction is front or rear. In this way, the driver can be made aware that the amount of information in the left side video 203 and the right side video 204 of the overhead view video 200 is reduced.
  • the bird's-eye view video generation unit 46A receives the signals from the left side camera 13 and the right side camera 14 when the direction in which the obstacle is detected is the front or the rear of the vehicle.
  • the peripheral video may be hidden, and the overhead video 200 having a horizontally long shape may be generated from the peripheral video from the front camera 11 and the rear camera 12. In this case, the driver can visually confirm the left and right sides of the vehicle.
  • the control unit 41A may determine whether or not to start the overhead view video display based on, for example, whether or not an operation for starting the overhead view video display on the operation unit is detected.
  • the seventh embodiment four sensors, that is, the front sensor 21, the rear sensor 22, the left side sensor 23, and the right side sensor 24 are provided as the sensors that are obstacle detection units, but the number of sensors is not limited. . If it is desired to detect the direction of an obstacle more finely, the number of sensors may be increased.
  • Overhead video generation system 11 Front camera (camera) 12 Rear camera (camera) 13 Left-side camera (camera) 14 Right-side camera (camera) 21A Front left sensor (obstacle detection unit) 21B Front center sensor (obstacle detection unit) 21C Front right sensor (obstacle detection unit) 22A Rear left sensor (obstacle detection unit) 22B Rear center sensor (obstacle detection unit) 22C Rear right sensor (obstacle detection unit) Reference Signs List 31 Display panel 40 Overhead video generation device 41 Control unit 42 Video acquisition unit 43 Obstacle information acquisition unit 44 Vehicle information acquisition unit 46 Overhead video generation unit (video generation unit) 47 Superimposed video generator (video generator) 48 Display control unit 49 Storage unit 100 Overhead video 120 Obstacle notification icon (information indicating an obstacle) 121 Front left icon (information indicating an obstacle) 122 Front center icon (information indicating an obstacle) 123 Front right icon (information indicating an obstacle) 124 Rear left icon (information indicating an obstacle) 125 Rear center icon (information indicating an obstacle) 126 Re

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

La présente invention est caractérisée en ce qu'elle comprend : une unité d'acquisition d'image vidéo pour acquérir une image vidéo périphérique dans laquelle la périphérie d'un véhicule est prise sur une vidéo; une unité d'acquisition d'informations d'obstacle pour acquérir des informations d'obstacle concernant un obstacle détecté dans la périphérie du véhicule; une unité de génération d'image vidéo de vue aérienne pour générer une image vidéo de vue aérienne à partir de l'image vidéo périphérique acquise par l'unité d'acquisition d'image vidéo, dans lequel un processus de conversion de point de vue a été réalisé de façon à surregarder le véhicule à partir de ce qui précède; une unité de génération d'image vidéo superposée pour générer, sur la base d'informations d'obstacle acquises par l'unité d'acquisition d'informations d'obstacle, une image vidéo de vue aérienne dans laquelle les informations indiquant l'obstacle sont superposées dans une section centrale enfermée par l'image vidéo de vue aérienne; et une unité de commande d'affichage pour amener l'image vidéo de vue aérienne générée par l'unité de génération d'image vidéo superposée à être affichée sur un panneau d'affichage.
PCT/JP2017/024558 2016-09-20 2017-07-04 Dispositif de génération d'image vidéo de vue aérienne, système de génération d'image vidéo de vue aérienne, procédé de génération d'image vidéo de vue aérienne, et programme WO2018055873A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP17852654.7A EP3462733B1 (fr) 2016-09-20 2017-07-04 Dispositif de génération d'image vidéo de vue aérienne, système de génération d'image vidéo de vue aérienne, procédé de génération d'image vidéo de vue aérienne, et programme
CN201780023741.3A CN109076195B (zh) 2016-09-20 2017-07-04 俯瞰影像生成装置、系统、方法以及存储介质
US16/208,650 US10587801B2 (en) 2016-09-20 2018-12-04 Bird's-eye view video generation device, bird'S-eye view video generation system, bird's-eye view video generation method, and non-transitory storage medium

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
JP2016183103 2016-09-20
JP2016-183103 2016-09-20
JP2016184461A JP6644264B2 (ja) 2016-09-21 2016-09-21 俯瞰映像生成装置、俯瞰映像生成システム、俯瞰映像生成方法およびプログラム
JP2016-184461 2016-09-21
JP2017073368A JP6730617B2 (ja) 2016-09-20 2017-04-03 俯瞰映像生成装置、俯瞰映像生成システム、俯瞰映像生成方法およびプログラム
JP2017-073368 2017-04-03

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/208,650 Continuation US10587801B2 (en) 2016-09-20 2018-12-04 Bird's-eye view video generation device, bird'S-eye view video generation system, bird's-eye view video generation method, and non-transitory storage medium

Publications (1)

Publication Number Publication Date
WO2018055873A1 true WO2018055873A1 (fr) 2018-03-29

Family

ID=61689394

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/024558 WO2018055873A1 (fr) 2016-09-20 2017-07-04 Dispositif de génération d'image vidéo de vue aérienne, système de génération d'image vidéo de vue aérienne, procédé de génération d'image vidéo de vue aérienne, et programme

Country Status (1)

Country Link
WO (1) WO2018055873A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112470465A (zh) * 2018-07-30 2021-03-09 Jvc建伍株式会社 俯瞰图像生成装置、俯瞰图像生成方法以及程序

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006341641A (ja) * 2005-06-07 2006-12-21 Nissan Motor Co Ltd 映像表示装置及び映像表示方法
JP2008174076A (ja) * 2007-01-18 2008-07-31 Xanavi Informatics Corp 車両周辺監視装置、その表示方法
WO2012144076A1 (fr) * 2011-04-22 2012-10-26 トヨタ自動車株式会社 Dispositif de surveillance de la périphérie d'un véhicule et procédé d'affichage d'informations de surveillance de la périphérie d'un véhicule
JP2015065515A (ja) * 2013-09-24 2015-04-09 日立建機株式会社 車両周囲移動物体検知システム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006341641A (ja) * 2005-06-07 2006-12-21 Nissan Motor Co Ltd 映像表示装置及び映像表示方法
JP2008174076A (ja) * 2007-01-18 2008-07-31 Xanavi Informatics Corp 車両周辺監視装置、その表示方法
WO2012144076A1 (fr) * 2011-04-22 2012-10-26 トヨタ自動車株式会社 Dispositif de surveillance de la périphérie d'un véhicule et procédé d'affichage d'informations de surveillance de la périphérie d'un véhicule
JP2015065515A (ja) * 2013-09-24 2015-04-09 日立建機株式会社 車両周囲移動物体検知システム

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112470465A (zh) * 2018-07-30 2021-03-09 Jvc建伍株式会社 俯瞰图像生成装置、俯瞰图像生成方法以及程序
EP3817373A4 (fr) * 2018-07-30 2021-07-14 JVCKenwood Corporation Appareil, procédé et programme de génération d'images en vue de dessus
US11498484B2 (en) 2018-07-30 2022-11-15 Jvckenwood Corporation Overhead image generation device, overhead image generation method, and program
CN112470465B (zh) * 2018-07-30 2023-09-29 Jvc建伍株式会社 俯瞰图像生成装置以及计算机可读存储介质

Similar Documents

Publication Publication Date Title
CN109076195B (zh) 俯瞰影像生成装置、系统、方法以及存储介质
JP6379779B2 (ja) 車両用表示装置
JP7196881B2 (ja) 俯瞰映像生成装置および俯瞰映像生成方法
JP2009239674A (ja) 車両周辺表示装置
WO2018159016A1 (fr) Dispositif, système et procédé de génération d'image en vue à vol d'oiseau et programme
WO2018159017A1 (fr) Dispositif de commande d'affichage de véhicule, système d'affichage de véhicule, programme et procédé de commande d'affichage de véhicule
WO2018030491A1 (fr) Dispositif d'inspection périphérique
JP2006246430A (ja) 車両周辺監視装置
WO2011013813A1 (fr) Dispositif embarqué à bord d’un véhicule et programme de traitement d’image
JP2018144554A (ja) 車両用ヘッドアップディスプレイ装置
WO2018096792A1 (fr) Dispositif, système et procédé de génération d'image vidéo de vue à vol d'oiseau et programme
WO2018055873A1 (fr) Dispositif de génération d'image vidéo de vue aérienne, système de génération d'image vidéo de vue aérienne, procédé de génération d'image vidéo de vue aérienne, et programme
JP2008003659A (ja) 車両用周囲表示装置
JP2011211432A (ja) 車載用撮像装置
JP2018148314A (ja) 俯瞰映像生成装置、俯瞰映像生成システム、俯瞰映像生成方法およびプログラム
JP4713033B2 (ja) 車両周辺環境表示装置
JP2017114218A (ja) 俯瞰映像生成装置、俯瞰映像生成システム、俯瞰映像生成方法およびプログラム
JP2010245701A (ja) 表示装置
JP2010009491A (ja) 運転支援装置、運転支援方法、及び運転支援プログラム
JP2018050180A (ja) 俯瞰映像生成装置、俯瞰映像生成システム、俯瞰映像生成方法およびプログラム
JP6644256B2 (ja) 俯瞰映像生成装置、俯瞰映像生成システム、俯瞰映像生成方法およびプログラム
JP6590538B2 (ja) 画像処理装置および画像処理方法
JP2016063352A (ja) 車載用表示装置
JP2018019176A (ja) 車両用表示制御装置、車両用表示システム、車両用表示制御方法およびプログラム
JP2019207632A (ja) 表示装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17852654

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017852654

Country of ref document: EP

Effective date: 20181228

NENP Non-entry into the national phase

Ref country code: DE