WO2018055873A1 - Overhead view video image generation device, overhead view video image generation system, overhead view video image generation method, and program - Google Patents

Overhead view video image generation device, overhead view video image generation system, overhead view video image generation method, and program Download PDF

Info

Publication number
WO2018055873A1
WO2018055873A1 PCT/JP2017/024558 JP2017024558W WO2018055873A1 WO 2018055873 A1 WO2018055873 A1 WO 2018055873A1 JP 2017024558 W JP2017024558 W JP 2017024558W WO 2018055873 A1 WO2018055873 A1 WO 2018055873A1
Authority
WO
WIPO (PCT)
Prior art keywords
obstacle
video
overhead
image
vehicle
Prior art date
Application number
PCT/JP2017/024558
Other languages
French (fr)
Japanese (ja)
Inventor
恒夫 里見
浅山　学
俊夫 森
Original Assignee
株式会社Ｊｖｃケンウッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2016184461A external-priority patent/JP6644264B2/en
Priority claimed from JP2017073368A external-priority patent/JP6730617B2/en
Application filed by 株式会社Ｊｖｃケンウッド filed Critical 株式会社Ｊｖｃケンウッド
Priority to CN201780023741.3A priority Critical patent/CN109076195B/en
Priority to EP17852654.7A priority patent/EP3462733B1/en
Publication of WO2018055873A1 publication Critical patent/WO2018055873A1/en
Priority to US16/208,650 priority patent/US10587801B2/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/301—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8093—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for obstacle warning

Definitions

  • the present invention relates to an overhead video generation device, an overhead video generation system, an overhead video generation method, and a program.
  • Patent Document 1 A technique related to a vehicle periphery display device that displays a vehicle overhead view image together with a vehicle image is known (see, for example, Patent Document 1).
  • the display panel of an in-vehicle device such as a navigation device that displays a bird's-eye view image of the vehicle is often horizontally long. For this reason, a vertically long overhead image is not displayed on the entire surface of the display panel. Therefore, a technique is known in which, when an obstacle close to the host vehicle is detected, a bird's-eye view image of the vehicle and a video image of the direction of the obstacle are displayed side by side (for example, see Patent Document 2).
  • the present invention has been made in view of the above, and an object thereof is to enable appropriate confirmation of obstacles around a vehicle.
  • an overhead image generation apparatus includes a video acquisition unit that acquires a peripheral video obtained by photographing the periphery of a vehicle, and obstacles detected in the vicinity of the vehicle.
  • An obstacle information acquisition unit that acquires obstacle information, and an overhead view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit,
  • a video generation unit that generates a bird's-eye view image obtained by synthesizing information indicating the obstacle acquired by the obstacle information acquisition unit, and a display control unit that causes the display unit to display the bird's-eye view image generated by the video generation unit. It is characterized by.
  • An overhead image generation system includes the above-described overhead image generation device, a camera that captures the periphery of the vehicle and supplies a periphery image to the image acquisition unit, detects an obstacle around the vehicle, and detects the obstacle An obstacle detection unit that supplies obstacle information to the object information acquisition unit, and at least one of the display unit.
  • the bird's-eye view video generation method includes a video acquisition step of acquiring a peripheral video obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring obstacle information of an obstacle detected in the periphery of the vehicle, A bird's-eye view image is generated from the surrounding image acquired in the image acquisition step so that the vehicle is viewed from above, and the obstacle acquired in the obstacle information acquisition step is shown in the center of the bird's-eye view image.
  • the program according to the present invention includes a video acquisition step of acquiring a peripheral video obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring obstacle information of an obstacle detected in the vicinity of the vehicle, and the video acquisition step. Generates an overhead view image obtained by performing viewpoint conversion processing so that the vehicle is looked down from above, and synthesizes information indicating the obstacle acquired in the obstacle information acquisition step in the center of the overhead view image
  • the computer operating as the overhead video generation device executes a video generation step for generating the overhead video and a display control step for displaying the overhead video generated in the video generation step on the display unit.
  • FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment.
  • FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment.
  • FIG. 3 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the first embodiment.
  • FIG. 4 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment.
  • FIG. 5 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the first embodiment.
  • FIG. 6 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the second embodiment.
  • FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment.
  • FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment.
  • FIG. 3 is
  • FIG. 7 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the second embodiment.
  • FIG. 8 is a flowchart showing the flow of processing in the overhead video generation device of the overhead video generation system according to the third embodiment.
  • FIG. 9 is a diagram showing an example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 10 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 11 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 12 is a graph showing an example of the relationship between the distance between the arcs and the distance to the obstacle in the obstacle notification icon.
  • FIG. 13 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 14 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 15 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 16 is a block diagram illustrating a configuration example of the overhead view video generation system according to the seventh embodiment.
  • FIG. 17 is a diagram illustrating a display panel that displays an overhead video generated by the overhead video generation system according to the seventh embodiment.
  • FIG. 18 is a diagram showing a display panel that displays a horizontally long overhead image generated by the overhead image generation system according to the seventh embodiment.
  • FIG. 19 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the seventh embodiment.
  • FIG. 20 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 21 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the seventh embodiment.
  • FIG. 22 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eighth embodiment.
  • FIG. 23 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 24 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the ninth embodiment.
  • FIG. 20 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 21 is a flowchart showing a flow of processing in the overhead video generation device of the overhead
  • FIG. 25 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 26 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the tenth embodiment.
  • FIG. 27 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 28 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eleventh embodiment.
  • FIG. 29 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • FIG. 30 is a diagram illustrating another example of a method for generating a horizontally long overhead image in the overhead image generation system.
  • FIG. 31 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 30.
  • FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment.
  • the bird's-eye view video generation system 1 generates a bird's-eye view video 100 (see FIG. 2).
  • the overhead image generation device 40 and the overhead image generation system 1 are mounted on a vehicle. In addition to what is mounted on the vehicle, the overhead image generation device 40 and the overhead image generation system 1 may be a portable device that can be used in the vehicle.
  • the overhead view video generation system 1 includes a front camera (camera) 11, a rear camera (camera) 12, a left side camera (camera) 13, a right side camera (camera) 14, and a front left sensor (obstacle detection unit).
  • 21A front center sensor (obstacle detection unit) 21B, front right sensor (obstacle detection unit) 21C, rear left sensor (obstacle detection unit) 22A, rear center sensor (obstacle detection unit) 22B , A rear right sensor (obstacle detection unit) 22C, a display panel 31, and an overhead image generation device 40.
  • the front camera 11 is arranged in front of the vehicle and photographs the periphery around the front of the vehicle.
  • the front camera 11 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.
  • the rear camera 12 is arranged at the rear of the vehicle and photographs the periphery around the rear of the vehicle.
  • the rear camera 12 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.
  • the left side camera 13 is arranged on the left side of the vehicle, and takes a picture around the left side of the vehicle.
  • the left side camera 13 outputs the captured video to the video acquisition unit 42 of the overhead view video generation device 40.
  • the right-side camera 14 is arranged on the right side of the vehicle and photographs the periphery around the right side of the vehicle.
  • the right side camera 14 outputs the captured video to the video acquisition unit 42 of the overhead view video generation device 40.
  • the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 photograph all directions of the vehicle.
  • the front left sensor 21A is disposed on the left front side of the vehicle and detects an obstacle on the left front side of the vehicle.
  • the front left sensor 21A is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the front left sensor 21 ⁇ / b> A detects an object having a height from the ground that may come into contact with the vehicle. For example, the front left sensor 21A detects an obstacle at a distance of about 5 m from the vehicle.
  • the front left sensor 21 ⁇ / b> A detects an obstacle in a range of, for example, about 40 ° with the center of the sensor as the center when viewed in the vertical direction.
  • the detection range of the front left sensor 21A may overlap with a part of the detection range of the front center sensor 21B.
  • the front left sensor 21 ⁇ / b> A outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front left sensor 21A, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the front center sensor 21B is arranged at the front center of the vehicle and detects an obstacle at the front center of the vehicle.
  • the front center sensor 21B is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the front center sensor 21B detects a sensor having a height from the ground that may come into contact with the vehicle.
  • the front center sensor 21B detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the front center sensor 21B detects an obstacle in a range of, for example, about 40 ° with the center portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the front center sensor 21B may overlap with a part of the detection range of the front left sensor 21A and the front right sensor 21C.
  • the front center sensor 21B outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front center sensor 21B, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the front right sensor 21C is arranged on the front right side of the vehicle and detects an obstacle on the front right side of the vehicle.
  • the front right sensor 21C is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the front right sensor 21 ⁇ / b> C detects a thing having a height from the ground that may come into contact with the vehicle.
  • the front right sensor 21C detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the front right sensor 21C detects an obstacle in a range of, for example, about 40 ° with the center portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the front right sensor 21C may overlap with a part of the detection range of the front center sensor 21B.
  • the front right sensor 21 ⁇ / b> C outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front right sensor 21C, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the front left sensor 21A, the front center sensor 21B, and the front right sensor 21C detect an obstacle ahead of the vehicle.
  • the rear left sensor 22A is disposed on the rear left side of the vehicle and detects an obstacle on the rear left side of the vehicle.
  • the rear left sensor 22A is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the rear left sensor 22A detects an object having a height from the ground that may come into contact with the vehicle.
  • the rear left sensor 22A detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the rear left sensor 22A detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the rear left sensor 22A may overlap with a part of the detection range of the rear center sensor 22B.
  • the rear left sensor 22A outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear left sensor 22A, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the rear center sensor 22B is disposed at the rear center of the vehicle and detects an obstacle at the rear center of the vehicle.
  • the rear center sensor 22B is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • Back center sensor 22B detects what has a height from the ground which may be in contact with a vehicle.
  • the rear center sensor 22B detects an obstacle at a distance of about 5 m from the vehicle.
  • the rear center sensor 22B detects an obstacle in a range of, for example, about 40 ° with the center of the sensor as the center when viewed in the vertical direction.
  • the detection range of the rear center sensor 22B may overlap with a part of the detection range of the rear left sensor 22A and the rear right sensor 22C.
  • the rear center sensor 22B outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead image generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear center sensor 22B, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the rear right sensor 22C is disposed on the rear right side of the vehicle and detects an obstacle on the rear right side of the vehicle.
  • the rear right sensor 22C is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof.
  • the rear right sensor 22C detects an object having a height from the ground that may come into contact with the vehicle.
  • the rear right sensor 22C detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the rear right sensor 22C detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the rear right sensor 22C may overlap with a part of the detection range of the rear center sensor 22B.
  • the rear right sensor 22C outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40.
  • Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear right sensor 22C, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
  • the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C detect an obstacle behind the vehicle.
  • the display panel 31 is a display including, for example, a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminescence) display.
  • the display panel 31 displays the overhead video 100 based on the video signal output from the overhead video generation device 40 of the overhead video generation system 1.
  • the display panel 31 may be dedicated to the overhead view video generation system 1 or may be used jointly with other systems including a navigation system, for example.
  • the display panel 31 is disposed at a position that is easily visible to the driver.
  • the overhead view video generation device 40 includes a control unit 41 and a storage unit 49.
  • the control unit 41 is an arithmetic processing unit configured with, for example, a CPU (Central Processing Unit).
  • the control unit 41 loads the program stored in the storage unit 49 into the memory and executes instructions included in the program.
  • the control unit 41 includes a video acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, an overhead video generation unit (video generation unit) 46, a superimposed video generation unit (video generation unit) 47, Display control unit 48.
  • the video acquisition unit 42 acquires a peripheral video obtained by photographing the periphery of the vehicle. More specifically, the video acquisition unit 42 acquires video output from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The video acquisition unit 42 outputs the acquired video to the overhead view video generation unit 46.
  • the obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in the vicinity of the vehicle.
  • the obstacle information acquisition unit 43 acquires obstacle information including the distance to the detected obstacle. More specifically, the obstacle information acquisition unit 43 acquires the obstacle information output by the front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C. To do.
  • the obstacle information acquisition unit 43 outputs the acquired obstacle information to the superimposed video generation unit 47.
  • the vehicle information acquisition unit 44 acquires vehicle information serving as a trigger for displaying a bird's-eye view video, such as vehicle gear operation information, from a CAN (Controller Area Network) or various sensors that sense the state of the vehicle.
  • vehicle information acquisition unit 44 outputs the acquired vehicle information to the overhead view video generation unit 46.
  • the bird's-eye view video generation unit 46 generates the bird's-eye view video 100 that has been subjected to the viewpoint conversion process so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. More specifically, the bird's-eye view image generation unit 46 generates the bird's-eye view image 100 based on images taken by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the bird's-eye view image 100 may be any known method and is not limited.
  • the overhead image generation unit 46 outputs the generated overhead image 100 to the display control unit 47.
  • the overhead view image 100 will be described with reference to FIG.
  • FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment.
  • the bird's-eye view image 100 has a vertically long rectangular shape.
  • the bird's-eye view image 100 is located at the center surrounded by the front image 101, the rear image 102, the left side image 103, the right side image 104, the front image 101, the rear image 102, the left side image 103, and the right side image 104.
  • the front video 101, the rear video 102, the left side video 103, the right side video 104, and the central video 105 may be separated by a frame-like boundary line.
  • the central video 105 is generated in a vertically long rectangular shape. In the central video 105, the boundaries of the front video 101, the rear video 102, the left video 103, and the right video 104 are separated by lines.
  • the center image 105 shows the position of the vehicle in the overhead view image 100.
  • FIG. 2 diagonal broken lines indicating boundaries between the front video 101, the rear video 102, the left side video 103, and the right side video 104 are illustrated for explanation, but are actually displayed on the display panel 31.
  • the broken line is not displayed in the overhead view video 100. The same applies to the other figures.
  • the superimposed video generation unit 47 generates an overhead video 100 in which information indicating an obstacle is superimposed on the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43.
  • the superimposed video generation unit 47 superimposes information indicating the direction in which the obstacle is detected on the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. 100 is generated.
  • the superimposed video generation unit 47 indicates information indicating the direction in which the obstacle is detected as an obstacle notification icon (information indicating the obstacle) 120.
  • the obstacle notification icon 120 schematically shows the detection direction in the horizontal direction of each sensor that detects an obstacle, and the arrangement direction of the plurality of arcs is the detection direction starting from the sensor mounting position, or the vehicle. For example, a radial arrangement with the center.
  • the width of the arc constituting the obstacle notification icon 120 may indicate the detection range of each sensor that detects the obstacle, or may be a fixed width corresponding to the detection direction regardless of the detection range of each sensor. .
  • the obstacle notification icon 120 is an icon for notifying an obstacle.
  • the obstacle notification icon 120 indicates the distance and direction to the obstacle.
  • the obstacle notification icon 120 includes a front left icon (information indicating an obstacle) 121, a front center icon (information indicating an obstacle) 122, a front right icon (information indicating an obstacle) 123, and a rear left icon (an obstacle is displayed).
  • the front left icon 121 is an icon for notifying an obstacle on the left front side of the vehicle. More specifically, the front left icon 121 is an icon that notifies that an obstacle has been detected by the front left sensor 21A. The front left icon 121 is superimposed on the upper left of the central image 105 of the overhead view image 100 in FIG.
  • the front left icon 121 is composed of a triple arc-shaped curve.
  • the arc-shaped curve bulges toward the outside of the overhead view image 100.
  • the triple arc-shaped curve has a radius that decreases from the outside of the bird's-eye view image 100 toward the center.
  • the triple arc-shaped curve is shortened from the outside of the overhead view image 100 toward the center side.
  • the front left icon 121 may change the color according to the distance to the obstacle. For example, the front left icon 121 is displayed in green when the distance to the obstacle is greater than or equal to the first predetermined distance. The front left icon 121 is displayed in yellow when the distance to the obstacle is equal to or greater than a second predetermined distance that is less than the first predetermined distance and smaller than the first predetermined distance. The front left icon 121 is displayed in red when the distance to the obstacle is less than the second predetermined distance.
  • the front left icon 121 may change the number of arcuate curves according to the distance to the obstacle. For example, the front left icon 121 displays the outermost arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is equal to or greater than the first predetermined distance. The front left icon 121 displays the outermost arc-shaped curve and the center arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is less than the first predetermined distance and greater than or equal to the second predetermined distance. To do. The front left icon 121 displays all triple arcuate curves when the distance to the obstacle is less than the second predetermined distance.
  • the front center icon 122 is an icon for notifying an obstacle at the front center of the vehicle. More specifically, the front center icon 122 is an icon that notifies that an obstacle has been detected by the front center sensor 21B. The front center icon 122 is superimposed on the upper center of the center image 105 of the overhead image 100 in FIG. The front center icon 122 is configured in the same manner as the front left icon 121.
  • the front right icon 123 is an icon for notifying an obstacle on the right front side of the vehicle. More specifically, the front right icon 123 is an icon that notifies that an obstacle has been detected by the front right sensor 21C. The front right icon 123 is superimposed on the upper right of the central image 105 of the overhead image 100 in FIG. The front right icon 123 is configured in the same manner as the front left icon 121.
  • the rear left icon 124 is an icon for notifying an obstacle on the left rear side of the vehicle. More specifically, the rear left icon 124 is an icon for notifying that an obstacle has been detected by the rear left sensor 22A.
  • the rear left icon 124 is superimposed on the lower left of the central image 105 of the overhead image 100 in FIG.
  • the rear left icon 124 is configured in the same manner as the front left icon 121.
  • the rear center icon 125 is an icon that notifies an obstacle at the rear center of the vehicle. More specifically, the rear center icon 125 is an icon for notifying that an obstacle has been detected by the rear center sensor 22B. The rear center icon 125 is superimposed on the lower center of the center image 105 of the overhead image 100 in FIG. The rear center icon 125 is configured in the same manner as the front left icon 121.
  • the rear right icon 126 is an icon for notifying an obstacle on the rear right side of the vehicle. More specifically, the rear right icon 126 is an icon for notifying that an obstacle has been detected by the rear right sensor 22C. The rear right icon 126 is superimposed on the lower right of the central image 105 of the overhead image 100 in FIG. The rear right icon 126 is configured in the same manner as the front left icon 121.
  • the display control unit 48 causes the display panel 31 to display the overhead video 100 generated by the superimposed video generation unit 47.
  • the storage unit 49 stores data required for various processes in the overhead view video generation device 40 and various processing results.
  • the storage unit 49 is, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.
  • FIG. 3 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the first embodiment.
  • the control unit 41 determines whether or not to start the overhead view video display (step S11). As an example of the determination to start the overhead view video display, the control unit 41 determines whether or not to disclose the overhead view video display based on the presence or absence of a reverse trigger.
  • the reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle is rearward in the front-rear direction of the vehicle.
  • the control unit 41 determines not to start the overhead view video display (No in step S11), and executes the process of step S11 again.
  • the control unit 41 determines to start the overhead view video display (Yes in Step S11), and proceeds to Step S12.
  • the control unit 41 generates and displays the bird's-eye view video 100 (step S12). More specifically, the control unit 41 causes the overhead view video generation unit 46 to generate the overhead view video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the control unit 41 determines whether an obstacle has been detected (step S13). More specifically, the control unit 41 determines whether or not obstacle information has been acquired by the obstacle information acquisition unit 43. When it is determined that the obstacle information acquisition unit 43 has acquired the obstacle information (Yes in step S13), the control unit 41 proceeds to step S14. When it is determined that the obstacle information acquisition unit 43 has not acquired the obstacle information (No in step S13), the control unit 41 proceeds to step S15.
  • the control unit 41 superimposes and displays an obstacle notification icon 120 indicating an obstacle at the center of the overhead view video 100 (step S14). More specifically, the control unit 41 causes the superimposed video generation unit 47 to indicate the direction in which the obstacle is detected in the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. The overhead view video 100 in which the obstacle notification icon 120 is superimposed on is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the generation unit 47 When the control unit 41 acquires obstacle information from a plurality of sensors including the front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C,
  • the generation unit 47 generates the overhead image 100 in which a plurality of obstacle notification icons 120 are superimposed on the central image 105 of the overhead image 100.
  • the control unit 41 determines whether or not to end the overhead view video display (step S15). More specifically, the control unit 41 determines whether or not to end the bird's-eye view video display based on the presence or absence of the reverse trigger. When there is no backward trigger, in other words, when the backward trigger is released, the control unit 41 determines to end the overhead view video display (Yes in step S15), and ends the process. When there is a reverse trigger, the control unit 41 determines not to end the overhead view video display (No in step S15), returns to step S13, and continues the process.
  • the overhead image generation system 1 superimposes the obstacle notification icon 120 on the center of the overhead image 100 so as to indicate the direction in which the obstacle is detected, and displays the display panel. 31 outputs a video signal.
  • the display panel 31 displays the bird's-eye view video 100 together with the navigation screen, for example, based on the video signal output from the bird's-eye view video generation system 1.
  • FIG. 4 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment.
  • the rear image 102 of the overhead view image 100 includes an obstacle image 130 of the obstacle on the left rear side.
  • Control unit 41 determines that an obstacle has been detected in step S13. Then, in step S14, the control unit 41 detects the obstacle in the central image 105 of the overhead view video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43 in the superimposed image generation unit 47, and then left An overhead view image 100 in which the obstacle notification icon 120 is superimposed so as to indicate the direction is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • FIG. 5 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the first embodiment.
  • a vehicle icon 140 that is a vehicle icon indicating a vehicle when the vehicle is viewed from above is synthesized with the central image 105 of the overhead view image 100.
  • the control unit 41 causes the superimposed video generation unit 47 to generate an overhead video 100 in which the obstacle notification icon 120 is superimposed on the vehicle icon 140 so as to indicate the left direction after the obstacle is detected.
  • the overhead notification icon 120 when an obstacle is detected around the vehicle, the overhead notification icon 120 is superimposed on the central image 105 of the overhead image 100 so as to indicate the direction in which the obstacle is detected.
  • the video 100 is displayed on the display panel 31.
  • the obstacle notification icon 120 since the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100, the obstacle notification icon 120 and the obstacle reflected in the overhead image 100 are not superimposed.
  • the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
  • this embodiment can make it possible to appropriately check obstacles around the vehicle.
  • the obstacle notification icon 120 can notify the direction and distance in which the obstacle is detected.
  • this embodiment can further improve the visibility of the obstacle notification icon 120 when the vehicle icon 140 is not displayed on the central image 105 of the overhead view image 100.
  • this embodiment can make it possible to more appropriately check obstacles around the vehicle.
  • the vehicle icon 140 may be displayed on the center image 105 of the overhead view image 100.
  • this embodiment can display the bird's-eye view image 100 which can recognize the direction of the obstacle with respect to a vehicle more easily.
  • this embodiment can make it possible to appropriately check obstacles around the vehicle.
  • FIG. 6 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the second embodiment.
  • FIG. 7 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the second embodiment.
  • the overhead image generation system 1 has the same basic configuration as the overhead image generation system 1 of the first embodiment. In the following description, components similar to those in the overhead view video generation system 1 are denoted by the same reference numerals or corresponding reference numerals, and detailed description thereof is omitted.
  • the overhead image generation system 1 of the present embodiment is different from the overhead image generation system 1 of the first embodiment in the processing of the obstacle information acquisition unit 43, the superimposed image generation unit 47, and the control unit 41 of the overhead image generation device 40. .
  • the obstacle notification icon 120 includes a second front left icon 121, a front center icon 122, a front right icon 123, a rear left icon 124, a rear center icon 125, and a rear right icon 126 in the first embodiment.
  • Icon information indicating obstacle
  • second front center icon information indicating obstacle
  • second front right icon information indicating obstacle
  • second rear left icon information indicating obstacle
  • second A rear center icon information indicating an obstacle
  • a second rear right icon information indicating an obstacle
  • the front left icon 121 is an icon for notifying an obstacle on the left front side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front left icon 121 is an icon for notifying that the front left sensor 21A has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the front center icon 122 is an icon for notifying an obstacle at the front center of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front center icon 122 is an icon for notifying that the front center sensor 21B has detected an obstacle whose distance to the obstacle is less than a threshold value.
  • the front right icon 123 is an icon for notifying an obstacle on the right front side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front right icon 123 is an icon for informing that the front right sensor 21C has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the rear left icon 124 is an icon for notifying an obstacle on the left rear side of the vehicle whose distance to the obstacle is less than the threshold. More specifically, the rear left icon 124 is an icon that notifies that the rear left sensor 22A has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the rear center icon 125 is an icon for notifying an obstacle at the rear center of the vehicle whose distance to the obstacle is less than a threshold value. More specifically, the rear center icon 125 is an icon for informing that the rear center sensor 22B has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the rear right icon 126 is an icon for notifying an obstacle on the right rear side of the vehicle whose distance to the obstacle is less than a threshold value. More specifically, the rear right icon 126 is an icon for notifying that the rear right sensor 22C has detected an obstacle whose distance to the obstacle is less than a threshold.
  • the second front left icon is an icon for notifying an obstacle on the left front side of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front left icon is an icon for notifying that the front left sensor 21A has detected an obstacle whose distance to the obstacle is not less than a threshold value.
  • the second front left icon is superimposed on the upper left outside the central image 105 of the overhead image 100.
  • the second front left icon is configured in the same manner as the front left icon 121.
  • the second front center icon is an icon for notifying an obstacle at the front center of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front center icon is an icon for notifying that the front center sensor 21B has detected an obstacle whose distance to the obstacle is a threshold value or more.
  • the second front center icon is superimposed on the upper center outside the center image 105 of the overhead view image 100.
  • the second front center icon is configured in the same manner as the front left icon 121.
  • the second front right icon is an icon for notifying an obstacle on the right front side of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front right icon is an icon for informing that the front right sensor 21C has detected an obstacle whose distance to the obstacle is not less than a threshold value.
  • the second front right icon is superimposed on the upper right outside the center image 105 of the overhead view image 100.
  • the second front right icon is configured in the same manner as the front left icon 121.
  • the second rear left icon is an icon for notifying an obstacle on the left rear side of the vehicle whose distance to the obstacle is equal to or greater than a threshold value. More specifically, the second rear left icon is an icon for notifying that the rear left sensor 22A has detected an obstacle whose distance to the obstacle is a threshold value or more.
  • the second rear left icon is superimposed on the lower left outside the center image 105 of the overhead view image 100.
  • the second rear left icon is configured in the same manner as the front left icon 121.
  • the second rear center icon is an icon for notifying an obstacle at the rear center of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second rear center icon is an icon for informing that the rear center sensor 22B has detected an obstacle whose distance to the obstacle is a threshold value or more. The second rear center icon is superimposed on the lower center outside the center image 105 of the overhead image 100. The second rear center icon is configured in the same manner as the front left icon 121.
  • the second rear right icon is an icon for notifying an obstacle on the right rear side of the vehicle whose distance to the obstacle is a threshold value or more. More specifically, the second rear right icon is an icon for notifying that the rear right sensor 22C has detected an obstacle whose distance to the obstacle is not less than a threshold value.
  • the second rear right icon is superimposed on the lower right outside the center image 105 of the overhead view image 100.
  • the second rear right icon is configured in the same manner as the front left icon 121.
  • the obstacle information acquisition unit 43 acquires information including the distance to the detected obstacle.
  • the obstacle information acquisition unit 43 outputs information including the acquired distance to the obstacle to the superimposed video generation unit 47.
  • the superimposed image generation unit 47 displays the vehicle icon 140 in the central image 105 of the overhead image 100. Is displayed, and an overhead video 100 is generated by superimposing an obstacle notification icon 120, which is information indicating an obstacle, on the outside of the central portion of the overhead video 100.
  • the superimposed video generation unit 47 displays the obstacle notification icon 120 in the center of the overhead view video 100.
  • the overhead view video 100 superimposed on the video 105 is generated.
  • step S21, step S23, step S26, and step S27 in the flowchart shown in FIG. 6 is the same as the processing in step S11, step S13, step S14, and step S15 in the flowchart shown in FIG.
  • the control unit 41 generates and displays the bird's-eye view image 100 on which the host vehicle icon 140 is superimposed (step S22). More specifically, the control unit 41 causes the overhead view video generation unit 46 to generate the overhead view video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. The control unit 41 synthesizes the vehicle icon 140 in the center of the generated bird's-eye view image 100 with the display control unit 48. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the control unit 41 determines whether or not the distance to the obstacle is greater than or equal to a threshold value (step S24). More specifically, when the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold (Yes in step S24), the control unit 41 proceeds to step S25. When the distance to the detected obstacle acquired by the obstacle information acquiring unit 43 is less than the predetermined threshold (No in step S24), the control unit 41 proceeds to step S26.
  • the predetermined threshold indicates that the position at which the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon are displayed is an overhead image 100. It is set to a value that does not overlap with the obstacle video. More specifically, the predetermined threshold is the outermost arc shape of the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon. Any value that is equal to or greater than the distance from the vehicle corresponding to the position of the curve is acceptable. For example, the predetermined threshold may be about 2 m.
  • the control unit 41 superimposes and displays the obstacle notification icon 120 on the bird's-eye view image 100 outside the host vehicle icon 140 (step S25). More specifically, the control unit 41 uses the superimposed image generation unit 47 to display the second front left icon and the overhead image 100 on the outside of the vehicle icon 140 based on the obstacle information acquired by the obstacle information acquisition unit 43. An overhead view image 100 in which at least one of a second front center icon, a second front right icon, a second rear left icon, a second rear center icon, and a second rear right icon is superimposed is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the overhead view video generation system 1 displays the second forward left icon, the second forward center icon, and the first forward icon 100 on the overhead view video 100 outside the vehicle icon 140.
  • a bird's-eye view image 100 in which at least one of a second front right icon, a second rear left icon, a second rear center icon, and a second rear right icon is superimposed is generated, and a video signal is output to the display panel 31.
  • the overhead view video generation system 1 includes a front left icon 121, a front center icon 122, a front right icon 123, a rear left icon 124, and a rear side.
  • An overhead view image 100 in which at least one of the center icon 125 and the rear right icon 126 is superimposed is generated, and a video signal is output to the display panel 31.
  • Control unit 41 determines that an obstacle has been detected in step S23.
  • step S24 the control unit 41 determines that the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold (Yes in step S24).
  • step S25 the control unit 41 superimposes the second rear left icon 127 indicating the left direction after detecting the obstacle on the overhead view video 100 outside the host vehicle icon 140, as shown in FIG. Is generated.
  • the control unit 41 displays the generated overhead image 100 on the display panel 31.
  • the present embodiment when the distance to the obstacle is equal to or greater than the predetermined threshold, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the overhead image 100 outside the host vehicle icon 140 is generated and displayed. Display on the panel 31.
  • the obstacle notification icon 120 when the distance to the obstacle is equal to or greater than a predetermined threshold, the obstacle notification icon 120 is superimposed on the overhead view image 100 outside the host vehicle icon 140, so that the obstacle notification icon 120 and the overhead view image 100 are reflected. Obstacles do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.
  • the present embodiment when the distance to the obstacle is less than the predetermined threshold, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100 is generated and displayed on the display panel 31.
  • the obstacle notification icon 120 when the distance to the obstacle is less than the predetermined threshold, the obstacle notification icon 120 is superimposed on the central image 105 of the overhead view video 100, so that the obstacle reflected in the obstacle notification icon 120 and the overhead view image 100 is displayed. And do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
  • the display position of the obstacle notification icon 120 in the overhead view video 100 changes according to the distance to the obstacle. For this reason, this embodiment can make it easier to grasp the distance to the obstacle.
  • FIG. 8 is a flowchart showing the flow of processing in the overhead video generation device of the overhead video generation system according to the third embodiment.
  • the overhead image generation system 1 of the present embodiment is different from the overhead image generation system 1 of the second embodiment in the processing of the obstacle information acquisition unit 43, the superimposed image generation unit 47, and the control unit 41 of the overhead image generation device 40. .
  • the front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C have a detection range that is farther from the range displayed as the overhead image 100.
  • the obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in a range far from the range displayed as the overhead view video 100.
  • the superimposed image generation unit 47 is a central portion of the overhead image 100 if the distance to the detected obstacle is far from the range displayed as the overhead image 100.
  • the vehicle icon 140 is displayed on the vehicle, and the overhead image 100 is generated by superimposing the obstacle notification icon 120 on the bird's-eye image 100 outside the vehicle icon 140.
  • the superimposed video generation unit 47 is based on the obstacle information acquired by the obstacle information acquisition unit 43. If the distance to the detected obstacle is within the range displayed as the overhead image 100, the obstacle notification icon The overhead view image 100 is generated by superimposing 120 on the central image 105 of the overhead view image 100 so as to indicate the direction in which the obstacle is detected.
  • steps S31 to S33 and steps S35 to S37 in the flowchart shown in FIG. 8 are the same as the processes in steps S21 to S23 and steps S25 to S27 in the flowchart shown in FIG.
  • the control unit 41 determines whether or not the distance to the obstacle is greater than or equal to the display range of the bird's-eye view image 100 (step S34).
  • the control unit 41 proceeds to step S35.
  • the control unit 41 proceeds to step S36.
  • the overhead view video generation system 1 displays the obstacle notification icon 120 on the overhead view video 100 outside the vehicle icon 140 when the distance to the obstacle is a range farther than the display range of the overhead view video 100.
  • the superimposed bird's-eye view image 100 is generated and a video signal is output to the display panel 31.
  • the overhead image generation system 1 When the distance to the obstacle is included in the display range of the overhead image 100, the overhead image generation system 1 generates the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100, and the display panel 31 outputs a video signal.
  • the obstacle notification icon 120 is superimposed on the bird's-eye view image 100 outside the host vehicle icon 140 when the distance to the obstacle is a range farther than the display range of the bird's-eye view image 100.
  • An overhead image 100 is generated and displayed on the display panel 31.
  • the obstacle notification icon 120 is superimposed on the outside of the vehicle icon 140, so the obstacle notification icon 120 and the overhead view video are displayed.
  • the obstacle reflected in 100 does not overlap.
  • the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
  • this embodiment can make it possible to appropriately check obstacles around the vehicle.
  • the present embodiment when the distance to the obstacle is included in the display range of the overhead image 100, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100 is generated and displayed on the display panel 31.
  • the obstacle notification icon 120 when the distance to the obstacle is included in the display range of the overhead image 100, the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100.
  • the obstacles reflected in the do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
  • This embodiment can display the obstacle notification icon 120 even when there is an obstacle in a range far from the display range of the overhead view video 100. For this reason, this embodiment can notify the obstacle of the range far from the display range of the bird's-eye view image 100.
  • the overhead view video generation system 1 according to the present embodiment will be described.
  • the overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in processing in the superimposed video generation unit 47 and the control unit 41 of the overhead video generation device 40. More specifically, it differs from the bird's-eye view image generation system 1 of the first embodiment in that an obstacle located in the traveling direction of the vehicle is detected in determining whether an obstacle is detected in the control unit 41.
  • the superimposed image generation unit 47 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 47 superimposes the obstacle notification icon 120 on the central image 105 of the overhead view video 100 for the obstacle located in the traveling direction of the vehicle.
  • the overhead view video 100 is generated.
  • step S13 the control unit 41 determines whether an obstacle is detected in the traveling direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43. When it is determined that the obstacle is detected in the traveling direction of the vehicle (Yes in Step S13), the control unit 41 proceeds to Step S14. When it is determined that no obstacle is detected in the traveling direction of the vehicle (No in step S13), the control unit 41 proceeds to step S15.
  • control unit 41 displays the obstacle notification icon 120 for obstacles approaching the vehicle, and hides the obstacle notification icon 120 for obstacles moving away from the vehicle.
  • this embodiment can make it possible to more appropriately check obstacles around the vehicle.
  • FIG. 9 is a diagram showing an example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 10 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 11 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment.
  • FIG. 12 is a graph showing an example of the relationship between the distance between the arcs and the distance to the obstacle in the obstacle notification icon.
  • the overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in the processing in the superimposed video generation unit 47 of the overhead video generation device 40.
  • the obstacle notification icon 120 has a plurality of components facing in the direction in which the obstacle is detected, and indicates the distance to the obstacle by the interval between the plurality of components.
  • the obstacle notification icon 120 includes a triple arc as a constituent element.
  • the obstacle notification icon 120 indicates the distance to the obstacle at intervals of arcs.
  • the obstacle notification icon 120 indicates that the arc interval is wider as the distance to the obstacle is larger, and the arc interval is narrower as the distance to the obstacle is smaller.
  • the interval between the arcs of the obstacle notification icon 120 may be set in a straight line so as to increase as the distance to the obstacle increases as shown by a solid line in FIG.
  • the interval between the arcs of the obstacle notification icon 120 may be set in a staircase shape so as to increase as the distance to the obstacle increases as indicated by a broken line in FIG. More specifically, when the distance to the obstacle is not less than the first threshold, for example, not less than 2 m, the arc interval is set as the first interval d1.
  • the first threshold may be a distance of a boundary included in a range to be displayed as the overhead view video 100. When the distance to the obstacle is equal to or greater than the first threshold, the outermost arc-shaped curve may be displayed outside the central image 105 of the overhead image 100.
  • the arc interval is set to a second interval d2 that is smaller than the first interval d1.
  • the arc interval is set to a third interval d3 that is smaller than the second interval d2.
  • the arcs may be overlapped with the interval between the arcs set to zero.
  • the distance to the obstacle is the distance from each sensor that detects the obstacle to the obstacle.
  • the distance to the obstacle substantially coincides with the distance from the end of the vehicle in the obstacle direction to the obstacle.
  • the superimposed video generation unit 47 displays the obstacle notification icon 120 in which the arc interval is changed based on the detected distance to the obstacle acquired by the obstacle information acquisition unit 43 in the central video 105 of the overhead view video 100.
  • the superimposed bird's-eye view image 100 is generated.
  • step S14 the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc.
  • the overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated.
  • the arc interval is the first interval d1.
  • the outermost arc-shaped curve is located outside the central image 105.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 9 on the display panel 31. Since the arc interval of the obstacle notification icon 120 and the outermost arc-shaped curve are located outside the central image 105, the obstacle is located near the boundary of the range to be displayed as the overhead image 100. I understand that.
  • step S14 the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc.
  • the overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated.
  • the arc interval is the second interval d2.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 10 on the display panel 31. It can be seen from the arc interval of the obstacle notification icon 120 that the obstacle is located within the range displayed as the bird's-eye view video 100.
  • step S14 the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc.
  • the overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated.
  • the arc interval is the third interval d3.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 11 on the display panel 31. Since the arc interval of the obstacle notification icon 120 is narrow, it can be seen that the obstacle is located in the vicinity of the rear end of the vehicle.
  • control unit 41 generates the overhead image 100 in which the obstacle notification icon 120 with the arc interval changed is superimposed on the central image 105 of the overhead image 100 based on the detected distance to the obstacle. .
  • the center image 105 of the overhead view image 100 indicates the direction in which the obstacle is detected, and the distance to the obstacle is indicated by the arc interval.
  • the overhead image 100 on which the obstacle notification icon 120 is superimposed is displayed on the display panel 31.
  • the obstacle notification icon 120 can display the direction of the obstacle and the distance to the obstacle.
  • FIG. 13 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 14 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • FIG. 15 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment.
  • the overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in the processing in the superimposed video generation unit 47 of the overhead video generation device 40.
  • the obstacle notification icon 120 has a horizontal detection range of the sensor detecting the obstacle and a horizontal direction of the detected obstacle depending on the horizontal width of the icon itself. Indicates the width.
  • the obstacle notification icon 120 has a larger arc length as the detection range in the horizontal direction of the sensor detecting the obstacle is larger, and the detection range in the horizontal direction of the sensor detecting the obstacle. The narrower is the smaller the arc length.
  • the obstacle notification icon 120 indicates that the arc length increases as the obstacle's horizontal width increases, and the arc length decreases as the obstacle's horizontal width decreases.
  • the obstacle notification icon 120 may have a projected width from the center of the overhead image 100 to both ends of the obstacle in the horizontal direction as the length of the arc.
  • the superimposed image generation unit 47 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 47 sets the width of the information indicating the obstacle as a width corresponding to the horizontal width of the detected obstacle. An overhead image 100 superimposed on the central image 105 is generated.
  • the control unit 41 includes a detection range in the horizontal direction of the sensor that detects the obstacle, based on the horizontal width of the icon itself, in the central video 105 of the overhead video 100 in the superimposed video generation unit 47.
  • An overhead view image 100 on which an obstacle notification icon 120 indicating the horizontal width of the detected obstacle is superimposed is generated.
  • the horizontal width of the obstacle here is the apparent width detected by the sensor, that is, the projected width.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead image 100 illustrated in FIG. 13 on the display panel 31. It can be seen from the horizontal width of the obstacle notification icon 120 that the detection range in the horizontal direction of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrow.
  • step S ⁇ b> 14 the control unit 41 causes the superimposed video generation unit 47 to generate the overhead video 100 in which the obstacle notification icon 120 is superimposed on the central video 105 of the overhead video 100. Since the projected width from the center of the overhead view image 100 to both ends in the horizontal direction of the obstacle is wide, the horizontal width of the obstacle notification icon 120 is the horizontal width of the obstacle notification icon 120 of FIG. Wider.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 14 on the display panel 31. Since the horizontal width of the obstacle notification icon 120 is wider than that in FIG. 13, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are widened. That is, it can be seen that the obstacle is approaching the rear end of the vehicle.
  • the superimposed video generation unit 47 displays the horizontal width of the obstacle notification icon 120 as shown in FIGS. 13 to 14 in accordance with the proximity state of the obstacle.
  • a change in the width of the obstacle notification icon 120 in the horizontal direction indicates whether the obstacle is approaching or moving away from the vehicle.
  • step S ⁇ b> 14 the control unit 41 causes the superimposed video generation unit 47 to generate the overhead video 100 in which the obstacle notification icon 120 is superimposed on the central video 105 of the overhead video 100.
  • the obstacle notification icon 120 has a horizontal width that is narrower than the horizontal width of the obstacle notification icon 120 of FIG.
  • the control unit 41 causes the superimposed video generation unit 47 to display the generated overhead image 100 illustrated in FIG. 15 on the display panel 31. Since the horizontal width of the obstacle notification icon 120 is narrower than that in FIG. 14, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrowed. That is, it can be seen that the horizontal width of the obstacle is narrow.
  • the center image 105 of the bird's-eye view image 100 indicates the direction in which the obstacle is detected, and the obstacle width is indicated by the horizontal width of the arc.
  • An overhead video 100 on which the obstacle notification icon 120 is superimposed is displayed on the display panel 31 so as to indicate the width in the horizontal direction.
  • the obstacle notification icon 120 can display the direction of the obstacle and the horizontal width of the obstacle.
  • the horizontal width of the obstacle notification icon 120 is changed and displayed as shown in FIGS. 13 to 14, for example.
  • FIG. 16 is a block diagram illustrating a configuration example of the overhead view video generation system according to the seventh embodiment.
  • the overhead image generation system 1A has the same basic configuration as the overhead image generation system 1 of the first embodiment.
  • the overhead view video generation system 1A will be described with reference to FIG.
  • the overhead image generation system 1A includes a front camera 11, a rear camera 12, a left side camera 13, a right side camera 14, a front sensor (obstacle detection unit) 21, and a rear sensor (obstacle detection unit) 22.
  • a left side sensor (obstacle detection unit) 23 a right side sensor (obstacle detection unit) 24, a display panel 31, and an overhead image generation device 40A.
  • the front sensor 21 is disposed in front of the vehicle and detects an obstacle in front of the vehicle.
  • the front sensor 21 is, for example, an infrared sensor or an ultrasonic sensor.
  • the front sensor 21 detects an object having a height from the ground that may come into contact with the vehicle.
  • the front sensor 21 detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the front sensor 21 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the front sensor 21 may overlap with a part of the detection range of the left side sensor 23 and the right side sensor 24.
  • the front sensor 21 may be configured by a combination of a plurality of sensors.
  • the front sensor 21 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
  • the rear sensor 22 is disposed behind the vehicle and detects an obstacle behind the vehicle.
  • the rear sensor 22 is, for example, an infrared sensor or an ultrasonic sensor.
  • the rear sensor 22 detects an object having a height from the ground that may come into contact with the vehicle.
  • the rear sensor 22 detects an obstacle at a distance of about 5 m from the vehicle.
  • the rear sensor 22 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the rear sensor 22 may overlap with a part of the detection range of the left side sensor 23 and the right side sensor 24.
  • the rear sensor 22 may be configured by a combination of a plurality of sensors.
  • the rear sensor 22 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
  • the left side sensor 23 is arranged on the left side of the vehicle and detects an obstacle on the left side of the vehicle.
  • the left side sensor 23 is, for example, an infrared sensor or an ultrasonic sensor.
  • the left side sensor 23 detects an object having a height from the ground that may come into contact with the vehicle.
  • the left side sensor 23 detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the left side sensor 23 detects an obstacle in a range of, for example, about 30 ° with the center of the sensor as the center when viewed in the vertical direction.
  • the detection range of the left side sensor 23 may overlap with a part of the detection range of the front sensor 21 and the rear sensor 22.
  • the left side sensor 23 may be configured by a combination of a plurality of sensors.
  • the left side sensor 23 outputs the obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead image generation device 40A.
  • the right side sensor 24 is arranged on the right side of the vehicle and detects an obstacle on the right side of the vehicle.
  • the right side sensor 24 is, for example, an infrared sensor or an ultrasonic sensor.
  • the right side sensor 24 detects an object having a height from the ground that may come into contact with the vehicle.
  • the right side sensor 24 detects an obstacle at a distance of about 5 m from the vehicle, for example.
  • the right side sensor 24 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction.
  • the detection range of the right side sensor 24 may overlap with a part of the detection range of the front sensor 21 and the rear sensor 22.
  • the right side sensor 24 may be configured by a combination of a plurality of sensors.
  • the right side sensor 24 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
  • the display panel 31 has a horizontally long rectangular shape. On the display panel 31, a vertically long overhead view image 100 as shown in FIG. 17 or a horizontally long overhead view image 200 as shown in FIG. 18 is displayed.
  • the overhead view image 200 having a horizontally long shape is a horizontally long overhead view image that matches the aspect ratio of the display surface of the display panel 31.
  • the control unit 41A includes a video acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, a video generation unit 45A, and a display control unit 48.
  • the obstacle information acquisition unit 43 acquires the obstacle information output by the front sensor 21, the rear sensor 22, the left side sensor 23, and the right side sensor 24.
  • the video generation unit 45A generates an overhead video 100 and an overhead video 200.
  • the video generation unit 45A includes an overhead video generation unit 46A and a video synthesis unit 47A.
  • the overhead video generation unit 46A generates the overhead video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. More specifically, the overhead view video generation unit 46A generates the overhead view video 100 based on the peripheral video captured by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the bird's-eye view image 100 may be any known method and is not limited. The overhead view video generation unit 46A outputs the generated overhead view video 100 to the display control unit 48.
  • FIG. 17 is a diagram illustrating a display panel that displays an overhead video generated by the overhead video generation system according to the seventh embodiment.
  • the bird's-eye view image 100 has a vertically long rectangular shape.
  • the bird's-eye view image 100 includes a front image 101, a rear image 102, a left side image 103, and a right side image 104.
  • a range surrounded by the front video 101, the rear video 102, the left side video 103, and the right side video 104 is referred to as a central portion of the overhead view video 100.
  • the central part of the overhead image 100 is the position of the vehicle in the overhead image 100.
  • the front video 101, the rear video 102, the left video 103, the right video 104, and the central portion of the overhead video 100 may be separated by a frame-like boundary line 110.
  • FIG. 17 diagonal broken lines indicating boundaries between the front video 101, the rear video 102, the left side video 103, and the right side video 104 are illustrated for explanation, but are actually displayed on the display panel 31.
  • the broken line may or may not be displayed in the overhead view video 100. The same applies to the other figures.
  • the overhead image generation unit 46A Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead image generation unit 46A does not change the display direction of the generated overhead image 100 when the direction in which the obstacle is detected is the front or the rear of the vehicle.
  • the overhead view image 200 having a horizontally long shape is generated.
  • the display direction is not changed means that the vertical direction of the bird's-eye view image 100 and the vertical direction of the bird's-eye view image 200 are kept aligned with the longitudinal direction of the vehicle.
  • that the display orientation is not changed is that, in the overhead view video 200 having a horizontally long shape, the front video 201 is arranged on the upper side, the rear video 202 is arranged on the lower side, the left side video 203 is arranged on the left side, and the right side In other words, it means that the right side image 204 is maintained.
  • the vertical direction of the bird's-eye view image 200 matches the longitudinal direction of the vehicle, so that the recognizability of the horizontally-viewed bird's-eye view image 200 can be maintained.
  • FIG. 18 is a diagram showing a display panel that displays a horizontally long overhead image generated by the overhead image generation system according to the seventh embodiment.
  • the bird's-eye view image 200 includes a front image 201, a rear image 202, a left side image 203, and a right side image 204.
  • a range surrounded by the front video 201, the rear video 202, the left side video 203, and the right side video 204 is referred to as a central portion of the overhead view video 200.
  • the central part of the overhead view image 200 is the position of the vehicle in the overhead view image 200.
  • the front video 201, the rear video 202, the left video 203, the right video 204, and the central portion of the overhead video 200 may be separated by a frame-like boundary line 210.
  • FIG. 19 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the seventh embodiment.
  • FIG. 20 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the bird's-eye view image generation unit 46A generates a bird's-eye view image 200 having a horizontally long shape by expanding a part of the surrounding image from the front camera 11 or the rear camera 12 with a fixed aspect ratio. More specifically, the overhead view video generation unit 46A expands a part of the front image 101 and a part of the rear image 102 from the generated vertically long overhead image 100 with a fixed aspect ratio to obtain a horizontally long shape. A video 200 is generated.
  • the overhead view video generation unit 46A generates a horizontally long overhead view video 200 from the cutout range A1, the cutout range A2, the cutout range A3, and the cutout range A4 of the overhead view video 100. More specifically, the overhead view video generation unit 46A expands the aspect ratio of the cutout range A1 and the cutout range A2 of the overhead view video 100 while fixing the aspect ratio to the aspect ratio of the display surface of the display panel 31. The bird's-eye view video 200 is generated.
  • the cutout range A1 is a rectangular range including a range in front of the vehicle that the driver needs to check.
  • the cut-out range A1 includes a part of the front image 101 of the overhead image 100, a part of the left side image 103, and a part of the right side image 104.
  • the cut-out range A1 includes a boundary between the front video 101 and the left side video 103 and a boundary between the front video 101 and the right side video 104.
  • the cut-out range A1 includes from the front end of the vehicle to about a few tens of centimeters forward.
  • the cutout range A1 includes from the left end of the vehicle to the left side of about several tens of cm.
  • the cutout range A1 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the cut-out range A2 is a rectangular range including a range behind the vehicle that the driver needs to check.
  • the cutout range A ⁇ b> 2 includes a part of the rear image 102 of the overhead image 100, a part of the left side image 103, and a part of the right side image 104.
  • the cutout range A ⁇ b> 2 includes a boundary between the rear image 102 and the left side image 103 and a boundary between the rear image 102 and the right side image 104.
  • the cut-out range A2 includes from the rear end of the vehicle to the rear of about several tens of centimeters.
  • the cutout range A2 includes from the left end of the vehicle to the left side of about several tens of cm.
  • the cutout range A2 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the cutout range A2 has the same shape and the same area as the cutout range A1.
  • the cutout area A3 is a rectangular area located between the cutout area A1 and the cutout area A2 of the overhead view image 100.
  • the cutout range A3 includes from the left end of the vehicle to the left side of about several tens of cm.
  • the left end portion of the cutout range A3 is positioned on the same straight line as the left end portion of the cutout range A1 and the left end portion of the cutout range A2.
  • the cutout range A4 is a rectangular range located between the cutout range A1 and the cutout range A2 of the overhead view image 100.
  • the cutout range A4 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the right end of the cutout range A4 is located on the same straight line as the right end of the cutout range A1 and the right end of the cutout range A2.
  • the cutout range A4 has the same shape and the same area as the cutout range A3.
  • a horizontally long overhead image 200 generated from such a bird's-eye view image 100 includes a cut-out range A1 image 211, a cut-out range A2 image 212, a cut-out range A3 image 213, and a cut-out range.
  • A4 video 214 The horizontally long overhead image 200 generated from such a bird's-eye view image 100 includes a cut-out range A1 image 211, a cut-out range A2 image 212, a cut-out range A3 image 213, and a cut-out range.
  • the image 211 is an image obtained by enlarging the image of the cutout range A1 of the overhead view image 100 until the aspect ratio is fixed and the width is the same as the width of the display panel 31.
  • the image of the tree displayed in the image 211 is an image obtained by fixing and expanding the image of the tree displayed in the cutout area A1 of the overhead image 100 shown in FIG.
  • the video 212 is a video obtained by enlarging the video of the cutout range A2 of the overhead video 100 until the aspect ratio is fixed and the horizontal width becomes the same as the horizontal width of the display panel 31.
  • the image of the tree displayed in the image 212 is an image obtained by expanding and fixing the aspect ratio of the image of the tree displayed in the cutout range A2 of the overhead image 100 shown in FIG.
  • the video 213 is a video in which the size and shape of the video in the cutout range A3 of the overhead video 100 are changed in accordance with the display panel 31 so that the boundary between the video 211 and the video 212 is smoothly continuous.
  • the video 213 may be a video obtained by compressing the video in the cutout range A3 of the overhead video 100 in the vertical direction and expanding in the horizontal direction.
  • the video 214 is a video in which the size and shape of the video in the cutout range A4 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 211 and the video 212 is smoothly continuous.
  • the video 214 may be a video obtained by compressing the video in the cutout range A4 of the overhead video 100 and vertically expanding the video.
  • the bird's-eye view image 200 includes a front image 201, a rear image 202, a left side image 203, and a right side image 204.
  • the front image 201 is a part of the image 211.
  • the rear video 202 is a part of the video 212.
  • the left side image 203 includes a lower left portion of the image 211, an upper left portion of the image 212, and an image 213.
  • the right side video 204 includes a lower right portion of the video 211, an upper right portion of the video 212, and a video 214.
  • the video synthesizing unit 47A is located at the center of the overhead view video 100 and the overhead view video 200 generated by the overhead view video generation unit 46A, and corresponds to the direction in which the obstacle information acquisition unit 43 detects the obstacle, the front camera 11 or the rear camera 12.
  • the bird's-eye view video 100 and the bird's-eye view image 200 are generated by synthesizing the peripheral images from the left side camera 13 or the right side camera 14.
  • the peripheral video from the front camera 11, the rear camera 12, the left side camera 13, or the right side camera 14 to be synthesized is a video before viewpoint conversion.
  • the video composition unit 47A outputs the generated overhead view video 100 and the overhead view video 200 to the display control unit 48.
  • the video composition unit 47A detects the obstacle in the central portion of the overhead image 100.
  • the overhead view image 100 is generated by combining the vertically long peripheral images from the left side camera 13 or the right side camera 14 corresponding to.
  • the peripheral video synthesized at the center of the overhead view video 100 is a peripheral video that has not been subjected to viewpoint conversion processing.
  • the video compositing unit 47A may display the boundary line 110 of the overhead view video 100 by thickening or changing the color of the direction in which the obstacle is detected.
  • the video synthesis unit 47A synthesizes the vertically long peripheral video 103A from the left side camera 13 in the center of the overhead view video 100 as shown in FIG. To do.
  • the rear image 102 and the surrounding image 103A of the overhead image 100 include an obstacle image 300 of a rear obstacle.
  • a boundary line 110A on the left side of the overhead view image 100 is a thick line.
  • the video composition unit 47A detects an obstacle at the center of the horizontally-viewed overhead image 200 when the direction in which the obstacle information acquisition unit 43 detects the obstacle is the front or the rear of the vehicle.
  • An overhead view image 200 is generated by combining horizontally long peripheral images from the front camera 11 or the rear camera 12 corresponding to the determined direction.
  • the peripheral video synthesized in the central portion of the overhead video 200 is a peripheral video that has not been subjected to the viewpoint conversion process.
  • the video compositing unit 47A may display the boundary line 210 of the overhead view video 200 by thickening or changing the color of the direction in which the obstacle is detected.
  • the video composition unit 47A synthesizes the horizontally long peripheral image 202A from the rear camera 12 in the center of the overhead view image 200 as shown in FIG.
  • the rear image 202 and the surrounding image 202A of the overhead image 200 include an obstacle image 300 of a rear obstacle.
  • a lower boundary line 210A of the bird's-eye view image 200 is a thick line.
  • FIG. 21 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the seventh embodiment.
  • Control unit 41A determines whether or not to start bird's-eye view video display (step ST11). In the present embodiment, the control unit 41A determines whether to start the overhead view video display based on the presence / absence of the backward trigger.
  • the reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle is the rear of the vehicle. If there is no reverse trigger, control unit 41A determines not to start the overhead view video display (No in step ST11), and executes the process of step ST11 again. If there is a reverse trigger, control unit 41A determines to start the overhead view video display (Yes in step ST11), and proceeds to step ST12.
  • the control unit 41A generates and displays the overhead view video 100 (step ST12). More specifically, the control unit 41A causes the overhead video generation unit 46A to generate the overhead video 100 that has been subjected to the viewpoint conversion processing so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. Then, the control unit 41A causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • Control unit 41A determines whether an obstacle has been detected (step ST13). More specifically, the control unit 41A determines whether or not obstacle information has been acquired by the obstacle information acquisition unit 43. If the control unit 41A determines that the obstacle information acquisition unit 43 has acquired the obstacle information (Yes in step ST13), the control unit 41A proceeds to step ST14. If the controller 41A determines that the obstacle information acquisition unit 43 has not acquired the obstacle information (No in step ST13), the control unit 41A proceeds to step ST17.
  • the control unit 41A determines whether an obstacle located in the traveling direction of the vehicle is detected based on the obstacle information acquired by the obstacle information acquisition unit 43.
  • Positioning in the traveling direction of the vehicle means positioning in a direction in which the distance to the vehicle decreases as the vehicle travels, and includes a range in which an obstacle interferes with the vehicle as the vehicle travels.
  • the control unit 41A determines whether or not an obstacle within a range centered on the traveling direction of the vehicle is detected, and the vehicle steers.
  • it is determined whether or not obstacles are detected in a range including the range in which the obstacle interferes with the vehicle by the traveling direction and steering of the vehicle.
  • the control unit 41A determines whether the obstacle detection direction is the left direction or the right direction (step ST14). In the present embodiment, the control unit 41A determines whether the detection direction of the obstacle with the shortest distance to the vehicle is the left direction or the right direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43. Determine whether. More specifically, the control unit 41A determines whether the obstacle information of the obstacle with the minimum distance to the vehicle acquired by the obstacle information acquisition unit 43 is the obstacle information from the left side sensor 23 or the right side sensor 24. Determine whether or not.
  • step ST14 When it is determined that the obstacle information of the obstacle with the shortest distance to the vehicle is the obstacle information from the left side sensor 23 or the right side sensor 24 (Yes in step ST14), the control unit 41A proceeds to step ST15. move on. If the controller 41A determines that the obstacle information of the obstacle with the shortest distance to the vehicle is not the obstacle information from the left side sensor 23 or the right side sensor 24 (No in step ST14), the control unit 41A goes to step ST16. move on.
  • the control unit 41A generates the bird's-eye view image 100 in which the peripheral image in the obstacle detection direction is displayed at the center of the bird's-eye view image 100 (step ST15). More specifically, the control unit 41 corresponds to the direction in which the video composition unit 47A detects the obstacle with the shortest distance to the vehicle at the center of the overhead image 100 generated by the overhead image generation unit 46A.
  • An overhead view image 100 is generated by synthesizing a vertically long peripheral image from the camera 13 or the right side camera 14. Then, the control unit 41A causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
  • the control unit 41A generates a bird's-eye view image 200 in which a peripheral image in the obstacle detection direction is displayed at the center of the horizontally long bird's-eye view image 200 (step ST16). More specifically, the control unit 41A causes the overhead view video generation unit 46A to generate the overhead view video 200 having a horizontally long shape without changing the display direction of the overhead view video 100 generated in step ST12. Then, the control unit 41A receives from the front camera 11 or the rear camera 12 corresponding to the direction in which the obstacle with the shortest distance to the vehicle is detected at the center of the horizontally long overhead image 200 in the video composition unit 47A. A bird's-eye view image 200 obtained by synthesizing a horizontally long peripheral image is generated. Then, the control unit 41 ⁇ / b> A causes the display control unit 48 to cause the display panel 31 to display the generated overhead image 200 having a horizontally long shape.
  • Control unit 41A determines whether or not to continue the overhead video display (step ST17). More specifically, the control unit 41A determines whether or not to continue the overhead view video display based on the presence / absence of the backward trigger. If there is a reverse trigger, control unit 41A determines to continue the overhead view video display (Yes in step ST17), and returns to step ST13 to continue the process. When there is no reverse trigger, in other words, when the reverse trigger is released, control unit 41A determines that the overhead view video display is not continued (No in step ST17), and ends the process.
  • the overhead view video generation system 1A has a left-side camera 13 corresponding to the direction in which the obstacle is detected in the center of the overhead view video 100 or A bird's-eye view image 100 is generated by synthesizing a vertically long peripheral image from the right-side camera 14.
  • the overhead view video generation system 1A When an obstacle is detected in the front or rear direction of the vehicle, the overhead view video generation system 1A has a front camera 11 or a rear camera corresponding to the direction in which the obstacle is detected at the center of the horizontally overhead view 200. 12 generates a bird's-eye view image 200 obtained by synthesizing a horizontally long peripheral image from 12.
  • the bird's-eye view video 100 and the bird's-eye view image 200 are generated by synthesizing the surrounding images corresponding to the direction in which the obstacle is detected in the center of the bird's-eye view image 100 and the bird's-eye view image 200.
  • this embodiment can also confirm the surrounding image of the direction which detected the obstacle with the bird's-eye view image 100 and the bird's-eye view image 200, without moving a big eyes
  • the present embodiment can suppress the movement of the driver's line of sight and appropriately check obstacles around the vehicle.
  • a vertically long image from the left side camera 13 or the right side camera 14 corresponding to the direction in which the obstacle is detected is displayed at the center of the overhead view image 100.
  • a bird's-eye view image 100 is generated by synthesizing the surrounding image of the shape.
  • the front camera 11 or the rear camera 12 corresponding to the direction in which the obstacle is detected is displayed at the center of the horizontally-viewed overhead view image 200.
  • a bird's-eye view video 200 is generated by synthesizing the surrounding video.
  • this embodiment can maintain the state in which the vertical direction of the bird's-eye view image 100 and the vertical direction of the bird's-eye view image 200 are combined with the longitudinal direction of the vehicle, regardless of the detection direction of the obstacle. For this reason, according to this embodiment, when the driver confirms the bird's-eye view image 100 or the bird's-eye view image 200, the periphery of the vehicle can be easily confirmed.
  • the overhead image 200 is obtained by synthesizing a peripheral image corresponding to the direction in which the obstacle is detected in the center of the horizontally overhead bird's eye image 200. Is generated and displayed on the display panel 31.
  • the bird's-eye view image 200 having a horizontally long shape can be displayed on the entire display surface of the display panel 31. Thereby, this embodiment can display a surrounding image largely in the center part of the bird's-eye view image 200, when an obstacle is detected in the front direction or the rear direction of the vehicle.
  • the video 211 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A1 of the overhead view video 100.
  • the front image 201 of the overhead image 200 has the same aspect ratio as the image of the cutout range A1 of the overhead image 100.
  • the video 212 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A2 of the overhead view video 100.
  • the rear video 202 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A2 of the overhead view video 100.
  • the front video 201 and the rear video 202 of the overhead view video 200 have the same aspect ratio and are not deformed. For this reason, according to the present embodiment, the driver can check the front video 201 and the rear video 202 of the bird's-eye view video 200 without a sense of incongruity.
  • FIG. 22 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eighth embodiment.
  • FIG. 23 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the overhead image generation system 1A has the same basic configuration as the overhead image generation system 1A of the seventh embodiment.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the seventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46A Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead view video generation unit 46A, when the direction in which the obstacle is detected is the front or the rear of the vehicle, the peripheral video from the front camera 11 or the rear camera 12 Is vertically compressed to generate a bird's-eye view image 200 having a horizontally long shape. More specifically, the overhead view video generation unit 46A compresses the front video 101 and the rear video 102 in the vertical direction from the vertically elongated overhead view video 100 that has been generated, and generates the overhead view video 200 having a horizontally long shape.
  • the cut-out range A1 includes a boundary between the front video 101 and the left side video 103 and a boundary between the front video 101 and the right side video 104.
  • the cut-out range A1 includes from the left end of the vehicle to the left side of about several tens of centimeters.
  • the cut-out range A1 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the cutout range A2 has the same vertical width as the vertical width of the rear video 102 of the overhead view video 100.
  • the cutout range A ⁇ b> 2 includes a boundary between the rear image 102 and the left side image 103 and a boundary between the rear image 102 and the right side image 104.
  • the cut-out range A2 includes from the left end of the vehicle to the left side of about several tens of centimeters.
  • the cutout range A2 includes from the right end of the vehicle to the right side of about several tens of centimeters.
  • the cutout area A2 has the same shape and the same area as the cutout area A1.
  • the cutout range A3 and the cutout range A4 are the same as in the seventh embodiment.
  • the overhead view video generation unit 46A compresses the cutout range A1 and the cutout range A2 of the overhead view video 100 in the vertical direction, and generates the overhead view video 200 having a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31. .
  • the video 211 is a video obtained by compressing the video of the cutout range A1 of the overhead video 100 in the vertical direction.
  • the image of the tree displayed in the image 211 is an image obtained by compressing the image of the tree displayed in the cutout range A1 of the overhead image 100 illustrated in FIG. 22 in the vertical direction.
  • the tree image displayed in the image 211 includes the entire tree image displayed in the front image 101 of the overhead image 100.
  • the video 212 is a video obtained by compressing the video of the cutout range A2 of the overhead video 100 in the vertical direction.
  • the image of the tree displayed in the image 212 is an image obtained by compressing the image of the tree displayed in the cutout range A2 of the overhead image 100 shown in FIG. 22 in the vertical direction.
  • the tree image displayed in the image 212 includes the entire tree image displayed in the rear image 102 of the overhead image 100.
  • the video 213 and the video 214 are the same as in the seventh embodiment.
  • the video 211 of the overhead video 200 is a video obtained by compressing the video of the cutout range A1 of the overhead video 100 in the vertical direction.
  • the video 211 of the overhead video 200 includes the entire vertical direction of the front video 101.
  • the video 212 of the bird's-eye view video 200 is a video obtained by compressing the video of the cutout range A2 of the bird's-eye view video 100 in the vertical direction.
  • the video 212 of the overhead video 200 includes the entire vertical direction of the rear video 102.
  • the driver can confirm the same range in the overhead image 200 as the vertical direction of the front image 101 of the overhead image 100 and the vertical direction of the rear image 102.
  • FIG. 24 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the ninth embodiment.
  • FIG. 25 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the seventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46 ⁇ / b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle.
  • a vertically long peripheral image is compressed in the vertical direction to generate a bird's-eye view image 200 having a horizontally long shape.
  • the overhead view video generation unit 46A generates the overhead view video 200 by compressing the cutout range A3 and the cutout range A4 of the generated overhead view video 100 in the vertical direction.
  • the cutout range A3 includes a boundary between the left side video 103 and the front video 101 and a boundary between the left side video 103 and the rear video 102.
  • the cut-out range A3 includes from the front end of the vehicle to about a few meters ahead.
  • the cut-out range A3 includes from the rear end of the vehicle to the rear about several meters.
  • the cutout range A4 has the same lateral width as the lateral width of the right-side video 104 of the overhead view video 100.
  • the cutout range A4 includes a boundary between the right side video 104 and the front video 101 and a boundary between the right side video 104 and the rear video 102.
  • the cut-out range A4 includes a distance of about several meters from the front end of the vehicle.
  • the cutout range A4 includes from the rear end of the vehicle to the rear about several meters.
  • the cutout area A4 has the same shape and the same area as the cutout area A3.
  • the cutout range A1 is a rectangular range located between the cutout range A3 and the cutout range A4.
  • the cut-out range A1 includes from the front end of the vehicle to about a few meters ahead.
  • the front end portion of the cutout range A1 is located on the same straight line as the front end portion of the cutout range A3 and the front end portion of the cutout range A4.
  • the cutout range A2 is a rectangular range located between the cutout range A3 and the cutout range A4.
  • the cut-out range A2 includes from the front end of the vehicle to about a few meters ahead.
  • the front end portion of the cutout range A2 is located on the same straight line as the front end portion of the cutout range A3 and the front end portion of the cutout range A4.
  • the overhead view video generation unit 46A compresses the cut-out range A1 and the cut-out range A2 of the generated overhead view video 100 in the vertical direction so as to have a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31. Is generated.
  • the video 213 is a video obtained by compressing the video in the cutout range A3 of the overhead view video 100 in the vertical direction.
  • the image of the tree displayed in the image 213 is an image obtained by compressing the image of the tree displayed in the cutout range A3 of the overhead image 100 in the vertical direction.
  • the tree image displayed in the image 213 includes the entire tree image displayed in the left-side image 103 of the overhead image 100.
  • the video 214 is a video obtained by compressing the video of the cutout range A4 of the overhead view video 100 in the vertical direction.
  • the image of the tree displayed in the image 214 is an image obtained by compressing the image of the tree displayed in the cutout range A4 of the overhead image 100 in the vertical direction.
  • the tree image displayed in the image 214 includes the entire tree image displayed in the right-side image 104 of the overhead image 100.
  • the video 211 is a video in which the size and shape of the video in the cutout range A1 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous.
  • the video 211 may be a video obtained by enlarging the video in the cutout range A1 of the overhead view video 100 in the horizontal direction.
  • the video 212 is a video in which the size and shape of the video in the cutout range A2 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous.
  • the video 212 may be a video obtained by enlarging the video in the cutout range A2 of the overhead view video 100 in the horizontal direction.
  • the video 213 of the overhead video 200 is a video obtained by compressing the video of the cutout range A3 of the overhead video 100 in the vertical direction.
  • the video 213 of the overhead video 200 includes the entire horizontal direction of the left side video 103.
  • the video 214 of the bird's-eye view video 200 is a video obtained by compressing the video of the cutout range A4 of the bird's-eye view video 100 in the vertical direction.
  • the image 214 of the bird's-eye view image 200 includes the entire horizontal direction of the right-side image 104.
  • the driver can confirm the same range in the overhead view video 200 as the horizontal direction of the left side video 103 and the horizontal direction of the right side video 104 of the overhead view video 100.
  • FIG. 26 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the tenth embodiment.
  • FIG. 27 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the ninth embodiment in the method of generating the overhead view video 200 having a horizontally long shape by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46 ⁇ / b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle.
  • the top and bottom portions are synthesized except for the center portion of the vertically long peripheral image, and the overhead view image 200 having a horizontally long shape is generated.
  • the 26 includes an upper part A31, a central part A32, and a lower part A33.
  • the upper part A31 includes a boundary between the left side image 103 and the front image 101.
  • the lower part A33 includes a boundary between the left side video 103 and the rear video 102.
  • the cutout range A4 includes an upper part A41, a central part A42, and a lower part A43.
  • the upper part A41 includes a boundary between the right side image 104 and the front image 101.
  • the lower part A43 includes a boundary between the right side image 104 and the rear image 102.
  • the cutout area A4 has the same shape and the same area as the cutout area A3.
  • the cutout range A1 and the cutout range A2 are the same as in the ninth embodiment.
  • the overhead image generation unit 46A combines the upper portion A31 and the lower portion A33 of the cutout range A3 of the generated overhead view image 100, and combines the upper portion A41 and the lower portion A43 of the cutout range A4 to display the display surface of the display panel 31.
  • a bird's-eye view image 200 having a horizontally long shape matching the aspect ratio is generated.
  • the video 213 is a video obtained by combining the upper part A31 and the lower part A33 excluding the central part A32 of the cutout range A3 of the overhead view video 100.
  • the video 213 is a video that does not include the video of the tree displayed in the central portion A32 of the cutout range A3 of the overhead view video 100 shown in FIG.
  • a line may be displayed at the boundary between the upper part A31 and the lower part A33 in order to show that the central part A32 of the cutout range A3 of the overhead view video 100 is omitted.
  • the video 214 is a video obtained by synthesizing the upper part A41 and the lower part A43 excluding the central part A42 of the cutout range A4 of the overhead view video 100.
  • the image 214 is an image that does not include the image of the tree displayed in the central portion A42 of the cutout range A4 of the overhead image 100 shown in FIG.
  • a line may be displayed at the boundary between the upper part A41 and the lower part A43 in order to show that the central part A42 of the cutout range A4 of the overhead view video 100 is omitted.
  • the video 211 and the video 212 are the same as in the ninth embodiment.
  • the video 213 of the bird's-eye view video 200 is composed of the upper part A31 and the lower part A33 excluding the central part A32 of the cutout range A3 of the bird's-eye view picture 100.
  • the video 213 of the bird's-eye view video 200 can largely display the front and rear of the left side video 103 of the bird's-eye view video 100.
  • the image 214 of the bird's-eye view image 200 combines the upper part A41 and the lower part A43 excluding the central part A42 of the cutout range A4 of the bird's-eye view image 100.
  • the video 214 of the bird's-eye view video 200 can largely display the front and rear of the right-side video 104 of the bird's-eye view video 100.
  • FIG. 28 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eleventh embodiment.
  • FIG. 29 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the ninth embodiment in the method of generating the overhead view video 200 having a horizontally long shape by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46 ⁇ / b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle.
  • the vertically long peripheral image is biased in the direction in which the obstacle is detected, and the overhead image 200 having a horizontally long shape is generated.
  • the direction in which the obstacle is detected will be described as the rear.
  • the cutout range A3 shown in FIG. 28 is a range that is biased backward in the obstacle detection direction in the left side image 103 of the overhead view image 100.
  • the cut-out range A3 includes the lower side from the center in the vertical direction of the overhead view image 100.
  • the cut-out range A4 is a range that is biased toward the rear in the obstacle detection direction in the right-side video 104 of the overhead view video 100.
  • the cut-out range A4 includes the lower side from the central portion in the vertical direction.
  • the cutout area A4 has the same shape and the same area as the cutout area A3.
  • the cutout range A5 is a rectangular range located between the cutout range A3 and the cutout range A4 of the overhead view image 100.
  • the upper end portion of the cutout range A5 is located on the same straight line as the upper end portion of the cutout range A1 and the upper end portion of the cutout range A2.
  • the lower end portion of the cutout range A5 is located on the same straight line as the lower end portion of the cutout range A1 and the lower end portion of the cutout range A2.
  • the overhead view video generation unit 46A generates an overhead view video 200 having a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31 from the cutout range A3, the cutout range A4, and the cutout range A5 of the generated overhead view video 100. Generate.
  • the video 213 is a video of the cutout range A3 of the overhead video 100.
  • the image of the tree displayed in the image 213 includes the entire image of the tree displayed in the cutout area A3 of the overhead image 100 shown in FIG.
  • the image 214 is an image of the cutout range A4 of the overhead image 100.
  • the tree image displayed in the image 214 includes the entire tree image displayed in the cutout range A4 of the overhead view image 100 shown in FIG.
  • the video 215 is a video in which the size and shape of the video in the cutout range A5 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous.
  • the video 215 may be a video obtained by enlarging the video in the cutout range A5 of the overhead view video 100 in the horizontal direction.
  • the left-side image 103 and the right-side image 104 of the overhead image 100 are biased in the obstacle detection direction to have a horizontally long shape.
  • An overhead image 200 is generated.
  • the bird's-eye view image 200 can display a larger direction in which the obstacle is detected in the bird's-eye view image 100.
  • the overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the eleventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
  • the overhead view video generation unit 46 ⁇ / b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle.
  • An overhead image 200 having a horizontally long shape is generated by deviating a vertically long peripheral image in the traveling direction of the vehicle.
  • the left side image 103 and the right side image 104 of the overhead view image 100 are biased in the traveling direction of the vehicle so as to have a horizontally long shape.
  • a video 200 is generated.
  • the bird's-eye view image 200 can display the traveling direction of the vehicle larger in the bird's-eye view image 100.
  • the constituent elements of the overhead view video generation system 1 shown in the figure are functionally conceptual, and need not be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or a part of them is functionally or physically distributed or integrated in arbitrary units according to the processing load or usage status of each device. May be.
  • the configuration of the overhead view video generation system 1 is realized by, for example, a program loaded in a memory as software.
  • the above embodiment has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.
  • the superimposed video generation unit 47 may display the distance to the obstacle numerically together with the obstacle notification icon 120 on the central video 105 of the overhead video 100. Good.
  • the superimposed image generation unit 47 displays the shape or the shape according to the distance to the obstacle together with the obstacle notification icon 120 on the central image 105 of the overhead view image 100. A figure whose color changes may be displayed.
  • the overhead view video generation system 1 can notify the distance to the obstacle in addition to the change of the obstacle notification icon 120 according to the distance to the obstacle. Thereby, the bird's-eye view video generation system 1 can make it possible to more appropriately check obstacles around the vehicle.
  • the control unit 41 may display the obstacle notification icon 120 for the obstacles with high priority.
  • the priority order may be the highest priority order with the smallest distance to the obstacle.
  • the priority may be the highest priority when the obstacle is a moving object.
  • the priority may be the highest priority when the obstacle is moving and is approaching the vehicle.
  • an icon indicating that the obstacle is a moving object may be displayed in the central image 105 of the overhead view image 100.
  • the icon indicating a moving object may be, for example, an icon indicating a pedestrian or an icon indicating a vehicle.
  • the obstacle notification icon 120 has been described as being composed of a triple arc-shaped curve, it is not limited to this.
  • the obstacle notification icon 120 may be, for example, an arrow-shaped figure indicating the direction of the obstacle.
  • the obstacle notification icon 120 may be an arrow-shaped figure whose thickness and size are changed according to the distance to the obstacle.
  • the sensor serving as the obstacle detection unit includes six sensors including a front left sensor 21A, a front center sensor 21B, a front right sensor 21C, a rear left sensor 22A, a rear center sensor 22B, and a rear right sensor 26.
  • the number of sensors is not limited. If it is desired to detect the direction of an obstacle more finely, the number of sensors may be increased.
  • the obstacle notification icon 120 corresponds to the number of sensors, the front left icon 121, the front center icon 122, the front right icon 123, the rear left icon 124, the rear center icon 125, and the rear right icon 126.
  • the present invention is not limited to this. In other words, the obstacle notification icon 120 only needs to associate the detection result of each sensor with the icon to be displayed.
  • the bird's-eye view image generation unit 46 may generate a bird's-eye view image 200 having a horizontally long shape by compressing the central portion of the peripheral image from the front camera 11 or the rear camera 12 in the vertical direction. This will be described in more detail with reference to FIGS.
  • FIG. 30 is a diagram illustrating another example of a method for generating a horizontally long overhead image in the overhead image generation system.
  • FIG. 31 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 30.
  • the overhead image generation unit 46 compresses the cutout range A6 at the center of the left side video 103 of the overhead view video 100 and the cutout range A7 at the center of the right side video 104 in the vertical direction, and displays the display panel 31 on the display panel 31.
  • a bird's-eye view image 200 having a horizontally long shape matching the aspect ratio of the surface is generated.
  • the control unit 41A synthesizes the peripheral video corresponding to the direction in which the obstacle with the shortest distance to the vehicle is detected at the center of the overhead video 100 and the overhead video 200.
  • the peripheral video may be selectable by the user.
  • peripheral images of the overhead view video 100 and the overhead view video 200 at the position touched by the user may be synthesized on the display surface of the display panel 31.
  • the control unit 41A may reduce the color information of the left-side image 203 and the right-side image 204 of the overhead image 200, for example, when the obstacle detection direction is forward or backward. Alternatively, when the obstacle detection direction is forward or backward, the control unit 41A may display the left side video 203 and the right side video 204 of the overhead view video 200, for example, in a single color display. Alternatively, the control unit 41A may reduce the color information of the left side video 203 and the right side video 204 of the overhead view video 200, for example, when the obstacle detection direction is front or rear. Alternatively, the control unit 41A may reduce the luminance of the left side video 203 and the right side video 204 of the overhead view video 200, for example, when the obstacle detection direction is front or rear. In this way, the driver can be made aware that the amount of information in the left side video 203 and the right side video 204 of the overhead view video 200 is reduced.
  • the bird's-eye view video generation unit 46A receives the signals from the left side camera 13 and the right side camera 14 when the direction in which the obstacle is detected is the front or the rear of the vehicle.
  • the peripheral video may be hidden, and the overhead video 200 having a horizontally long shape may be generated from the peripheral video from the front camera 11 and the rear camera 12. In this case, the driver can visually confirm the left and right sides of the vehicle.
  • the control unit 41A may determine whether or not to start the overhead view video display based on, for example, whether or not an operation for starting the overhead view video display on the operation unit is detected.
  • the seventh embodiment four sensors, that is, the front sensor 21, the rear sensor 22, the left side sensor 23, and the right side sensor 24 are provided as the sensors that are obstacle detection units, but the number of sensors is not limited. . If it is desired to detect the direction of an obstacle more finely, the number of sensors may be increased.
  • Overhead video generation system 11 Front camera (camera) 12 Rear camera (camera) 13 Left-side camera (camera) 14 Right-side camera (camera) 21A Front left sensor (obstacle detection unit) 21B Front center sensor (obstacle detection unit) 21C Front right sensor (obstacle detection unit) 22A Rear left sensor (obstacle detection unit) 22B Rear center sensor (obstacle detection unit) 22C Rear right sensor (obstacle detection unit) Reference Signs List 31 Display panel 40 Overhead video generation device 41 Control unit 42 Video acquisition unit 43 Obstacle information acquisition unit 44 Vehicle information acquisition unit 46 Overhead video generation unit (video generation unit) 47 Superimposed video generator (video generator) 48 Display control unit 49 Storage unit 100 Overhead video 120 Obstacle notification icon (information indicating an obstacle) 121 Front left icon (information indicating an obstacle) 122 Front center icon (information indicating an obstacle) 123 Front right icon (information indicating an obstacle) 124 Rear left icon (information indicating an obstacle) 125 Rear center icon (information indicating an obstacle) 126 Re

Abstract

The present invention is characterized by having: a video image acquisition unit 42 for acquiring a peripheral video image in which the periphery of a vehicle is taken on video; an obstacle information acquisition unit 43 for acquiring obstacle information pertaining to an obstacle detected in the periphery of the vehicle; an overhead view video image generation unit 46 for generating an overhead view video image from the peripheral video image acquired by the video image acquisition unit 42, in which a viewpoint conversion process was performed so as to overlook the vehicle from above; a superimposed video image generation unit 47 for generating, on the basis of obstacle information acquired by the obstacle information acquisition unit 43, an overhead view video image in which the information indicating the obstacle is superimposed in a middle section enclosed by the overhead view video image; and a display control unit 48 for causing the overhead view video image generated by the superimposed video image generation unit 47 to be displayed on a display panel 31.

Description

俯瞰映像生成装置、俯瞰映像生成システム、俯瞰映像生成方法およびプログラムOverhead video generation device, overhead video generation system, overhead video generation method, and program
　本発明は、俯瞰映像生成装置、俯瞰映像生成システム、俯瞰映像生成方法およびプログラムに関する。 The present invention relates to an overhead video generation device, an overhead video generation system, an overhead video generation method, and a program.
　車両の俯瞰映像を車両画像と共に表示する車両周辺表示装置に関する技術が知られている（例えば、特許文献１参照）。 A technique related to a vehicle periphery display device that displays a vehicle overhead view image together with a vehicle image is known (see, for example, Patent Document 1).
　また、車両の俯瞰映像を表示する、ナビゲーション装置などの車載装置の表示パネルは、横長形状であることが多い。このため、縦長形状の俯瞰映像は、表示パネルの全面に表示されない。そこで、自車両に近接する障害物を検出した場合、車両の俯瞰映像と、障害物の方向を撮像した映像とを並べて表示する技術が知られている（例えば、特許文献２参照）。 Also, the display panel of an in-vehicle device such as a navigation device that displays a bird's-eye view image of the vehicle is often horizontally long. For this reason, a vertically long overhead image is not displayed on the entire surface of the display panel. Therefore, a technique is known in which, when an obstacle close to the host vehicle is detected, a bird's-eye view image of the vehicle and a video image of the direction of the obstacle are displayed side by side (for example, see Patent Document 2).
特開２０１５－０７６６４５号公報Japanese Patent Laying-Open No. 2015-0776645 特開２０１１－２５１６８１号公報JP 2011-251681 A
　車両の周辺をより確認しやすくするため、車両の周辺において検出した障害物の障害物情報を俯瞰映像に重畳させて表示する技術がある。ところが、障害物情報を俯瞰映像に重畳させると、俯瞰映像に映った障害物の視認性が阻害されるおそれがある。 In order to make it easier to check the surroundings of the vehicle, there is a technology for displaying obstacle information of obstacles detected in the vicinity of the vehicle superimposed on the overhead image. However, when the obstacle information is superimposed on the overhead view video, the visibility of the obstacle reflected in the overhead view video may be hindered.
　また、特許文献２に記載の技術では、障害物を検出した場合、運転者は、俯瞰映像と障害物の検出方向の映像とを視線を移動し確認することになる。 In the technique described in Patent Document 2, when an obstacle is detected, the driver moves the line of sight and confirms the overhead view image and the image in the obstacle detection direction.
　本発明は、上記に鑑みてなされたものであって、車両周辺の障害物を適切に確認可能にすることを目的とする。 The present invention has been made in view of the above, and an object thereof is to enable appropriate confirmation of obstacles around a vehicle.
　上述した課題を解決し、目的を達成するために、本発明に係る俯瞰映像生成装置は、車両の周辺を撮影した周辺映像を取得する映像取得部と、前記車両の周辺において検出した障害物の障害物情報を取得する障害物情報取得部と、前記映像取得部が取得した周辺映像から前記車両を上方から見下ろすように視点変換処理を行った俯瞰映像を生成し、前記俯瞰映像の中央部に、前記障害物情報取得部が取得した障害物を示す情報を合成した俯瞰映像を生成する映像生成部と、前記映像生成部が生成した俯瞰映像を表示部に表示させる表示制御部とを有することを特徴とする。 In order to solve the above-described problems and achieve the object, an overhead image generation apparatus according to the present invention includes a video acquisition unit that acquires a peripheral video obtained by photographing the periphery of a vehicle, and obstacles detected in the vicinity of the vehicle. An obstacle information acquisition unit that acquires obstacle information, and an overhead view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit, A video generation unit that generates a bird's-eye view image obtained by synthesizing information indicating the obstacle acquired by the obstacle information acquisition unit, and a display control unit that causes the display unit to display the bird's-eye view image generated by the video generation unit. It is characterized by.
　本発明に係る俯瞰映像生成システムは、上記の俯瞰映像生成装置と、前記車両の周辺を撮影し前記映像取得部に周辺映像を供給するカメラと、前記車両の周辺において障害物を検出し前記障害物情報取得部に障害物情報を供給する障害物検出部と、前記表示部との少なくともいずれかとを備える。 An overhead image generation system according to the present invention includes the above-described overhead image generation device, a camera that captures the periphery of the vehicle and supplies a periphery image to the image acquisition unit, detects an obstacle around the vehicle, and detects the obstacle An obstacle detection unit that supplies obstacle information to the object information acquisition unit, and at least one of the display unit.
　本発明に係る俯瞰映像生成方法は、車両の周辺を撮影した周辺映像を取得する映像取得ステップと、前記車両の周辺において検出した障害物の障害物情報を取得する障害物情報取得ステップと、前記映像取得ステップで取得した周辺映像から前記車両を上方から見下ろすように視点変換処理を行った俯瞰映像を生成し、前記俯瞰映像の中央部に、前記障害物情報取得ステップで取得した障害物を示す情報を合成した俯瞰映像を生成する映像生成ステップと、前記映像生成ステップで生成した俯瞰映像を表示部に表示させる表示制御ステップとを含む。 The bird's-eye view video generation method according to the present invention includes a video acquisition step of acquiring a peripheral video obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring obstacle information of an obstacle detected in the periphery of the vehicle, A bird's-eye view image is generated from the surrounding image acquired in the image acquisition step so that the vehicle is viewed from above, and the obstacle acquired in the obstacle information acquisition step is shown in the center of the bird's-eye view image. A video generation step of generating a bird's-eye view video obtained by combining information; and a display control step of displaying the overhead view video generated in the video generation step on a display unit.
　本発明に係るプログラムは、車両の周辺を撮影した周辺映像を取得する映像取得ステップと、前記車両の周辺において検出した障害物の障害物情報を取得する障害物情報取得ステップと、前記映像取得ステップで取得した周辺映像から前記車両を上方から見下ろすように視点変換処理を行った俯瞰映像を生成し、前記俯瞰映像の中央部に、前記障害物情報取得ステップで取得した障害物を示す情報を合成した俯瞰映像を生成する映像生成ステップと、前記映像生成ステップで生成した俯瞰映像を表示部に表示させる表示制御ステップとを俯瞰映像生成装置として動作するコンピュータに実行させる。 The program according to the present invention includes a video acquisition step of acquiring a peripheral video obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring obstacle information of an obstacle detected in the vicinity of the vehicle, and the video acquisition step. Generates an overhead view image obtained by performing viewpoint conversion processing so that the vehicle is looked down from above, and synthesizes information indicating the obstacle acquired in the obstacle information acquisition step in the center of the overhead view image The computer operating as the overhead video generation device executes a video generation step for generating the overhead video and a display control step for displaying the overhead video generated in the video generation step on the display unit.
　本発明によれば、車両周辺の障害物を適切に確認可能にすることができるという効果を奏する。 According to the present invention, there is an effect that obstacles around the vehicle can be appropriately confirmed.
図１は、第一実施形態に係る俯瞰映像生成システムの構成例を示すブロック図である。FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment. 図２は、第一実施形態に係る俯瞰映像生成システムで生成した俯瞰映像および障害物報知アイコンを示す図である。FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment. 図３は、第一実施形態に係る俯瞰映像生成システムの俯瞰映像生成装置における処理の流れを示すフローチャートである。FIG. 3 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the first embodiment. 図４は、第一実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の一例を示す図である。FIG. 4 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment. 図５は、第一実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。FIG. 5 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the first embodiment. 図６は、第二実施形態に係る俯瞰映像生成システムの俯瞰映像生成装置における処理の流れを示すフローチャートである。FIG. 6 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the second embodiment. 図７は、第二実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の一例を示す図である。FIG. 7 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the second embodiment. 図８は、第三実施形態に係る俯瞰映像生成システムの俯瞰映像生成装置における処理の流れを示すフローチャートである。FIG. 8 is a flowchart showing the flow of processing in the overhead video generation device of the overhead video generation system according to the third embodiment. 図９は、第五実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の一例を示す図である。FIG. 9 is a diagram showing an example of an overhead video generated by the overhead video generation system according to the fifth embodiment. 図１０は、第五実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。FIG. 10 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment. 図１１は、第五実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。FIG. 11 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment. 図１２は、障害物報知アイコンにおける、円弧間の間隔と障害物までの距離との関係の一例を示すグラフである。FIG. 12 is a graph showing an example of the relationship between the distance between the arcs and the distance to the obstacle in the obstacle notification icon. 図１３は、第六実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の一例を示す図である。FIG. 13 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the sixth embodiment. 図１４は、第六実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。FIG. 14 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment. 図１５は、第六実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。FIG. 15 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment. 図１６は、第七実施形態に係る俯瞰映像生成システムの構成例を示すブロック図である。FIG. 16 is a block diagram illustrating a configuration example of the overhead view video generation system according to the seventh embodiment. 図１７は、第七実施形態に係る俯瞰映像生成システムで生成した俯瞰映像を表示した表示パネルを示す図である。FIG. 17 is a diagram illustrating a display panel that displays an overhead video generated by the overhead video generation system according to the seventh embodiment. 図１８は、第七実施形態に係る俯瞰映像生成システムで生成した、横長形状の俯瞰映像を表示した表示パネルを示す図である。FIG. 18 is a diagram showing a display panel that displays a horizontally long overhead image generated by the overhead image generation system according to the seventh embodiment. 図１９は、第七実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。FIG. 19 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the seventh embodiment. 図２０は、図１９の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。FIG. 20 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 図２１は、第七実施形態に係る俯瞰映像生成システムの俯瞰映像生成装置における処理の流れを示すフローチャートである。FIG. 21 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the seventh embodiment. 図２２は、第八実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。FIG. 22 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eighth embodiment. 図２３は、図２２の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。FIG. 23 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 図２４は、第九実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。FIG. 24 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the ninth embodiment. 図２５は、図２４の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。FIG. 25 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 図２６は、第十実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。FIG. 26 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the tenth embodiment. 図２７は、図２６の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。FIG. 27 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 図２８は、第十一実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。FIG. 28 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eleventh embodiment. 図２９は、図２８の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。FIG. 29 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 図３０は、俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法の他の例を説明する図である。FIG. 30 is a diagram illustrating another example of a method for generating a horizontally long overhead image in the overhead image generation system. 図３１は、図３０の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。FIG. 31 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 30.
　以下に添付図面を参照して、本発明に係る俯瞰映像生成装置４０、俯瞰映像生成システム１、俯瞰映像生成方法およびプログラムの実施形態を詳細に説明する。なお、以下の実施形態により本発明が限定されるものではない。 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an overhead view video generation device 40, an overhead view video generation system 1, an overhead view video generation method, and a program according to the present invention will be described in detail below with reference to the accompanying drawings. In addition, this invention is not limited by the following embodiment.
［第一実施形態］
　図１は、第一実施形態に係る俯瞰映像生成システムの構成例を示すブロック図である。俯瞰映像生成システム１は、車両の俯瞰映像１００（図２参照）を生成する。俯瞰映像生成装置４０および俯瞰映像生成システム１は、車両に搭載されている。俯瞰映像生成装置４０および俯瞰映像生成システム１は、車両に載置されているものに加えて、可搬型で車両において利用可能な装置であってもよい。 [First embodiment]
FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment. The bird's-eye view video generation system 1 generates a bird's-eye view video 100 (see FIG. 2). The overhead image generation device 40 and the overhead image generation system 1 are mounted on a vehicle. In addition to what is mounted on the vehicle, the overhead image generation device 40 and the overhead image generation system 1 may be a portable device that can be used in the vehicle.
　図１を用いて、俯瞰映像生成システム１について説明する。俯瞰映像生成システム１は、前方カメラ（カメラ）１１と、後方カメラ（カメラ）１２と、左側方カメラ（カメラ）１３と、右側方カメラ（カメラ）１４と、前方左センサ（障害物検出部）２１Ａと、前方中央センサ（障害物検出部）２１Ｂと、前方右センサ（障害物検出部）２１Ｃと、後方左センサ（障害物検出部）２２Ａと、後方中央センサ（障害物検出部）２２Ｂと、後方右センサ（障害物検出部）２２Ｃと、表示パネル３１と、俯瞰映像生成装置４０とを有する。 Referring to FIG. 1, the overhead video generation system 1 will be described. The overhead view video generation system 1 includes a front camera (camera) 11, a rear camera (camera) 12, a left side camera (camera) 13, a right side camera (camera) 14, and a front left sensor (obstacle detection unit). 21A, front center sensor (obstacle detection unit) 21B, front right sensor (obstacle detection unit) 21C, rear left sensor (obstacle detection unit) 22A, rear center sensor (obstacle detection unit) 22B , A rear right sensor (obstacle detection unit) 22C, a display panel 31, and an overhead image generation device 40.
　前方カメラ１１は、車両の前方に配置され、車両の前方を中心とした周辺を撮影する。前方カメラ１１は、撮影した映像を俯瞰映像生成装置４０の映像取得部４２へ出力する。 The front camera 11 is arranged in front of the vehicle and photographs the periphery around the front of the vehicle. The front camera 11 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.
　後方カメラ１２は、車両の後方に配置され、車両の後方を中心とした周辺を撮影する。後方カメラ１２は、撮影した映像を俯瞰映像生成装置４０の映像取得部４２へ出力する。 The rear camera 12 is arranged at the rear of the vehicle and photographs the periphery around the rear of the vehicle. The rear camera 12 outputs the captured video to the video acquisition unit 42 of the overhead video generation device 40.
　左側方カメラ１３は、車両の左側方に配置され、車両の左側方を中心とした周辺を撮影する。左側方カメラ１３は、撮影した映像を俯瞰映像生成装置４０の映像取得部４２へ出力する。 The left side camera 13 is arranged on the left side of the vehicle, and takes a picture around the left side of the vehicle. The left side camera 13 outputs the captured video to the video acquisition unit 42 of the overhead view video generation device 40.
　右側方カメラ１４は、車両の右側方に配置され、車両の右側方を中心とした周辺を撮影する。右側方カメラ１４は、撮影した映像を俯瞰映像生成装置４０の映像取得部４２へ出力する。 The right-side camera 14 is arranged on the right side of the vehicle and photographs the periphery around the right side of the vehicle. The right side camera 14 outputs the captured video to the video acquisition unit 42 of the overhead view video generation device 40.
　前方カメラ１１と後方カメラ１２と左側方カメラ１３と右側方カメラ１４とで、車両の全方位を撮影する。 The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 photograph all directions of the vehicle.
　前方左センサ２１Ａは、車両の前方左側に配置され、車両の前方左側における障害物を検出する。前方左センサ２１Ａは、例えば、赤外線センサまたは超音波センサ、ミリ波レーダなどであり、これらの組合せで構成されてもよい。前方左センサ２１Ａは、車両と接触するおそれがある、地上から高さを有するものを検出する。前方左センサ２１Ａは、例えば、車両から５ｍ程度までの距離の障害物を検出する。前方左センサ２１Ａは、鉛直方向視において、センサの中央部を中心とした、例えば、４０°程度の範囲の障害物を検出する。前方左センサ２１Ａの検出範囲は、前方中央センサ２１Ｂの検出範囲の一部と重複していてもよい。前方左センサ２１Ａは、検出した障害物の障害物情報を俯瞰映像生成装置４０の障害物情報取得部４３へ出力する。障害物情報の例としては、前方左センサ２１Ａの検出範囲における障害物の有無、障害物までの距離、水平方向における障害物の存在範囲などである。 The front left sensor 21A is disposed on the left front side of the vehicle and detects an obstacle on the left front side of the vehicle. The front left sensor 21A is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The front left sensor 21 </ b> A detects an object having a height from the ground that may come into contact with the vehicle. For example, the front left sensor 21A detects an obstacle at a distance of about 5 m from the vehicle. The front left sensor 21 </ b> A detects an obstacle in a range of, for example, about 40 ° with the center of the sensor as the center when viewed in the vertical direction. The detection range of the front left sensor 21A may overlap with a part of the detection range of the front center sensor 21B. The front left sensor 21 </ b> A outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front left sensor 21A, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
　前方中央センサ２１Ｂは、車両の前方中央に配置され、車両の前方中央における障害物を検出する。前方中央センサ２１Ｂは、例えば、赤外線センサまたは超音波センサ、ミリ波レーダなどであり、これらの組合せで構成されてもよい。前方中央センサ２１Ｂは、車両と接触するおそれがある、地上から高さを有するものを検出する。前方中央センサ２１Ｂは、例えば、車両から５ｍ程度までの距離の障害物を検出する。前方中央センサ２１Ｂは、鉛直方向視において、センサの中央部を中心とした、例えば、４０°程度の範囲の障害物を検出する。前方中央センサ２１Ｂの検出範囲は、前方左センサ２１Ａおよび前方右センサ２１Ｃの検出範囲の一部と重複していてもよい。前方中央センサ２１Ｂは、検出した障害物の障害物情報を俯瞰映像生成装置４０の障害物情報取得部４３へ出力する。障害物情報の例としては、前方中央センサ２１Ｂの検出範囲における障害物の有無、障害物までの距離、水平方向における障害物の存在範囲などである。 The front center sensor 21B is arranged at the front center of the vehicle and detects an obstacle at the front center of the vehicle. The front center sensor 21B is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The front center sensor 21B detects a sensor having a height from the ground that may come into contact with the vehicle. The front center sensor 21B detects an obstacle at a distance of about 5 m from the vehicle, for example. The front center sensor 21B detects an obstacle in a range of, for example, about 40 ° with the center portion of the sensor as the center when viewed in the vertical direction. The detection range of the front center sensor 21B may overlap with a part of the detection range of the front left sensor 21A and the front right sensor 21C. The front center sensor 21B outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front center sensor 21B, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
　前方右センサ２１Ｃは、車両の前方右側に配置され、車両の前方右側における障害物を検出する。前方右センサ２１Ｃは、例えば、赤外線センサまたは超音波センサ、ミリ波レーダなどであり、これらの組合せで構成されてもよい。前方右センサ２１Ｃは、車両と接触するおそれがある、地上から高さを有するものを検出する。前方右センサ２１Ｃは、例えば、車両から５ｍ程度までの距離の障害物を検出する。前方右センサ２１Ｃは、鉛直方向視において、センサの中央部を中心とした、例えば、４０°程度の範囲の障害物を検出する。前方右センサ２１Ｃの検出範囲は、前方中央センサ２１Ｂの検出範囲の一部と重複していてもよい。前方右センサ２１Ｃは、検出した障害物の障害物情報を俯瞰映像生成装置４０の障害物情報取得部４３へ出力する。障害物情報の例としては、前方右センサ２１Ｃの検出範囲における障害物の有無、障害物までの距離、水平方向における障害物の存在範囲などである。 The front right sensor 21C is arranged on the front right side of the vehicle and detects an obstacle on the front right side of the vehicle. The front right sensor 21C is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The front right sensor 21 </ b> C detects a thing having a height from the ground that may come into contact with the vehicle. The front right sensor 21C detects an obstacle at a distance of about 5 m from the vehicle, for example. The front right sensor 21C detects an obstacle in a range of, for example, about 40 ° with the center portion of the sensor as the center when viewed in the vertical direction. The detection range of the front right sensor 21C may overlap with a part of the detection range of the front center sensor 21B. The front right sensor 21 </ b> C outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the front right sensor 21C, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
　前方左センサ２１Ａと前方中央センサ２１Ｂと前方右センサ２１Ｃとで、車両の前方の障害物を検出する。 The front left sensor 21A, the front center sensor 21B, and the front right sensor 21C detect an obstacle ahead of the vehicle.
　後方左センサ２２Ａは、車両の後方左側に配置され、車両の後方左側における障害物を検出する。後方左センサ２２Ａは、例えば、赤外線センサまたは超音波センサ、ミリ波レーダなどであり、これらの組合せで構成されてもよい。後方左センサ２２Ａは、車両と接触するおそれがある、地上から高さを有するものを検出する。後方左センサ２２Ａは、例えば、車両から５ｍ程度までの距離の障害物を検出する。後方左センサ２２Ａは、鉛直方向視において、センサの中央部を中心とした、例えば、４０°程度の範囲の障害物を検出する。後方左センサ２２Ａの検出範囲は、後方中央センサ２２Ｂの検出範囲の一部と重複していてもよい。後方左センサ２２Ａは、検出した障害物の障害物情報を俯瞰映像生成装置４０の障害物情報取得部４３へ出力する。障害物情報の例としては、後方左センサ２２Ａの検出範囲における障害物の有無、障害物までの距離、水平方向における障害物の存在範囲などである。 The rear left sensor 22A is disposed on the rear left side of the vehicle and detects an obstacle on the rear left side of the vehicle. The rear left sensor 22A is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The rear left sensor 22A detects an object having a height from the ground that may come into contact with the vehicle. The rear left sensor 22A detects an obstacle at a distance of about 5 m from the vehicle, for example. The rear left sensor 22A detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the rear left sensor 22A may overlap with a part of the detection range of the rear center sensor 22B. The rear left sensor 22A outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear left sensor 22A, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
　後方中央センサ２２Ｂは、車両の後方中央に配置され、車両の後方中央における障害物を検出する。後方中央センサ２２Ｂは、例えば、赤外線センサまたは超音波センサ、ミリ波レーダなどであり、これらの組合せで構成されてもよい。後方中央センサ２２Ｂは、車両と接触するおそれがある、地上から高さを有するものを検出する。後方中央センサ２２Ｂは、例えば、車両から５ｍ程度までの距離の障害物を検出する。後方中央センサ２２Ｂは、鉛直方向視において、センサの中央部を中心とした、例えば、４０°程度の範囲の障害物を検出する。後方中央センサ２２Ｂの検出範囲は、後方左センサ２２Ａおよび後方右センサ２２Ｃの検出範囲の一部と重複していてもよい。後方中央センサ２２Ｂは、検出した障害物の障害物情報を俯瞰映像生成装置４０の障害物情報取得部４３へ出力する。障害物情報の例としては、後方中央センサ２２Ｂの検出範囲における障害物の有無、障害物までの距離、水平方向における障害物の存在範囲などである。 The rear center sensor 22B is disposed at the rear center of the vehicle and detects an obstacle at the rear center of the vehicle. The rear center sensor 22B is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. Back center sensor 22B detects what has a height from the ground which may be in contact with a vehicle. For example, the rear center sensor 22B detects an obstacle at a distance of about 5 m from the vehicle. The rear center sensor 22B detects an obstacle in a range of, for example, about 40 ° with the center of the sensor as the center when viewed in the vertical direction. The detection range of the rear center sensor 22B may overlap with a part of the detection range of the rear left sensor 22A and the rear right sensor 22C. The rear center sensor 22B outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the overhead image generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear center sensor 22B, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
　後方右センサ２２Ｃは、車両の後方右側に配置され、車両の後方右側における障害物を検出する。後方右センサ２２Ｃは、例えば、赤外線センサまたは超音波センサ、ミリ波レーダなどであり、これらの組合せで構成されてもよい。後方右センサ２２Ｃは、車両と接触するおそれがある、地上から高さを有するものを検出する。後方右センサ２２Ｃは、例えば、車両から５ｍ程度までの距離の障害物を検出する。後方右センサ２２Ｃは、鉛直方向視において、センサの中央部を中心とした、例えば、４０°程度の範囲の障害物を検出する。後方右センサ２２Ｃの検出範囲は、後方中央センサ２２Ｂの検出範囲の一部と重複していてもよい。後方右センサ２２Ｃは、検出した障害物の障害物情報を俯瞰映像生成装置４０の障害物情報取得部４３へ出力する。障害物情報の例としては、後方右センサ２２Ｃの検出範囲における障害物の有無、障害物までの距離、水平方向における障害物の存在範囲などである。 The rear right sensor 22C is disposed on the rear right side of the vehicle and detects an obstacle on the rear right side of the vehicle. The rear right sensor 22C is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be configured by a combination thereof. The rear right sensor 22C detects an object having a height from the ground that may come into contact with the vehicle. The rear right sensor 22C detects an obstacle at a distance of about 5 m from the vehicle, for example. The rear right sensor 22C detects an obstacle in a range of, for example, about 40 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the rear right sensor 22C may overlap with a part of the detection range of the rear center sensor 22B. The rear right sensor 22C outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40. Examples of the obstacle information include the presence / absence of an obstacle in the detection range of the rear right sensor 22C, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.
　後方左センサ２２Ａと後方中央センサ２２Ｂと後方右センサ２２Ｃとで、車両の後方の障害物を検出する。 The rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C detect an obstacle behind the vehicle.
　表示パネル３１は、例えば、液晶ディスプレイ（ＬＣＤ：Ｌｉｑｕｉｄ　Ｃｒｙｓｔａｌ　Ｄｉｓｐｌａｙ）または有機ＥＬ（Ｏｒｇａｎｉｃ　Ｅｌｅｃｔｒｏ－Ｌｕｍｉｎｅｓｃｅｎｃｅ）ディスプレイを含むディスプレイである。表示パネル３１は、俯瞰映像生成システム１の俯瞰映像生成装置４０から出力された映像信号に基づいて、俯瞰映像１００を表示する。表示パネル３１は、俯瞰映像生成システム１に専用のものであっても、例えば、ナビゲーションシステムを含む他のシステムと共同で使用するものであってもよい。表示パネル３１は、運転者から視認容易な位置に配置されている。 The display panel 31 is a display including, for example, a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminescence) display. The display panel 31 displays the overhead video 100 based on the video signal output from the overhead video generation device 40 of the overhead video generation system 1. The display panel 31 may be dedicated to the overhead view video generation system 1 or may be used jointly with other systems including a navigation system, for example. The display panel 31 is disposed at a position that is easily visible to the driver.
　俯瞰映像生成装置４０は、制御部４１と、記憶部４９とを有する。 The overhead view video generation device 40 includes a control unit 41 and a storage unit 49.
　制御部４１は、例えば、ＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）などで構成された演算処理装置である。制御部４１は、記憶部４９に記憶されているプログラムをメモリにロードして、プログラムに含まれる命令を実行する。制御部４１は、映像取得部４２と、障害物情報取得部４３と、車両情報取得部４４と、俯瞰映像生成部（映像生成部）４６と、重畳映像生成部（映像生成部）４７と、表示制御部４８とを有する。 The control unit 41 is an arithmetic processing unit configured with, for example, a CPU (Central Processing Unit). The control unit 41 loads the program stored in the storage unit 49 into the memory and executes instructions included in the program. The control unit 41 includes a video acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, an overhead video generation unit (video generation unit) 46, a superimposed video generation unit (video generation unit) 47, Display control unit 48.
　映像取得部４２は、車両の周辺を撮影した周辺映像を取得する。より詳しくは、映像取得部４２は、前方カメラ１１と後方カメラ１２と左側方カメラ１３と右側方カメラ１４とが出力した映像を取得する。映像取得部４２は、取得した映像を俯瞰映像生成部４６に出力する。 The video acquisition unit 42 acquires a peripheral video obtained by photographing the periphery of the vehicle. More specifically, the video acquisition unit 42 acquires video output from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The video acquisition unit 42 outputs the acquired video to the overhead view video generation unit 46.
　障害物情報取得部４３は、車両の周辺において検出した障害物の障害物情報を取得する。本実施形態では、障害物情報取得部４３は、検出した障害物までの距離を含む障害物情報を取得する。より詳しくは、障害物情報取得部４３は、前方左センサ２１Ａと前方中央センサ２１Ｂと前方右センサ２１Ｃと後方左センサ２２Ａと後方中央センサ２２Ｂと後方右センサ２２Ｃとが出力した障害物情報を取得する。障害物情報取得部４３は、取得した障害物情報を重畳映像生成部４７に出力する。 The obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in the vicinity of the vehicle. In the present embodiment, the obstacle information acquisition unit 43 acquires obstacle information including the distance to the detected obstacle. More specifically, the obstacle information acquisition unit 43 acquires the obstacle information output by the front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C. To do. The obstacle information acquisition unit 43 outputs the acquired obstacle information to the superimposed video generation unit 47.
　車両情報取得部４４は、車両のギア操作情報など、俯瞰映像を表示させるためのトリガとなる車両情報を、ＣＡＮ（Ｃｏｎｔｒｏｌｌｅｒ　Ａｒｅａ　Ｎｅｔｗｏｒｋ）や車両の状態をセンシングする各種センサなどから取得する。車両情報取得部４４は、取得した車両情報を俯瞰映像生成部４６に出力する。 The vehicle information acquisition unit 44 acquires vehicle information serving as a trigger for displaying a bird's-eye view video, such as vehicle gear operation information, from a CAN (Controller Area Network) or various sensors that sense the state of the vehicle. The vehicle information acquisition unit 44 outputs the acquired vehicle information to the overhead view video generation unit 46.
　俯瞰映像生成部４６は、映像取得部４２で取得した周辺映像から車両を上方から見下ろすように視点変換処理を行った俯瞰映像１００を生成する。より詳しくは、俯瞰映像生成部４６は、前方カメラ１１と後方カメラ１２と左側方カメラ１３と右側方カメラ１４とで撮影した映像に基づいて、俯瞰映像１００を生成する。俯瞰映像１００を生成する方法は、公知のいずれの方法でもよく、限定されない。俯瞰映像生成部４６は、生成した俯瞰映像１００を表示制御部４７に出力する。 The bird's-eye view video generation unit 46 generates the bird's-eye view video 100 that has been subjected to the viewpoint conversion process so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. More specifically, the bird's-eye view image generation unit 46 generates the bird's-eye view image 100 based on images taken by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the bird's-eye view image 100 may be any known method and is not limited. The overhead image generation unit 46 outputs the generated overhead image 100 to the display control unit 47.
　図２を用いて、俯瞰映像１００について説明する。図２は、第一実施形態に係る俯瞰映像生成システムで生成した俯瞰映像および障害物報知アイコンを示す図である。俯瞰映像１００は、縦長の矩形状である。俯瞰映像１００は、前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４と、前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４とで囲まれた中央部に位置する中央映像１０５とを含む。前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４と、中央映像１０５とは、枠状の境界線で区切られていてもよい。 The overhead view image 100 will be described with reference to FIG. FIG. 2 is a diagram illustrating an overhead video and an obstacle notification icon generated by the overhead video generation system according to the first embodiment. The bird's-eye view image 100 has a vertically long rectangular shape. The bird's-eye view image 100 is located at the center surrounded by the front image 101, the rear image 102, the left side image 103, the right side image 104, the front image 101, the rear image 102, the left side image 103, and the right side image 104. And a central image 105 to be displayed. The front video 101, the rear video 102, the left side video 103, the right side video 104, and the central video 105 may be separated by a frame-like boundary line.
　中央映像１０５は、縦長の矩形状に生成される。中央映像１０５は、前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４との境界が線で区切られている。中央映像１０５は、俯瞰映像１００における車両の位置を示す。 The central video 105 is generated in a vertically long rectangular shape. In the central video 105, the boundaries of the front video 101, the rear video 102, the left video 103, and the right video 104 are separated by lines. The center image 105 shows the position of the vehicle in the overhead view image 100.
　図２においては、前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４との境界を示す斜めの破線を説明のために図示しているが、実際に表示パネル３１に表示される俯瞰映像１００には当該破線は表示されない。他の図も同様である。 In FIG. 2, diagonal broken lines indicating boundaries between the front video 101, the rear video 102, the left side video 103, and the right side video 104 are illustrated for explanation, but are actually displayed on the display panel 31. The broken line is not displayed in the overhead view video 100. The same applies to the other figures.
　重畳映像生成部４７は、障害物情報取得部４３が取得した障害物情報に基づいて、障害物を示す情報を、俯瞰映像１００の中央映像１０５に重畳した俯瞰映像１００を生成する。本実施形態では、重畳映像生成部４７は、障害物情報取得部４３が取得した障害物情報に基づいて、俯瞰映像１００の中央映像１０５に障害物を検出した方向を示す情報を重畳した俯瞰映像１００を生成する。本実施形態では、重畳映像生成部４７は、障害物を検出した方向を示す情報を障害物報知アイコン（障害物を示す情報）１２０で示す。障害物報知アイコン１２０は、障害物を検出する各センサの水平方向における検出方向を模式的に示しており、複数の円弧の配置方向は、センサの取付位置を起点とした検出方向、または車両を中心とした放射状の配置などである。障害物報知アイコン１２０を構成する円弧の幅は、障害物を検出する各センサの検出範囲を示していてもよく、各センサの検出範囲によらず検出方向に対応した固定幅であってもよい。 The superimposed video generation unit 47 generates an overhead video 100 in which information indicating an obstacle is superimposed on the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. In the present embodiment, the superimposed video generation unit 47 superimposes information indicating the direction in which the obstacle is detected on the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. 100 is generated. In the present embodiment, the superimposed video generation unit 47 indicates information indicating the direction in which the obstacle is detected as an obstacle notification icon (information indicating the obstacle) 120. The obstacle notification icon 120 schematically shows the detection direction in the horizontal direction of each sensor that detects an obstacle, and the arrangement direction of the plurality of arcs is the detection direction starting from the sensor mounting position, or the vehicle. For example, a radial arrangement with the center. The width of the arc constituting the obstacle notification icon 120 may indicate the detection range of each sensor that detects the obstacle, or may be a fixed width corresponding to the detection direction regardless of the detection range of each sensor. .
　障害物報知アイコン１２０は、障害物を報知するアイコンである。障害物報知アイコン１２０は、障害物までの距離と方向とを示す。障害物報知アイコン１２０は、前方左アイコン（障害物を示す情報）１２１と前方中央アイコン（障害物を示す情報）１２２と前方右アイコン（障害物を示す情報）１２３と後方左アイコン（障害物を示す情報）１２４と後方中央アイコン（障害物を示す情報）１２５と後方右アイコン（障害物を示す情報）１２６とを含む。 The obstacle notification icon 120 is an icon for notifying an obstacle. The obstacle notification icon 120 indicates the distance and direction to the obstacle. The obstacle notification icon 120 includes a front left icon (information indicating an obstacle) 121, a front center icon (information indicating an obstacle) 122, a front right icon (information indicating an obstacle) 123, and a rear left icon (an obstacle is displayed). Information) 124, a rear center icon (information indicating an obstacle) 125, and a rear right icon (information indicating an obstacle) 126.
　前方左アイコン１２１は、車両の前方左側の障害物を報知するアイコンである。より詳しくは、前方左アイコン１２１は、前方左センサ２１Ａで障害物を検出したことを報知するアイコンである。前方左アイコン１２１は、図２において、俯瞰映像１００の中央映像１０５の左上に重畳される。 The front left icon 121 is an icon for notifying an obstacle on the left front side of the vehicle. More specifically, the front left icon 121 is an icon that notifies that an obstacle has been detected by the front left sensor 21A. The front left icon 121 is superimposed on the upper left of the central image 105 of the overhead view image 100 in FIG.
　本実施形態では、前方左アイコン１２１は、３重の弧状の曲線で構成される。弧状の曲線は、俯瞰映像１００の外側に向かって膨出している。３重の弧状の曲線は、俯瞰映像１００の外側から中心側に向かって半径が小さくなっている。３重の弧状の曲線は、俯瞰映像１００の外側から中心側に向かって短くなっている。 In the present embodiment, the front left icon 121 is composed of a triple arc-shaped curve. The arc-shaped curve bulges toward the outside of the overhead view image 100. The triple arc-shaped curve has a radius that decreases from the outside of the bird's-eye view image 100 toward the center. The triple arc-shaped curve is shortened from the outside of the overhead view image 100 toward the center side.
　前方左アイコン１２１は、障害物までの距離に応じて、色を変化させてもよい。例えば、前方左アイコン１２１は、障害物までの距離が第一所定距離以上の場合、緑色で表示する。前方左アイコン１２１は、障害物までの距離が第一所定距離未満かつ第一所定距離より小さい第二所定距離以上の場合、黄色で表示する。前方左アイコン１２１は、障害物までの距離が第二所定距離未満の場合、赤色で表示する。 The front left icon 121 may change the color according to the distance to the obstacle. For example, the front left icon 121 is displayed in green when the distance to the obstacle is greater than or equal to the first predetermined distance. The front left icon 121 is displayed in yellow when the distance to the obstacle is equal to or greater than a second predetermined distance that is less than the first predetermined distance and smaller than the first predetermined distance. The front left icon 121 is displayed in red when the distance to the obstacle is less than the second predetermined distance.
　前方左アイコン１２１は、障害物までの距離に応じて、弧状の曲線の本数を変化させてもよい。例えば、前方左アイコン１２１は、障害物までの距離が第一所定距離以上の場合、３重の弧状の曲線のうち、最も外側の弧状の曲線を表示する。前方左アイコン１２１は、障害物までの距離が第一所定距離未満かつ第二所定距離以上の場合、３重の弧状の曲線のうち、最も外側の弧状の曲線と中央の弧状の曲線とを表示する。前方左アイコン１２１は、障害物までの距離が第二所定距離未満の場合、３重の弧状の曲線をすべて表示する。 The front left icon 121 may change the number of arcuate curves according to the distance to the obstacle. For example, the front left icon 121 displays the outermost arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is equal to or greater than the first predetermined distance. The front left icon 121 displays the outermost arc-shaped curve and the center arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is less than the first predetermined distance and greater than or equal to the second predetermined distance. To do. The front left icon 121 displays all triple arcuate curves when the distance to the obstacle is less than the second predetermined distance.
　前方中央アイコン１２２は、車両の前方中央の障害物を報知するアイコンである。より詳しくは、前方中央アイコン１２２は、前方中央センサ２１Ｂで障害物を検出したことを報知するアイコンである。前方中央アイコン１２２は、図２において、俯瞰映像１００の中央映像１０５の上側中央に重畳される。前方中央アイコン１２２は、前方左アイコン１２１と同様に構成されている。 The front center icon 122 is an icon for notifying an obstacle at the front center of the vehicle. More specifically, the front center icon 122 is an icon that notifies that an obstacle has been detected by the front center sensor 21B. The front center icon 122 is superimposed on the upper center of the center image 105 of the overhead image 100 in FIG. The front center icon 122 is configured in the same manner as the front left icon 121.
　前方右アイコン１２３は、車両の前方右側の障害物を報知するアイコンである。より詳しくは、前方右アイコン１２３は、前方右センサ２１Ｃで障害物を検出したことを報知するアイコンである。前方右アイコン１２３は、図２において、俯瞰映像１００の中央映像１０５の右上に重畳される。前方右アイコン１２３は、前方左アイコン１２１と同様に構成されている。 The front right icon 123 is an icon for notifying an obstacle on the right front side of the vehicle. More specifically, the front right icon 123 is an icon that notifies that an obstacle has been detected by the front right sensor 21C. The front right icon 123 is superimposed on the upper right of the central image 105 of the overhead image 100 in FIG. The front right icon 123 is configured in the same manner as the front left icon 121.
　後方左アイコン１２４は、車両の後方左側の障害物を報知するアイコンである。より詳しくは、後方左アイコン１２４は、後方左センサ２２Ａで障害物を検出したことを報知するアイコンである。後方左アイコン１２４は、図２において、俯瞰映像１００の中央映像１０５の左下に重畳される。後方左アイコン１２４は、前方左アイコン１２１と同様に構成されている。 The rear left icon 124 is an icon for notifying an obstacle on the left rear side of the vehicle. More specifically, the rear left icon 124 is an icon for notifying that an obstacle has been detected by the rear left sensor 22A. The rear left icon 124 is superimposed on the lower left of the central image 105 of the overhead image 100 in FIG. The rear left icon 124 is configured in the same manner as the front left icon 121.
　後方中央アイコン１２５は、車両の後方中央の障害物を報知するアイコンである。より詳しくは、後方中央アイコン１２５は、後方中央センサ２２Ｂで障害物を検出したことを報知するアイコンである。後方中央アイコン１２５は、図２において、俯瞰映像１００の中央映像１０５の下側中央に重畳される。後方中央アイコン１２５は、前方左アイコン１２１と同様に構成されている。 The rear center icon 125 is an icon that notifies an obstacle at the rear center of the vehicle. More specifically, the rear center icon 125 is an icon for notifying that an obstacle has been detected by the rear center sensor 22B. The rear center icon 125 is superimposed on the lower center of the center image 105 of the overhead image 100 in FIG. The rear center icon 125 is configured in the same manner as the front left icon 121.
　後方右アイコン１２６は、車両の後方右側の障害物を報知するアイコンである。より詳しくは、後方右アイコン１２６は、後方右センサ２２Ｃで障害物を検出したことを報知するアイコンである。後方右アイコン１２６は、図２において、俯瞰映像１００の中央映像１０５の右下に重畳される。後方右アイコン１２６は、前方左アイコン１２１と同様に構成されている。 The rear right icon 126 is an icon for notifying an obstacle on the rear right side of the vehicle. More specifically, the rear right icon 126 is an icon for notifying that an obstacle has been detected by the rear right sensor 22C. The rear right icon 126 is superimposed on the lower right of the central image 105 of the overhead image 100 in FIG. The rear right icon 126 is configured in the same manner as the front left icon 121.
　表示制御部４８は、重畳映像生成部４７が生成した俯瞰映像１００を表示パネル３１に表示させる。 The display control unit 48 causes the display panel 31 to display the overhead video 100 generated by the superimposed video generation unit 47.
　記憶部４９は、俯瞰映像生成装置４０における各種処理に要するデータおよび各種処理結果を記憶する。記憶部４９は、例えば、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）、フラッシュメモリ（Ｆｌａｓｈ　Ｍｅｍｏｒｙ）などの半導体メモリ素子、または、ハードディスク、光ディスクなどの記憶装置である。 The storage unit 49 stores data required for various processes in the overhead view video generation device 40 and various processing results. The storage unit 49 is, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.
　次に、図３を用いて、俯瞰映像生成システム１の俯瞰映像生成装置４０における処理の流れについて説明する。図３は、第一実施形態に係る俯瞰映像生成システムの俯瞰映像生成装置における処理の流れを示すフローチャートである。 Next, the flow of processing in the overhead video generation device 40 of the overhead video generation system 1 will be described with reference to FIG. FIG. 3 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the first embodiment.
　制御部４１は、俯瞰映像表示を開始するか否かを判定する（ステップＳ１１）。俯瞰映像表示を開始する判定の例として、制御部４１は、後退トリガの有無に基づいて、俯瞰映像表示を開示するか否かを判定する。後退トリガとは、例えば、シフトポジションが「リバース」とされたことをいう。または、後退トリガとは、車両の進行方向が車両の前後方向の後方となったことをいう。制御部４１は、後退トリガがない場合、俯瞰映像表示を開始しないと判定し（ステップＳ１１でＮｏ）、ステップＳ１１の処理を再度実行する。制御部４１は、後退トリガがある場合、俯瞰映像表示を開始すると判定し（ステップＳ１１でＹｅｓ）、ステップＳ１２に進む。 The control unit 41 determines whether or not to start the overhead view video display (step S11). As an example of the determination to start the overhead view video display, the control unit 41 determines whether or not to disclose the overhead view video display based on the presence or absence of a reverse trigger. The reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle is rearward in the front-rear direction of the vehicle. When there is no reverse trigger, the control unit 41 determines not to start the overhead view video display (No in step S11), and executes the process of step S11 again. When there is a backward trigger, the control unit 41 determines to start the overhead view video display (Yes in Step S11), and proceeds to Step S12.
　制御部４１は、俯瞰映像１００を生成し表示する（ステップＳ１２）。より詳しくは、制御部４１は、俯瞰映像生成部４６で、映像取得部４２が取得した周辺映像から車両を上方から見下ろすように視点変換処理を行った俯瞰映像１００を生成させる。そして、制御部４１は、表示制御部４８で、生成した俯瞰映像１００を表示パネル３１に表示させる。 The control unit 41 generates and displays the bird's-eye view video 100 (step S12). More specifically, the control unit 41 causes the overhead view video generation unit 46 to generate the overhead view video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
　制御部４１は、障害物を検出したか否かを判定する（ステップＳ１３）。より詳しくは、制御部４１は、障害物情報取得部４３で障害物情報が取得されたか否かを判定する。制御部４１は、障害物情報取得部４３で障害物情報が取得されたと判定した場合（ステップＳ１３でＹｅｓ）、ステップＳ１４に進む。制御部４１は、障害物情報取得部４３で障害物情報が取得されていないと判定した場合（ステップＳ１３でＮｏ）、ステップＳ１５に進む。 The control unit 41 determines whether an obstacle has been detected (step S13). More specifically, the control unit 41 determines whether or not obstacle information has been acquired by the obstacle information acquisition unit 43. When it is determined that the obstacle information acquisition unit 43 has acquired the obstacle information (Yes in step S13), the control unit 41 proceeds to step S14. When it is determined that the obstacle information acquisition unit 43 has not acquired the obstacle information (No in step S13), the control unit 41 proceeds to step S15.
　制御部４１は、俯瞰映像１００の中央部に障害物を示す障害物報知アイコン１２０を重畳表示する（ステップＳ１４）。より詳しくは、制御部４１は、重畳映像生成部４７で、障害物情報取得部４３が取得した障害物情報に基づいて、俯瞰映像１００の中央映像１０５に、障害物を検出した方向を示すように障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。そして、制御部４１は、表示制御部４８で、生成した俯瞰映像１００を表示パネル３１に表示させる。 The control unit 41 superimposes and displays an obstacle notification icon 120 indicating an obstacle at the center of the overhead view video 100 (step S14). More specifically, the control unit 41 causes the superimposed video generation unit 47 to indicate the direction in which the obstacle is detected in the central video 105 of the overhead video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. The overhead view video 100 in which the obstacle notification icon 120 is superimposed on is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
　制御部４１は、前方左センサ２１Ａと前方中央センサ２１Ｂと前方右センサ２１Ｃと後方左センサ２２Ａと後方中央センサ２２Ｂと後方右センサ２２Ｃの複数のセンサからの障害物情報を取得した場合、重畳映像生成部４７で、俯瞰映像１００の中央映像１０５に、複数の障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。 When the control unit 41 acquires obstacle information from a plurality of sensors including the front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C, The generation unit 47 generates the overhead image 100 in which a plurality of obstacle notification icons 120 are superimposed on the central image 105 of the overhead image 100.
　制御部４１は、俯瞰映像表示を終了するか否かを判定する（ステップＳ１５）。より詳しくは、制御部４１は、後退トリガの有無に基づいて、俯瞰映像表示を終了するか否かを判定する。制御部４１は、後退トリガがない場合、言い換えると、後退トリガが解除された場合、俯瞰映像表示を終了すると判定し（ステップＳ１５でＹｅｓ）、処理を終了する。制御部４１は、後退トリガがある場合、俯瞰映像表示を終了しないと判定し（ステップＳ１５でＮｏ）、ステップＳ１３に戻って処理を継続する。 The control unit 41 determines whether or not to end the overhead view video display (step S15). More specifically, the control unit 41 determines whether or not to end the bird's-eye view video display based on the presence or absence of the reverse trigger. When there is no backward trigger, in other words, when the backward trigger is released, the control unit 41 determines to end the overhead view video display (Yes in step S15), and ends the process. When there is a reverse trigger, the control unit 41 determines not to end the overhead view video display (No in step S15), returns to step S13, and continues the process.
　このようにして、俯瞰映像生成システム１は、障害物が検出されると、俯瞰映像１００の中央部に、障害物を検出した方向を示すように障害物報知アイコン１２０を重畳して、表示パネル３１に映像信号を出力する。表示パネル３１は、俯瞰映像生成システム１から出力された映像信号に基づいて、例えば、ナビゲーション画面とともに俯瞰映像１００を表示する。 In this way, when an obstacle is detected, the overhead image generation system 1 superimposes the obstacle notification icon 120 on the center of the overhead image 100 so as to indicate the direction in which the obstacle is detected, and displays the display panel. 31 outputs a video signal. The display panel 31 displays the bird's-eye view video 100 together with the navigation screen, for example, based on the video signal output from the bird's-eye view video generation system 1.
　例えば、車両の後退時、後方左側に障害物を検出した場合の一例を、図４を用いて説明する。図４は、第一実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の一例を示す図である。図４において、俯瞰映像１００の後方映像１０２には、後方左側の障害物の障害物映像１３０が含まれている。 For example, an example of detecting an obstacle on the left rear side when the vehicle is moving backward will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment. In FIG. 4, the rear image 102 of the overhead view image 100 includes an obstacle image 130 of the obstacle on the left rear side.
　制御部４１は、ステップＳ１３において、障害物を検出したと判定する。そして、制御部４１は、ステップＳ１４において、重畳映像生成部４７で、障害物情報取得部４３が取得した障害物情報に基づいて、俯瞰映像１００の中央映像１０５に、障害物を検出した後左方向を示すように障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。そして、制御部４１は、表示制御部４８で、生成した俯瞰映像１００を表示パネル３１に表示させる。 Control unit 41 determines that an obstacle has been detected in step S13. Then, in step S14, the control unit 41 detects the obstacle in the central image 105 of the overhead view video 100 based on the obstacle information acquired by the obstacle information acquisition unit 43 in the superimposed image generation unit 47, and then left An overhead view image 100 in which the obstacle notification icon 120 is superimposed so as to indicate the direction is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
　車両の後退時、後方左側に障害物を検出した場合の他の例を、図５を用いて説明する。図５は、第一実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。 Another example when an obstacle is detected on the left rear side when the vehicle is moving backward will be described with reference to FIG. FIG. 5 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the first embodiment.
　図５においては、俯瞰映像１００の中央映像１０５に、車両を上方から見た、車両を示す自車アイコンである自車アイコン１４０が合成されている。制御部４１は、ステップＳ１４において、重畳映像生成部４７で、自車アイコン１４０に、障害物を検出した後左方向を示すように障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。 In FIG. 5, a vehicle icon 140 that is a vehicle icon indicating a vehicle when the vehicle is viewed from above is synthesized with the central image 105 of the overhead view image 100. In step S <b> 14, the control unit 41 causes the superimposed video generation unit 47 to generate an overhead video 100 in which the obstacle notification icon 120 is superimposed on the vehicle icon 140 so as to indicate the left direction after the obstacle is detected.
　上述したように、本実施形態は、車両の周辺に障害物が検出されると、俯瞰映像１００の中央映像１０５に、障害物を検出した方向を示すように障害物報知アイコン１２０を重畳した俯瞰映像１００を表示パネル３１に表示させる。本実施形態は、障害物報知アイコン１２０が俯瞰映像１００の中央映像１０５に重畳されるので、障害物報知アイコン１２０と俯瞰映像１００に映った障害物とが重畳しない。言い換えると、本実施形態は、俯瞰映像１００に映った障害物の視認性を損なわず、障害物を明瞭に表示することができる。このように、本実施形態は、車両周辺の障害物を適切に確認可能にすることができる。 As described above, according to the present embodiment, when an obstacle is detected around the vehicle, the overhead notification icon 120 is superimposed on the central image 105 of the overhead image 100 so as to indicate the direction in which the obstacle is detected. The video 100 is displayed on the display panel 31. In the present embodiment, since the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100, the obstacle notification icon 120 and the obstacle reflected in the overhead image 100 are not superimposed. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.
　本実施形態は、障害物報知アイコン１２０で、障害物を検出した方向と距離とを報知することができる。 In this embodiment, the obstacle notification icon 120 can notify the direction and distance in which the obstacle is detected.
　本実施形態は、図４に示すように、俯瞰映像１００の中央映像１０５に自車アイコン１４０を表示しない場合、障害物報知アイコン１２０の視認性をより向上することができる。このように、本実施形態は、車両周辺の障害物をより適切に確認可能にすることができる。 As shown in FIG. 4, this embodiment can further improve the visibility of the obstacle notification icon 120 when the vehicle icon 140 is not displayed on the central image 105 of the overhead view image 100. Thus, this embodiment can make it possible to more appropriately check obstacles around the vehicle.
　本実施形態は、図５に示すように、俯瞰映像１００の中央映像１０５に自車アイコン１４０を表示してもよい。これにより、本実施形態は、車両に対する障害物の方向をより認識しやすい俯瞰映像１００を表示することができる。このように、本実施形態は、車両周辺の障害物を適切に確認可能にすることができる。 In the present embodiment, as shown in FIG. 5, the vehicle icon 140 may be displayed on the center image 105 of the overhead view image 100. Thereby, this embodiment can display the bird's-eye view image 100 which can recognize the direction of the obstacle with respect to a vehicle more easily. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.
［第二実施形態］
　図６、図７を参照しながら、本実施形態に係る俯瞰映像生成システム１について説明する。図６は、第二実施形態に係る俯瞰映像生成システムの俯瞰映像生成装置における処理の流れを示すフローチャートである。図７は、第二実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の一例を示す図である。俯瞰映像生成システム１は、基本的な構成は第一実施形態の俯瞰映像生成システム１と同様である。以下の説明においては、俯瞰映像生成システム１と同様の構成要素には、同一の符号または対応する符号を付し、その詳細な説明は省略する。本実施形態の俯瞰映像生成システム１は、俯瞰映像生成装置４０の障害物情報取得部４３と重畳映像生成部４７と制御部４１における処理とが、第一実施形態の俯瞰映像生成システム１と異なる。 [Second Embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. FIG. 6 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the second embodiment. FIG. 7 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the second embodiment. The overhead image generation system 1 has the same basic configuration as the overhead image generation system 1 of the first embodiment. In the following description, components similar to those in the overhead view video generation system 1 are denoted by the same reference numerals or corresponding reference numerals, and detailed description thereof is omitted. The overhead image generation system 1 of the present embodiment is different from the overhead image generation system 1 of the first embodiment in the processing of the obstacle information acquisition unit 43, the superimposed image generation unit 47, and the control unit 41 of the overhead image generation device 40. .
　障害物報知アイコン１２０は、第一実施形態の前方左アイコン１２１と前方中央アイコン１２２と前方右アイコン１２３と後方左アイコン１２４と後方中央アイコン１２５と後方右アイコン１２６とに加えて、第二前方左アイコン（障害物を示す情報）と第二前方中央アイコン（障害物を示す情報）と第二前方右アイコン（障害物を示す情報）と第二後方左アイコン（障害物を示す情報）と第二後方中央アイコン（障害物を示す情報）と第二後方右アイコン（障害物を示す情報）とを含む。図７においては、第二前方左アイコンと第二前方中央アイコンと第二前方右アイコンと第二後方中央アイコンと第二後方右アイコンは図示されておらず、第二後方左アイコン（障害物を示す情報）が示されている。 The obstacle notification icon 120 includes a second front left icon 121, a front center icon 122, a front right icon 123, a rear left icon 124, a rear center icon 125, and a rear right icon 126 in the first embodiment. Icon (information indicating obstacle), second front center icon (information indicating obstacle), second front right icon (information indicating obstacle), second rear left icon (information indicating obstacle) and second A rear center icon (information indicating an obstacle) and a second rear right icon (information indicating an obstacle) are included. In FIG. 7, the second front left icon, the second front center icon, the second front right icon, the second rear center icon, and the second rear right icon are not shown. Information).
　前方左アイコン１２１は、障害物までの距離が閾値未満の、車両の前方左側の障害物を報知するアイコンである。より詳しくは、前方左アイコン１２１は、前方左センサ２１Ａで、障害物までの距離が閾値未満の障害物を検出したことを報知するアイコンである。 The front left icon 121 is an icon for notifying an obstacle on the left front side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front left icon 121 is an icon for notifying that the front left sensor 21A has detected an obstacle whose distance to the obstacle is less than a threshold.
　前方中央アイコン１２２は、障害物までの距離が閾値未満の、車両の前方中央の障害物を報知するアイコンである。より詳しくは、前方中央アイコン１２２は、前方中央センサ２１Ｂで、障害物までの距離が閾値未満の障害物を検出したことを報知するアイコンである。 The front center icon 122 is an icon for notifying an obstacle at the front center of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front center icon 122 is an icon for notifying that the front center sensor 21B has detected an obstacle whose distance to the obstacle is less than a threshold value.
　前方右アイコン１２３は、障害物までの距離が閾値未満の、車両の前方右側の障害物を報知するアイコンである。より詳しくは、前方右アイコン１２３は、前方右センサ２１Ｃで、障害物までの距離が閾値未満の障害物を検出したことを報知するアイコンである。 The front right icon 123 is an icon for notifying an obstacle on the right front side of the vehicle whose distance to the obstacle is less than a threshold. More specifically, the front right icon 123 is an icon for informing that the front right sensor 21C has detected an obstacle whose distance to the obstacle is less than a threshold.
　後方左アイコン１２４は、障害物までの距離が閾値未満の、車両の後方左側の障害物を報知するアイコンである。より詳しくは、後方左アイコン１２４は、後方左センサ２２Ａで、障害物までの距離が閾値未満の障害物を検出したことを報知するアイコンである。 The rear left icon 124 is an icon for notifying an obstacle on the left rear side of the vehicle whose distance to the obstacle is less than the threshold. More specifically, the rear left icon 124 is an icon that notifies that the rear left sensor 22A has detected an obstacle whose distance to the obstacle is less than a threshold.
　後方中央アイコン１２５は、障害物までの距離が閾値未満の、車両の後方中央の障害物を報知するアイコンである。より詳しくは、後方中央アイコン１２５は、後方中央センサ２２Ｂで、障害物までの距離が閾値未満の障害物を検出したことを報知するアイコンである。 The rear center icon 125 is an icon for notifying an obstacle at the rear center of the vehicle whose distance to the obstacle is less than a threshold value. More specifically, the rear center icon 125 is an icon for informing that the rear center sensor 22B has detected an obstacle whose distance to the obstacle is less than a threshold.
　後方右アイコン１２６は、障害物までの距離が閾値未満の、車両の後方右側の障害物を報知するアイコンである。より詳しくは、後方右アイコン１２６は、後方右センサ２２Ｃで、障害物までの距離が閾値未満の障害物を検出したことを報知するアイコンである。 The rear right icon 126 is an icon for notifying an obstacle on the right rear side of the vehicle whose distance to the obstacle is less than a threshold value. More specifically, the rear right icon 126 is an icon for notifying that the rear right sensor 22C has detected an obstacle whose distance to the obstacle is less than a threshold.
　第二前方左アイコンは、障害物までの距離が閾値以上の、車両の前方左側の障害物を報知するアイコンである。より詳しくは、第二前方左アイコンは、前方左センサ２１Ａで、障害物までの距離が閾値以上の障害物を検出したことを報知するアイコンである。第二前方左アイコンは、俯瞰映像１００の中央映像１０５より外側の左上に重畳される。第二前方左アイコンは、前方左アイコン１２１と同様に構成されている。 The second front left icon is an icon for notifying an obstacle on the left front side of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front left icon is an icon for notifying that the front left sensor 21A has detected an obstacle whose distance to the obstacle is not less than a threshold value. The second front left icon is superimposed on the upper left outside the central image 105 of the overhead image 100. The second front left icon is configured in the same manner as the front left icon 121.
　第二前方中央アイコンは、障害物までの距離が閾値以上の、車両の前方中央の障害物を報知するアイコンである。より詳しくは、第二前方中央アイコンは、前方中央センサ２１Ｂで、障害物までの距離が閾値以上の障害物を検出したことを報知するアイコンである。第二前方中央アイコンは、俯瞰映像１００の中央映像１０５より外側の上側中央に重畳される。第二前方中央アイコンは、前方左アイコン１２１と同様に構成されている。 The second front center icon is an icon for notifying an obstacle at the front center of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front center icon is an icon for notifying that the front center sensor 21B has detected an obstacle whose distance to the obstacle is a threshold value or more. The second front center icon is superimposed on the upper center outside the center image 105 of the overhead view image 100. The second front center icon is configured in the same manner as the front left icon 121.
　第二前方右アイコンは、障害物までの距離が閾値以上の、車両の前方右側の障害物を報知するアイコンである。より詳しくは、第二前方右アイコンは、前方右センサ２１Ｃで、障害物までの距離が閾値以上の障害物を検出したことを報知するアイコンである。第二前方右アイコンは、俯瞰映像１００の中央映像１０５より外側の右上に重畳される。第二前方右アイコンは、前方左アイコン１２１と同様に構成されている。 The second front right icon is an icon for notifying an obstacle on the right front side of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second front right icon is an icon for informing that the front right sensor 21C has detected an obstacle whose distance to the obstacle is not less than a threshold value. The second front right icon is superimposed on the upper right outside the center image 105 of the overhead view image 100. The second front right icon is configured in the same manner as the front left icon 121.
　第二後方左アイコンは、障害物までの距離が閾値以上の、車両の後方左側の障害物を報知するアイコンである。より詳しくは、第二後方左アイコンは、後方左センサ２２Ａで、障害物までの距離が閾値以上の障害物を検出したことを報知するアイコンである。第二後方左アイコンは、俯瞰映像１００の中央映像１０５より外側の左下に重畳される。第二後方左アイコンは、前方左アイコン１２１と同様に構成されている。 The second rear left icon is an icon for notifying an obstacle on the left rear side of the vehicle whose distance to the obstacle is equal to or greater than a threshold value. More specifically, the second rear left icon is an icon for notifying that the rear left sensor 22A has detected an obstacle whose distance to the obstacle is a threshold value or more. The second rear left icon is superimposed on the lower left outside the center image 105 of the overhead view image 100. The second rear left icon is configured in the same manner as the front left icon 121.
　第二後方中央アイコンは、障害物までの距離が閾値以上の、車両の後方中央の障害物を報知するアイコンである。より詳しくは、第二後方中央アイコンは、後方中央センサ２２Ｂで、障害物までの距離が閾値以上の障害物を検出したことを報知するアイコンである。第二後方中央アイコンは、俯瞰映像１００の中央映像１０５より外側の下側中央に重畳される。第二後方中央アイコンは、前方左アイコン１２１と同様に構成されている。 The second rear center icon is an icon for notifying an obstacle at the rear center of the vehicle whose distance to the obstacle is not less than a threshold. More specifically, the second rear center icon is an icon for informing that the rear center sensor 22B has detected an obstacle whose distance to the obstacle is a threshold value or more. The second rear center icon is superimposed on the lower center outside the center image 105 of the overhead image 100. The second rear center icon is configured in the same manner as the front left icon 121.
　第二後方右アイコンは、障害物までの距離が閾値以上の、車両の後方右側の障害物を報知するアイコンである。より詳しくは、第二後方右アイコンは、後方右センサ２２Ｃで、障害物までの距離が閾値以上の障害物を検出したことを報知するアイコンである。第二後方右アイコンは、俯瞰映像１００の中央映像１０５より外側の右下に重畳される。第二後方右アイコンは、前方左アイコン１２１と同様に構成されている。 The second rear right icon is an icon for notifying an obstacle on the right rear side of the vehicle whose distance to the obstacle is a threshold value or more. More specifically, the second rear right icon is an icon for notifying that the rear right sensor 22C has detected an obstacle whose distance to the obstacle is not less than a threshold value. The second rear right icon is superimposed on the lower right outside the center image 105 of the overhead view image 100. The second rear right icon is configured in the same manner as the front left icon 121.
　障害物情報取得部４３は、検出した障害物までの距離を含む情報を取得する。障害物情報取得部４３は、取得した障害物までの距離を含む情報を重畳映像生成部４７に出力する。 The obstacle information acquisition unit 43 acquires information including the distance to the detected obstacle. The obstacle information acquisition unit 43 outputs information including the acquired distance to the obstacle to the superimposed video generation unit 47.
　重畳映像生成部４７は、障害物情報取得部４３で取得した障害物情報に基づいて、検出した障害物までの距離が所定閾値以上であれば、俯瞰映像１００の中央映像１０５に自車アイコン１４０を表示させ、障害物を示す情報である障害物報知アイコン１２０を俯瞰映像１００の中央部の外側に重畳させた俯瞰映像１００を生成する。 If the distance to the detected obstacle is greater than or equal to a predetermined threshold based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 47 displays the vehicle icon 140 in the central image 105 of the overhead image 100. Is displayed, and an overhead video 100 is generated by superimposing an obstacle notification icon 120, which is information indicating an obstacle, on the outside of the central portion of the overhead video 100.
　重畳映像生成部４７は、障害物情報取得部４３で取得した障害物情報に基づいて、検出した障害物までの距離が所定閾値未満であれば、障害物報知アイコン１２０を、俯瞰映像１００の中央映像１０５に重畳した俯瞰映像１００を生成する。 If the distance to the detected obstacle is less than a predetermined threshold based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed video generation unit 47 displays the obstacle notification icon 120 in the center of the overhead view video 100. The overhead view video 100 superimposed on the video 105 is generated.
　次に、図６を用いて、俯瞰映像生成システム１の俯瞰映像生成装置４０における処理の流れについて説明する。図６に示すフローチャートのステップＳ２１、ステップＳ２３、ステップＳ２６、ステップＳ２７の処理は、図３に示すフローチャートのステップＳ１１、ステップＳ１３、ステップＳ１４、ステップＳ１５の処理と同様である。 Next, the flow of processing in the overhead video generation device 40 of the overhead video generation system 1 will be described with reference to FIG. The processing in step S21, step S23, step S26, and step S27 in the flowchart shown in FIG. 6 is the same as the processing in step S11, step S13, step S14, and step S15 in the flowchart shown in FIG.
　制御部４１は、自車アイコン１４０を重畳した俯瞰映像１００を生成し表示する（ステップＳ２２）。より詳しくは、制御部４１は、俯瞰映像生成部４６で、映像取得部４２が取得した周辺映像から車両を上方から見下ろすように視点変換処理を行った俯瞰映像１００を生成させる。制御部４１は、表示制御部４８で、生成した俯瞰映像１００の中央部に自車アイコン１４０を合成する。そして、制御部４１は、表示制御部４８で、生成した俯瞰映像１００を表示パネル３１に表示させる。 The control unit 41 generates and displays the bird's-eye view image 100 on which the host vehicle icon 140 is superimposed (step S22). More specifically, the control unit 41 causes the overhead view video generation unit 46 to generate the overhead view video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. The control unit 41 synthesizes the vehicle icon 140 in the center of the generated bird's-eye view image 100 with the display control unit 48. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
　制御部４１は、障害物までの距離が閾値以上であるか否かを判定する（ステップＳ２４）。より詳しくは、制御部４１は、障害物情報取得部４３で取得された、検出した障害物までの距離が所定閾値以上である場合（ステップＳ２４でＹｅｓ）、ステップＳ２５に進む。制御部４１は、障害物情報取得部４３で取得された、検出した障害物までの距離が所定閾値未満である場合（ステップＳ２４でＮｏ）、ステップＳ２６に進む。 The control unit 41 determines whether or not the distance to the obstacle is greater than or equal to a threshold value (step S24). More specifically, when the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold (Yes in step S24), the control unit 41 proceeds to step S25. When the distance to the detected obstacle acquired by the obstacle information acquiring unit 43 is less than the predetermined threshold (No in step S24), the control unit 41 proceeds to step S26.
　所定閾値は、第二前方左アイコンと第二前方中央アイコンと第二前方右アイコンと第二後方左アイコンと第二後方中央アイコンと第二後方右アイコンとが表示される位置が、俯瞰映像１００の障害物の映像と重複しない値に設定される。より詳しくは、所定閾値は、第二前方左アイコンと第二前方中央アイコンと第二前方右アイコンと第二後方左アイコンと第二後方中央アイコンと第二後方右アイコンの最も外側の円弧状の曲線の位置に対応する車両からの距離以上の値であればよい。例えば、所定閾値は、２ｍ程度としてもよい。 The predetermined threshold indicates that the position at which the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon are displayed is an overhead image 100. It is set to a value that does not overlap with the obstacle video. More specifically, the predetermined threshold is the outermost arc shape of the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon. Any value that is equal to or greater than the distance from the vehicle corresponding to the position of the curve is acceptable. For example, the predetermined threshold may be about 2 m.
　制御部４１は、自車アイコン１４０の外側の俯瞰映像１００に、障害物報知アイコン１２０を重畳表示する（ステップＳ２５）。より詳しくは、制御部４１は、重畳映像生成部４７で、障害物情報取得部４３が取得した障害物情報に基づいて、自車アイコン１４０の外側の俯瞰映像１００に、第二前方左アイコンと第二前方中央アイコンと第二前方右アイコンと第二後方左アイコンと第二後方中央アイコンと第二後方右アイコンとの少なくともいずれかを重畳した俯瞰映像１００を生成させる。そして、制御部４１は、表示制御部４８で、生成した俯瞰映像１００を表示パネル３１に表示させる。 The control unit 41 superimposes and displays the obstacle notification icon 120 on the bird's-eye view image 100 outside the host vehicle icon 140 (step S25). More specifically, the control unit 41 uses the superimposed image generation unit 47 to display the second front left icon and the overhead image 100 on the outside of the vehicle icon 140 based on the obstacle information acquired by the obstacle information acquisition unit 43. An overhead view image 100 in which at least one of a second front center icon, a second front right icon, a second rear left icon, a second rear center icon, and a second rear right icon is superimposed is generated. Then, the control unit 41 causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
　このようにして、俯瞰映像生成システム１は、障害物までの距離が所定閾値以上であると、自車アイコン１４０の外側の俯瞰映像１００に、第二前方左アイコンと第二前方中央アイコンと第二前方右アイコンと第二後方左アイコンと第二後方中央アイコンと第二後方右アイコンとの少なくともいずれかを重畳した俯瞰映像１００を生成し、表示パネル３１に映像信号を出力する。俯瞰映像生成システム１は、障害物までの距離が所定閾値未満であると、俯瞰映像１００の中央映像１０５に、前方左アイコン１２１と前方中央アイコン１２２と前方右アイコン１２３と後方左アイコン１２４と後方中央アイコン１２５と後方右アイコン１２６との少なくともいずれかを重畳した俯瞰映像１００を生成し、表示パネル３１に映像信号を出力する。 Thus, when the distance to the obstacle is equal to or greater than the predetermined threshold, the overhead view video generation system 1 displays the second forward left icon, the second forward center icon, and the first forward icon 100 on the overhead view video 100 outside the vehicle icon 140. A bird's-eye view image 100 in which at least one of a second front right icon, a second rear left icon, a second rear center icon, and a second rear right icon is superimposed is generated, and a video signal is output to the display panel 31. When the distance to the obstacle is less than a predetermined threshold, the overhead view video generation system 1 includes a front left icon 121, a front center icon 122, a front right icon 123, a rear left icon 124, and a rear side. An overhead view image 100 in which at least one of the center icon 125 and the rear right icon 126 is superimposed is generated, and a video signal is output to the display panel 31.
　例えば、車両の後退時、後方左側の所定閾値以上の距離に障害物を検出した場合の一例を、図７を用いて説明する。 For example, an example in which an obstacle is detected at a distance greater than or equal to a predetermined threshold on the left rear side when the vehicle is reversing will be described with reference to FIG.
　制御部４１は、ステップＳ２３において、障害物を検出したと判定する。そして、制御部４１は、ステップＳ２４において、障害物情報取得部４３で取得された、検出した障害物までの距離が所定閾値以上であると判定する（ステップＳ２４でＹｅｓ）。そして、制御部４１は、ステップＳ２５において、自車アイコン１４０の外側の俯瞰映像１００に、障害物を検出した後左方向を示す第二後方左アイコン１２７を重畳した、図７に示す俯瞰映像１００を生成させる。そして、制御部４１は、生成した俯瞰映像１００を表示パネル３１に表示させる。 Control unit 41 determines that an obstacle has been detected in step S23. In step S24, the control unit 41 determines that the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold (Yes in step S24). Then, in step S25, the control unit 41 superimposes the second rear left icon 127 indicating the left direction after detecting the obstacle on the overhead view video 100 outside the host vehicle icon 140, as shown in FIG. Is generated. Then, the control unit 41 displays the generated overhead image 100 on the display panel 31.
　上述したように、本実施形態は、障害物までの距離が所定閾値以上であると、自車アイコン１４０の外側の俯瞰映像１００に、障害物報知アイコン１２０を重畳した俯瞰映像１００を生成し表示パネル３１に表示させる。本実施形態は、障害物までの距離が所定閾値以上の場合、障害物報知アイコン１２０が自車アイコン１４０の外側の俯瞰映像１００に重畳されるので、障害物報知アイコン１２０と俯瞰映像１００に映った障害物とが重畳しない。言い換えると、本実施形態は、俯瞰映像１００に映った障害物の視認性を損なわず、障害物を明瞭に表示することができる。このように、本実施形態は、車両周辺の障害物を適切に確認可能にすることができる。 As described above, according to the present embodiment, when the distance to the obstacle is equal to or greater than the predetermined threshold, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the overhead image 100 outside the host vehicle icon 140 is generated and displayed. Display on the panel 31. In the present embodiment, when the distance to the obstacle is equal to or greater than a predetermined threshold, the obstacle notification icon 120 is superimposed on the overhead view image 100 outside the host vehicle icon 140, so that the obstacle notification icon 120 and the overhead view image 100 are reflected. Obstacles do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.
　本実施形態は、障害物までの距離が所定閾値未満であると、俯瞰映像１００の中央映像１０５に、障害物報知アイコン１２０を重畳した俯瞰映像１００を生成し表示パネル３１に表示させる。本実施形態は、障害物までの距離が所定閾値未満の場合、障害物報知アイコン１２０が俯瞰映像１００の中央映像１０５に重畳されるので、障害物報知アイコン１２０と俯瞰映像１００に映った障害物とが重畳しない。言い換えると、本実施形態は、俯瞰映像１００に映った障害物の視認性を損なわず、障害物を明瞭に表示することができる。 In this embodiment, when the distance to the obstacle is less than the predetermined threshold, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100 is generated and displayed on the display panel 31. In the present embodiment, when the distance to the obstacle is less than the predetermined threshold, the obstacle notification icon 120 is superimposed on the central image 105 of the overhead view video 100, so that the obstacle reflected in the obstacle notification icon 120 and the overhead view image 100 is displayed. And do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
　このように、本実施形態は、障害物までの距離に応じて、車両周辺の障害物を適切に確認可能にすることができる。 As described above, according to the present embodiment, it is possible to appropriately check obstacles around the vehicle according to the distance to the obstacle.
　本実施形態は、障害物までの距離に応じて、俯瞰映像１００における障害物報知アイコン１２０の表示位置が変化する。このため、本実施形態は、障害物までの距離をより把握しやすくすることができる。 In the present embodiment, the display position of the obstacle notification icon 120 in the overhead view video 100 changes according to the distance to the obstacle. For this reason, this embodiment can make it easier to grasp the distance to the obstacle.
［第三実施形態］
　図８を参照しながら、本実施形態に係る俯瞰映像生成システム１について説明する。図８は、第三実施形態に係る俯瞰映像生成システムの俯瞰映像生成装置における処理の流れを示すフローチャートである。本実施形態の俯瞰映像生成システム１は、俯瞰映像生成装置４０の障害物情報取得部４３と重畳映像生成部４７と制御部４１における処理とが、第二実施形態の俯瞰映像生成システム１と異なる。 [Third embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing the flow of processing in the overhead video generation device of the overhead video generation system according to the third embodiment. The overhead image generation system 1 of the present embodiment is different from the overhead image generation system 1 of the second embodiment in the processing of the obstacle information acquisition unit 43, the superimposed image generation unit 47, and the control unit 41 of the overhead image generation device 40. .
　前方左センサ２１Ａと前方中央センサ２１Ｂと前方右センサ２１Ｃと後方左センサ２２Ａと後方中央センサ２２Ｂと後方右センサ２２Ｃとは、俯瞰映像１００として表示する範囲より遠方の範囲を検出範囲とする。 The front left sensor 21A, the front center sensor 21B, the front right sensor 21C, the rear left sensor 22A, the rear center sensor 22B, and the rear right sensor 22C have a detection range that is farther from the range displayed as the overhead image 100.
　障害物情報取得部４３は、俯瞰映像１００として表示する範囲より遠方の範囲において検出した障害物の障害物情報を取得する。 The obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in a range far from the range displayed as the overhead view video 100.
　重畳映像生成部４７は、障害物情報取得部４３で取得した障害物情報に基づいて、検出した障害物までの距離が俯瞰映像１００として表示する範囲より遠方であれば、俯瞰映像１００の中央部に自車アイコン１４０を表示させ、障害物報知アイコン１２０を自車アイコン１４０の外側の俯瞰映像１００に重畳させた俯瞰映像１００を生成する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 47 is a central portion of the overhead image 100 if the distance to the detected obstacle is far from the range displayed as the overhead image 100. The vehicle icon 140 is displayed on the vehicle, and the overhead image 100 is generated by superimposing the obstacle notification icon 120 on the bird's-eye image 100 outside the vehicle icon 140.
　重畳映像生成部４７は、障害物情報取得部４３で取得した障害物情報に基づいて、検出した障害物までの距離が俯瞰映像１００として表示する範囲に含まれる範囲であれば、障害物報知アイコン１２０を、俯瞰映像１００の中央映像１０５に障害物を検出した方向を示すように重畳した俯瞰映像１００を生成する。 The superimposed video generation unit 47 is based on the obstacle information acquired by the obstacle information acquisition unit 43. If the distance to the detected obstacle is within the range displayed as the overhead image 100, the obstacle notification icon The overhead view image 100 is generated by superimposing 120 on the central image 105 of the overhead view image 100 so as to indicate the direction in which the obstacle is detected.
　次に、図８を用いて、俯瞰映像生成システム１の俯瞰映像生成装置４０における処理の流れについて説明する。図８に示すフローチャートのステップＳ３１～ステップＳ３３、ステップＳ３５～ステップＳ３７の処理は、図６に示すフローチャートのステップＳ２１～ステップＳ２３、ステップＳ２５～ステップＳ２７の処理と同様である。 Next, the flow of processing in the overhead video generation device 40 of the overhead video generation system 1 will be described with reference to FIG. The processes in steps S31 to S33 and steps S35 to S37 in the flowchart shown in FIG. 8 are the same as the processes in steps S21 to S23 and steps S25 to S27 in the flowchart shown in FIG.
　制御部４１は、障害物までの距離は俯瞰映像１００の表示範囲以上であるか否かを判定する（ステップＳ３４）。制御部４１は、障害物情報取得部４３で取得された、検出した障害物までの距離が俯瞰映像１００の表示範囲より遠方の範囲である場合（ステップＳ３４でＹｅｓ）、ステップＳ３５に進む。制御部４１は、障害物情報取得部４３で取得された、検出した障害物までの距離が俯瞰映像１００の表示範囲に含まれる場合（ステップＳ３４でＮｏ）、ステップＳ３６に進む。 The control unit 41 determines whether or not the distance to the obstacle is greater than or equal to the display range of the bird's-eye view image 100 (step S34). When the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is a range far from the display range of the overhead view image 100 (Yes in step S34), the control unit 41 proceeds to step S35. When the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is included in the display range of the overhead image 100 (No in step S34), the control unit 41 proceeds to step S36.
　このようにして、俯瞰映像生成システム１は、障害物までの距離が俯瞰映像１００の表示範囲より遠方の範囲であると、自車アイコン１４０の外側の俯瞰映像１００に、障害物報知アイコン１２０を重畳した俯瞰映像１００を生成し、表示パネル３１に映像信号を出力する。俯瞰映像生成システム１は、障害物までの距離が俯瞰映像１００の表示範囲に含まれると、俯瞰映像１００の中央映像１０５に、障害物報知アイコン１２０を重畳した俯瞰映像１００を生成し、表示パネル３１に映像信号を出力する。 In this way, the overhead view video generation system 1 displays the obstacle notification icon 120 on the overhead view video 100 outside the vehicle icon 140 when the distance to the obstacle is a range farther than the display range of the overhead view video 100. The superimposed bird's-eye view image 100 is generated and a video signal is output to the display panel 31. When the distance to the obstacle is included in the display range of the overhead image 100, the overhead image generation system 1 generates the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100, and the display panel 31 outputs a video signal.
　上述したように、本実施形態は、障害物までの距離が俯瞰映像１００の表示範囲より遠方の範囲であると、自車アイコン１４０の外側の俯瞰映像１００に、障害物報知アイコン１２０を重畳した俯瞰映像１００を生成し表示パネル３１に表示させる。本実施形態は、障害物までの距離が俯瞰映像１００の表示範囲より遠方の範囲の場合、障害物報知アイコン１２０が自車アイコン１４０の外側に重畳されるので、障害物報知アイコン１２０と俯瞰映像１００に映った障害物とが重畳しない。言い換えると、本実施形態は、俯瞰映像１００に映った障害物の視認性を損なわず、障害物を明瞭に表示することができる。このように、本実施形態は、車両周辺の障害物を適切に確認可能にすることができる。 As described above, in the present embodiment, the obstacle notification icon 120 is superimposed on the bird's-eye view image 100 outside the host vehicle icon 140 when the distance to the obstacle is a range farther than the display range of the bird's-eye view image 100. An overhead image 100 is generated and displayed on the display panel 31. In the present embodiment, when the distance to the obstacle is a range farther than the display range of the overhead view image 100, the obstacle notification icon 120 is superimposed on the outside of the vehicle icon 140, so the obstacle notification icon 120 and the overhead view video are displayed. The obstacle reflected in 100 does not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.
　本実施形態は、障害物までの距離が俯瞰映像１００の表示範囲に含まれると、俯瞰映像１００の中央映像１０５に、障害物報知アイコン１２０を重畳した俯瞰映像１００を生成し表示パネル３１に表示させる。本実施形態は、障害物までの距離が俯瞰映像１００の表示範囲に含まれる場合、障害物報知アイコン１２０が俯瞰映像１００の中央映像１０５に重畳されるので、障害物報知アイコン１２０と俯瞰映像１００に映った障害物とが重畳しない。言い換えると、本実施形態は、俯瞰映像１００に映った障害物の視認性を損なわず、障害物を明瞭に表示することができる。 In the present embodiment, when the distance to the obstacle is included in the display range of the overhead image 100, the overhead image 100 in which the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100 is generated and displayed on the display panel 31. Let In the present embodiment, when the distance to the obstacle is included in the display range of the overhead image 100, the obstacle notification icon 120 is superimposed on the central image 105 of the overhead image 100. The obstacles reflected in the do not overlap. In other words, the present embodiment can clearly display an obstacle without impairing the visibility of the obstacle reflected in the overhead view video 100.
　このように、本実施形態は、障害物までの距離に応じて、車両周辺の障害物を適切に確認可能にすることができる。 As described above, according to the present embodiment, it is possible to appropriately check obstacles around the vehicle according to the distance to the obstacle.
　本実施形態は、俯瞰映像１００の表示範囲より遠方の範囲に障害物がある場合にも、障害物報知アイコン１２０を表示させることができる。このため、本実施形態は、俯瞰映像１００の表示範囲より遠方の範囲の障害物を報知することができる。 This embodiment can display the obstacle notification icon 120 even when there is an obstacle in a range far from the display range of the overhead view video 100. For this reason, this embodiment can notify the obstacle of the range far from the display range of the bird's-eye view image 100.
［第四実施形態］
　本実施形態に係る俯瞰映像生成システム１について説明する。本実施形態の俯瞰映像生成システム１は、俯瞰映像生成装置４０の重畳映像生成部４７と制御部４１における処理とが、第一実施形態の俯瞰映像生成システム１と異なる。より詳しくは、制御部４１における障害物を検出したか否かの判定において、車両の進行方向に位置する障害物を検出する点が第一実施形態の俯瞰映像生成システム１と異なる。 [Fourth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described. The overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in processing in the superimposed video generation unit 47 and the control unit 41 of the overhead video generation device 40. More specifically, it differs from the bird's-eye view image generation system 1 of the first embodiment in that an obstacle located in the traveling direction of the vehicle is detected in determining whether an obstacle is detected in the control unit 41.
　重畳映像生成部４７は、障害物情報取得部４３が取得した障害物情報に基づいて、車両の進行方向に位置する障害物について、障害物報知アイコン１２０を俯瞰映像１００の中央映像１０５に重畳した俯瞰映像１００を生成する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 47 superimposes the obstacle notification icon 120 on the central image 105 of the overhead view video 100 for the obstacle located in the traveling direction of the vehicle. The overhead view video 100 is generated.
　次に、俯瞰映像生成システム１の俯瞰映像生成装置４０における処理の流れについて説明する。 Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described.
　制御部４１は、ステップＳ１３において、障害物情報取得部４３で取得した障害物情報に基づいて、車両の進行方向に障害物が検出されたか否かを判定する。制御部４１は、車両の進行方向に障害物を検出したと判定した場合（ステップＳ１３でＹｅｓ）、ステップＳ１４に進む。制御部４１は、車両の進行方向に障害物が検出されていないと判定した場合（ステップＳ１３でＮｏ）、ステップＳ１５に進む。 In step S13, the control unit 41 determines whether an obstacle is detected in the traveling direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43. When it is determined that the obstacle is detected in the traveling direction of the vehicle (Yes in Step S13), the control unit 41 proceeds to Step S14. When it is determined that no obstacle is detected in the traveling direction of the vehicle (No in step S13), the control unit 41 proceeds to step S15.
　このようにして、制御部４１は、車両に近づく障害物に対して、障害物報知アイコン１２０を表示し、車両から遠ざかる障害物に対して、障害物報知アイコン１２０を非表示とする。 In this way, the control unit 41 displays the obstacle notification icon 120 for obstacles approaching the vehicle, and hides the obstacle notification icon 120 for obstacles moving away from the vehicle.
　上述したように、本実施形態は、車両周辺の障害物をより適切に確認可能にすることができる。 As described above, this embodiment can make it possible to more appropriately check obstacles around the vehicle.
［第五実施形態］
　図９ないし図１２を参照しながら、本実施形態に係る俯瞰映像生成システム１について説明する。図９は、第五実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の一例を示す図である。図１０は、第五実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。図１１は、第五実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。図１２は、障害物報知アイコンにおける、円弧間の間隔と障害物までの距離との関係の一例を示すグラフである。本実施形態の俯瞰映像生成システム１は、俯瞰映像生成装置４０の重畳映像生成部４７における処理が、第一実施形態の俯瞰映像生成システム１と異なる。 [Fifth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. 9 to 12. FIG. 9 is a diagram showing an example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 10 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 11 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the fifth embodiment. FIG. 12 is a graph showing an example of the relationship between the distance between the arcs and the distance to the obstacle in the obstacle notification icon. The overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in the processing in the superimposed video generation unit 47 of the overhead video generation device 40.
　図９ないし図１１に示すように、障害物報知アイコン１２０は、障害物を検出した方向を向いた複数の構成要素を有し、複数の構成要素の間隔によって障害物までの距離を示す。本実施形態では、障害物報知アイコン１２０は、三重の円弧を構成要素としている。本実施形態では、障害物報知アイコン１２０は、円弧の間隔で障害物までの距離を示す。本実施形態では、障害物報知アイコン１２０は、障害物までの距離が大きいほど円弧の間隔を広く、障害物までの距離が小さいほど円弧の間隔を狭く示す。 As shown in FIGS. 9 to 11, the obstacle notification icon 120 has a plurality of components facing in the direction in which the obstacle is detected, and indicates the distance to the obstacle by the interval between the plurality of components. In the present embodiment, the obstacle notification icon 120 includes a triple arc as a constituent element. In the present embodiment, the obstacle notification icon 120 indicates the distance to the obstacle at intervals of arcs. In the present embodiment, the obstacle notification icon 120 indicates that the arc interval is wider as the distance to the obstacle is larger, and the arc interval is narrower as the distance to the obstacle is smaller.
　障害物報知アイコン１２０の円弧の間隔は、図１２において実線で示すように、障害物までの距離が大きいほど広くなるように、直線状に設定されていてもよい。 The interval between the arcs of the obstacle notification icon 120 may be set in a straight line so as to increase as the distance to the obstacle increases as shown by a solid line in FIG.
　障害物報知アイコン１２０の円弧の間隔は、図１２において破線で示すように、障害物までの距離が大きいほど広くなるように、階段状に設定されていてもよい。より詳しくは、障害物までの距離が第一閾値以上の場合、例えば、２ｍ以上の場合、円弧の間隔を第一間隔ｄ１とする。第一閾値は、俯瞰映像１００として表示する範囲に含まれる境界の距理としてもよい。障害物までの距離が第一閾値以上の場合、最も外側の円弧状の曲線を俯瞰映像１００の中央映像１０５の外部に表示してもよい。障害物までの距離が第一閾値未満かつ第二閾値以上の場合、例えば、２ｍ未満１ｍ以上の場合、円弧の間隔を第一間隔ｄ１より小さい第二間隔ｄ２とする。障害物までの距離が第二閾値未満の場合、例えば、１ｍ未満の場合、円弧の間隔を第二間隔ｄ２より小さい第三間隔ｄ３とする。さらに障害物までの距離が小さくなった場合、円弧の間隔をゼロとして円弧を重ねてもよい。 The interval between the arcs of the obstacle notification icon 120 may be set in a staircase shape so as to increase as the distance to the obstacle increases as indicated by a broken line in FIG. More specifically, when the distance to the obstacle is not less than the first threshold, for example, not less than 2 m, the arc interval is set as the first interval d1. The first threshold may be a distance of a boundary included in a range to be displayed as the overhead view video 100. When the distance to the obstacle is equal to or greater than the first threshold, the outermost arc-shaped curve may be displayed outside the central image 105 of the overhead image 100. When the distance to the obstacle is less than the first threshold and greater than or equal to the second threshold, for example, less than 2 m and greater than or equal to 1 m, the arc interval is set to a second interval d2 that is smaller than the first interval d1. When the distance to the obstacle is less than the second threshold, for example, less than 1 m, the arc interval is set to a third interval d3 that is smaller than the second interval d2. Further, when the distance to the obstacle becomes smaller, the arcs may be overlapped with the interval between the arcs set to zero.
　障害物までの距離とは、障害物を検出する各センサから障害物までの距離である。障害物までの距離は、車両の障害物方向の端部から障害物までの距離とほぼ一致する。 The distance to the obstacle is the distance from each sensor that detects the obstacle to the obstacle. The distance to the obstacle substantially coincides with the distance from the end of the vehicle in the obstacle direction to the obstacle.
　重畳映像生成部４７は、障害物情報取得部４３で取得された、検出した障害物までの距離に基づいて、円弧の間隔を変化させた障害物報知アイコン１２０を俯瞰映像１００の中央映像１０５に重畳した俯瞰映像１００を生成する。 The superimposed video generation unit 47 displays the obstacle notification icon 120 in which the arc interval is changed based on the detected distance to the obstacle acquired by the obstacle information acquisition unit 43 in the central video 105 of the overhead view video 100. The superimposed bird's-eye view image 100 is generated.
　車両の後退時、後方左側に障害物を検出した場合の例を、図９ないし図１１を用いて説明する。 An example when an obstacle is detected on the rear left side when the vehicle is reversing will be described with reference to FIGS.
　例えば、車両の後退時、後方左側に障害物を検出し、障害物までの距離が２ｍである場合の一例を、図９を用いて説明する。ステップＳ１４において、制御部４１は、重畳映像生成部４７で、俯瞰映像１００の中央映像１０５に、円弧の膨出方向で障害物を検出した後左方向を示し、円弧の間隔で障害物までの距離を示す障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。障害物報知アイコン１２０は、円弧の間隔が第一間隔ｄ１である。障害物報知アイコン１２０は、最も外側の円弧状の曲線が中央映像１０５の外部に位置する。制御部４１は、重畳映像生成部４７で、図９に示す、生成した俯瞰映像１００を表示パネル３１に表示させる。障害物報知アイコン１２０の円弧の間隔と、最も外側の円弧状の曲線が中央映像１０５の外部に位置していることから、障害物が俯瞰映像１００として表示する範囲の境界付近に位置していることがわかる。 For example, an example in which an obstacle is detected on the rear left side when the vehicle is reversing and the distance to the obstacle is 2 m will be described with reference to FIG. In step S14, the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc. The overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated. In the obstacle notification icon 120, the arc interval is the first interval d1. In the obstacle notification icon 120, the outermost arc-shaped curve is located outside the central image 105. The control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 9 on the display panel 31. Since the arc interval of the obstacle notification icon 120 and the outermost arc-shaped curve are located outside the central image 105, the obstacle is located near the boundary of the range to be displayed as the overhead image 100. I understand that.
　例えば、車両の後退時、後方左側に障害物を検出し、障害物までの距離が１．５ｍである場合の一例を、図１０を用いて説明する。ステップＳ１４において、制御部４１は、重畳映像生成部４７で、俯瞰映像１００の中央映像１０５に、円弧の膨出方向で障害物を検出した後左方向を示し、円弧の間隔で障害物までの距離を示す障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。障害物報知アイコン１２０は、円弧の間隔が第二間隔ｄ２である。制御部４１は、重畳映像生成部４７で、図１０に示す、生成した俯瞰映像１００を表示パネル３１に表示させる。障害物報知アイコン１２０の円弧の間隔から、障害物が俯瞰映像１００として表示する範囲内に位置していることがわかる。 For example, an example in which an obstacle is detected on the rear left side when the vehicle is reversing and the distance to the obstacle is 1.5 m will be described with reference to FIG. In step S14, the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc. The overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated. In the obstacle notification icon 120, the arc interval is the second interval d2. The control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 10 on the display panel 31. It can be seen from the arc interval of the obstacle notification icon 120 that the obstacle is located within the range displayed as the bird's-eye view video 100.
　例えば、車両の後退時、後方左側に障害物を検出し、障害物までの距離が１ｍである場合の一例を、図１１を用いて説明する。ステップＳ１４において、制御部４１は、重畳映像生成部４７で、俯瞰映像１００の中央映像１０５に、円弧の膨出方向で障害物を検出した後左方向を示し、円弧の間隔で障害物までの距離を示す障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。障害物報知アイコン１２０は、円弧の間隔が第三間隔ｄ３である。制御部４１は、重畳映像生成部４７で、図１１に示す、生成した俯瞰映像１００を表示パネル３１に表示させる。障害物報知アイコン１２０の円弧の間隔が狭いことから、障害物が車両の後端部の近傍に位置していることがわかる。 For example, an example in which an obstacle is detected on the rear left side when the vehicle is reversing and the distance to the obstacle is 1 m will be described with reference to FIG. In step S14, the control unit 41 indicates the left direction after detecting the obstacle in the bulge direction of the arc in the central video 105 of the overhead video 100 by the superimposed video generation unit 47, and reaches the obstacle at intervals of the arc. The overhead view image 100 on which the obstacle notification icon 120 indicating the distance is superimposed is generated. In the obstacle notification icon 120, the arc interval is the third interval d3. The control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 11 on the display panel 31. Since the arc interval of the obstacle notification icon 120 is narrow, it can be seen that the obstacle is located in the vicinity of the rear end of the vehicle.
　このようにして、制御部４１は、検出した障害物までの距離に基づいて、円弧の間隔を変化させた障害物報知アイコン１２０を俯瞰映像１００の中央映像１０５に重畳した俯瞰映像１００を生成する。 In this way, the control unit 41 generates the overhead image 100 in which the obstacle notification icon 120 with the arc interval changed is superimposed on the central image 105 of the overhead image 100 based on the detected distance to the obstacle. .
　上述したように、本実施形態は、車両の周辺に障害物が検出されると、俯瞰映像１００の中央映像１０５に、障害物を検出した方向を示し、円弧の間隔で障害物までの距離を示すように障害物報知アイコン１２０を重畳した俯瞰映像１００を表示パネル３１に表示させる。本実施形態は、障害物報知アイコン１２０によって、障害物の方向と、障害物までの距離を表示することができる。このように、本実施形態は、車両周辺の障害物を適切に確認可能にすることができる。 As described above, in the present embodiment, when an obstacle is detected in the vicinity of the vehicle, the center image 105 of the overhead view image 100 indicates the direction in which the obstacle is detected, and the distance to the obstacle is indicated by the arc interval. As shown, the overhead image 100 on which the obstacle notification icon 120 is superimposed is displayed on the display panel 31. In this embodiment, the obstacle notification icon 120 can display the direction of the obstacle and the distance to the obstacle. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.
［第六実施形態］
　図１３ないし図１５を参照しながら、本実施形態に係る俯瞰映像生成システム１について説明する。図１３は、第六実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の一例を示す図である。図１４は、第六実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。図１５は、第六実施形態に係る俯瞰映像生成システムで生成した俯瞰映像の他の例を示す図である。本実施形態の俯瞰映像生成システム１は、俯瞰映像生成装置４０の重畳映像生成部４７における処理が、第一実施形態の俯瞰映像生成システム１と異なる。 [Sixth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIGS. 13 to 15. FIG. 13 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the sixth embodiment. FIG. 14 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment. FIG. 15 is a diagram illustrating another example of an overhead video generated by the overhead video generation system according to the sixth embodiment. The overhead video generation system 1 of the present embodiment is different from the overhead video generation system 1 of the first embodiment in the processing in the superimposed video generation unit 47 of the overhead video generation device 40.
　図１３ないし図１５に示すように、障害物報知アイコン１２０は、アイコン自身の水平方向の幅によって、障害物を検出しているセンサの水平方向における検出範囲と、検出した障害物の水平方向の幅を示す。本実施形態では、障害物報知アイコン１２０は、障害物を検出しているセンサの水平方向における検出範囲が広いほど円弧の長さを大きく、障害物を検出しているセンサの水平方向における検出範囲が狭いほど円弧の長さを小さく示す。本実施形態では、障害物報知アイコン１２０は、障害物の水平方向の幅が広いほど円弧の長さを大きく、障害物の水平方向の幅が狭いほど円弧の長さを小さく示す。 As shown in FIGS. 13 to 15, the obstacle notification icon 120 has a horizontal detection range of the sensor detecting the obstacle and a horizontal direction of the detected obstacle depending on the horizontal width of the icon itself. Indicates the width. In this embodiment, the obstacle notification icon 120 has a larger arc length as the detection range in the horizontal direction of the sensor detecting the obstacle is larger, and the detection range in the horizontal direction of the sensor detecting the obstacle. The narrower is the smaller the arc length. In the present embodiment, the obstacle notification icon 120 indicates that the arc length increases as the obstacle's horizontal width increases, and the arc length decreases as the obstacle's horizontal width decreases.
　より詳しくは、例えば、障害物報知アイコン１２０は、俯瞰映像１００の中央部から障害物の水平方向の両端部までの投影幅を円弧の長さとしてもよい。 More specifically, for example, the obstacle notification icon 120 may have a projected width from the center of the overhead image 100 to both ends of the obstacle in the horizontal direction as the length of the arc.
　重畳映像生成部４７は、障害物情報取得部４３が取得した障害物情報に基づいて、障害物を示す情報の幅を、検出した障害物の水平方向幅に対応させた幅として俯瞰映像１００の中央映像１０５に重畳した俯瞰映像１００を生成する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 47 sets the width of the information indicating the obstacle as a width corresponding to the horizontal width of the detected obstacle. An overhead image 100 superimposed on the central image 105 is generated.
　車両の後退時、後方左側に障害物を検出した場合の例を、図１３ないし図１５を用いて説明する。 An example when an obstacle is detected on the left rear side when the vehicle is reversing will be described with reference to FIGS.
　例えば、車両の後退時、後方左側に障害物を検出し、障害物までの距離が２ｍである場合の一例を、図１３を用いて説明する。ステップＳ１４において、制御部４１は、重畳映像生成部４７で、俯瞰映像１００の中央映像１０５に、アイコン自身の水平方向の幅によって、障害物を検出しているセンサの水平方向における検出範囲と、検出した障害物の水平方向の幅を示す障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。ここでいう障害物の水平方向の幅とは、センサで検出した見かけの幅、つまり投影幅である。制御部４１は、重畳映像生成部４７で、図１３に示す、生成した俯瞰映像１００を表示パネル３１に表示させる。障害物報知アイコン１２０の水平方向の幅によって、障害物を検出しているセンサの水平方向における検出範囲と、検出した障害物の水平方向の幅が狭いことがわかる。 For example, an example in which an obstacle is detected on the rear left side when the vehicle is reversing and the distance to the obstacle is 2 m will be described with reference to FIG. In step S <b> 14, the control unit 41 includes a detection range in the horizontal direction of the sensor that detects the obstacle, based on the horizontal width of the icon itself, in the central video 105 of the overhead video 100 in the superimposed video generation unit 47. An overhead view image 100 on which an obstacle notification icon 120 indicating the horizontal width of the detected obstacle is superimposed is generated. The horizontal width of the obstacle here is the apparent width detected by the sensor, that is, the projected width. The control unit 41 causes the superimposed video generation unit 47 to display the generated overhead image 100 illustrated in FIG. 13 on the display panel 31. It can be seen from the horizontal width of the obstacle notification icon 120 that the detection range in the horizontal direction of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrow.
　例えば、図１３に示す状態から、障害物までの距離が１ｍであった場合の一例を、図１４を用いて説明する。ステップＳ１４において、制御部４１は、重畳映像生成部４７で、俯瞰映像１００の中央映像１０５に、障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。俯瞰映像１００の中央部から障害物の水平方向の両端部までの投影幅が広くなるので、障害物報知アイコン１２０は、水平方向の幅が、図１３の障害物報知アイコン１２０の水平方向の幅より広い。制御部４１は、重畳映像生成部４７で、図１４に示す、生成した俯瞰映像１００を表示パネル３１に表示させる。障害物報知アイコン１２０の水平方向の幅が図１３より広くなっていることによって、障害物を検出しているセンサの水平方向における検出範囲と、検出した障害物の水平方向の幅が広がっている、すなわち、障害物が車両の後端に近づいていることがわかる。 For example, an example where the distance from the state shown in FIG. 13 to the obstacle is 1 m will be described with reference to FIG. In step S <b> 14, the control unit 41 causes the superimposed video generation unit 47 to generate the overhead video 100 in which the obstacle notification icon 120 is superimposed on the central video 105 of the overhead video 100. Since the projected width from the center of the overhead view image 100 to both ends in the horizontal direction of the obstacle is wide, the horizontal width of the obstacle notification icon 120 is the horizontal width of the obstacle notification icon 120 of FIG. Wider. The control unit 41 causes the superimposed video generation unit 47 to display the generated overhead video 100 illustrated in FIG. 14 on the display panel 31. Since the horizontal width of the obstacle notification icon 120 is wider than that in FIG. 13, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are widened. That is, it can be seen that the obstacle is approaching the rear end of the vehicle.
　このように、重畳映像生成部４７は、障害物の近接状況に応じて、障害物報知アイコン１２０の水平方向の幅を、図１３から図１４に示すように変化させて表示させる。障害物報知アイコン１２０の水平方向の幅の変化によって、障害物が車両に近づいているのか遠ざかっているのかがわかる。 As described above, the superimposed video generation unit 47 displays the horizontal width of the obstacle notification icon 120 as shown in FIGS. 13 to 14 in accordance with the proximity state of the obstacle. A change in the width of the obstacle notification icon 120 in the horizontal direction indicates whether the obstacle is approaching or moving away from the vehicle.
　例えば、車両の後退時、後方左側に、図１４に示す障害物より水平方向の幅が狭い障害物を検出した場合の一例を、図１５を用いて説明する。ステップＳ１４において、制御部４１は、重畳映像生成部４７で、俯瞰映像１００の中央映像１０５に、障害物報知アイコン１２０を重畳した俯瞰映像１００を生成させる。障害物報知アイコン１２０は、水平方向の幅が、図１４の障害物報知アイコン１２０の水平方向の幅より狭い。制御部４１は、重畳映像生成部４７で、図１５に示す、生成した俯瞰映像１００を表示パネル３１に表示させる。障害物報知アイコン１２０の水平方向の幅が図１４より狭くなっていることによって、障害物を検出しているセンサの水平方向における検出範囲と、検出した障害物の水平方向の幅が狭くなっている、すなわち、障害物の水平方向の幅が狭いことがわかる。 For example, an example in which an obstacle having a width smaller in the horizontal direction than the obstacle shown in FIG. 14 is detected on the left rear side when the vehicle is reversing will be described with reference to FIG. In step S <b> 14, the control unit 41 causes the superimposed video generation unit 47 to generate the overhead video 100 in which the obstacle notification icon 120 is superimposed on the central video 105 of the overhead video 100. The obstacle notification icon 120 has a horizontal width that is narrower than the horizontal width of the obstacle notification icon 120 of FIG. The control unit 41 causes the superimposed video generation unit 47 to display the generated overhead image 100 illustrated in FIG. 15 on the display panel 31. Since the horizontal width of the obstacle notification icon 120 is narrower than that in FIG. 14, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrowed. That is, it can be seen that the horizontal width of the obstacle is narrow.
　上述したように、本実施形態は、車両の周辺に障害物が検出されると、俯瞰映像１００の中央映像１０５に、障害物を検出した方向を示し、円弧の水平方向の幅で障害物の水平方向の幅を示すように障害物報知アイコン１２０を重畳した俯瞰映像１００を表示パネル３１に表示させる。本実施形態は、障害物報知アイコン１２０によって、障害物の方向と、障害物の水平方向の幅を表示することができる。このように、本実施形態は、車両周辺の障害物を適切に確認可能にすることができる。 As described above, in the present embodiment, when an obstacle is detected around the vehicle, the center image 105 of the bird's-eye view image 100 indicates the direction in which the obstacle is detected, and the obstacle width is indicated by the horizontal width of the arc. An overhead video 100 on which the obstacle notification icon 120 is superimposed is displayed on the display panel 31 so as to indicate the width in the horizontal direction. In the present embodiment, the obstacle notification icon 120 can display the direction of the obstacle and the horizontal width of the obstacle. Thus, this embodiment can make it possible to appropriately check obstacles around the vehicle.
　さらに、本実施形態は、障害物までの距離が変化すると、障害物報知アイコン１２０の水平方向の幅を、例えば、図１３から図１４に示すように変化させて表示させる。これにより、本実施形態によれば、障害物報知アイコン１２０の水平方向の幅の変化で、障害物の近接状況を確認可能にすることができる。 Furthermore, according to the present embodiment, when the distance to the obstacle changes, the horizontal width of the obstacle notification icon 120 is changed and displayed as shown in FIGS. 13 to 14, for example. Thereby, according to this embodiment, it is possible to make it possible to confirm the proximity state of the obstacle by a change in the horizontal width of the obstacle notification icon 120.
［第七実施形態］
　図１６を参照しながら、本実施形態に係る俯瞰映像生成システム１Ａについて説明する。図１６は、第七実施形態に係る俯瞰映像生成システムの構成例を示すブロック図である。俯瞰映像生成システム１Ａは、基本的な構成は第一実施形態の俯瞰映像生成システム１と同様である。 [Seventh embodiment]
The overhead view video generation system 1A according to the present embodiment will be described with reference to FIG. FIG. 16 is a block diagram illustrating a configuration example of the overhead view video generation system according to the seventh embodiment. The overhead image generation system 1A has the same basic configuration as the overhead image generation system 1 of the first embodiment.
　図１６を用いて、俯瞰映像生成システム１Ａについて説明する。俯瞰映像生成システム１Ａは、前方カメラ１１と、後方カメラ１２と、左側方カメラ１３と、右側方カメラ１４と、前方センサ（障害物検出部）２１と、後方センサ（障害物検出部）２２と、左側方センサ（障害物検出部）２３と、右側方センサ（障害物検出部）２４と、表示パネル３１と、俯瞰映像生成装置４０Ａとを有する。 The overhead view video generation system 1A will be described with reference to FIG. The overhead image generation system 1A includes a front camera 11, a rear camera 12, a left side camera 13, a right side camera 14, a front sensor (obstacle detection unit) 21, and a rear sensor (obstacle detection unit) 22. , A left side sensor (obstacle detection unit) 23, a right side sensor (obstacle detection unit) 24, a display panel 31, and an overhead image generation device 40A.
　前方センサ２１は、車両の前方に配置され、車両の前方における障害物を検出する。前方センサ２１は、例えば、赤外線センサまたは超音波センサである。前方センサ２１は、車両と接触するおそれがある、地上から高さを有するものを検出する。前方センサ２１は、例えば、車両から５ｍ程度までの距離の障害物を検出する。前方センサ２１は、鉛直方向視において、センサの中央部を中心とした、例えば、３０°程度の範囲の障害物を検出する。前方センサ２１の検出範囲は、左側方センサ２３および右側方センサ２４の検出範囲の一部と重複していてもよい。前方センサ２１は、複数のセンサの組み合わせで構成されていてもよい。前方センサ２１は、検出した障害物の障害物情報を俯瞰映像生成装置４０Ａの障害物情報取得部４３へ出力する。 The front sensor 21 is disposed in front of the vehicle and detects an obstacle in front of the vehicle. The front sensor 21 is, for example, an infrared sensor or an ultrasonic sensor. The front sensor 21 detects an object having a height from the ground that may come into contact with the vehicle. The front sensor 21 detects an obstacle at a distance of about 5 m from the vehicle, for example. The front sensor 21 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the front sensor 21 may overlap with a part of the detection range of the left side sensor 23 and the right side sensor 24. The front sensor 21 may be configured by a combination of a plurality of sensors. The front sensor 21 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
　後方センサ２２は、車両の後方に配置され、車両の後方における障害物を検出する。後方センサ２２は、例えば、赤外線センサまたは超音波センサである。後方センサ２２は、車両と接触するおそれがある、地上から高さを有するものを検出する。後方センサ２２は、例えば、車両から５ｍ程度までの距離の障害物を検出する。後方センサ２２は、鉛直方向視において、センサの中央部を中心とした、例えば、３０°程度の範囲の障害物を検出する。後方センサ２２の検出範囲は、左側方センサ２３および右側方センサ２４の検出範囲の一部と重複していてもよい。後方センサ２２は、複数のセンサの組み合わせで構成されていてもよい。後方センサ２２は、検出した障害物の障害物情報を俯瞰映像生成装置４０Ａの障害物情報取得部４３へ出力する。 The rear sensor 22 is disposed behind the vehicle and detects an obstacle behind the vehicle. The rear sensor 22 is, for example, an infrared sensor or an ultrasonic sensor. The rear sensor 22 detects an object having a height from the ground that may come into contact with the vehicle. For example, the rear sensor 22 detects an obstacle at a distance of about 5 m from the vehicle. The rear sensor 22 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the rear sensor 22 may overlap with a part of the detection range of the left side sensor 23 and the right side sensor 24. The rear sensor 22 may be configured by a combination of a plurality of sensors. The rear sensor 22 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
　左側方センサ２３は、車両の左側方に配置され、車両の左側方における障害物を検出する。左側方センサ２３は、例えば、赤外線センサまたは超音波センサである。左側方センサ２３は、車両と接触するおそれがある、地上から高さを有するものを検出する。左側方センサ２３は、例えば、車両から５ｍ程度までの距離の障害物を検出する。左側方センサ２３は、鉛直方向視において、センサの中央部を中心とした、例えば、３０°程度の範囲の障害物を検出する。左側方センサ２３の検出範囲は、前方センサ２１および後方センサ２２の検出範囲の一部と重複していてもよい。左側方センサ２３は、複数のセンサの組み合わせで構成されていてもよい。左側方センサ２３は、検出した障害物の障害物情報を俯瞰映像生成装置４０Ａの障害物情報取得部４３へ出力する。 The left side sensor 23 is arranged on the left side of the vehicle and detects an obstacle on the left side of the vehicle. The left side sensor 23 is, for example, an infrared sensor or an ultrasonic sensor. The left side sensor 23 detects an object having a height from the ground that may come into contact with the vehicle. The left side sensor 23 detects an obstacle at a distance of about 5 m from the vehicle, for example. The left side sensor 23 detects an obstacle in a range of, for example, about 30 ° with the center of the sensor as the center when viewed in the vertical direction. The detection range of the left side sensor 23 may overlap with a part of the detection range of the front sensor 21 and the rear sensor 22. The left side sensor 23 may be configured by a combination of a plurality of sensors. The left side sensor 23 outputs the obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead image generation device 40A.
　右側方センサ２４は、車両の右側方に配置され、車両の右側方における障害物を検出する。右側方センサ２４は、例えば、赤外線センサまたは超音波センサである。右側方センサ２４は、車両と接触するおそれがある、地上から高さを有するものを検出する。右側方センサ２４は、例えば、車両から５ｍ程度までの距離の障害物を検出する。右側方センサ２４は、鉛直方向視において、センサの中央部を中心とした、例えば、３０°程度の範囲の障害物を検出する。右側方センサ２４の検出範囲は、前方センサ２１および後方センサ２２の検出範囲の一部と重複していてもよい。右側方センサ２４は、複数のセンサの組み合わせで構成されていてもよい。右側方センサ２４は、検出した障害物の障害物情報を俯瞰映像生成装置４０Ａの障害物情報取得部４３へ出力する。 The right side sensor 24 is arranged on the right side of the vehicle and detects an obstacle on the right side of the vehicle. The right side sensor 24 is, for example, an infrared sensor or an ultrasonic sensor. The right side sensor 24 detects an object having a height from the ground that may come into contact with the vehicle. The right side sensor 24 detects an obstacle at a distance of about 5 m from the vehicle, for example. The right side sensor 24 detects an obstacle in a range of, for example, about 30 ° with the central portion of the sensor as the center when viewed in the vertical direction. The detection range of the right side sensor 24 may overlap with a part of the detection range of the front sensor 21 and the rear sensor 22. The right side sensor 24 may be configured by a combination of a plurality of sensors. The right side sensor 24 outputs obstacle information of the detected obstacle to the obstacle information acquisition unit 43 of the overhead view video generation device 40A.
　表示パネル３１は、横長形状の矩形状である。表示パネル３１には、図１７に示すような、縦長形状の俯瞰映像１００、または、図１８に示すような、横長形状とした俯瞰映像２００が表示される。横長形状とした俯瞰映像２００とは、表示パネル３１の表示面の縦横比に合わせた横長の矩形状の俯瞰映像である。 The display panel 31 has a horizontally long rectangular shape. On the display panel 31, a vertically long overhead view image 100 as shown in FIG. 17 or a horizontally long overhead view image 200 as shown in FIG. 18 is displayed. The overhead view image 200 having a horizontally long shape is a horizontally long overhead view image that matches the aspect ratio of the display surface of the display panel 31.
　制御部４１Ａは、映像取得部４２と、障害物情報取得部４３と、車両情報取得部４４と、映像生成部４５Ａと、表示制御部４８とを有する。 The control unit 41A includes a video acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, a video generation unit 45A, and a display control unit 48.
　障害物情報取得部４３は、前方センサ２１と後方センサ２２と左側方センサ２３と右側方センサ２４とが出力した障害物情報を取得する。 The obstacle information acquisition unit 43 acquires the obstacle information output by the front sensor 21, the rear sensor 22, the left side sensor 23, and the right side sensor 24.
　映像生成部４５Ａは、俯瞰映像１００、俯瞰映像２００を生成する。映像生成部４５Ａは、俯瞰映像生成部４６Ａと、映像合成部４７Ａとを有する。 The video generation unit 45A generates an overhead video 100 and an overhead video 200. The video generation unit 45A includes an overhead video generation unit 46A and a video synthesis unit 47A.
　俯瞰映像生成部４６Ａは、映像取得部４２で取得した周辺映像から車両を上方から見下ろすように視点変換処理を行った俯瞰映像１００を生成する。より詳しくは、俯瞰映像生成部４６Ａは、前方カメラ１１と後方カメラ１２と左側方カメラ１３と右側方カメラ１４とで撮影した周辺映像に基づいて、俯瞰映像１００を生成する。俯瞰映像１００を生成する方法は、公知のいずれの方法でもよく、限定されない。俯瞰映像生成部４６Ａは、生成した俯瞰映像１００を表示制御部４８に出力する。 The overhead video generation unit 46A generates the overhead video 100 in which the viewpoint conversion processing is performed so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. More specifically, the overhead view video generation unit 46A generates the overhead view video 100 based on the peripheral video captured by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the bird's-eye view image 100 may be any known method and is not limited. The overhead view video generation unit 46A outputs the generated overhead view video 100 to the display control unit 48.
　図１７を用いて、俯瞰映像１００について説明する。図１７は、第七実施形態に係る俯瞰映像生成システムで生成した俯瞰映像を表示した表示パネルを示す図である。俯瞰映像１００は、縦長の矩形状である。俯瞰映像１００は、前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４とを含む。前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４とで囲まれた範囲を、俯瞰映像１００の中央部という。俯瞰映像１００の中央部は、俯瞰映像１００における車両の位置である。前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４と、俯瞰映像１００の中央部とは、枠状の境界線１１０で区切られていてもよい。 The overhead view image 100 will be described with reference to FIG. FIG. 17 is a diagram illustrating a display panel that displays an overhead video generated by the overhead video generation system according to the seventh embodiment. The bird's-eye view image 100 has a vertically long rectangular shape. The bird's-eye view image 100 includes a front image 101, a rear image 102, a left side image 103, and a right side image 104. A range surrounded by the front video 101, the rear video 102, the left side video 103, and the right side video 104 is referred to as a central portion of the overhead view video 100. The central part of the overhead image 100 is the position of the vehicle in the overhead image 100. The front video 101, the rear video 102, the left video 103, the right video 104, and the central portion of the overhead video 100 may be separated by a frame-like boundary line 110.
　図１７においては、前方映像１０１と後方映像１０２と左側方映像１０３と右側方映像１０４との境界を示す斜めの破線を説明のために図示しているが、実際に表示パネル３１に表示される俯瞰映像１００には当該破線は表示されても表示されなくともよい。他の図も同様である。 In FIG. 17, diagonal broken lines indicating boundaries between the front video 101, the rear video 102, the left side video 103, and the right side video 104 are illustrated for explanation, but are actually displayed on the display panel 31. The broken line may or may not be displayed in the overhead view video 100. The same applies to the other figures.
　俯瞰映像生成部４６Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、障害物を検出した方向が車両の前方または後方である場合、生成した俯瞰映像１００の表示向きを変えずに横長形状とした俯瞰映像２００を生成する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead image generation unit 46A does not change the display direction of the generated overhead image 100 when the direction in which the obstacle is detected is the front or the rear of the vehicle. The overhead view image 200 having a horizontally long shape is generated.
　表示向きを変えないとは、俯瞰映像１００の縦方向および俯瞰映像２００の縦方向を、車両の前後方向に合せた状態を保つことをいう。言い換えると、表示向きを変えないとは、横長形状とした俯瞰映像２００において、上側に前方映像２０１が配置され、下側に後方映像２０２が配置され、左側に左側方映像２０３が配置され、右側に右側方映像２０４が配置された状態を保つことをいう。表示向きを変えないことで、俯瞰映像２００の縦方向と車両の前後方向とが一致しているため、横長形状とした俯瞰映像２００の認識性を保つことができる。 “The display direction is not changed” means that the vertical direction of the bird's-eye view image 100 and the vertical direction of the bird's-eye view image 200 are kept aligned with the longitudinal direction of the vehicle. In other words, that the display orientation is not changed is that, in the overhead view video 200 having a horizontally long shape, the front video 201 is arranged on the upper side, the rear video 202 is arranged on the lower side, the left side video 203 is arranged on the left side, and the right side In other words, it means that the right side image 204 is maintained. By not changing the display orientation, the vertical direction of the bird's-eye view image 200 matches the longitudinal direction of the vehicle, so that the recognizability of the horizontally-viewed bird's-eye view image 200 can be maintained.
　図１８を用いて、俯瞰映像２００について説明する。図１８は、第七実施形態に係る俯瞰映像生成システムで生成した、横長形状の俯瞰映像を表示した表示パネルを示す図である。俯瞰映像２００は、前方映像２０１と後方映像２０２と左側方映像２０３と右側方映像２０４とを含む。前方映像２０１と後方映像２０２と左側方映像２０３と右側方映像２０４とで囲まれた範囲を、俯瞰映像２００の中央部という。俯瞰映像２００の中央部は、俯瞰映像２００における車両の位置である。前方映像２０１と後方映像２０２と左側方映像２０３と右側方映像２０４と、俯瞰映像２００の中央部とは、枠状の境界線２１０で区切られていてもよい。 The overhead image 200 will be described with reference to FIG. FIG. 18 is a diagram showing a display panel that displays a horizontally long overhead image generated by the overhead image generation system according to the seventh embodiment. The bird's-eye view image 200 includes a front image 201, a rear image 202, a left side image 203, and a right side image 204. A range surrounded by the front video 201, the rear video 202, the left side video 203, and the right side video 204 is referred to as a central portion of the overhead view video 200. The central part of the overhead view image 200 is the position of the vehicle in the overhead view image 200. The front video 201, the rear video 202, the left video 203, the right video 204, and the central portion of the overhead video 200 may be separated by a frame-like boundary line 210.
　図１９、図２０を用いて、俯瞰映像生成部４６Ａで、横長形状とした俯瞰映像２００を生成する方法について説明する。図１９は、第七実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。図２０は、図１９の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。 19 and 20, a method for generating a horizontally-elevated overhead view image 200 by the overhead view image generation unit 46A will be described. FIG. 19 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the seventh embodiment. FIG. 20 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG.
　図１９においては、切出範囲Ａ１と切出範囲Ａ２と切出範囲Ａ３と切出範囲Ａ４とを示す実線を説明のために図示しているが、実際に表示パネル３１に表示される俯瞰映像１００には当該実線は表示されない。他の図も同様である。 In FIG. 19, solid lines indicating the cutout range A1, the cutout range A2, the cutout range A3, and the cutout range A4 are illustrated for explanation, but an overhead view image that is actually displayed on the display panel 31 is illustrated. 100 does not display the solid line. The same applies to the other figures.
　本実施形態では、俯瞰映像生成部４６Ａは、前方カメラ１１または後方カメラ１２からの周辺映像の一部を縦横比を固定して拡大し、横長形状とした俯瞰映像２００を生成する。より詳しくは、俯瞰映像生成部４６Ａは、生成した縦長形状の俯瞰映像１００から、前方映像１０１の一部と後方映像１０２の一部とを縦横比を固定して拡大し、横長形状とした俯瞰映像２００を生成する。 In this embodiment, the bird's-eye view image generation unit 46A generates a bird's-eye view image 200 having a horizontally long shape by expanding a part of the surrounding image from the front camera 11 or the rear camera 12 with a fixed aspect ratio. More specifically, the overhead view video generation unit 46A expands a part of the front image 101 and a part of the rear image 102 from the generated vertically long overhead image 100 with a fixed aspect ratio to obtain a horizontally long shape. A video 200 is generated.
　俯瞰映像生成部４６Ａは、俯瞰映像１００の切出範囲Ａ１と切出範囲Ａ２と切出範囲Ａ３と切出範囲Ａ４とから、横長形状とした俯瞰映像２００を生成する。より詳しくは、俯瞰映像生成部４６Ａは、俯瞰映像１００の切出範囲Ａ１と切出範囲Ａ２との縦横比を固定して拡大し、表示パネル３１の表示面の縦横比に合わせた横長形状とした俯瞰映像２００を生成する。 The overhead view video generation unit 46A generates a horizontally long overhead view video 200 from the cutout range A1, the cutout range A2, the cutout range A3, and the cutout range A4 of the overhead view video 100. More specifically, the overhead view video generation unit 46A expands the aspect ratio of the cutout range A1 and the cutout range A2 of the overhead view video 100 while fixing the aspect ratio to the aspect ratio of the display surface of the display panel 31. The bird's-eye view video 200 is generated.
　切出範囲Ａ１は、運転者が確認を必要とする車両の前方の範囲を含む矩形状の範囲である。切出範囲Ａ１は、俯瞰映像１００の前方映像１０１の一部と、左側方映像１０３の一部と、右側方映像１０４の一部とを含む。切出範囲Ａ１は、前方映像１０１と左側方映像１０３との境界および前方映像１０１と右側方映像１０４との境界を含む。本実施形態では、切出範囲Ａ１は、車両の前端部から数１０ｃｍ程度前方までを含む。本実施形態では、切出範囲Ａ１は、車両の左側端部から数１０ｃｍ程度左側方までを含む。本実施形態では、切出範囲Ａ１は、車両の右側端部から数１０ｃｍ程度右側方までを含む。 The cutout range A1 is a rectangular range including a range in front of the vehicle that the driver needs to check. The cut-out range A1 includes a part of the front image 101 of the overhead image 100, a part of the left side image 103, and a part of the right side image 104. The cut-out range A1 includes a boundary between the front video 101 and the left side video 103 and a boundary between the front video 101 and the right side video 104. In the present embodiment, the cut-out range A1 includes from the front end of the vehicle to about a few tens of centimeters forward. In the present embodiment, the cutout range A1 includes from the left end of the vehicle to the left side of about several tens of cm. In the present embodiment, the cutout range A1 includes from the right end of the vehicle to the right side of about several tens of centimeters.
　切出範囲Ａ２は、運転者が確認を必要とする車両の後方の範囲を含む矩形状の範囲である。切出範囲Ａ２は、俯瞰映像１００の後方映像１０２の一部と、左側方映像１０３の一部と、右側方映像１０４の一部とを含む。切出範囲Ａ２は、後方映像１０２と左側方映像１０３との境界および後方映像１０２と右側方映像１０４との境界を含む。本実施形態では、切出範囲Ａ２は、車両の後端部から数１０ｃｍ程度後方までを含む。本実施形態では、切出範囲Ａ２は、車両の左側端部から数１０ｃｍ程度左側方までを含む。本実施形態では、切出範囲Ａ２は、車両の右側端部から数１０ｃｍ程度右側方までを含む。本実施形態では、切出範囲Ａ２は、切出範囲Ａ１と同一形状、同一面積を有する。 The cut-out range A2 is a rectangular range including a range behind the vehicle that the driver needs to check. The cutout range A <b> 2 includes a part of the rear image 102 of the overhead image 100, a part of the left side image 103, and a part of the right side image 104. The cutout range A <b> 2 includes a boundary between the rear image 102 and the left side image 103 and a boundary between the rear image 102 and the right side image 104. In the present embodiment, the cut-out range A2 includes from the rear end of the vehicle to the rear of about several tens of centimeters. In the present embodiment, the cutout range A2 includes from the left end of the vehicle to the left side of about several tens of cm. In the present embodiment, the cutout range A2 includes from the right end of the vehicle to the right side of about several tens of centimeters. In the present embodiment, the cutout range A2 has the same shape and the same area as the cutout range A1.
　切出範囲Ａ３は、俯瞰映像１００の切出範囲Ａ１と切出範囲Ａ２との間に位置する矩形状の範囲である。本実施形態では、切出範囲Ａ３は、車両の左側端部から数１０ｃｍ程度左側方までを含む。切出範囲Ａ３の左側端部は、切出範囲Ａ１の左側端部と切出範囲Ａ２の左側端部と同一直線上に位置する。 The cutout area A3 is a rectangular area located between the cutout area A1 and the cutout area A2 of the overhead view image 100. In the present embodiment, the cutout range A3 includes from the left end of the vehicle to the left side of about several tens of cm. The left end portion of the cutout range A3 is positioned on the same straight line as the left end portion of the cutout range A1 and the left end portion of the cutout range A2.
　切出範囲Ａ４は、俯瞰映像１００の切出範囲Ａ１と切出範囲Ａ２との間に位置する矩形状の範囲である。本実施形態では、切出範囲Ａ４は、車両の右側端部から数１０ｃｍ程度右側方までを含む。切出範囲Ａ４の右側端部は、切出範囲Ａ１の右側端部と切出範囲Ａ２の右側端部と同一直線上に位置する。本実施形態では、切出範囲Ａ４は、切出範囲Ａ３と同一形状、同一面積を有する。 The cutout range A4 is a rectangular range located between the cutout range A1 and the cutout range A2 of the overhead view image 100. In the present embodiment, the cutout range A4 includes from the right end of the vehicle to the right side of about several tens of centimeters. The right end of the cutout range A4 is located on the same straight line as the right end of the cutout range A1 and the right end of the cutout range A2. In the present embodiment, the cutout range A4 has the same shape and the same area as the cutout range A3.
　このような俯瞰映像１００から生成された、横長形状とした俯瞰映像２００は、切出範囲Ａ１の映像２１１と、切出範囲Ａ２の映像２１２と、切出範囲Ａ３の映像２１３と、切出範囲Ａ４の映像２１４とを含む。 A horizontally long overhead image 200 generated from such a bird's-eye view image 100 includes a cut-out range A1 image 211, a cut-out range A2 image 212, a cut-out range A3 image 213, and a cut-out range. A4 video 214.
　映像２１１は、俯瞰映像１００の切出範囲Ａ１の映像を、縦横比を固定し、横幅が表示パネル３１の横幅と同じ幅になるまで拡大した映像である。図２０において、映像２１１に表示された樹木の映像は、図１９に示す俯瞰映像１００の切出範囲Ａ１に表示された樹木の映像を、縦横比を固定し拡大した映像になっている。 The image 211 is an image obtained by enlarging the image of the cutout range A1 of the overhead view image 100 until the aspect ratio is fixed and the width is the same as the width of the display panel 31. In FIG. 20, the image of the tree displayed in the image 211 is an image obtained by fixing and expanding the image of the tree displayed in the cutout area A1 of the overhead image 100 shown in FIG.
　映像２１２は、俯瞰映像１００の切出範囲Ａ２の映像を、縦横比を固定し、横幅が表示パネル３１の横幅と同じ幅になるまで拡大した映像である。図２０において、映像２１２に表示された樹木の映像は、図１９に示す俯瞰映像１００の切出範囲Ａ２に表示された樹木の映像を、縦横比を固定し拡大した映像になっている。 The video 212 is a video obtained by enlarging the video of the cutout range A2 of the overhead video 100 until the aspect ratio is fixed and the horizontal width becomes the same as the horizontal width of the display panel 31. In FIG. 20, the image of the tree displayed in the image 212 is an image obtained by expanding and fixing the aspect ratio of the image of the tree displayed in the cutout range A2 of the overhead image 100 shown in FIG.
　映像２１３は、俯瞰映像１００の切出範囲Ａ３の映像を、映像２１１と映像２１２との境界が滑らかに連続するように、表示パネル３１に合わせて大きさと形状とを変えた映像である。例えば、映像２１３は、俯瞰映像１００の切出範囲Ａ３の映像を、縦方向に圧縮し、横方向に拡大した映像でもよい。 The video 213 is a video in which the size and shape of the video in the cutout range A3 of the overhead video 100 are changed in accordance with the display panel 31 so that the boundary between the video 211 and the video 212 is smoothly continuous. For example, the video 213 may be a video obtained by compressing the video in the cutout range A3 of the overhead video 100 in the vertical direction and expanding in the horizontal direction.
　映像２１４は、俯瞰映像１００の切出範囲Ａ４の映像を、映像２１１と映像２１２との境界が滑らかに連続するように、表示パネル３１に合わせて大きさと形状とを変えた映像である。例えば、映像２１４は、俯瞰映像１００の切出範囲Ａ４の映像を、縦方向に圧縮し、横方向に拡大した映像でもよい。 The video 214 is a video in which the size and shape of the video in the cutout range A4 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 211 and the video 212 is smoothly continuous. For example, the video 214 may be a video obtained by compressing the video in the cutout range A4 of the overhead video 100 and vertically expanding the video.
　俯瞰映像２００は、前方映像２０１と後方映像２０２と左側方映像２０３と右側方映像２０４とを含む。前方映像２０１は、映像２１１の一部である。後方映像２０２は、映像２１２の一部である。左側方映像２０３は、映像２１１の左下部と映像２１２の左上部と映像２１３とを含む。右側方映像２０４は、映像２１１の右下部と映像２１２の右上部と映像２１４とを含む。 The bird's-eye view image 200 includes a front image 201, a rear image 202, a left side image 203, and a right side image 204. The front image 201 is a part of the image 211. The rear video 202 is a part of the video 212. The left side image 203 includes a lower left portion of the image 211, an upper left portion of the image 212, and an image 213. The right side video 204 includes a lower right portion of the video 211, an upper right portion of the video 212, and a video 214.
　映像合成部４７Ａは、俯瞰映像生成部４６Ａで生成した俯瞰映像１００、俯瞰映像２００の中央部に、障害物情報取得部４３が障害物を検出した方向に対応する、前方カメラ１１または後方カメラ１２または左側方カメラ１３または右側方カメラ１４からの周辺映像を合成した俯瞰映像１００、俯瞰映像２００を生成する。合成する前方カメラ１１または後方カメラ１２または左側方カメラ１３または右側方カメラ１４からの周辺映像は、視点変換前の映像である。映像合成部４７Ａは、生成した俯瞰映像１００、俯瞰映像２００を表示制御部４８に出力する。 The video synthesizing unit 47A is located at the center of the overhead view video 100 and the overhead view video 200 generated by the overhead view video generation unit 46A, and corresponds to the direction in which the obstacle information acquisition unit 43 detects the obstacle, the front camera 11 or the rear camera 12. Alternatively, the bird's-eye view video 100 and the bird's-eye view image 200 are generated by synthesizing the peripheral images from the left side camera 13 or the right side camera 14. The peripheral video from the front camera 11, the rear camera 12, the left side camera 13, or the right side camera 14 to be synthesized is a video before viewpoint conversion. The video composition unit 47A outputs the generated overhead view video 100 and the overhead view video 200 to the display control unit 48.
　本実施形態では、映像合成部４７Ａは、障害物情報取得部４３が障害物を検出した方向が車両の左方向または右方向である場合、俯瞰映像１００の中央部に、障害物を検出した方向に対応する左側方カメラ１３または右側方カメラ１４からの縦長形状の周辺映像を合成した俯瞰映像１００を生成する。俯瞰映像１００の中央部に合成される周辺映像は、視点変換処理が行われていない周辺映像である。映像合成部４７Ａは、俯瞰映像１００の境界線１１０の、障害物を検出した方向の線を太くしたり色を変えたりして表示してもよい。 In the present embodiment, when the direction in which the obstacle information acquisition unit 43 detects the obstacle is the left direction or the right direction of the vehicle, the video composition unit 47A detects the obstacle in the central portion of the overhead image 100. The overhead view image 100 is generated by combining the vertically long peripheral images from the left side camera 13 or the right side camera 14 corresponding to. The peripheral video synthesized at the center of the overhead view video 100 is a peripheral video that has not been subjected to viewpoint conversion processing. The video compositing unit 47A may display the boundary line 110 of the overhead view video 100 by thickening or changing the color of the direction in which the obstacle is detected.
　例えば、映像合成部４７Ａは、障害物を検出した方向が左方向である場合、図１７に示すように、俯瞰映像１００の中央部に、左側方カメラ１３からの縦長形状の周辺映像１０３Ａを合成する。俯瞰映像１００の後方映像１０２と周辺映像１０３Ａとには、後方の障害物の障害物映像３００が含まれている。俯瞰映像１００の左側の境界線１１０Ａは、太線である。 For example, when the direction in which the obstacle is detected is the left direction, the video synthesis unit 47A synthesizes the vertically long peripheral video 103A from the left side camera 13 in the center of the overhead view video 100 as shown in FIG. To do. The rear image 102 and the surrounding image 103A of the overhead image 100 include an obstacle image 300 of a rear obstacle. A boundary line 110A on the left side of the overhead view image 100 is a thick line.
　本実施形態では、映像合成部４７Ａは、障害物情報取得部４３が障害物を検出した方向が車両の前方または後方である場合、横長形状とした俯瞰映像２００の中央部に、障害物を検出した方向に対応する前方カメラ１１または後方カメラ１２からの横長形状の周辺映像を合成した俯瞰映像２００を生成する。俯瞰映像２００の中央部に合成される周辺映像は、視点変換処理が行われていない周辺映像である。映像合成部４７Ａは、俯瞰映像２００の境界線２１０の、障害物を検出した方向の線を太くしたり色を変えたりして表示してもよい。 In the present embodiment, the video composition unit 47A detects an obstacle at the center of the horizontally-viewed overhead image 200 when the direction in which the obstacle information acquisition unit 43 detects the obstacle is the front or the rear of the vehicle. An overhead view image 200 is generated by combining horizontally long peripheral images from the front camera 11 or the rear camera 12 corresponding to the determined direction. The peripheral video synthesized in the central portion of the overhead video 200 is a peripheral video that has not been subjected to the viewpoint conversion process. The video compositing unit 47A may display the boundary line 210 of the overhead view video 200 by thickening or changing the color of the direction in which the obstacle is detected.
　例えば、映像合成部４７Ａは、障害物を検出した方向が後方である場合、図１８に示すように、俯瞰映像２００の中央部に、後方カメラ１２からの横長形状の周辺映像２０２Ａを合成する。俯瞰映像２００の後方映像２０２と周辺映像２０２Ａとには、後方の障害物の障害物映像３００が含まれている。俯瞰映像２００の下側の境界線２１０Ａは、太線である。 For example, when the direction in which the obstacle is detected is backward, the video composition unit 47A synthesizes the horizontally long peripheral image 202A from the rear camera 12 in the center of the overhead view image 200 as shown in FIG. The rear image 202 and the surrounding image 202A of the overhead image 200 include an obstacle image 300 of a rear obstacle. A lower boundary line 210A of the bird's-eye view image 200 is a thick line.
　次に、図２１を用いて、俯瞰映像生成システム１Ａの俯瞰映像生成装置４０Ａにおける処理の流れについて説明する。図２１は、第七実施形態に係る俯瞰映像生成システムの俯瞰映像生成装置における処理の流れを示すフローチャートである。 Next, the flow of processing in the overhead video generation device 40A of the overhead video generation system 1A will be described with reference to FIG. FIG. 21 is a flowchart showing a flow of processing in the overhead video generation device of the overhead video generation system according to the seventh embodiment.
　制御部４１Ａは、俯瞰映像表示を開始するか否かを判定する（ステップＳＴ１１）。本実施形態では、制御部４１Ａは、後退トリガの有無に基づいて、俯瞰映像表示を開始するか否かを判定する。後退トリガとは、例えば、シフトポジションが「リバース」とされたことをいう。または、後退トリガとは、車両の進行方向が車両の後方となったことをいう。制御部４１Ａは、後退トリガがない場合、俯瞰映像表示を開始しないと判定し（ステップＳＴ１１でＮｏ）、ステップＳＴ１１の処理を再度実行する。制御部４１Ａは、後退トリガがある場合、俯瞰映像表示を開始すると判定し（ステップＳＴ１１でＹｅｓ）、ステップＳＴ１２に進む。 Control unit 41A determines whether or not to start bird's-eye view video display (step ST11). In the present embodiment, the control unit 41A determines whether to start the overhead view video display based on the presence / absence of the backward trigger. The reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle is the rear of the vehicle. If there is no reverse trigger, control unit 41A determines not to start the overhead view video display (No in step ST11), and executes the process of step ST11 again. If there is a reverse trigger, control unit 41A determines to start the overhead view video display (Yes in step ST11), and proceeds to step ST12.
　制御部４１Ａは、俯瞰映像１００を生成し表示する（ステップＳＴ１２）。より詳しくは、制御部４１Ａは、俯瞰映像生成部４６Ａで、映像取得部４２で取得した周辺映像から車両を上方から見下ろすように視点変換処理を行った俯瞰映像１００を生成させる。そして、制御部４１Ａは、表示制御部４８で、生成した俯瞰映像１００を表示パネル３１に表示させる。 The control unit 41A generates and displays the overhead view video 100 (step ST12). More specifically, the control unit 41A causes the overhead video generation unit 46A to generate the overhead video 100 that has been subjected to the viewpoint conversion processing so that the vehicle is looked down from above from the peripheral video acquired by the video acquisition unit 42. Then, the control unit 41A causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
　制御部４１Ａは、障害物を検出したか否かを判定する（ステップＳＴ１３）。より詳しくは、制御部４１Ａは、障害物情報取得部４３で障害物情報が取得されたか否かを判定する。制御部４１Ａは、障害物情報取得部４３で障害物情報が取得されたと判定した場合（ステップＳＴ１３でＹｅｓ）、ステップＳＴ１４に進む。制御部４１Ａは、障害物情報取得部４３で障害物情報が取得されていないと判定した場合（ステップＳＴ１３でＮｏ）、ステップＳＴ１７に進む。 Control unit 41A determines whether an obstacle has been detected (step ST13). More specifically, the control unit 41A determines whether or not obstacle information has been acquired by the obstacle information acquisition unit 43. If the control unit 41A determines that the obstacle information acquisition unit 43 has acquired the obstacle information (Yes in step ST13), the control unit 41A proceeds to step ST14. If the controller 41A determines that the obstacle information acquisition unit 43 has not acquired the obstacle information (No in step ST13), the control unit 41A proceeds to step ST17.
　本実施形態では、制御部４１Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、車両の進行方向に位置する障害物を検出したか否かを判定する。車両の進行方向に位置するとは、車両の進行につれて車両までの距離が減少する方向に位置することをいい、車両の進行につれて車両に障害物が干渉する範囲を含む。具体的には、制御部４１Ａは、車両の進行につれて車両が進行方向に直進している場合は、車両の進行方向を中心とした範囲の障害物の検出の有無を判定し、車両が操舵しながら進行している場合は、車両の進行方向および操舵によって障害物が車両に干渉する範囲を含めた範囲の障害物の検出の有無を判定する。 In the present embodiment, the control unit 41A determines whether an obstacle located in the traveling direction of the vehicle is detected based on the obstacle information acquired by the obstacle information acquisition unit 43. Positioning in the traveling direction of the vehicle means positioning in a direction in which the distance to the vehicle decreases as the vehicle travels, and includes a range in which an obstacle interferes with the vehicle as the vehicle travels. Specifically, when the vehicle is traveling straight in the traveling direction as the vehicle travels, the control unit 41A determines whether or not an obstacle within a range centered on the traveling direction of the vehicle is detected, and the vehicle steers. When the vehicle is traveling, it is determined whether or not obstacles are detected in a range including the range in which the obstacle interferes with the vehicle by the traveling direction and steering of the vehicle.
　制御部４１Ａは、障害物の検出方向が左方向または右方向であるか否かを判定する（ステップＳＴ１４）。本実施形態では、制御部４１Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、車両までの距離が最小の障害物の検出方向が車両の左方向または右方向であるか否かを判定する。より詳しくは、制御部４１Ａは、障害物情報取得部４３で取得した車両までの距離が最小の障害物の障害物情報が、左側方センサ２３または右側方センサ２４からの障害物情報であるか否かを判定する。 The control unit 41A determines whether the obstacle detection direction is the left direction or the right direction (step ST14). In the present embodiment, the control unit 41A determines whether the detection direction of the obstacle with the shortest distance to the vehicle is the left direction or the right direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43. Determine whether. More specifically, the control unit 41A determines whether the obstacle information of the obstacle with the minimum distance to the vehicle acquired by the obstacle information acquisition unit 43 is the obstacle information from the left side sensor 23 or the right side sensor 24. Determine whether or not.
　制御部４１Ａは、車両までの距離が最小の障害物の障害物情報が、左側方センサ２３または右側方センサ２４からの障害物情報であると判定した場合（ステップＳＴ１４でＹｅｓ）、ステップＳＴ１５に進む。制御部４１Ａは、車両までの距離が最小の障害物の障害物情報が、左側方センサ２３または右側方センサ２４からの障害物情報ではないと判定した場合（ステップＳＴ１４でＮｏ）、ステップＳＴ１６に進む。 When it is determined that the obstacle information of the obstacle with the shortest distance to the vehicle is the obstacle information from the left side sensor 23 or the right side sensor 24 (Yes in step ST14), the control unit 41A proceeds to step ST15. move on. If the controller 41A determines that the obstacle information of the obstacle with the shortest distance to the vehicle is not the obstacle information from the left side sensor 23 or the right side sensor 24 (No in step ST14), the control unit 41A goes to step ST16. move on.
　制御部４１Ａは、俯瞰映像１００の中央部に障害物検出方向の周辺映像を表示した俯瞰映像１００を生成する（ステップＳＴ１５）。より詳しくは、制御部４１は、映像合成部４７Ａで、俯瞰映像生成部４６Ａが生成した俯瞰映像１００の中央部に、車両までの距離が最小の障害物を検出した方向に対応する、左側方カメラ１３または右側方カメラ１４からの縦長形状の周辺映像を合成した俯瞰映像１００を生成させる。そして、制御部４１Ａは、表示制御部４８で、生成した俯瞰映像１００を表示パネル３１に表示させる。 The control unit 41A generates the bird's-eye view image 100 in which the peripheral image in the obstacle detection direction is displayed at the center of the bird's-eye view image 100 (step ST15). More specifically, the control unit 41 corresponds to the direction in which the video composition unit 47A detects the obstacle with the shortest distance to the vehicle at the center of the overhead image 100 generated by the overhead image generation unit 46A. An overhead view image 100 is generated by synthesizing a vertically long peripheral image from the camera 13 or the right side camera 14. Then, the control unit 41A causes the display control unit 48 to display the generated overhead image 100 on the display panel 31.
　制御部４１Ａは、横長形状とした俯瞰映像２００の中央部に障害物検出方向の周辺映像を表示した俯瞰映像２００を生成する（ステップＳＴ１６）。より詳しくは、制御部４１Ａは、俯瞰映像生成部４６Ａで、ステップＳＴ１２で生成した俯瞰映像１００の表示向きを変えずに横長形状とした俯瞰映像２００を生成させる。そして、制御部４１Ａは、映像合成部４７Ａで、横長形状とした俯瞰映像２００の中央部に、車両までの距離が最小の障害物を検出した方向に対応する前方カメラ１１または後方カメラ１２からの横長形状の周辺映像を合成した俯瞰映像２００を生成させる。そして、制御部４１Ａは、表示制御部４８で、生成した横長形状とした俯瞰映像２００を表示パネル３１に表示させる。 The control unit 41A generates a bird's-eye view image 200 in which a peripheral image in the obstacle detection direction is displayed at the center of the horizontally long bird's-eye view image 200 (step ST16). More specifically, the control unit 41A causes the overhead view video generation unit 46A to generate the overhead view video 200 having a horizontally long shape without changing the display direction of the overhead view video 100 generated in step ST12. Then, the control unit 41A receives from the front camera 11 or the rear camera 12 corresponding to the direction in which the obstacle with the shortest distance to the vehicle is detected at the center of the horizontally long overhead image 200 in the video composition unit 47A. A bird's-eye view image 200 obtained by synthesizing a horizontally long peripheral image is generated. Then, the control unit 41 </ b> A causes the display control unit 48 to cause the display panel 31 to display the generated overhead image 200 having a horizontally long shape.
　制御部４１Ａは、俯瞰映像表示を継続するか否かを判定する（ステップＳＴ１７）。より詳しくは、制御部４１Ａは、後退トリガの有無に基づいて、俯瞰映像表示を継続するか否かを判定する。制御部４１Ａは、後退トリガがある場合、俯瞰映像表示を継続すると判定し（ステップＳＴ１７でＹｅｓ）、ステップＳＴ１３に戻って処理を継続する。制御部４１Ａは、後退トリガがない場合、言い換えると、後退トリガが解除された場合、俯瞰映像表示を継続しないと判定し（ステップＳＴ１７でＮｏ）、処理を終了する。 Control unit 41A determines whether or not to continue the overhead video display (step ST17). More specifically, the control unit 41A determines whether or not to continue the overhead view video display based on the presence / absence of the backward trigger. If there is a reverse trigger, control unit 41A determines to continue the overhead view video display (Yes in step ST17), and returns to step ST13 to continue the process. When there is no reverse trigger, in other words, when the reverse trigger is released, control unit 41A determines that the overhead view video display is not continued (No in step ST17), and ends the process.
　このようにして、俯瞰映像生成システム１Ａは、車両の左方向または右方向に障害物が検出されると、俯瞰映像１００の中央部に、障害物を検出した方向に対応する左側方カメラ１３または右側方カメラ１４からの縦長形状の周辺映像を合成した俯瞰映像１００を生成する。 Thus, when an obstacle is detected in the left direction or the right direction of the vehicle, the overhead view video generation system 1A has a left-side camera 13 corresponding to the direction in which the obstacle is detected in the center of the overhead view video 100 or A bird's-eye view image 100 is generated by synthesizing a vertically long peripheral image from the right-side camera 14.
　俯瞰映像生成システム１Ａは、車両の前方向または後方向に障害物が検出されると、横長形状とした俯瞰映像２００の中央部に、障害物を検出した方向に対応する前方カメラ１１または後方カメラ１２からの横長形状の周辺映像を合成した俯瞰映像２００を生成する。 When an obstacle is detected in the front or rear direction of the vehicle, the overhead view video generation system 1A has a front camera 11 or a rear camera corresponding to the direction in which the obstacle is detected at the center of the horizontally overhead view 200. 12 generates a bird's-eye view image 200 obtained by synthesizing a horizontally long peripheral image from 12.
　上述したように、本実施形態は、俯瞰映像１００、俯瞰映像２００の中央部に、障害物を検出した方向に対応する周辺映像を合成した俯瞰映像１００、俯瞰映像２００を生成する。これにより、本実施形態は、大きな視線の移動をせずに、俯瞰映像１００、俯瞰映像２００とともに、障害物を検出した方向の周辺映像も確認することができる。このように、本実施形態は、運転者の視線の移動を抑制し、車両周辺の障害物を適切に確認可能にすることができる。 As described above, in the present embodiment, the bird's-eye view video 100 and the bird's-eye view image 200 are generated by synthesizing the surrounding images corresponding to the direction in which the obstacle is detected in the center of the bird's-eye view image 100 and the bird's-eye view image 200. Thereby, this embodiment can also confirm the surrounding image of the direction which detected the obstacle with the bird's-eye view image 100 and the bird's-eye view image 200, without moving a big eyes | visual_axis. As described above, the present embodiment can suppress the movement of the driver's line of sight and appropriately check obstacles around the vehicle.
　本実施形態は、車両の左方向または右方向に障害物が検出されると、俯瞰映像１００の中央部に、障害物を検出した方向に対応する左側方カメラ１３または右側方カメラ１４からの縦長形状の周辺映像を合成した俯瞰映像１００を生成させる。本実施形態は、車両の前方向または後方向に障害物が検出されると、横長形状とした俯瞰映像２００の中央部に、障害物を検出した方向に対応する前方カメラ１１または後方カメラ１２からの周辺映像を合成した俯瞰映像２００を生成する。このように、本実施形態は、障害物の検出方向によらず、俯瞰映像１００の縦方向および俯瞰映像２００の縦方向と、車両の前後方向とを合せた状態を保つことができる。このため、本実施形態によれば、運転者が俯瞰映像１００または俯瞰映像２００を確認した際に、車両の周辺を容易に確認することができる。 In this embodiment, when an obstacle is detected in the left direction or the right direction of the vehicle, a vertically long image from the left side camera 13 or the right side camera 14 corresponding to the direction in which the obstacle is detected is displayed at the center of the overhead view image 100. A bird's-eye view image 100 is generated by synthesizing the surrounding image of the shape. In the present embodiment, when an obstacle is detected in the front direction or the rear direction of the vehicle, the front camera 11 or the rear camera 12 corresponding to the direction in which the obstacle is detected is displayed at the center of the horizontally-viewed overhead view image 200. A bird's-eye view video 200 is generated by synthesizing the surrounding video. Thus, this embodiment can maintain the state in which the vertical direction of the bird's-eye view image 100 and the vertical direction of the bird's-eye view image 200 are combined with the longitudinal direction of the vehicle, regardless of the detection direction of the obstacle. For this reason, according to this embodiment, when the driver confirms the bird's-eye view image 100 or the bird's-eye view image 200, the periphery of the vehicle can be easily confirmed.
　本実施形態は、車両の前方向または後方向に障害物が検出されると、横長形状とした俯瞰映像２００の中央部に、障害物を検出した方向に対応する周辺映像を合成した俯瞰映像２００を生成し、表示パネル３１に表示する。本実施形態は、横長形状とした俯瞰映像２００を、表示パネル３１の表示面の全面に表示することができる。これにより、本実施形態は、車両の前方向または後方向に障害物が検出された場合、俯瞰映像２００の中央部に、周辺映像を大きく表示することができる。 In the present embodiment, when an obstacle is detected in the front or rear direction of the vehicle, the overhead image 200 is obtained by synthesizing a peripheral image corresponding to the direction in which the obstacle is detected in the center of the horizontally overhead bird's eye image 200. Is generated and displayed on the display panel 31. In the present embodiment, the bird's-eye view image 200 having a horizontally long shape can be displayed on the entire display surface of the display panel 31. Thereby, this embodiment can display a surrounding image largely in the center part of the bird's-eye view image 200, when an obstacle is detected in the front direction or the rear direction of the vehicle.
　本実施形態は、俯瞰映像２００の映像２１１は、俯瞰映像１００の切出範囲Ａ１の映像と縦横比が同じである。言い換えると、俯瞰映像２００の前方映像２０１は、俯瞰映像１００の切出範囲Ａ１の映像と縦横比が同じである。俯瞰映像２００の映像２１２は、俯瞰映像１００の切出範囲Ａ２の映像と縦横比が同じである。言い換えると、俯瞰映像２００の後方映像２０２は、俯瞰映像１００の切出範囲Ａ２の映像と縦横比が同じである。このように、本実施形態は、俯瞰映像２００の前方映像２０１と後方映像２０２とは、縦横比が変わらず、変形されていない。このため、本実施形態によれば、運転者が俯瞰映像２００の前方映像２０１と後方映像２０２とを違和感なく確認することができる。 In this embodiment, the video 211 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A1 of the overhead view video 100. In other words, the front image 201 of the overhead image 200 has the same aspect ratio as the image of the cutout range A1 of the overhead image 100. The video 212 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A2 of the overhead view video 100. In other words, the rear video 202 of the overhead view video 200 has the same aspect ratio as the video of the cutout range A2 of the overhead view video 100. Thus, in the present embodiment, the front video 201 and the rear video 202 of the overhead view video 200 have the same aspect ratio and are not deformed. For this reason, according to the present embodiment, the driver can check the front video 201 and the rear video 202 of the bird's-eye view video 200 without a sense of incongruity.
［第八実施形態］
　図２２、図２３を参照しながら、本実施形態に係る俯瞰映像生成システム１Ａについて説明する。図２２は、第八実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。図２３は、図２２の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。俯瞰映像生成システム１Ａは、基本的な構成は第七実施形態の俯瞰映像生成システム１Ａと同様である。本実施形態の俯瞰映像生成システム１Ａは、俯瞰映像生成部４６Ａで横長形状とした俯瞰映像２００を生成する方法が、第七実施形態の俯瞰映像生成システム１Ａと異なる。 [Eighth embodiment]
With reference to FIGS. 22 and 23, a bird's-eye view video generation system 1A according to the present embodiment will be described. FIG. 22 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eighth embodiment. FIG. 23 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. The overhead image generation system 1A has the same basic configuration as the overhead image generation system 1A of the seventh embodiment. The overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the seventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
　俯瞰映像生成部４６Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、障害物を検出した方向が車両の前方または後方である場合、前方カメラ１１または後方カメラ１２からの周辺映像を縦方向に圧縮し、横長形状とした俯瞰映像２００を生成する。より詳しくは、俯瞰映像生成部４６Ａは、生成した縦長形状の俯瞰映像１００から、前方映像１０１と後方映像１０２とを縦方向に圧縮し、横長形状とした俯瞰映像２００を生成する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead view video generation unit 46A, when the direction in which the obstacle is detected is the front or the rear of the vehicle, the peripheral video from the front camera 11 or the rear camera 12 Is vertically compressed to generate a bird's-eye view image 200 having a horizontally long shape. More specifically, the overhead view video generation unit 46A compresses the front video 101 and the rear video 102 in the vertical direction from the vertically elongated overhead view video 100 that has been generated, and generates the overhead view video 200 having a horizontally long shape.
　図２２に示す切出範囲Ａ１は、俯瞰映像１００の前方映像１０１の縦方向の幅と同じ縦方向の幅を有する。切出範囲Ａ１は、前方映像１０１と左側方映像１０３との境界および前方映像１０１と右側方映像１０４との境界を含む。切出範囲Ａ１は、車両の左側端部から数１０ｃｍ程度左側方までを含む。切出範囲Ａ１は、車両の右側端部から数１０ｃｍ程度右側方までを含む。 22 has a vertical width that is the same as the vertical width of the front video 101 of the overhead view video 100. The cut-out range A1 includes a boundary between the front video 101 and the left side video 103 and a boundary between the front video 101 and the right side video 104. The cut-out range A1 includes from the left end of the vehicle to the left side of about several tens of centimeters. The cut-out range A1 includes from the right end of the vehicle to the right side of about several tens of centimeters.
　切出範囲Ａ２は、俯瞰映像１００の後方映像１０２の縦方向の幅と同じ縦方向の幅を有する。切出範囲Ａ２は、後方映像１０２と左側方映像１０３との境界および後方映像１０２と右側方映像１０４との境界を含む。切出範囲Ａ２は、車両の左側端部から数１０ｃｍ程度左側方までを含む。切出範囲Ａ２は、車両の右側端部から数１０ｃｍ程度右側方までを含む。切出範囲Ａ２は、切出範囲Ａ１と同一形状、同一面積を有する。 The cutout range A2 has the same vertical width as the vertical width of the rear video 102 of the overhead view video 100. The cutout range A <b> 2 includes a boundary between the rear image 102 and the left side image 103 and a boundary between the rear image 102 and the right side image 104. The cut-out range A2 includes from the left end of the vehicle to the left side of about several tens of centimeters. The cutout range A2 includes from the right end of the vehicle to the right side of about several tens of centimeters. The cutout area A2 has the same shape and the same area as the cutout area A1.
　切出範囲Ａ３と切出範囲Ａ４とは、第七実施形態と同様である。 The cutout range A3 and the cutout range A4 are the same as in the seventh embodiment.
　俯瞰映像生成部４６Ａは、俯瞰映像１００の切出範囲Ａ１と切出範囲Ａ２とを縦方向に圧縮し、表示パネル３１の表示面の縦横比に合わせた横長形状とした俯瞰映像２００を生成する。 The overhead view video generation unit 46A compresses the cutout range A1 and the cutout range A2 of the overhead view video 100 in the vertical direction, and generates the overhead view video 200 having a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31. .
　このような俯瞰映像１００から生成された、横長形状とした俯瞰映像２００について説明する。 A description will be given of an overhead view image 200 having a horizontally long shape generated from such an overhead view image 100.
　映像２１１は、俯瞰映像１００の切出範囲Ａ１の映像を、縦方向に圧縮した映像である。図２３において、映像２１１に表示された樹木の映像は、図２２に示す俯瞰映像１００の切出範囲Ａ１に表示された樹木の映像を、縦方向に圧縮した映像になっている。映像２１１に表示された樹木の映像は、俯瞰映像１００の前方映像１０１に表示された樹木の映像の全体を含んでいる。 The video 211 is a video obtained by compressing the video of the cutout range A1 of the overhead video 100 in the vertical direction. In FIG. 23, the image of the tree displayed in the image 211 is an image obtained by compressing the image of the tree displayed in the cutout range A1 of the overhead image 100 illustrated in FIG. 22 in the vertical direction. The tree image displayed in the image 211 includes the entire tree image displayed in the front image 101 of the overhead image 100.
　映像２１２は、俯瞰映像１００の切出範囲Ａ２の映像を、縦方向に圧縮した映像である。図２３において、映像２１２に表示された樹木の映像は、図２２に示す俯瞰映像１００の切出範囲Ａ２に表示された樹木の映像を、縦方向に圧縮した映像になっている。映像２１２に表示された樹木の映像は、俯瞰映像１００の後方映像１０２に表示された樹木の映像の全体を含んでいる。 The video 212 is a video obtained by compressing the video of the cutout range A2 of the overhead video 100 in the vertical direction. In FIG. 23, the image of the tree displayed in the image 212 is an image obtained by compressing the image of the tree displayed in the cutout range A2 of the overhead image 100 shown in FIG. 22 in the vertical direction. The tree image displayed in the image 212 includes the entire tree image displayed in the rear image 102 of the overhead image 100.
　映像２１３と映像２１４とは、第七実施形態と同様である。 The video 213 and the video 214 are the same as in the seventh embodiment.
　上述したように、本実施形態によれば、俯瞰映像２００の映像２１１は、俯瞰映像１００の切出範囲Ａ１の映像を縦方向に圧縮した映像である。俯瞰映像２００の映像２１１は、前方映像１０１の縦方向の全体を含む。本実施形態は、俯瞰映像２００の映像２１２は、俯瞰映像１００の切出範囲Ａ２の映像を縦方向に圧縮した映像である。俯瞰映像２００の映像２１２は、後方映像１０２の縦方向の全体を含む。このため、本実施形態によれば、運転者が俯瞰映像２００で、俯瞰映像１００の前方映像１０１の縦方向と後方映像１０２の縦方向と同じ範囲を確認することができる。 As described above, according to the present embodiment, the video 211 of the overhead video 200 is a video obtained by compressing the video of the cutout range A1 of the overhead video 100 in the vertical direction. The video 211 of the overhead video 200 includes the entire vertical direction of the front video 101. In the present embodiment, the video 212 of the bird's-eye view video 200 is a video obtained by compressing the video of the cutout range A2 of the bird's-eye view video 100 in the vertical direction. The video 212 of the overhead video 200 includes the entire vertical direction of the rear video 102. For this reason, according to the present embodiment, the driver can confirm the same range in the overhead image 200 as the vertical direction of the front image 101 of the overhead image 100 and the vertical direction of the rear image 102.
［第九実施形態］
　図２４、図２５を参照しながら、本実施形態に係る俯瞰映像生成システム１Ａについて説明する。図２４は、第九実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。図２５は、図２４の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。本実施形態の俯瞰映像生成システム１Ａは、俯瞰映像生成部４６Ａで横長形状とした俯瞰映像２００を生成する方法が、第七実施形態の俯瞰映像生成システム１Ａと異なる。 [Ninth embodiment]
With reference to FIG. 24 and FIG. 25, the overhead video generation system 1 </ b> A according to the present embodiment will be described. FIG. 24 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the ninth embodiment. FIG. 25 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. The overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the seventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
　俯瞰映像生成部４６Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、障害物を検出した方向が車両の前方または後方である場合、左側方カメラ１３または右側方カメラ１４からの縦長形状の周辺映像を縦方向に圧縮し、横長形状とした俯瞰映像２００を生成する。より詳しくは、俯瞰映像生成部４６Ａは、生成した俯瞰映像１００の切出範囲Ａ３と切出範囲Ａ４とを縦方向に圧縮して俯瞰映像２００を生成する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead view video generation unit 46 </ b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle. A vertically long peripheral image is compressed in the vertical direction to generate a bird's-eye view image 200 having a horizontally long shape. More specifically, the overhead view video generation unit 46A generates the overhead view video 200 by compressing the cutout range A3 and the cutout range A4 of the generated overhead view video 100 in the vertical direction.
　図２４に示す切出範囲Ａ３は、俯瞰映像１００の左側方映像１０３の横方向の幅と同じ横方向の幅を有する。切出範囲Ａ３は、左側方映像１０３と前方映像１０１との境界および左側方映像１０３と後方映像１０２との境界を含む。切出範囲Ａ３は、車両の前端部から数ｍ程度前方までを含む。切出範囲Ａ３は、車両の後端部から数ｍ程度後方までを含む。 24 has a horizontal width that is the same as the horizontal width of the left-side video 103 of the overhead view video 100. The cutout range A3 includes a boundary between the left side video 103 and the front video 101 and a boundary between the left side video 103 and the rear video 102. The cut-out range A3 includes from the front end of the vehicle to about a few meters ahead. The cut-out range A3 includes from the rear end of the vehicle to the rear about several meters.
　切出範囲Ａ４は、俯瞰映像１００の右側方映像１０４の横方向の幅と同じ横方向の幅を有する。切出範囲Ａ４は、右側方映像１０４と前方映像１０１との境界および右側方映像１０４と後方映像１０２との境界を含む。切出範囲Ａ４は、車両の前端部から数ｍ程度前方までを含む。切出範囲Ａ４は、車両の後端部から数ｍ程度後方までを含む。切出範囲Ａ４は、切出範囲Ａ３と同一形状、同一面積を有する。 The cutout range A4 has the same lateral width as the lateral width of the right-side video 104 of the overhead view video 100. The cutout range A4 includes a boundary between the right side video 104 and the front video 101 and a boundary between the right side video 104 and the rear video 102. The cut-out range A4 includes a distance of about several meters from the front end of the vehicle. The cutout range A4 includes from the rear end of the vehicle to the rear about several meters. The cutout area A4 has the same shape and the same area as the cutout area A3.
　切出範囲Ａ１は、切出範囲Ａ３と切出範囲Ａ４との間に位置する矩形状の範囲である。切出範囲Ａ１は、車両の前端部から数ｍ程度前方までを含む。切出範囲Ａ１の前端部は、切出範囲Ａ３の前端部と切出範囲Ａ４の前端部と同一直線上に位置する。 The cutout range A1 is a rectangular range located between the cutout range A3 and the cutout range A4. The cut-out range A1 includes from the front end of the vehicle to about a few meters ahead. The front end portion of the cutout range A1 is located on the same straight line as the front end portion of the cutout range A3 and the front end portion of the cutout range A4.
　切出範囲Ａ２は、切出範囲Ａ３と切出範囲Ａ４との間に位置する矩形状の範囲である。切出範囲Ａ２は、車両の前端部から数ｍ程度前方までを含む。切出範囲Ａ２の前端部は、切出範囲Ａ３の前端部と切出範囲Ａ４の前端部と同一直線上に位置する。 The cutout range A2 is a rectangular range located between the cutout range A3 and the cutout range A4. The cut-out range A2 includes from the front end of the vehicle to about a few meters ahead. The front end portion of the cutout range A2 is located on the same straight line as the front end portion of the cutout range A3 and the front end portion of the cutout range A4.
　俯瞰映像生成部４６Ａは、生成した俯瞰映像１００の切出範囲Ａ１と切出範囲Ａ２とを縦方向に圧縮して、表示パネル３１の表示面の縦横比に合わせた横長形状とした俯瞰映像２００を生成する。 The overhead view video generation unit 46A compresses the cut-out range A1 and the cut-out range A2 of the generated overhead view video 100 in the vertical direction so as to have a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31. Is generated.
　このような俯瞰映像１００から生成された、横長形状とした俯瞰映像２００について説明する。 A description will be given of an overhead view image 200 having a horizontally long shape generated from such an overhead view image 100.
　映像２１３は、俯瞰映像１００の切出範囲Ａ３の映像を、縦方向に圧縮した映像である。図２５において、映像２１３に表示された樹木の映像は、俯瞰映像１００の切出範囲Ａ３に表示された樹木の映像を、縦方向に圧縮した映像になっている。映像２１３に表示された樹木の映像は、俯瞰映像１００の左側方映像１０３に表示された樹木の映像の全体を含んでいる。 The video 213 is a video obtained by compressing the video in the cutout range A3 of the overhead view video 100 in the vertical direction. In FIG. 25, the image of the tree displayed in the image 213 is an image obtained by compressing the image of the tree displayed in the cutout range A3 of the overhead image 100 in the vertical direction. The tree image displayed in the image 213 includes the entire tree image displayed in the left-side image 103 of the overhead image 100.
　映像２１４は、俯瞰映像１００の切出範囲Ａ４の映像を、縦方向に圧縮した映像である。図２５において、映像２１４に表示された樹木の映像は、俯瞰映像１００の切出範囲Ａ４に表示された樹木の映像を、縦方向に圧縮した映像になっている。映像２１４に表示された樹木の映像は、俯瞰映像１００の右側方映像１０４に表示された樹木の映像の全体を含んでいる。 The video 214 is a video obtained by compressing the video of the cutout range A4 of the overhead view video 100 in the vertical direction. In FIG. 25, the image of the tree displayed in the image 214 is an image obtained by compressing the image of the tree displayed in the cutout range A4 of the overhead image 100 in the vertical direction. The tree image displayed in the image 214 includes the entire tree image displayed in the right-side image 104 of the overhead image 100.
　映像２１１は、俯瞰映像１００の切出範囲Ａ１の映像を、映像２１３と映像２１４との境界が滑らかに連続するように、表示パネル３１に合わせて大きさと形状とを変えた映像である。例えば、映像２１１は、俯瞰映像１００の切出範囲Ａ１の映像を、横方向に拡大した映像でもよい。 The video 211 is a video in which the size and shape of the video in the cutout range A1 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous. For example, the video 211 may be a video obtained by enlarging the video in the cutout range A1 of the overhead view video 100 in the horizontal direction.
　映像２１２は、俯瞰映像１００の切出範囲Ａ２の映像を、映像２１３と映像２１４との境界が滑らかに連続するように、表示パネル３１に合わせて大きさと形状とを変えた映像である。例えば、映像２１２は、俯瞰映像１００の切出範囲Ａ２の映像を、横方向に拡大した映像でもよい。 The video 212 is a video in which the size and shape of the video in the cutout range A2 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous. For example, the video 212 may be a video obtained by enlarging the video in the cutout range A2 of the overhead view video 100 in the horizontal direction.
　上述したように、本実施形態によれば、俯瞰映像２００の映像２１３は、俯瞰映像１００の切出範囲Ａ３の映像を縦方向に圧縮した映像である。俯瞰映像２００の映像２１３は、左側方映像１０３の横方向の全体を含む。本実施形態は、俯瞰映像２００の映像２１４は、俯瞰映像１００の切出範囲Ａ４の映像を縦方向に圧縮した映像である。俯瞰映像２００の映像２１４は、右側方映像１０４の横方向の全体を含む。このため、本実施形態によれば、運転者が俯瞰映像２００で、俯瞰映像１００の左側方映像１０３の横方向と右側方映像１０４の横方向と同じ範囲を確認することができる。 As described above, according to the present embodiment, the video 213 of the overhead video 200 is a video obtained by compressing the video of the cutout range A3 of the overhead video 100 in the vertical direction. The video 213 of the overhead video 200 includes the entire horizontal direction of the left side video 103. In this embodiment, the video 214 of the bird's-eye view video 200 is a video obtained by compressing the video of the cutout range A4 of the bird's-eye view video 100 in the vertical direction. The image 214 of the bird's-eye view image 200 includes the entire horizontal direction of the right-side image 104. For this reason, according to the present embodiment, the driver can confirm the same range in the overhead view video 200 as the horizontal direction of the left side video 103 and the horizontal direction of the right side video 104 of the overhead view video 100.
［第十実施形態］
　図２６、図２７を参照しながら、本実施形態に係る俯瞰映像生成システム１Ａについて説明する。図２６は、第十実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。図２７は、図２６の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。本実施形態の俯瞰映像生成システム１Ａは、俯瞰映像生成部４６Ａで横長形状とした俯瞰映像２００を生成する方法が、第九実施形態の俯瞰映像生成システム１Ａと異なる。 [Tenth embodiment]
With reference to FIG. 26 and FIG. 27, an overhead video generation system 1A according to the present embodiment will be described. FIG. 26 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the tenth embodiment. FIG. 27 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. The overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the ninth embodiment in the method of generating the overhead view video 200 having a horizontally long shape by the overhead view video generation unit 46A.
　俯瞰映像生成部４６Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、障害物を検出した方向が車両の前方または後方である場合、左側方カメラ１３または右側方カメラ１４からの縦長形状の周辺映像の中央部を除いて上部と下部を合成し、横長形状とした俯瞰映像２００を生成する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead view video generation unit 46 </ b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle. The top and bottom portions are synthesized except for the center portion of the vertically long peripheral image, and the overhead view image 200 having a horizontally long shape is generated.
　図２６に示す切出範囲Ａ３は、上部Ａ３１と中央部Ａ３２と下部Ａ３３とを含む。上部Ａ３１は、左側方映像１０３と前方映像１０１との境界を含む。下部Ａ３３は、左側方映像１０３と後方映像１０２との境界を含む。 26 includes an upper part A31, a central part A32, and a lower part A33. The upper part A31 includes a boundary between the left side image 103 and the front image 101. The lower part A33 includes a boundary between the left side video 103 and the rear video 102.
　切出範囲Ａ４は、上部Ａ４１と中央部Ａ４２と下部Ａ４３とを含む。上部Ａ４１は、右側方映像１０４と前方映像１０１との境界を含む。下部Ａ４３は、右側方映像１０４と後方映像１０２との境界を含む。切出範囲Ａ４は、切出範囲Ａ３と同一形状、同一面積を有する。 The cutout range A4 includes an upper part A41, a central part A42, and a lower part A43. The upper part A41 includes a boundary between the right side image 104 and the front image 101. The lower part A43 includes a boundary between the right side image 104 and the rear image 102. The cutout area A4 has the same shape and the same area as the cutout area A3.
　切出範囲Ａ１と切出範囲Ａ２とは、第九実施形態と同様である。 The cutout range A1 and the cutout range A2 are the same as in the ninth embodiment.
　俯瞰映像生成部４６Ａは、生成した俯瞰映像１００の切出範囲Ａ３の上部Ａ３１と下部Ａ３３とを合成し、切出範囲Ａ４の上部Ａ４１と下部Ａ４３とを合成して、表示パネル３１の表示面の縦横比に合わせた横長形状とした俯瞰映像２００を生成する。 The overhead image generation unit 46A combines the upper portion A31 and the lower portion A33 of the cutout range A3 of the generated overhead view image 100, and combines the upper portion A41 and the lower portion A43 of the cutout range A4 to display the display surface of the display panel 31. A bird's-eye view image 200 having a horizontally long shape matching the aspect ratio is generated.
　このような俯瞰映像１００から生成された、横長形状とした俯瞰映像２００について説明する。 A description will be given of an overhead view image 200 having a horizontally long shape generated from such an overhead view image 100.
　映像２１３は、俯瞰映像１００の切出範囲Ａ３の中央部Ａ３２を除いた、上部Ａ３１と下部Ａ３３とを合成した映像である。図２７において、映像２１３は、図２６に示す俯瞰映像１００の切出範囲Ａ３の中央部Ａ３２に表示された樹木の映像を含まない映像になっている。映像２１３には、俯瞰映像１００の切出範囲Ａ３の中央部Ａ３２が省略されていることを示すため、上部Ａ３１と下部Ａ３３との境界に線を表示してもよい。 The video 213 is a video obtained by combining the upper part A31 and the lower part A33 excluding the central part A32 of the cutout range A3 of the overhead view video 100. In FIG. 27, the video 213 is a video that does not include the video of the tree displayed in the central portion A32 of the cutout range A3 of the overhead view video 100 shown in FIG. In the video 213, a line may be displayed at the boundary between the upper part A31 and the lower part A33 in order to show that the central part A32 of the cutout range A3 of the overhead view video 100 is omitted.
　映像２１４は、俯瞰映像１００の切出範囲Ａ４の中央部Ａ４２を除いた、上部Ａ４１と下部Ａ４３とを合成した映像である。図２７において、映像２１４は、図２６に示す俯瞰映像１００の切出範囲Ａ４の中央部Ａ４２に表示された樹木の映像を含まない映像になっている。映像２１４には、俯瞰映像１００の切出範囲Ａ４の中央部Ａ４２が省略されていることを示すため、上部Ａ４１と下部Ａ４３との境界に線を表示してもよい。 The video 214 is a video obtained by synthesizing the upper part A41 and the lower part A43 excluding the central part A42 of the cutout range A4 of the overhead view video 100. In FIG. 27, the image 214 is an image that does not include the image of the tree displayed in the central portion A42 of the cutout range A4 of the overhead image 100 shown in FIG. In the video 214, a line may be displayed at the boundary between the upper part A41 and the lower part A43 in order to show that the central part A42 of the cutout range A4 of the overhead view video 100 is omitted.
　映像２１１と映像２１２とは、第九実施形態と同様である。 The video 211 and the video 212 are the same as in the ninth embodiment.
　上述したように、本実施形態によれば、俯瞰映像２００の映像２１３は、俯瞰映像１００の切出範囲Ａ３の中央部Ａ３２を除いた上部Ａ３１と下部Ａ３３とを合成している。これにより、俯瞰映像２００の映像２１３は、俯瞰映像１００の左側方映像１０３の前方および後方を大きく表示することができる。俯瞰映像２００の映像２１４は、俯瞰映像１００の切出範囲Ａ４の中央部Ａ４２を除いた上部Ａ４１と下部Ａ４３とを合成している。これにより、俯瞰映像２００の映像２１４は、俯瞰映像１００の右側方映像１０４の前方および後方を大きく表示することができる。 As described above, according to the present embodiment, the video 213 of the bird's-eye view video 200 is composed of the upper part A31 and the lower part A33 excluding the central part A32 of the cutout range A3 of the bird's-eye view picture 100. Thereby, the video 213 of the bird's-eye view video 200 can largely display the front and rear of the left side video 103 of the bird's-eye view video 100. The image 214 of the bird's-eye view image 200 combines the upper part A41 and the lower part A43 excluding the central part A42 of the cutout range A4 of the bird's-eye view image 100. Thereby, the video 214 of the bird's-eye view video 200 can largely display the front and rear of the right-side video 104 of the bird's-eye view video 100.
［第十一実施形態］
　図２８、図２９を参照しながら、本実施形態に係る俯瞰映像生成システム１Ａについて説明する。図２８は、第十一実施形態に係る俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法を説明する図である。図２９は、図２８の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。本実施形態の俯瞰映像生成システム１Ａは、俯瞰映像生成部４６Ａで横長形状とした俯瞰映像２００を生成する方法が、第九実施形態の俯瞰映像生成システム１Ａと異なる。 [Eleventh embodiment]
With reference to FIG. 28 and FIG. 29, the overhead video generation system 1A according to the present embodiment will be described. FIG. 28 is a diagram for explaining a method for generating a horizontally long overhead image in the overhead image generation system according to the eleventh embodiment. FIG. 29 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. The overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the ninth embodiment in the method of generating the overhead view video 200 having a horizontally long shape by the overhead view video generation unit 46A.
　俯瞰映像生成部４６Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、障害物を検出した方向が車両の前方または後方である場合、左側方カメラ１３または右側方カメラ１４からの縦長形状の周辺映像を、障害物を検出した方向に偏重し、横長形状とした俯瞰映像２００を生成する。本実施形態では、障害物を検出した方向は、後方として説明する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead view video generation unit 46 </ b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle. The vertically long peripheral image is biased in the direction in which the obstacle is detected, and the overhead image 200 having a horizontally long shape is generated. In the present embodiment, the direction in which the obstacle is detected will be described as the rear.
　図２８に示す切出範囲Ａ３は、俯瞰映像１００の左側方映像１０３において、障害物の検出方向である後方に偏重した範囲である。切出範囲Ａ３は、俯瞰映像１００の縦方向の中央部から下側を含む。 The cutout range A3 shown in FIG. 28 is a range that is biased backward in the obstacle detection direction in the left side image 103 of the overhead view image 100. The cut-out range A3 includes the lower side from the center in the vertical direction of the overhead view image 100.
　切出範囲Ａ４は、俯瞰映像１００の右側方映像１０４において、障害物の検出方向である後方に偏重した範囲である。切出範囲Ａ４は、縦方向の中央部から下側を含む。切出範囲Ａ４は、切出範囲Ａ３と同一形状、同一面積を有する。 The cut-out range A4 is a range that is biased toward the rear in the obstacle detection direction in the right-side video 104 of the overhead view video 100. The cut-out range A4 includes the lower side from the central portion in the vertical direction. The cutout area A4 has the same shape and the same area as the cutout area A3.
　切出範囲Ａ５は、俯瞰映像１００の切出範囲Ａ３と切出範囲Ａ４との間に位置する矩形状の範囲である。切出範囲Ａ５の上端部は、切出範囲Ａ１の上端部と切出範囲Ａ２の上端部と同一直線上に位置する。切出範囲Ａ５の下端部は、切出範囲Ａ１の下端部と切出範囲Ａ２の下端部と同一直線上に位置する。 The cutout range A5 is a rectangular range located between the cutout range A3 and the cutout range A4 of the overhead view image 100. The upper end portion of the cutout range A5 is located on the same straight line as the upper end portion of the cutout range A1 and the upper end portion of the cutout range A2. The lower end portion of the cutout range A5 is located on the same straight line as the lower end portion of the cutout range A1 and the lower end portion of the cutout range A2.
　俯瞰映像生成部４６Ａは、生成した俯瞰映像１００の切出範囲Ａ３と切出範囲Ａ４と切出範囲Ａ５とから、表示パネル３１の表示面の縦横比に合わせた横長形状とした俯瞰映像２００を生成する。 The overhead view video generation unit 46A generates an overhead view video 200 having a horizontally long shape that matches the aspect ratio of the display surface of the display panel 31 from the cutout range A3, the cutout range A4, and the cutout range A5 of the generated overhead view video 100. Generate.
　このような俯瞰映像１００から生成された、横長形状とした俯瞰映像２００について説明する。 A description will be given of an overhead view image 200 having a horizontally long shape generated from such an overhead view image 100.
　映像２１３は、俯瞰映像１００の切出範囲Ａ３の映像である。図２９において、映像２１３に表示された樹木の映像は、図２８に示す俯瞰映像１００の切出範囲Ａ３に表示された樹木の映像の全体を含んでいる。 The video 213 is a video of the cutout range A3 of the overhead video 100. In FIG. 29, the image of the tree displayed in the image 213 includes the entire image of the tree displayed in the cutout area A3 of the overhead image 100 shown in FIG.
　映像２１４は、俯瞰映像１００の切出範囲Ａ４の映像である。図２９において、映像２１４に表示された樹木の映像は、図２８に示す俯瞰映像１００の切出範囲Ａ４に表示された樹木の映像の全体を含んでいる。 The image 214 is an image of the cutout range A4 of the overhead image 100. In FIG. 29, the tree image displayed in the image 214 includes the entire tree image displayed in the cutout range A4 of the overhead view image 100 shown in FIG.
　映像２１５は、俯瞰映像１００の切出範囲Ａ５の映像を、映像２１３と映像２１４との境界が滑らかに連続するように、表示パネル３１に合わせて大きさと形状とを変えた映像である。例えば、映像２１５は、俯瞰映像１００の切出範囲Ａ５の映像を、横方向に拡大した映像でもよい。 The video 215 is a video in which the size and shape of the video in the cutout range A5 of the overhead video 100 are changed according to the display panel 31 so that the boundary between the video 213 and the video 214 is smoothly continuous. For example, the video 215 may be a video obtained by enlarging the video in the cutout range A5 of the overhead view video 100 in the horizontal direction.
　上述したように、本実施形態は、障害物の検出方向が前方または後方である場合、俯瞰映像１００の左側方映像１０３と右側方映像１０４を障害物の検出方向に偏重し、横長形状とした俯瞰映像２００を生成する。これにより、俯瞰映像２００は、俯瞰映像１００において障害物を検出した方向を大きく表示することができる。 As described above, in the present embodiment, when the obstacle detection direction is forward or backward, the left-side image 103 and the right-side image 104 of the overhead image 100 are biased in the obstacle detection direction to have a horizontally long shape. An overhead image 200 is generated. Thereby, the bird's-eye view image 200 can display a larger direction in which the obstacle is detected in the bird's-eye view image 100.
［第十二実施形態］
　本実施形態に係る俯瞰映像生成システム１Ａについて説明する。本実施形態の俯瞰映像生成システム１Ａは、俯瞰映像生成部４６Ａで横長形状とした俯瞰映像２００を生成する方法が、第十一実施形態の俯瞰映像生成システム１Ａと異なる。 [Twelfth embodiment]
An overhead video generation system 1A according to the present embodiment will be described. The overhead view video generation system 1A of the present embodiment is different from the overhead view video generation system 1A of the eleventh embodiment in the method of generating a horizontally long overhead view video 200 by the overhead view video generation unit 46A.
　俯瞰映像生成部４６Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、障害物を検出した方向が車両の前方または後方である場合、左側方カメラ１３または右側方カメラ１４からの縦長形状の周辺映像を車両の進行方向に偏重し、横長形状とした俯瞰映像２００を生成する。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the overhead view video generation unit 46 </ b> A receives from the left side camera 13 or the right side camera 14 when the direction in which the obstacle is detected is front or rear of the vehicle. An overhead image 200 having a horizontally long shape is generated by deviating a vertically long peripheral image in the traveling direction of the vehicle.
　上述したように、本実施形態は、障害物の検出方向が前方または後方である場合、俯瞰映像１００の左側方映像１０３と右側方映像１０４を車両の進行方向に偏重し、横長形状とした俯瞰映像２００を生成する。これにより、俯瞰映像２００は、俯瞰映像１００において車両の進行方向を大きく表示することができる。 As described above, in the present embodiment, when the obstacle detection direction is forward or backward, the left side image 103 and the right side image 104 of the overhead view image 100 are biased in the traveling direction of the vehicle so as to have a horizontally long shape. A video 200 is generated. Thereby, the bird's-eye view image 200 can display the traveling direction of the vehicle larger in the bird's-eye view image 100.
　さて、これまで本発明に係る俯瞰映像生成システム１について説明したが、上述した実施形態以外にも種々の異なる形態にて実施されてよいものである。 Now, the overhead video generation system 1 according to the present invention has been described so far, but may be implemented in various different forms other than the above-described embodiment.
　図示した俯瞰映像生成システム１の各構成要素は、機能概念的なものであり、必ずしも物理的に図示の如く構成されていなくてもよい。すなわち、各装置の具体的形態は、図示のものに限られず、各装置の処理負担や使用状況などに応じて、その全部または一部を任意の単位で機能的または物理的に分散または統合してもよい。 The constituent elements of the overhead view video generation system 1 shown in the figure are functionally conceptual, and need not be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or a part of them is functionally or physically distributed or integrated in arbitrary units according to the processing load or usage status of each device. May be.
　俯瞰映像生成システム１の構成は、例えば、ソフトウェアとして、メモリにロードされたプログラムなどによって実現される。上記実施形態では、これらのハードウェアまたはソフトウェアの連携によって実現される機能ブロックとして説明した。すなわち、これらの機能ブロックについては、ハードウェアのみ、ソフトウェアのみ、または、それらの組み合わせによって種々の形で実現できる。 The configuration of the overhead view video generation system 1 is realized by, for example, a program loaded in a memory as software. The above embodiment has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.
　上記した構成要素には、当業者が容易に想定できるもの、実質的に同一のものを含む。さらに、上記した構成は適宜組み合わせが可能である。また、本発明の要旨を逸脱しない範囲において構成の種々の省略、置換または変更が可能である。 The above-mentioned constituent elements include those that can be easily assumed by those skilled in the art and those that are substantially the same. Furthermore, the above-described configurations can be appropriately combined. In addition, various omissions, substitutions, or changes in the configuration can be made without departing from the scope of the present invention.
　重畳映像生成部４７は、例えば、最も近い障害物までの距離が所定閾値未満の場合、俯瞰映像１００の中央映像１０５に、障害物報知アイコン１２０とともに障害物までの距離を数値で表示してもよい。または、重畳映像生成部４７は、例えば、最も近い障害物までの距離が所定閾値未満の場合、俯瞰映像１００の中央映像１０５に、障害物報知アイコン１２０とともに障害物までの距離に応じて形状や色が変化する図形を表示してもよい。俯瞰映像生成システム１は、障害物までの距離に応じた障害物報知アイコン１２０の変化に加えて、障害物までの距離を報知することができる。これにより、俯瞰映像生成システム１は、車両周辺の障害物をより適切に確認可能にすることができる。 For example, when the distance to the nearest obstacle is less than a predetermined threshold, the superimposed video generation unit 47 may display the distance to the obstacle numerically together with the obstacle notification icon 120 on the central video 105 of the overhead video 100. Good. Alternatively, for example, when the distance to the nearest obstacle is less than a predetermined threshold, the superimposed image generation unit 47 displays the shape or the shape according to the distance to the obstacle together with the obstacle notification icon 120 on the central image 105 of the overhead view image 100. A figure whose color changes may be displayed. The overhead view video generation system 1 can notify the distance to the obstacle in addition to the change of the obstacle notification icon 120 according to the distance to the obstacle. Thereby, the bird's-eye view video generation system 1 can make it possible to more appropriately check obstacles around the vehicle.
　制御部４１は、障害物が複数ある場合、優先順位の高い障害物について、障害物報知アイコン１２０を表示するようにしてもよい。例えば、優先順位は、障害物までの距離が最も小さいものを最も高い優先順位としてもよい。または、例えば、優先順位は、障害物が動体であるものを最も高い優先順位としてもよい。または、例えば、優先順位は、障害物が動体で、かつ、車両に近づいてきているものを最も高い優先順位としてもよい。 When there are a plurality of obstacles, the control unit 41 may display the obstacle notification icon 120 for the obstacles with high priority. For example, the priority order may be the highest priority order with the smallest distance to the obstacle. Alternatively, for example, the priority may be the highest priority when the obstacle is a moving object. Alternatively, for example, the priority may be the highest priority when the obstacle is moving and is approaching the vehicle.
　障害物が動体であることが検出された場合、俯瞰映像１００の中央映像１０５に、動体であることを示すアイコンを表示してもよい。動体であることを示すアイコンは、例えば、歩行者を示すアイコンや車両であることを示すアイコンとしてもよい。これにより、俯瞰映像生成システム１は、車両周辺の障害物をより適切に確認可能にすることができる。 When it is detected that the obstacle is a moving object, an icon indicating that the obstacle is a moving object may be displayed in the central image 105 of the overhead view image 100. The icon indicating a moving object may be, for example, an icon indicating a pedestrian or an icon indicating a vehicle. Thereby, the bird's-eye view video generation system 1 can make it possible to more appropriately check obstacles around the vehicle.
　障害物報知アイコン１２０は、３重の弧状の曲線で構成されるものとして説明したが、これに限定されない。障害物報知アイコン１２０は、例えば、障害物の方向を示す矢印状の図形でもよい。この場合、障害物報知アイコン１２０は、障害物までの距離に応じて、太さや大きさを変えた矢印状の図形としてもよい。 Although the obstacle notification icon 120 has been described as being composed of a triple arc-shaped curve, it is not limited to this. The obstacle notification icon 120 may be, for example, an arrow-shaped figure indicating the direction of the obstacle. In this case, the obstacle notification icon 120 may be an arrow-shaped figure whose thickness and size are changed according to the distance to the obstacle.
　本実施形態では、障害物検出部であるセンサとして、前方左センサ２１Ａと前方中央センサ２１Ｂと前方右センサ２１Ｃと後方左センサ２２Ａと後方中央センサ２２Ｂと後方右センサ２６との６つのセンサを有するものとしたが、センサの数は限定されない。障害物の方向をより細分化して検出したい場合、センサの数を増やしてもよい。 In the present embodiment, the sensor serving as the obstacle detection unit includes six sensors including a front left sensor 21A, a front center sensor 21B, a front right sensor 21C, a rear left sensor 22A, a rear center sensor 22B, and a rear right sensor 26. However, the number of sensors is not limited. If it is desired to detect the direction of an obstacle more finely, the number of sensors may be increased.
　本実施形態では、障害物報知アイコン１２０は、センサの数に対応して、前方左アイコン１２１と前方中央アイコン１２２と前方右アイコン１２３と後方左アイコン１２４と後方中央アイコン１２５と後方右アイコン１２６とを含むものとしたが、これに限定されない。言い換えると、障害物報知アイコン１２０は、各センサの検出結果と表示させるアイコンとが対応付けられていればよい。 In the present embodiment, the obstacle notification icon 120 corresponds to the number of sensors, the front left icon 121, the front center icon 122, the front right icon 123, the rear left icon 124, the rear center icon 125, and the rear right icon 126. However, the present invention is not limited to this. In other words, the obstacle notification icon 120 only needs to associate the detection result of each sensor with the icon to be displayed.
　第八実施形態において、俯瞰映像生成部４６は、前方カメラ１１または後方カメラ１２からの周辺映像の中央部を縦方向に圧縮し、横長形状とした俯瞰映像２００を生成してもよい。図３０、図３１を用いて、より詳しく説明する。図３０は、俯瞰映像生成システムにおける、横長形状の俯瞰映像の生成方法の他の例を説明する図である。図３１は、図３０の俯瞰映像から生成した、横長形状とした俯瞰映像の一例を示す図である。俯瞰映像生成部４６は、俯瞰映像１００の左側方映像１０３の中央部の切出範囲Ａ６と、右側方映像１０４の中央部の切出範囲Ａ７とを縦方向に圧縮し、表示パネル３１の表示面の縦横比に合わせた横長形状とした俯瞰映像２００を生成する。 In the eighth embodiment, the bird's-eye view image generation unit 46 may generate a bird's-eye view image 200 having a horizontally long shape by compressing the central portion of the peripheral image from the front camera 11 or the rear camera 12 in the vertical direction. This will be described in more detail with reference to FIGS. FIG. 30 is a diagram illustrating another example of a method for generating a horizontally long overhead image in the overhead image generation system. FIG. 31 is a diagram illustrating an example of a horizontally long overhead image generated from the overhead image of FIG. 30. The overhead image generation unit 46 compresses the cutout range A6 at the center of the left side video 103 of the overhead view video 100 and the cutout range A7 at the center of the right side video 104 in the vertical direction, and displays the display panel 31 on the display panel 31. A bird's-eye view image 200 having a horizontally long shape matching the aspect ratio of the surface is generated.
　本実施形態では、制御部４１Ａは、俯瞰映像１００、俯瞰映像２００の中央部に、車両までの距離が最小の障害物を検出した方向に対応する周辺映像を合成するものとしたが、合成する周辺映像は、ユーザが選択できるようにしてもよい。例えば、表示パネル３１の表示面で、ユーザがタッチした位置の俯瞰映像１００、俯瞰映像２００の周辺映像を合成してもよい。 In the present embodiment, the control unit 41A synthesizes the peripheral video corresponding to the direction in which the obstacle with the shortest distance to the vehicle is detected at the center of the overhead video 100 and the overhead video 200. The peripheral video may be selectable by the user. For example, peripheral images of the overhead view video 100 and the overhead view video 200 at the position touched by the user may be synthesized on the display surface of the display panel 31.
　制御部４１Ａは、障害物の検出方向が前方または後方である場合、俯瞰映像２００の左側方映像２０３と右側方映像２０４とを、例えば、色情報を低減してもよい。または、制御部４１Ａは、障害物の検出方向が前方または後方である場合、俯瞰映像２００の左側方映像２０３と右側方映像２０４とを、例えば、単色表示にしてもよい。または、制御部４１Ａは、障害物の検出方向が前方または後方である場合、俯瞰映像２００の左側方映像２０３と右側方映像２０４とを、例えば、色情報を低減してもよい。または、制御部４１Ａは、障害物の検出方向が前方または後方である場合、俯瞰映像２００の左側方映像２０３と右側方映像２０４とを、例えば、輝度を低減してもよい。このようにすることで、俯瞰映像２００の左側方映像２０３と右側方映像２０４との情報量が低減されていることを、運転者に認識させることができる。 The control unit 41A may reduce the color information of the left-side image 203 and the right-side image 204 of the overhead image 200, for example, when the obstacle detection direction is forward or backward. Alternatively, when the obstacle detection direction is forward or backward, the control unit 41A may display the left side video 203 and the right side video 204 of the overhead view video 200, for example, in a single color display. Alternatively, the control unit 41A may reduce the color information of the left side video 203 and the right side video 204 of the overhead view video 200, for example, when the obstacle detection direction is front or rear. Alternatively, the control unit 41A may reduce the luminance of the left side video 203 and the right side video 204 of the overhead view video 200, for example, when the obstacle detection direction is front or rear. In this way, the driver can be made aware that the amount of information in the left side video 203 and the right side video 204 of the overhead view video 200 is reduced.
　俯瞰映像生成部４６Ａは、障害物情報取得部４３で取得した障害物情報に基づいて、障害物を検出した方向が車両の前方または後方である場合、左側方カメラ１３および右側方カメラ１４からの周辺映像を非表示とし、前方カメラ１１および後方カメラ１２からの周辺映像で、横長形状とした俯瞰映像２００を生成してもよい。この場合、車両の左右は、運転者が目視で確認することができる。 Based on the obstacle information acquired by the obstacle information acquisition unit 43, the bird's-eye view video generation unit 46A receives the signals from the left side camera 13 and the right side camera 14 when the direction in which the obstacle is detected is the front or the rear of the vehicle. The peripheral video may be hidden, and the overhead video 200 having a horizontally long shape may be generated from the peripheral video from the front camera 11 and the rear camera 12. In this case, the driver can visually confirm the left and right sides of the vehicle.
　制御部４１Ａは、例えば、操作部に対する、俯瞰映像表示開始の操作の検出の有無で、俯瞰映像表示を開始するか否かを判定してもよい。 The control unit 41A may determine whether or not to start the overhead view video display based on, for example, whether or not an operation for starting the overhead view video display on the operation unit is detected.
　第七実施形態では、障害物検出部であるセンサとして、前方センサ２１と後方センサ２２と左側方センサ２３と右側方センサ２４との４つのセンサを有するものとしたが、センサの数は限定されない。障害物の方向をより細分化して検出したい場合、センサの数を増やしてもよい。 In the seventh embodiment, four sensors, that is, the front sensor 21, the rear sensor 22, the left side sensor 23, and the right side sensor 24 are provided as the sensors that are obstacle detection units, but the number of sensors is not limited. . If it is desired to detect the direction of an obstacle more finely, the number of sensors may be increased.
　１　　　俯瞰映像生成システム
　１１　　前方カメラ（カメラ）
　１２　　後方カメラ（カメラ）
　１３　　左側方カメラ（カメラ）
　１４　　右側方カメラ（カメラ）
　２１Ａ　　前方左センサ（障害物検出部）
　２１Ｂ　　前方中央センサ（障害物検出部）
　２１Ｃ　　前方右センサ（障害物検出部）
　２２Ａ　　後方左センサ（障害物検出部）
　２２Ｂ　　後方中央センサ（障害物検出部）
　２２Ｃ　　後方右センサ（障害物検出部）
　３１　　表示パネル
　４０　　俯瞰映像生成装置
　４１　　制御部
　４２　　映像取得部
　４３　　障害物情報取得部
　４４　　車両情報取得部
　４６　　俯瞰映像生成部（映像生成部）
　４７　　重畳映像生成部（映像生成部）
　４８　　表示制御部
　４９　　記憶部
　１００　俯瞰映像
　１２０　障害物報知アイコン（障害物を示す情報）
　１２１　前方左アイコン（障害物を示す情報）
　１２２　前方中央アイコン（障害物を示す情報）
　１２３　前方右アイコン（障害物を示す情報）
　１２４　後方左アイコン（障害物を示す情報）
　１２５　後方中央アイコン（障害物を示す情報）
　１２６　後方右アイコン（障害物を示す情報）
　１４０　自車アイコン 1 Overhead video generation system 11 Front camera (camera)
12 Rear camera (camera)
13 Left-side camera (camera)
14 Right-side camera (camera)
21A Front left sensor (obstacle detection unit)
21B Front center sensor (obstacle detection unit)
21C Front right sensor (obstacle detection unit)
22A Rear left sensor (obstacle detection unit)
22B Rear center sensor (obstacle detection unit)
22C Rear right sensor (obstacle detection unit)
Reference Signs List 31 Display panel 40 Overhead video generation device 41 Control unit 42 Video acquisition unit 43 Obstacle information acquisition unit 44 Vehicle information acquisition unit 46 Overhead video generation unit (video generation unit)
47 Superimposed video generator (video generator)
48 Display control unit 49 Storage unit 100 Overhead video 120 Obstacle notification icon (information indicating an obstacle)
121 Front left icon (information indicating an obstacle)
122 Front center icon (information indicating an obstacle)
123 Front right icon (information indicating an obstacle)
124 Rear left icon (information indicating an obstacle)
125 Rear center icon (information indicating an obstacle)
126 Rear right icon (information indicating an obstacle)
140 Car icon

Claims (25)

  1. 　車両の周辺を撮影した周辺映像を取得する映像取得部と、
    　前記車両の周辺において検出した障害物の障害物情報を取得する障害物情報取得部と、
    　前記映像取得部が取得した周辺映像から前記車両を上方から見下ろすように視点変換処理を行った俯瞰映像を生成し、前記俯瞰映像の中央部に、前記障害物情報取得部が取得した障害物を示す情報を合成した俯瞰映像を生成する映像生成部と、
    　前記映像生成部が生成した俯瞰映像を表示部に表示させる表示制御部と
    　を有することを特徴とする俯瞰映像生成装置。     A video acquisition unit that acquires a peripheral video of the periphery of the vehicle;
    An obstacle information acquisition unit for acquiring obstacle information of obstacles detected in the vicinity of the vehicle;
    From the surrounding video acquired by the video acquisition unit, an overhead view video is generated by performing viewpoint conversion processing so that the vehicle is looked down from above, and the obstacle acquired by the obstacle information acquisition unit is displayed at the center of the overhead video. A video generation unit that generates a bird's-eye view video that combines the information shown;
    A bird's-eye-view video generation device, comprising: a display controller configured to display the bird's-eye-view video generated by the video generation unit on a display unit.
  2. 　前記映像生成部は、前記障害物を示す情報を、前記俯瞰映像で囲まれた中央部に重畳した俯瞰映像を生成する、
    　請求項１に記載の俯瞰映像生成装置。     The video generation unit generates an overhead video in which information indicating the obstacle is superimposed on a central portion surrounded by the overhead video,
    The overhead image generation apparatus according to claim 1.
  3. 　前記障害物情報取得部は、検出した障害物までの距離を含む障害物情報を取得し、
    　前記映像生成部は、前記障害物情報取得部が取得した障害物情報に基づいて、検出した障害物までの距離が所定閾値以上であれば、前記俯瞰映像で囲まれた中央部に前記車両を示す自車アイコンを表示させ、前記障害物を示す情報を前記俯瞰映像に重畳させた俯瞰映像を生成し、前記障害物情報取得部が取得した障害物情報に基づいて、検出した障害物までの距離が所定閾値未満であれば、前記障害物を示す情報を前記俯瞰映像で囲まれた中央部に重畳した俯瞰映像を生成する、
    　請求項２に記載の俯瞰映像生成装置。     The obstacle information acquisition unit acquires obstacle information including a distance to the detected obstacle,
    If the distance to the detected obstacle is greater than or equal to a predetermined threshold based on the obstacle information acquired by the obstacle information acquisition unit, the video generation unit is configured to place the vehicle in a central portion surrounded by the overhead video. The vehicle icon is displayed, and an overhead view image in which information indicating the obstacle is superimposed on the overhead view image is generated. Based on the obstacle information acquired by the obstacle information acquisition unit, the detected obstacle is displayed. If the distance is less than a predetermined threshold, generate an overhead image in which information indicating the obstacle is superimposed on a central portion surrounded by the overhead image,
    The overhead image generation apparatus according to claim 2.
  4. 　前記障害物情報取得部は、前記俯瞰映像として表示する範囲より遠方の範囲において検出した障害物の障害物情報を取得し、
    　前記映像生成部は、前記障害物情報取得部が取得した障害物情報に基づいて、検出した障害物までの距離が前記俯瞰映像として表示する範囲より遠方であれば、前記俯瞰映像で囲まれた中央部に前記車両を示す自車アイコンを表示させ、前記障害物を示す情報を前記俯瞰映像に重畳させた俯瞰映像を生成し、前記障害物情報取得部が取得した障害物情報に基づいて、検出した障害物までの距離が前記俯瞰映像として表示する範囲に含まれる範囲であれば、前記障害物を示す情報を前記俯瞰映像で囲まれた中央部に重畳した俯瞰映像を生成する、
    　請求項３に記載の俯瞰映像生成装置。     The obstacle information acquisition unit acquires obstacle information of an obstacle detected in a range farther than a range to be displayed as the overhead image,
    If the distance to the detected obstacle is farther than the range to be displayed as the overhead image based on the obstacle information acquired by the obstacle information acquisition unit, the video generation unit is surrounded by the overhead image. Based on the obstacle information acquired by the obstacle information acquisition unit, displaying a vehicle icon indicating the vehicle in the center, generating an overhead image in which information indicating the obstacle is superimposed on the overhead image, If the distance to the detected obstacle is a range included in the range to be displayed as the overhead image, an overhead image is generated by superimposing information indicating the obstacle on a central portion surrounded by the overhead image.
    The overhead view video generation device according to claim 3.
  5. 　前記障害物を示す情報は、前記障害物を検出した方向を示す情報である、
    　請求項２から４のいずれか一項に記載の俯瞰映像生成装置。     The information indicating the obstacle is information indicating a direction in which the obstacle is detected.
    The overhead view video generation device according to any one of claims 2 to 4.
  6. 　前記障害物を示す情報は、検出した障害物までの距離に応じて変化する、
    　請求項２から５のいずれか一項に記載の俯瞰映像生成装置。     The information indicating the obstacle changes according to the distance to the detected obstacle.
    The bird's-eye view image generation device according to any one of claims 2 to 5.
  7. 　前記障害物を示す情報は、前記障害物を検出した方向を向いた複数の構成要素を有し、前記複数の構成要素の間隔によって前記障害物までの距離を示し、
    　前記映像生成部は、検出した障害物までの距離に基づいて、前記複数の構成要素の間隔を変化させる、
    　請求項２から６のいずれか一項に記載の俯瞰映像生成装置。     The information indicating the obstacle has a plurality of components facing the direction in which the obstacle is detected, and indicates the distance to the obstacle by the interval of the plurality of components,
    The video generation unit changes the interval between the plurality of components based on the distance to the detected obstacle.
    The bird's-eye view image generation device according to any one of claims 2 to 6.
  8. 　前記障害物情報取得部は、検出した障害物の水平方向幅を含む障害物情報を取得し、
    　前記障害物を示す情報は、検出した障害物の水平方向幅を示す情報を含み、
    　前記映像生成部は、前記障害物情報取得部が取得した障害物情報に基づいて、前記障害物を示す情報の幅を、検出した障害物の水平方向幅に対応させた幅として前記俯瞰映像で囲まれた中央部に重畳した俯瞰映像を生成する、
    　請求項２から７のいずれか一項に記載の俯瞰映像生成装置。     The obstacle information acquisition unit acquires obstacle information including a horizontal width of the detected obstacle,
    The information indicating the obstacle includes information indicating a horizontal width of the detected obstacle,
    The video generation unit uses the overhead image as the width corresponding to the horizontal width of the detected obstacle based on the obstacle information acquired by the obstacle information acquisition unit. Generate a bird's-eye view image superimposed on the enclosed central part,
    The overhead image generation apparatus according to any one of claims 2 to 7.
  9. 　前記映像生成部は、前記障害物情報取得部が取得した障害物情報に基づいて、前記車両の進行方向に位置する前記障害物について、前記障害物を示す情報を前記俯瞰映像で囲まれた中央部に重畳した俯瞰映像を生成する、
    　請求項２から８のいずれか一項に記載の俯瞰映像生成装置。     The video generation unit is a center in which information indicating the obstacle is surrounded by the overhead video with respect to the obstacle located in the traveling direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit. Generate a bird's-eye view image superimposed on the part,
    The overhead image generation apparatus according to any one of claims 2 to 8.
  10. 　前記映像取得部は、車両の前後左右を撮影する複数のカメラからの周辺映像を取得し、
    　前記映像生成部は、前記俯瞰映像で囲まれた中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの周辺映像を合成した俯瞰映像を生成する、
    　請求項１に記載の俯瞰映像生成装置。     The video acquisition unit acquires peripheral video from a plurality of cameras that shoot front, rear, left, and right of the vehicle,
    The video generation unit generates a bird's-eye view image obtained by synthesizing peripheral images from the camera corresponding to the direction in which the obstacle information acquisition unit has detected an obstacle in a central portion surrounded by the bird's-eye view video.
    The overhead image generation apparatus according to claim 1.
  11. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の前方または後方である場合、表示向きを変えずに横長形状とした俯瞰映像を生成し、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの横長形状の周辺映像を合成した俯瞰映像を生成し、
    　前記表示制御部は、前記映像生成部が生成した俯瞰映像を横長形状の表示面を備える表示部に表示させる、
    　請求項１０に記載の俯瞰映像生成装置。     When the direction in which the obstacle information acquisition unit has detected the obstacle is the front or the rear of the vehicle, the video generation unit generates a horizontal overhead image without changing the display direction, and generates the overhead video In the central part of the, to generate a bird's-eye view image that combines a horizontally long peripheral image from the camera corresponding to the direction in which the obstacle information acquisition unit has detected the obstacle,
    The display control unit displays the overhead view video generated by the video generation unit on a display unit having a horizontally long display surface.
    The overhead view video generation apparatus according to claim 10.
  12. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の左方向または右方向である場合、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの縦長形状の周辺映像を合成した俯瞰映像を生成する、
    　請求項１０に記載の俯瞰映像生成装置。     When the direction in which the obstacle information acquisition unit detects the obstacle is the left direction or the right direction of the vehicle, the image generation unit has the obstacle information acquisition unit at the center of the generated overhead image. To generate a bird's-eye view image that is a combination of vertically long peripheral images from the camera corresponding to the direction of detecting
    The overhead view video generation apparatus according to claim 10.
  13. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の前方または後方である場合、前記車両の前方向と後方向とに対応する前記カメラからの周辺映像の一部を縦横比を固定して拡大し、横長形状とした俯瞰映像を生成し、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの横長形状の周辺映像を合成した俯瞰映像を生成する、
    　請求項１１に記載の俯瞰映像生成装置。     The video generation unit is a peripheral video from the camera corresponding to the front direction and the rear direction of the vehicle when the direction in which the obstacle information acquisition unit detects the obstacle is forward or backward of the vehicle. The section is enlarged with a fixed aspect ratio to generate a horizontal overhead view image, and the obstacle information acquisition unit from the camera corresponding to the direction in which the obstacle is detected is formed at the center of the generated overhead view video. Generate a bird's-eye view image that combines horizontally long peripheral images.
    The overhead view video generation apparatus according to claim 11.
  14. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の前方または後方である場合、前記車両の前方向と後方向とに対応する前記カメラからの周辺映像を縦方向に圧縮し、横長形状とした俯瞰映像を生成し、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの横長形状の周辺映像を合成した俯瞰映像を生成する、
    　請求項１１に記載の俯瞰映像生成装置。     When the obstacle information acquisition unit detects the obstacle in front or behind the vehicle, the image generation unit vertically displays a peripheral image from the camera corresponding to the front direction and the rear direction of the vehicle. A horizontal overhead image from the camera corresponding to the direction in which the obstacle information acquisition unit has detected an obstacle is displayed at the center of the generated overhead image. Generate a synthesized bird's-eye view video,
    The overhead view video generation apparatus according to claim 11.
  15. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の前方または後方である場合、前記車両の左方向と右方向とに対応する前記カメラからの縦長形状の周辺映像を縦方向に圧縮し、横長形状とした俯瞰映像を生成し、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの横長形状の周辺映像を合成した俯瞰映像を生成する、
    　請求項１１に記載の俯瞰映像生成装置。     When the direction in which the obstacle information acquisition unit has detected the obstacle is the front or the rear of the vehicle, the video generation unit has a vertically long periphery from the camera corresponding to the left direction and the right direction of the vehicle. The video is compressed in the vertical direction to generate a horizontally-elevated overhead view image, and the oblong information from the camera corresponding to the direction in which the obstacle information acquisition unit detects the obstacle is formed at the center of the generated overhead view image. Generate a bird's-eye view image that combines surrounding images.
    The overhead view video generation apparatus according to claim 11.
  16. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の前方または後方である場合、前記車両の左方向と右方向とに対応する前記カメラからの縦長形状の周辺映像の中央部を除いて上部と下部を合成し、横長形状とした俯瞰映像を生成し、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの横長形状の周辺映像を合成した俯瞰映像を生成する、
    　請求項１１に記載の俯瞰映像生成装置。     When the direction in which the obstacle information acquisition unit has detected the obstacle is the front or the rear of the vehicle, the video generation unit has a vertically long periphery from the camera corresponding to the left direction and the right direction of the vehicle. The top and bottom are synthesized excluding the central part of the video to generate a horizontally long overhead image, and the obstacle information acquisition unit corresponds to the direction in which the obstacle information is detected in the central part of the generated overhead image. Generate a bird's-eye view image that combines the horizontally long surrounding images from the camera.
    The overhead view video generation apparatus according to claim 11.
  17. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の前方または後方である場合、前記車両の左方向と右方向とに対応する前記カメラからの縦長形状の周辺映像を前記障害物情報取得部が障害物を検出した方向に偏重し、横長形状とした俯瞰映像を生成し、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの横長形状の周辺映像を合成した俯瞰映像を生成する、
    　請求項１１に記載の俯瞰映像生成装置。     When the direction in which the obstacle information acquisition unit has detected the obstacle is the front or the rear of the vehicle, the video generation unit has a vertically long periphery from the camera corresponding to the left direction and the right direction of the vehicle. The obstacle information acquisition unit deviates in the direction in which the obstacle information acquisition unit detected the obstacle, generates a horizontally long overhead image, and the obstacle information acquisition unit detects an obstacle in the center of the generated overhead image. Generating a bird's-eye view image by combining a horizontally long peripheral image from the camera corresponding to the direction;
    The overhead view video generation apparatus according to claim 11.
  18. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の前方または後方である場合、前記車両の左方向と右方向とに対応する前記カメラからの縦長形状の周辺映像を前記車両の進行方向に偏重し、横長形状とした俯瞰映像を生成し、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの横長形状の周辺映像を合成した俯瞰映像を生成する、
    　請求項１１に記載の俯瞰映像生成装置。     When the direction in which the obstacle information acquisition unit has detected the obstacle is the front or the rear of the vehicle, the video generation unit has a vertically long periphery from the camera corresponding to the left direction and the right direction of the vehicle. From the camera corresponding to the direction in which the obstacle information acquisition unit detects the obstacle in the center of the generated overhead view video, the video is biased in the traveling direction of the vehicle to generate a horizontally long overhead view image. Generate a bird's-eye view image that combines horizontally long peripheral images.
    The overhead view video generation apparatus according to claim 11.
  19. 　前記映像生成部は、前記障害物情報取得部が障害物を検出した方向が前記車両の前方または後方である場合、前記車両の前方向と後方向とに対応する前記カメラからの周辺映像の中央部を縦方向に圧縮し、横長形状とした俯瞰映像を生成し、生成した俯瞰映像の中央部に、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの横長形状の周辺映像を合成した俯瞰映像を生成する、
    　請求項１１に記載の俯瞰映像生成装置。     The video generation unit, when the direction in which the obstacle information acquisition unit has detected the obstacle is the front or the rear of the vehicle, the center of the peripheral video from the camera corresponding to the front direction and the rear direction of the vehicle A vertical overhead view image is generated by compressing the portion in the vertical direction, and the oblong information from the camera corresponding to the direction in which the obstacle information acquisition unit detects the obstacle is formed at the center of the generated overhead view video. Generate a bird's-eye view image that combines surrounding images.
    The overhead view video generation apparatus according to claim 11.
  20. 　前記俯瞰映像の中央部に合成する映像は、前記障害物情報取得部が障害物を検出した方向に対応する前記カメラからの周辺映像の視点変換前の映像である、
    　請求項１０から１９のいずれか一項に記載の俯瞰映像生成装置。     The video to be combined with the central portion of the overhead video is a video before the viewpoint conversion of the peripheral video from the camera corresponding to the direction in which the obstacle information acquisition unit detects the obstacle,
    The overhead view video generation device according to any one of claims 10 to 19.
  21. 　前記映像生成部は、生成した俯瞰映像の中央部に、前記障害物情報取得部が検出した、車両の進行方向に位置する障害物を検出した方向に対応する前記カメラからの周辺映像を合成した俯瞰映像を生成する、
    　請求項１０から１９のいずれか一項に記載の俯瞰映像生成装置。     The video generation unit synthesizes a peripheral video from the camera corresponding to a direction in which an obstacle located in the traveling direction of the vehicle, detected by the obstacle information acquisition unit, is detected at the center of the generated overhead view video. Generate overhead video,
    The overhead view video generation device according to any one of claims 10 to 19.
  22. 　前記映像生成部は、生成した俯瞰映像の中央部に、前記障害物情報取得部が検出した、車両に最も近い障害物を検出した方向に対応する前記カメラからの周辺映像を合成した俯瞰映像を生成する、
    　請求項１０から１９のいずれか一項に記載の俯瞰映像生成装置。     The video generation unit generates a bird's-eye view video obtained by synthesizing a peripheral video from the camera corresponding to the direction in which the obstacle closest to the vehicle is detected, which is detected by the obstacle information acquisition unit, in a central portion of the generated overhead view video Generate,
    The overhead view video generation device according to any one of claims 10 to 19.
  23. 　請求項１から２２のいずれか一項に記載の俯瞰映像生成装置と、
    　前記車両の周辺を撮影し前記映像取得部に周辺映像を供給するカメラと、前記車両の周辺において障害物を検出し前記障害物情報取得部に障害物情報を供給する障害物検出部と、前記表示部との少なくともいずれかとを備える、
    　俯瞰映像生成システム。     An overhead view video generation device according to any one of claims 1 to 22,
    A camera that captures the periphery of the vehicle and supplies a peripheral image to the video acquisition unit; an obstacle detection unit that detects an obstacle in the vicinity of the vehicle and supplies the obstacle information to the obstacle information acquisition unit; Comprising at least one of a display unit and
    Overhead video generation system.
  24. 　車両の周辺を撮影した周辺映像を取得する映像取得ステップと、
    　前記車両の周辺において検出した障害物の障害物情報を取得する障害物情報取得ステップと、
    　前記映像取得ステップで取得した周辺映像から前記車両を上方から見下ろすように視点変換処理を行った俯瞰映像を生成し、前記俯瞰映像の中央部に、前記障害物情報取得ステップで取得した障害物を示す情報を合成した俯瞰映像を生成する映像生成ステップと、
    　前記映像生成ステップで生成した俯瞰映像を表示部に表示させる表示制御ステップと
    　を有することを特徴とする俯瞰映像生成方法。     A video acquisition step for acquiring a peripheral video of the periphery of the vehicle;
    Obstacle information acquisition step for acquiring obstacle information of obstacles detected in the vicinity of the vehicle;
    From the peripheral video acquired in the video acquisition step, an overhead view video that has been subjected to viewpoint conversion processing so as to look down on the vehicle from above is generated, and the obstacle acquired in the obstacle information acquisition step is located in the center of the overhead video A video generation step of generating a bird's-eye view video that combines the information shown;
    And a display control step for displaying the overhead view video generated in the video generation step on a display unit.
  25. 　車両の周辺を撮影した周辺映像を取得する映像取得ステップと、
    　前記車両の周辺において検出した障害物の障害物情報を取得する障害物情報取得ステップと、
    　前記映像取得ステップで取得した周辺映像から前記車両を上方から見下ろすように視点変換処理を行った俯瞰映像を生成し、前記俯瞰映像の中央部に、前記障害物情報取得ステップで取得した障害物を示す情報を合成した俯瞰映像を生成する映像生成ステップと、
    　前記映像生成ステップで生成した俯瞰映像を表示部に表示させる表示制御ステップと
    　を俯瞰映像生成装置として動作するコンピュータに実行させるためのプログラム。     A video acquisition step for acquiring a peripheral video of the periphery of the vehicle;
    Obstacle information acquisition step for acquiring obstacle information of obstacles detected in the vicinity of the vehicle;
    From the peripheral video acquired in the video acquisition step, an overhead view video that has been subjected to viewpoint conversion processing so as to look down on the vehicle from above is generated, and the obstacle acquired in the obstacle information acquisition step is located in the center of the overhead video A video generation step of generating a bird's-eye view video that combines the information shown;
    A program for causing a computer that operates as an overhead video generation device to execute a display control step of displaying the overhead video generated in the video generation step on a display unit.
PCT/JP2017/024558 2016-09-20 2017-07-04 Overhead view video image generation device, overhead view video image generation system, overhead view video image generation method, and program WO2018055873A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201780023741.3A CN109076195B (en) 2016-09-20 2017-07-04 Overhead image generation device, system, method and storage medium
EP17852654.7A EP3462733B1 (en) 2016-09-20 2017-07-04 Overhead view video image generation device, overhead view video image generation system, overhead view video image generation method, and program
US16/208,650 US10587801B2 (en) 2016-09-20 2018-12-04 Bird's-eye view video generation device, bird'S-eye view video generation system, bird's-eye view video generation method, and non-transitory storage medium

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
JP2016183103 2016-09-20
JP2016-183103 2016-09-20
JP2016184461A JP6644264B2 (en) 2016-09-21 2016-09-21 Overhead video generation device, overhead video generation system, overhead video generation method and program
JP2016-184461 2016-09-21
JP2017-073368 2017-04-03
JP2017073368A JP6730617B2 (en) 2016-09-20 2017-04-03 Overhead video generation device, overhead video generation system, overhead video generation method and program

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/208,650 Continuation US10587801B2 (en) 2016-09-20 2018-12-04 Bird's-eye view video generation device, bird'S-eye view video generation system, bird's-eye view video generation method, and non-transitory storage medium

Publications (1)

Publication Number Publication Date
WO2018055873A1 true WO2018055873A1 (en) 2018-03-29

Family

ID=61689394

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/024558 WO2018055873A1 (en) 2016-09-20 2017-07-04 Overhead view video image generation device, overhead view video image generation system, overhead view video image generation method, and program

Country Status (1)

Country Link
WO (1) WO2018055873A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112470465A (en) * 2018-07-30 2021-03-09 Jvc建伍株式会社 Bird's-eye view image generation device, bird's-eye view image generation method, and program

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006341641A (en) * 2005-06-07 2006-12-21 Nissan Motor Co Ltd Image display apparatus and image display method
JP2008174076A (en) * 2007-01-18 2008-07-31 Xanavi Informatics Corp Vehicle periphery monitoring device, and its displaying method
WO2012144076A1 (en) * 2011-04-22 2012-10-26 トヨタ自動車株式会社 Vehicle periphery monitoring device and display method for vehicle periphery monitoring information
JP2015065515A (en) * 2013-09-24 2015-04-09 日立建機株式会社 System for detecting moving object around vehicle

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006341641A (en) * 2005-06-07 2006-12-21 Nissan Motor Co Ltd Image display apparatus and image display method
JP2008174076A (en) * 2007-01-18 2008-07-31 Xanavi Informatics Corp Vehicle periphery monitoring device, and its displaying method
WO2012144076A1 (en) * 2011-04-22 2012-10-26 トヨタ自動車株式会社 Vehicle periphery monitoring device and display method for vehicle periphery monitoring information
JP2015065515A (en) * 2013-09-24 2015-04-09 日立建機株式会社 System for detecting moving object around vehicle

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112470465A (en) * 2018-07-30 2021-03-09 Jvc建伍株式会社 Bird's-eye view image generation device, bird's-eye view image generation method, and program
EP3817373A4 (en) * 2018-07-30 2021-07-14 JVCKenwood Corporation Overhead image generation apparatus, overhead image generation method, and program
US11498484B2 (en) 2018-07-30 2022-11-15 Jvckenwood Corporation Overhead image generation device, overhead image generation method, and program
CN112470465B (en) * 2018-07-30 2023-09-29 Jvc建伍株式会社 Overhead image generation device and computer-readable storage medium

Similar Documents

Publication Publication Date Title
CN109076195B (en) Overhead image generation device, system, method and storage medium
JP6379779B2 (en) Vehicle display device
JP7196881B2 (en) OVERALL VIEW VIDEO GENERATION DEVICE AND OVERALL VIEW VIDEO GENERATION METHOD
WO2017110144A1 (en) Overhead view video image generation device, overhead view video image generation system, overhead view video image generation method, and program
EP3142066A1 (en) Image synthesizer for a surround monitoring system
JP2009239674A (en) Vehicular periphery display device
WO2018159017A1 (en) Vehicle display control device, vehicle display system, vehicle display control method and program
WO2018030491A1 (en) Peripheral inspection device
JP2006246430A (en) Vehicle vicinity monitoring apparatus
WO2011013813A1 (en) In-vehicle device and image processing program
JP2018144554A (en) Head-up display device for vehicle
WO2018055873A1 (en) Overhead view video image generation device, overhead view video image generation system, overhead view video image generation method, and program
JP2008003659A (en) Surroundings display device for vehicle
JP2011211432A (en) In-vehicle imaging apparatus
JP6730606B2 (en) Overhead video generation device, overhead video generation system, overhead video generation method and program
JP2018148314A (en) Bird&#39;s eye view image generation apparatus, bird&#39;s eye view image generation system, bird&#39;s eye view image generation method, and program
JP4713033B2 (en) Vehicle surrounding environment display device
JP2017114218A (en) Bird&#39;s-eye video generation device, bird&#39;s-eye video generation system, bird&#39;s-eye video generation method and program
JP2010245701A (en) Display
JP2010009491A (en) Driving assist system, driving assist method, and driving assist program
JP2018050180A (en) Overhead video generating device, overhead video generating system, overhead video generating method and program
JP6644256B2 (en) Overhead video generation device, overhead video generation system, overhead video generation method and program
JP6590538B2 (en) Image processing apparatus and image processing method
JP2016063352A (en) On-vehicle display device
JP2018019176A (en) Display controller for vehicle, display system for vehicle, display control method for vehicle and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17852654

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017852654

Country of ref document: EP

Effective date: 20181228

NENP Non-entry into the national phase

Ref country code: DE