JP2018142882A - Bird's eye video creation device, bird's eye video creation system, bird's eye video creation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display a vehicle and detected obstacles such that a positional relationship between the vehicle and obstacles can be intuitively grasped.SOLUTION: A bird's eye video creation device comprises: a video data acquisition part 42 that acquires periphery video data obtained by photographing the periphery of a vehicle; an obstacle information acquisition part 43 that acquires positional information on objects detected on the periphery of the vehicle; a bird's eye video creation part 45 that creates a composed bird's eye video obtained by performing viewpoint conversion processing such that a virtual viewpoint is located separated from a position right above the vehicle in at least either one of a vehicle width direction or a fore-and-aft direction, so that a bird's eye view of the vehicle and obstacles can be seen in the periphery video acquired by the video data acquisition part 42, on the basis of the positional information on the obstacles acquired by the obstacle information acquisition part 43; and a display control part 48 that displays the bird's eye video created by the bird's eye video creation part 45 on a display panel 30.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an overhead video generation device, an overhead video generation system, an overhead video generation method, and a program.

  A technique is known in which a plurality of cameras installed around the vehicle are photographed around the vehicle, a viewpoint conversion process is performed on the plurality of photographed images, and a combined overhead image is displayed on a monitor (see, for example, Patent Document 1). ). In this technique, when predetermined conditions regarding the vehicle speed and the obstacle are satisfied, the peripheral image displayed on the display unit is switched to another peripheral image having a different display range.

JP 2012-227699 A

  When the display range of the peripheral image displayed on the display unit is switched and enlarged, the obstacle is displayed relatively small. As the displayed obstacles become smaller, it may be difficult to correctly recognize the obstacles. When the displayed obstacles are small, it may be difficult to correctly grasp the positional relationship between the obstacles and the vehicle. Thus, there is room for improvement in displaying the detected obstacle in a bird's-eye view image.

  The present invention has been made in view of the above, and an object thereof is to display the positional relationship between a detected obstacle and a vehicle so that the positional relationship can be intuitively understood.

  In order to solve the above-described problems and achieve the object, an overhead image generation apparatus according to the present invention includes a video data acquisition unit that acquires peripheral video data obtained by photographing the periphery of a vehicle, and a failure detected in the vicinity of the vehicle. An obstacle information acquisition unit that acquires position information of an object, and a peripheral image acquired by the video data acquisition unit based on the position information of the obstacle acquired by the obstacle information acquisition unit, the vehicle and the obstacle A bird's-eye view image generated by performing a viewpoint conversion process so that a virtual viewpoint is located at a distance from the position directly above the vehicle in at least one of the vehicle width direction and the front-rear direction so that an object can be seen overhead It has a video production | generation part and the display control part which displays the overhead view video which the said overhead view video production | generation part produced | generated on the display part.

  An overhead view video generation system according to the present invention includes at least one of the above-described overhead view video generation device, a plurality of cameras that capture the periphery of the vehicle, the display unit, and a detection unit that detects an obstacle and outputs obstacle information. Is provided.

  The bird's-eye view image generation method according to the present invention includes a video data acquisition step of acquiring peripheral video data obtained by photographing the periphery of the vehicle, an obstacle information acquisition step of acquiring position information of the obstacle detected in the periphery of the vehicle, Based on the position information of the obstacle acquired in the obstacle information acquisition step, the position directly above the vehicle so that the vehicle and the obstacle can overlook the peripheral video acquired in the video data acquisition step. A bird's-eye view video generation step for generating a bird's-eye view image synthesized by performing viewpoint conversion processing so that the virtual viewpoint is located at a distance from the vehicle width direction or at least in the front-rear direction; And a display control step for displaying the video on the display unit.

  The program according to the present invention includes a video data acquisition step of acquiring peripheral video data obtained by photographing the periphery of a vehicle, an obstacle information acquisition step of acquiring position information of an obstacle detected in the vicinity of the vehicle, and the obstacle Based on the position information of the obstacle acquired in the information acquisition step, the vehicle width from the position directly above the vehicle so that the vehicle and the obstacle can overlook the peripheral image acquired in the video data acquisition step. A bird's-eye view video generation step for generating a bird's-eye view video synthesized by performing viewpoint conversion processing so that the virtual viewpoint is located at least one of the direction and the front-rear direction; The display control step to be displayed on the screen is executed by a computer that operates as an overhead video generation device.

  According to the present invention, there is an effect that the positional relationship between the detected obstacle and the vehicle can be displayed in an intuitive manner.

FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment. FIG. 2 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. FIG. 3 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment. FIG. 4 is a schematic diagram illustrating the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. FIG. 5 is a schematic diagram illustrating the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. FIG. 6 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. FIG. 7 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. FIG. 8 is a flowchart showing the flow of processing in the overhead view video generation system according to the first embodiment. FIG. 9 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the second embodiment. FIG. 10 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the third embodiment. FIG. 11 is a schematic diagram illustrating another example of the position of the virtual viewpoint in the overhead view video generation system according to the third embodiment. FIG. 12 is a schematic diagram illustrating another example of the position of the virtual viewpoint in the overhead view video generation system according to the third embodiment. FIG. 13 is a schematic diagram illustrating another example of the position of the virtual viewpoint in the overhead view video generation system according to the third embodiment. FIG. 14 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the fourth embodiment. FIG. 15 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the fifth embodiment.

  DESCRIPTION OF EMBODIMENTS Embodiments of an overhead video generation device 40, an overhead video generation system 1, an overhead video generation method, and a program according to the present invention will be described below in detail with reference to the accompanying drawings. In addition, this invention is not limited by the following embodiment.

  In the following description, the front-rear direction is a direction parallel to the traveling direction when the vehicle is traveling straight, and the front side from the driver's seat toward the windshield is “front” in the front-rear direction, and the side from the windshield toward the driver's seat is “After” the direction. The front-rear direction is the X-axis direction. The left-right direction is a direction that is horizontally orthogonal to the front-rear direction. To the windshield, the left hand side is “left” and the right hand side is “right”. The left-right direction is the Y-axis direction. The vertical direction is a direction orthogonal to the front-rear direction and the left-right direction. The vertical direction is the Z-axis direction. Therefore, the front-rear direction, the left-right direction, and the vertical direction are orthogonal in three dimensions. In the following description, front and rear, left and right, and top and bottom are front and rear, left and right, and top and bottom in a state where the overhead view video generation system 1 is mounted on a vehicle.

[First embodiment]
FIG. 1 is a block diagram illustrating a configuration example of an overhead video generation system according to the first embodiment. The overhead image generation system 1 is mounted on the vehicle V. In addition to what is mounted on the vehicle V, the overhead image generation system 1 may be a portable device that can be used in the vehicle V.

  The overhead view video generation system 1 will be described with reference to FIG. The overhead view video generation system 1 includes a front camera 11, a rear camera 12, a left side camera 13, a right side camera 14, a sensor group (obstacle sensor) 20, a display panel (display unit) 30, and an overhead view video. And a generation device 40.

  The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 will be described with reference to FIG. FIG. 2 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. The front camera 11 is disposed in front of the vehicle V and captures a periphery around the front of the vehicle V. For example, the front camera 11 captures an imaging range A1 of about 180 °. The front camera 11 outputs the captured video to the video data acquisition unit 42 of the overhead video generation device 40.

  The rear camera 12 is disposed behind the vehicle V and photographs the periphery around the rear of the vehicle V. For example, the rear camera 12 captures an imaging range A2 of about 180 °. The rear camera 12 outputs the captured video to the video data acquisition unit 42 of the overhead video generation device 40.

  The left-side camera 13 is disposed on the left side of the vehicle V and photographs the periphery around the left side of the vehicle V. The left side camera 13 captures an imaging range A3 of about 180 °, for example. The left side camera 13 outputs the captured video to the video data acquisition unit 42 of the overhead video generation device 40.

  The right-side camera 14 is disposed on the right side of the vehicle V and photographs the periphery around the right side of the vehicle V. The right side camera 14 captures an imaging range A4 (not shown) of about 180 °, for example. The right-side camera 14 outputs the captured video to the video data acquisition unit 42 of the overhead view video generation device 40.

  The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 photograph all directions of the vehicle V.

  Returning to FIG. 1, the sensor group 20 is a detection unit that detects an obstacle Q around the vehicle V. The sensor group 20 can detect the obstacle Q in a range including the display range A of the overhead view video. In the present embodiment, the sensor group 20 includes a front sensor, a rear sensor, a left side sensor, and a right side sensor. The sensor group 20 is capable of sensing from a distance of several tens of meters to several hundreds of meters depending on the sensing method. However, when used for this purpose, the sensor group 20 detects an obstacle Q at a distance of about 5 m from the vehicle V. For the sensor group 20, various methods such as an infrared sensor, an ultrasonic sensor, a millimeter wave sensor, a sensor based on image recognition, and the like can be applied.

  The front sensor is disposed in front of the vehicle V and detects an obstacle Q existing in a range centered on the front of the vehicle V. The front sensor detects an object having a height from the ground that may come into contact with the vehicle V when the vehicle V is moving forward. The front sensor detects an obstacle Q at a distance of about 5 m from the vehicle V, for example. The detection range of the front sensor overlaps with the shooting range A1 of the front camera 11. The detection range of the front sensor may overlap with a part of the detection range of the left side sensor and the right side sensor. The front sensor is composed of a combination of a plurality of sensors. Thereby, the direction of the obstacle Q is subdivided and detected. The front sensor outputs the obstacle information of the detected obstacle Q to the obstacle information acquisition unit 43 of the overhead view video generation device 40.

  The rear sensor is disposed behind the vehicle V and detects an obstacle Q existing in a range centered on the rear of the vehicle V. The rear sensor detects an object having a height from the ground that may come into contact with the vehicle V when the vehicle V is moving backward. The rear sensor detects an obstacle Q at a distance of about 5 m from the vehicle V, for example. The detection range of the rear sensor overlaps with the shooting range A2 of the rear camera 12. The detection range of the rear sensor may overlap with a part of the detection range of the left side sensor and the right side sensor. The rear sensor is composed of a combination of a plurality of sensors. Thereby, the direction of the obstacle Q is subdivided and detected. The rear sensor outputs the obstacle information of the detected obstacle Q to the obstacle information acquisition unit 43 of the overhead view video generation device 40.

  The left side sensor is disposed on the left side of the vehicle V and detects an obstacle Q existing in a range centered on the left side of the vehicle V. The left side sensor detects an object having a height from the ground that may come into contact with the vehicle V when the vehicle V is moving forward or backward while steering. The left side sensor detects an obstacle Q at a distance of about 5 m from the vehicle V, for example. The detection range of the left side sensor overlaps with the shooting range A3 of the left side camera 13. The detection range of the left side sensor may overlap with a part of the detection range of the front sensor and the rear sensor. The left side sensor is composed of a combination of a plurality of sensors. Thereby, the direction of the obstacle Q is subdivided and detected. The left side sensor outputs the detected obstacle information of the obstacle Q to the obstacle information acquisition unit 43 of the overhead view video generation device 40.

  The right side sensor is disposed on the right side of the vehicle V and detects an obstacle Q existing in a range centered on the right side of the vehicle V. The right side sensor detects an object having a height from the ground that may come into contact with the vehicle V when the vehicle V is moving forward or backward while steering. The right side sensor detects an obstacle Q at a distance of about 5 m from the vehicle V, for example. The detection range of the right side sensor overlaps the shooting range A4 of the right side camera 14. The detection range of the right side sensor may overlap with a part of the detection range of the front sensor and the rear sensor. The right side sensor is composed of a combination of a plurality of sensors. Thereby, the direction of the obstacle Q is subdivided and detected. The right side sensor outputs the detected obstacle information of the obstacle Q to the obstacle information acquisition unit 43 of the overhead view video generation device 40.

  The display panel 30 is a display including, for example, a liquid crystal display (LCD) or an organic EL-Organic Electro-Luminescence (EL) display. The display panel 30 displays the overhead view video 100 (see FIG. 3) based on the video signal output from the overhead view video generation device 40 of the overhead view video generation system 1. The display panel 30 may be dedicated to the overhead view video generation system 1 or may be used jointly with other systems including a navigation system, for example. The display panel 30 is disposed at a position that is easily visible to the driver.

  When the shape of the display panel 30 is a horizontally long rectangle, the display panel 30 may be divided into a plurality of display ranges. For example, the display panel 30 has a display range for displaying the bird's-eye view video 100 and a display range for displaying a navigation screen or an audio screen arranged on the side of the display range of the bird's-eye view video 100. The display range for displaying the bird's-eye view video 100 is a vertically long rectangular shape.

  The overhead video generation device 40 includes a control unit 41 and a storage unit 49.

  The control unit 41 is an arithmetic processing unit configured with, for example, a CPU (Central Processing Unit). The control unit 41 loads the program stored in the storage unit 49 into the memory and executes instructions included in the program. The control unit 41 includes a video data acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, an overhead video generation unit 45, and a display control unit 48.

  The video data acquisition unit 42 acquires peripheral video data obtained by photographing the periphery of the vehicle V. More specifically, the video data acquisition unit 42 acquires peripheral video data output by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The video data acquisition unit 42 outputs the acquired peripheral video data to the overhead video generation unit 45.

  The obstacle information acquisition unit 43 acquires the obstacle information of the obstacle Q detected in the vicinity of the vehicle V, and specifies the position of the detected obstacle Q on the overhead view video. More specifically, the obstacle information acquisition unit 43 acquires the obstacle information output by the sensor group 20. The obstacle information acquisition unit 43 subdivides and acquires the direction of the obstacle Q based on the attachment position of the sensor that outputs the obstacle information and the detection range of the obstacle Q by the sensor. The obstacle information acquisition unit 43 acquires obstacle information including the distance to the detected obstacle Q. The obstacle information acquisition unit 43 specifies the position of the obstacle Q on the overhead view video based on the acquired direction and distance of the obstacle Q.

  The vehicle information acquisition unit 44 acquires vehicle information serving as a trigger for displaying the bird's-eye view image 100 such as gear operation information of the vehicle V from a CAN (Controller Area Network) or various sensors that sense the state of the vehicle V. . In the present embodiment, the vehicle information includes information indicating the traveling direction of the vehicle V. The vehicle information acquisition unit 44 outputs the acquired vehicle information to the overhead view video generation unit 45.

  The bird's-eye view video generation unit 45 generates a bird's-eye view video 100 with the virtual viewpoint P above the vehicle V by performing a viewpoint conversion process on the peripheral video data acquired by the video data acquisition unit 42 and combining it.

  The virtual viewpoint P will be described with reference to FIG. The virtual viewpoint P is located above the center of the vehicle V. The virtual viewpoint P is a viewpoint that looks down from directly above the center of the vehicle V. The center of the vehicle V is the center in the vehicle width direction of the vehicle V and the center in the front-rear direction. “Directly above the vehicle V” is a position on the vertical line of the reference plane of the vehicle V. The reference plane is a plane that is horizontal to the road surface when the vehicle V is positioned on a horizontal and flat road surface. The position of the virtual viewpoint P is assumed to be (x, y, z).

  The generated overhead image 100 will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of an overhead video generated by the overhead video generation system according to the first embodiment. The bird's-eye view video 100 combines and displays a video obtained by performing viewpoint conversion processing on the surrounding video in the display range A as a virtual viewpoint P. The bird's-eye view image 100 includes a front image 101, a rear image 102, a left side image 103, and a right side image 104. The display area of the front image 101 and the display area of the rear image 102 are the same area. The display area of the left side image 103 and the display area of the right side image 104 are the same. In the central part of the bird's-eye view video 100, a host vehicle icon 200 indicating the vehicle V is displayed. The host vehicle icon 200 indicates a form in which the vehicle V is looked down from directly above.

  In FIG. 3, a composite boundary B1 between the front video 101 and the left video 103, a composite boundary B2 between the front video 101 and the right video 104, a composite boundary B3 between the rear video 102 and the left video 103, Although an oblique broken line indicating the composite boundary B4 of the rear image 102 and the right-side image 104 is illustrated for explanation, the broken line is displayed in the overhead image 100 actually displayed on the display panel 30. May not be displayed. The same applies to the other figures. In the following description, when it is not necessary to particularly distinguish the composite boundary B1, the composite boundary B2, the composite boundary B3, and the composite boundary B4, the composite boundary B will be described.

  The bird's-eye view video generation unit 45 uses the position information of the obstacle Q acquired by the obstacle information acquisition unit 43 to display the surrounding video acquired by the video data acquisition unit 42 when the obstacle satisfying the predetermined condition exists. The viewpoint conversion process is performed so that the virtual viewpoint PA is located at a distance from the virtual viewpoint P located directly above the vehicle V in at least one of the vehicle width direction and the front-rear direction so that the V and the obstacle Q can be seen from above. Go and compose a bird's-eye view video.

  The predetermined condition is a condition for detecting the obstacle Q. In the present embodiment, the predetermined condition is that the obstacle Q is positioned in the traveling direction of the vehicle V. When the obstacle Q is located in the traveling direction of the vehicle V, it is determined that the predetermined condition is satisfied, and is detected as the obstacle Q. For example, when the vehicle V is moving backward and the obstacle Q is located behind the front end of the vehicle V, it is determined that the predetermined condition is satisfied, and is detected as the obstacle Q. More specifically, when the vehicle V is moving backward, the control unit 41 sets a predetermined condition when a sensor that detects an obstacle Q behind the front end of the vehicle V in the sensor group 20 detects the obstacle Q. It determines with satisfy | filling, and detects as the obstruction Q. In the present embodiment, a case where the vehicle V is moving backward will be described.

  The bird's-eye view video generation unit 45 includes a viewpoint conversion processing unit 451, a cut-out processing unit 452, and a composition processing unit 453.

  The viewpoint conversion processing unit 451 performs viewpoint conversion processing on the surrounding video data acquired by the video data acquisition unit 42 so that the vehicle V is looked down from the upper virtual viewpoint P. More specifically, the viewpoint conversion processing unit 451 generates a video on which the viewpoint conversion processing has been performed based on peripheral video data captured by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method of viewpoint conversion processing may be any known method and is not limited. The viewpoint conversion processing unit 451 outputs the peripheral video data that has undergone the viewpoint conversion processing to the clipping processing unit 452.

  Based on the position information of the obstacle Q acquired by the obstacle information acquisition unit 43, the viewpoint conversion processing unit 451 can overlook the vehicle V and the obstacle Q when there is an obstacle Q that satisfies a predetermined condition. An image is generated by performing viewpoint conversion processing so that the virtual viewpoint PA is positioned at least in either the vehicle width direction or the front-rear direction from the virtual viewpoint P positioned directly above the vehicle V. More specifically, the viewpoint conversion processing unit 451 determines the position of the virtual viewpoint so that the relative positional relationship between the vehicle V and the obstacle Q can be recognized when the obstacle Q located in the traveling direction of the vehicle V exists. An image that has undergone viewpoint conversion processing is generated as a virtual viewpoint PA that is separated from the virtual viewpoint P. The viewpoint conversion processing unit 451 outputs the peripheral video data that has undergone the viewpoint conversion processing to the clipping processing unit 452.

  Being able to recognize the relative positional relationship between the vehicle V and the obstacle Q means that the vehicle V and the obstacle Q are represented without overlapping in the overhead view image. Furthermore, being able to recognize the relative positional relationship between the vehicle V and the obstacle Q means that when the vehicle V or the obstacle Q moves, according to a change in the relative distance between the vehicle V and the obstacle Q, This means that the relative positional relationship between the vehicle V and the obstacle Q in the overhead view image changes.

  The virtual viewpoint PA is preferably at a position where the angle of view of the overhead view video in the direction in which the obstacle is detected is approximately the same as the angle of view of the overhead view video 100 of the virtual viewpoint P. The overhead video 100 of the virtual viewpoint P and the overhead video obtained by moving the virtual viewpoint to the virtual viewpoint PA are the same video, but the region on the appropriate virtual projection plane corresponding to the position of the virtual viewpoint is used for the overhead video. . For this reason, when the change in the angle of view of the overhead view video in the direction in which the obstacle is detected is large, it is easy to grasp the position of the obstacle on the overhead view video when the virtual viewpoint position is changed. Specifically, when the virtual viewpoint PA is the rear of the vehicle V, the rear video in the direction in which the virtual viewpoint PA is located is approximately the same as the cut-out angle of the rear video 102 that is the virtual viewpoint P, and the virtual viewpoint PA And the front image in the opposite direction across the vehicle V has a cut-out angle of view wider than the front image 101 with the virtual viewpoint P in front of the vehicle. In this way, it is possible to generate a bird's-eye view video as if looking down from the virtual viewpoint PA.

  The position of the virtual viewpoint PA that is separated from the virtual viewpoint P in the vehicle width direction or the front-rear direction is located on a curved surface centered on the vehicle V as shown in FIGS. FIG. 4 is a schematic diagram illustrating the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. FIG. 5 is a schematic diagram illustrating the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. Since the angle of view of the overhead view video with the virtual viewpoint PA on the curved surface is not different from the angle of view of the overhead view video 100 with the virtual viewpoint P, the size of the obstacle Q displayed on the overhead view video does not change. For this reason, even if the position of the virtual viewpoint is changed, a bird's-eye view video that makes it easy to intuitively grasp the relative positional relationship between the obstacle Q and the vehicle V is generated.

  The virtual viewpoint PA is preferably located at a position farther from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q. Thereby, the bird's-eye view image in which the relative positional relationship between the obstacle Q and the vehicle V can be easily recognized is generated.

  In the present embodiment, as illustrated in FIGS. 6 and 7, the viewpoint conversion processing unit 451 is configured so that the left rear virtual viewpoint with respect to the virtual viewpoint P is detected when the obstacle Q located at the left rear of the vehicle V is detected. An image that has undergone viewpoint conversion processing is generated as PA1. FIG. 6 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. FIG. 7 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the first embodiment. The position of the virtual viewpoint PA1 is represented by (xA1, yA1, zA1).

  The virtual viewpoint PA1 will be described. In the present embodiment, the virtual viewpoint PA1 is located on the left rear side of the vehicle V. The virtual viewpoint PA1 is located on the left front side from the obstacle Q in the Z-axis direction view. The virtual viewpoint PA1 is located on the left rear side of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PA1 is further away from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q. The virtual viewpoint PA1 is a viewpoint in which the vehicle V is looked down obliquely from the left rear.

  The cutout processing unit 452 performs a cutout process for cutting out a predetermined range of video from the peripheral video data that has undergone the viewpoint conversion processing. The cutout processing unit 452 cuts out the front cutout range from the peripheral video data from the front camera 11 that has undergone the viewpoint conversion process. The cutout processing unit 452 cuts out the rear cutout range from the peripheral video data from the rear camera 12 that has performed the viewpoint conversion process. The cut-out processing unit 452 cuts out the left-side cut-out range from the peripheral video data from the left-side camera 13 that has undergone the viewpoint conversion process. The cut-out processing unit 452 cuts out the right-side cut-out range from the peripheral video data from the right-side camera 14 that has undergone the viewpoint conversion process. The cutout processing unit 452 outputs the video data of the video that has undergone the cutout processing to the composition processing unit 453.

  The front cutout range is a range in which the front video 101 is cut out from the peripheral video data subjected to the viewpoint conversion process. The front cutout range is a range ahead from the front end of the vehicle V. The rear clipping range is a range in which the rear video 102 is cut out from the peripheral video data subjected to the viewpoint conversion process. The rear cutout range is a range behind the rear end of the vehicle V. The left side cutout range is a range where the left side video 103 is cut out from the peripheral video data subjected to the viewpoint conversion process. The left side cut-out range is a left range from the left side of the vehicle V. The right side cutout range is a range in which the right side video 104 is cut out from the peripheral video data subjected to the viewpoint conversion process. The right side cutout range is a rightward range from the right side portion of the vehicle V.

  The composition processing unit 453 synthesizes a plurality of videos cut out by the cut-out processing unit 452 to generate an overhead view video. The composition processing unit 453 outputs the generated overhead view video to the display control unit 48.

  The composition processing unit 453 synthesizes a host vehicle icon imitating the form of the host vehicle viewed from the virtual viewpoint in the center of the synthesized overhead view video when synthesizing the video subjected to the viewpoint conversion processing. The own vehicle icon when the virtual viewpoint position is the virtual viewpoint P when looking down from directly above the vehicle V is such that the own vehicle is viewed from directly above like the own vehicle icon 200 shown in FIG. . On the other hand, when the virtual viewpoint position is set at a position separated from the virtual viewpoint P, it is preferable that the host vehicle is viewed obliquely from the virtual viewpoint. When such a form is adopted, it is natural to display a bird's-eye view image on the opposite side across the host vehicle from a virtual viewpoint with a display close to the host vehicle behind the host vehicle icon. Even in such a case, it is preferable that the host vehicle icon has a display form that does not cause a blind spot due to the host vehicle icon. For example, the host vehicle icon may be translucent. For example, the host vehicle icon may have a frame shape indicating the outer shape.

  Returning to FIG. 1, the display control unit 48 causes the display panel 30 to display the overhead view video.

  The storage unit 49 stores data and various processing results required for various processes in the overhead view video generation device 40. The storage unit 49 is, for example, a semiconductor memory device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described with reference to FIG. FIG. 8 is a flowchart showing the flow of processing in the overhead view video generation system according to the first embodiment.

  When the overhead video generation system 1 is activated, the control unit 41 causes the video data acquisition unit 42 to acquire peripheral video data. The control unit 41 causes the obstacle information acquisition unit 43 to acquire obstacle information.

  The control unit 41 determines whether or not to start the overhead view video display (step S11). In the present embodiment, the control unit 41 determines whether to start bird's-eye view video display based on the presence or absence of a reverse trigger. The reverse trigger means that, for example, the shift position is “reverse”. Alternatively, the reverse trigger means that the traveling direction of the vehicle V is the rear of the vehicle V. When there is no reverse trigger, the control unit 41 determines not to start the overhead view video display (No in step S11), and executes the process of step S11 again. When there is a backward trigger, the control unit 41 determines to start the overhead view video display (Yes in Step S11), and proceeds to Step S12. The trigger for starting the bird's-eye view video display is not limited to the reverse trigger, and any trigger such as a user operation, an obstacle detection result, or a stop is applied.

  The control unit 41 determines whether an obstacle Q has been detected (step S12). More specifically, the control unit 41 determines whether or not the obstacle information acquisition unit 43 has acquired the obstacle information of the obstacle Q that satisfies the predetermined condition. When it is determined that the obstacle information of the obstacle Q that satisfies the predetermined condition has been acquired (Yes in Step S12), the control unit 41 proceeds to Step S13. When it is determined that the obstacle information of the obstacle Q that satisfies the predetermined condition has not been acquired (No in step S12), the control unit 41 proceeds to step S17.

  The control part 41 acquires the positional information on the obstacle Q with respect to the vehicle V (step S13). More specifically, the control unit 41 causes the obstacle information acquisition unit 43 to acquire the position of the obstacle Q on the bird's-eye view video based on the acquired obstacle information. The control unit 41 proceeds to step S14.

  The control unit 41 determines whether or not the obstacle Q is located in the display range A of the overhead view video (step S14). More specifically, the control unit 41 determines whether or not the obstacle Q is in the overhead image display range A based on the position information of the obstacle Q acquired in step S13. When the control unit 41 determines that the obstacle Q is within the display range A of the overhead view video (Yes in step S14), the control unit 41 proceeds to step S16. If the control unit 41 determines that the obstacle Q is not within the overhead image display range A (No in step S14), the control unit 41 proceeds to step S15.

  The control unit 41 generates and displays the overhead view video 100 (step S15). More specifically, the control unit 41 generates a bird's-eye view image 100 by performing a viewpoint conversion process so that the vehicle V is looked down from above with a virtual viewpoint P from the surrounding image data acquired by the image data acquisition unit 42 in the overhead view image generation unit 45. Display on the display panel 30. The control unit 41 proceeds to step S13.

  The control unit 41 changes the virtual viewpoint position (step S16). More specifically, the control unit 41 causes the overhead view video generation unit 45 to change the position of the virtual viewpoint P of the overhead view video 100 to generate the overhead view video as the virtual viewpoint PA, and display it on the display panel 30. The control unit 41 proceeds to step S18.

  In the present embodiment, the control unit 41 causes the overhead view video generation unit 45 to generate a video in which the viewpoint conversion processing is performed as the left rear virtual viewpoint PA1 with respect to the virtual viewpoint P, and displays the video on the display panel 30.

  The control unit 41 generates and displays the overhead view video 100 (step S17). The control unit 41 performs the same process as step S15.

  The controller 41 determines whether or not to end the overhead view video display (step S18). More specifically, the control unit 41 determines whether or not to end the bird's-eye view video display based on the presence or absence of a reverse end trigger. The reverse end trigger means, for example, that the shift position has changed from “reverse” to another position. If there is a reverse end trigger, the control unit 41 determines to end the overhead view video display (Yes in step S18), and ends the process. If there is no reverse end trigger, the control unit 41 determines not to end the overhead view video display (No in step S18), returns to step S12, and continues the process.

  In this way, the overhead view video generation system 1 changes the position of the virtual viewpoint to the virtual viewpoint PA separated from the virtual viewpoint P located directly above the vehicle V when the obstacle Q is detected in the traveling direction. Generate video.

  As described above, in the present embodiment, when the obstacle Q is detected in the traveling direction, the overhead view video in which the position of the virtual viewpoint is changed to the virtual viewpoint PA is generated. Accordingly, in the present embodiment, when the obstacle Q is detected in the traveling direction, the position of the virtual viewpoint is shifted, and the overhead view image showing the relative positional relationship between the obstacle Q and the vehicle V is displayed. be able to. According to the present embodiment, it is possible to intuitively recognize whether or not the obstacle Q is in contact with the vehicle V from the overhead view video. Thus, this embodiment can display the obstacle Q appropriately in the bird's-eye view video. The present embodiment can improve the visibility of the obstacle Q in the overhead view video. In the present embodiment, the obstacle Q around the vehicle can be displayed so as to be appropriately confirmed.

  In the present embodiment, for example, a bird's-eye view image is set as a virtual viewpoint PA that stands near the position where the obstacle Q is detected and guides the approaching vehicle V while confirming both the obstacle Q and the vehicle V. Can be displayed. Accordingly, it is possible to easily recognize whether or not the obstacle Q is in contact with the vehicle V as compared with the overhead view image 100 that is the virtual viewpoint P located directly above the vehicle V.

  According to the present embodiment, the virtual viewpoint PA is a position where the angle of view of the generated overhead view video is approximately the same as the angle of view of the overhead view video 100 of the virtual viewpoint P. Thereby, the size of the obstacle Q displayed on the overhead view video as the virtual viewpoint PA is not different from the size of the obstacle Q displayed on the overhead view video 100. According to the present embodiment, it is possible to generate a bird's-eye view image in which the relative distance between the obstacle Q and the vehicle V can be easily recognized even if the position of the virtual viewpoint is changed.

[Second Embodiment]
With reference to FIG. 9, a bird's-eye view video generation system 1 according to the present embodiment will be described. FIG. 9 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the second embodiment. The overhead image generation system 1 has the same basic configuration as the overhead image generation system 1 of the first embodiment. In the following description, components similar to those in the overhead view video generation system 1 are denoted by the same reference numerals or corresponding reference numerals, and detailed description thereof is omitted. The overhead view video generation system 1 of the present embodiment differs from the overhead view video generation system 1 of the first embodiment in the processing in the viewpoint conversion processing unit 451 of the overhead view video generation unit 45.

  When the position of the obstacle Q acquired by the obstacle information acquisition unit 43 is located in the traveling direction of the vehicle V, the viewpoint conversion processing unit 451 has a straight line L12 orthogonal to the extension line L11 from the center of the vehicle V to the obstacle Q. An image that has undergone viewpoint conversion processing is generated as a virtual viewpoint PB positioned based on the above. In this embodiment, when the position of the obstacle Q is positioned in the traveling direction of the vehicle V, the viewpoint conversion processing unit 451 is positioned on a straight line L12 orthogonal to the extension line L11 from the center of the vehicle V to the obstacle Q. An image that has undergone viewpoint conversion processing is generated as a virtual viewpoint PB.

  In the present embodiment, the viewpoint conversion processing unit 451 generates an image obtained by performing viewpoint conversion processing on the virtual viewpoint P as the left front virtual viewpoint PB when the obstacle Q located behind is detected. The position of the virtual viewpoint PB is represented by (xB, yB, zB).

  The virtual viewpoint PB will be described. In the present embodiment, the virtual viewpoint PB is located on the left front side of the vehicle V. The virtual viewpoint PB is located on the left front side from the obstacle Q in the Z-axis direction view. The virtual viewpoint PB is located on the left front side of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PB is farther from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q. The virtual viewpoint PB is a viewpoint that looks down at the vehicle V obliquely from the left front.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described. The bird's-eye view video generation device 40 performs processing according to the flowchart shown in FIG. In this embodiment, the process in step S16 is different from that in the first embodiment, and the processes in steps S11 to S15, step S17, and step S18 are the same as those in the first embodiment.

  The control unit 41 changes the virtual viewpoint position (step S16). More specifically, the control unit 41 causes the overhead image generation unit 45 to generate an overhead image as a virtual viewpoint PB located on a straight line L12 orthogonal to the extension line L11 from the vehicle V to the obstacle Q, and causes the display panel 30 to generate the overhead image. Display. The control unit 41 proceeds to step S18.

  As described above, in the present embodiment, when the obstacle Q in the traveling direction is detected, the overhead view image of the virtual viewpoint PB located on the straight line L12 orthogonal to the extension line L11 from the center of the vehicle V to the obstacle Q. Is generated. In the present embodiment, when an obstacle Q in the traveling direction is detected, the position of the virtual viewpoint can be shifted and a bird's-eye view video showing the relative positional relationship between the obstacle Q and the vehicle V can be displayed. . According to the present embodiment, it is possible to intuitively recognize whether or not the obstacle Q is in contact with the vehicle V from the overhead view video. Thus, this embodiment can display the obstacle Q appropriately in the bird's-eye view video. The present embodiment can improve the visibility of the obstacle Q in the overhead view video. In the present embodiment, the obstacle Q around the vehicle can be displayed so as to be appropriately confirmed. Further, in the present embodiment, the position of the virtual viewpoint PB is set so that the position of the virtual viewpoint PB is positioned on the straight line L12 orthogonal to the extension line L11 from the center of the vehicle V to the obstacle Q. It can be set as the position which can confirm V and the obstruction Q.

  In the present embodiment, for example, a bird's-eye view image is displayed as a virtual viewpoint PB that stands on the side where the obstacle Q is detected and guides the vehicle V moving away while checking both the obstacle Q and the vehicle V. be able to. Accordingly, it is possible to easily recognize whether or not the obstacle Q is in contact with the vehicle V as compared with the overhead view image 100 that is the virtual viewpoint P located directly above the vehicle V.

[Third embodiment]
The overhead view video generation system 1 according to this embodiment will be described with reference to FIGS. 10 to 13. FIG. 10 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the third embodiment. FIG. 11 is a schematic diagram illustrating another example of the position of the virtual viewpoint in the overhead view video generation system according to the third embodiment. FIG. 12 is a schematic diagram illustrating another example of the position of the virtual viewpoint in the overhead view video generation system according to the third embodiment. FIG. 13 is a schematic diagram illustrating another example of the position of the virtual viewpoint in the overhead view video generation system according to the third embodiment. The overhead view video generation system 1 of the present embodiment differs from the overhead view video generation system 1 of the first embodiment in the processing in the viewpoint conversion processing unit 451 of the overhead view video generation unit 45. In the present embodiment, there are an obstacle Q1 and an obstacle Q2.

  Based on the positional information of the plurality of obstacles Q acquired by the obstacle information acquisition unit 43, the viewpoint conversion processing unit 451 looks down the surrounding video acquired by the video data acquisition unit 42 from the vehicle V and the plurality of obstacles Q. An image is generated by performing viewpoint conversion processing so that a virtual viewpoint separated from a virtual viewpoint P positioned directly above the vehicle V is positioned as possible.

  In the present embodiment, as shown in FIG. 10, when a plurality of obstacles Q located on the left rear side of the vehicle V are detected, the viewpoint conversion processing unit 451 has a virtual viewpoint PC1 on the left side with respect to the virtual viewpoint P. As shown in FIG. The position of the virtual viewpoint PC1 is represented by (xC1, yC1, zC1).

  The virtual viewpoint PC1 will be described. In the present embodiment, the virtual viewpoint PC1 is located on the left side of the vehicle V. The virtual viewpoint PC1 is located in front of the obstacle Q1 and the obstacle Q2 when viewed in the Z-axis direction. The virtual viewpoint PA1 is located on the left side of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PC1 is further away from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q1 and the distance between the center of the vehicle V and the obstacle Q2. The virtual viewpoint PC1 is a viewpoint that looks down at the vehicle V obliquely from the left side.

  Alternatively, as illustrated in FIG. 11, when the obstacle Q1 and the obstacle Q2 located at the left rear of the vehicle V are detected, the viewpoint conversion processing unit 451 extends the extension line L21 from the center of the vehicle V to the obstacle Q1. The overhead view video may be generated as the virtual viewpoint PC2 located on the bisector L25 of the straight line L22 orthogonal to the vehicle L and the straight line L24 orthogonal to the extension line L23 from the center of the vehicle V to the obstacle Q2. The position of the virtual viewpoint PC2 is represented by (xC2, yC2, zC2).

  The virtual viewpoint PC2 will be described. In the present embodiment, the virtual viewpoint PC2 is located on the left front side of the vehicle V. The virtual viewpoint PC2 is located on the left front side of the obstacle Q1 and the obstacle Q2 in the Z-axis direction view. The virtual viewpoint PC2 is located on the left front side of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PC2 is further away from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q1 and the distance between the center of the vehicle V and the obstacle Q2. The virtual viewpoint PC2 is a viewpoint that looks down at the vehicle V obliquely from the left front.

  Alternatively, as illustrated in FIG. 12, the viewpoint conversion processing unit 451 changes to the virtual viewpoint P when an obstacle Q1 located on the right side of the vehicle V and an obstacle Q2 located on the left rear side of the vehicle V are detected. On the other hand, an image obtained by performing viewpoint conversion processing may be generated as the virtual viewpoint PC 3 on the right rear side. The virtual viewpoint PC3 exists on a bisector of a line segment connecting the center of the vehicle V and the obstacle Q1 and a line segment connecting the center of the vehicle V and the obstacle Q2. The position of the virtual viewpoint PC3 is represented by (xC3, yC3, zC3).

  The virtual viewpoint PC3 will be described. In this embodiment, the virtual viewpoint PC3 is located on the right rear side of the vehicle V. The virtual viewpoint PC3 is located on the left rear side from the obstacle Q1 and on the right rear side from the obstacle Q2 when viewed in the Z-axis direction. The virtual viewpoint PC3 is located on the right rear side of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PC3 is further away from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q1 and the distance between the center of the vehicle V and the obstacle Q2. The virtual viewpoint PC3 is a viewpoint in which the vehicle V is looked down obliquely from the right rear.

  Alternatively, as illustrated in FIG. 13, the viewpoint conversion processing unit 451 detects the center of the vehicle V when the obstacle Q1 located on the right side of the vehicle V and the obstacle Q2 located on the left rear side of the vehicle V are detected. Is located on a bisector L30 having an angle between a straight line L27 orthogonal to the extension line L26 from the vehicle to the obstacle Q1 and a straight line L29 orthogonal to the extension line L28 from the center of the vehicle V to the obstacle Q2. An overhead view video may be generated as the viewpoint PC 4. The position of the virtual viewpoint PC4 is represented by (xC4, yC4, zC4).

  The virtual viewpoint PC4 will be described. In this embodiment, the virtual viewpoint PC4 is located on the right rear side of the vehicle V. The virtual viewpoint PC4 is located on the left rear side of the obstacle Q1 and on the right front side of the obstacle Q2 when viewed in the Z-axis direction. The virtual viewpoint PC4 is located on the right rear side of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PC4 is further away from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q1 and the distance between the center of the vehicle V and the obstacle Q2. The virtual viewpoint PC4 is a viewpoint that looks down the vehicle V obliquely from the right rear.

  In the following description, when it is not necessary to particularly distinguish the virtual viewpoint PC1, the virtual viewpoint PC2, the virtual viewpoint PC3, and the virtual viewpoint PC4, they will be described as a virtual viewpoint PC.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described. The bird's-eye view video generation device 40 performs processing according to the flowchart shown in FIG. In this embodiment, the process in step S16 is different from that in the first embodiment, and the processes in steps S11 to S15, step S17, and step S18 are the same as those in the first embodiment.

  The control unit 41 changes the virtual viewpoint position (step S16). More specifically, the control unit 41 is an overhead view video generation unit 45 that provides a virtual viewpoint PC that is separated from the virtual viewpoint P positioned directly above the vehicle V so that the vehicle V and the plurality of obstacles Q can be seen from above. Are generated and displayed on the display panel 30. The control unit 41 proceeds to step S18.

  As described above, in the present embodiment, when a plurality of obstacles Q in the traveling direction are detected, the vehicle V and the plurality of obstacles Q are separated from the virtual viewpoint P located immediately above the vehicle V so that the overhead view can be seen. A bird's-eye view video of the virtual viewpoint PC is generated. In the present embodiment, when a plurality of obstacles Q in the traveling direction are detected, the position of the virtual viewpoint is shifted so that the plurality of obstacles Q do not overlap, but the relative positions of the plurality of obstacles Q and the vehicle V An overhead video showing the relationship can be displayed. According to the present embodiment, whether or not a plurality of obstacles Q are in contact with the vehicle V can be intuitively recognized from the overhead view video. Thus, this embodiment can display the several obstruction Q appropriately in a bird's-eye view image | video. This embodiment can improve the visibility of a plurality of obstacles Q in a bird's-eye view video. In the present embodiment, a plurality of obstacles Q around the vehicle can be displayed so as to be appropriately confirmed.

  In the present embodiment, for example, a bird's-eye view video as a virtual viewpoint PC that guides the approaching vehicle V while confirming both the plurality of obstacles Q and the vehicle V while standing at a position where the plurality of obstacles Q can be visually recognized. Can be displayed. Accordingly, it is possible to easily recognize whether or not a plurality of obstacles Q are in contact with the vehicle V as compared with the overhead view image 100 that is the virtual viewpoint P located directly above the vehicle V.

[Fourth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 14 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the fourth embodiment. The overhead view video generation system 1 of the present embodiment differs from the overhead view video generation system 1 of the first embodiment in the processing in the overhead view video generation unit 45. In the present embodiment, the obstacle Q1 is located on the left rear side of the vehicle V, and the obstacle Q2 is located on the right rear side of the vehicle V.

  The bird's-eye view video generation unit 45 uses the peripheral information acquired by the video data acquisition unit 42 based on the positional information of the plurality of obstacles Q acquired by the obstacle information acquisition unit 43 as at least a part of the vehicle V and the obstacle Q. Is generated as a virtual viewpoint PD1 that is different from the virtual viewpoint P so that the viewpoint conversion process is performed. Furthermore, the overhead view video generation unit 45 has a virtual viewpoint different from the virtual viewpoint P so that at least the remaining part of the vehicle V and the obstacle Q can be seen from the surrounding video based on the positional information of the plurality of obstacles Q. An image subjected to viewpoint conversion processing is generated as PD2. Then, the overhead video generation unit 45 combines the two videos that have undergone the viewpoint conversion processing to generate an overhead video.

  The viewpoint conversion processing unit 451 performs viewpoint conversion processing as a virtual viewpoint PD1 on the left side with respect to the virtual viewpoint P when the obstacle Q1 and the obstacle Q2 located at the left rear and right rear of the vehicle V are detected. A video and a video obtained by performing viewpoint conversion processing as a virtual viewpoint PD2 on the right side of the virtual viewpoint P are generated. The position of the virtual viewpoint PD1 is represented by (xD1, yD1, zD1). The position of the virtual viewpoint PD2 is represented by (xD2, yD2, zD2).

  The virtual viewpoint PD1 will be described. In the present embodiment, the virtual viewpoint PD1 is located on the left front side of the vehicle V. The virtual viewpoint PD1 is located on the left front side of the obstacle Q1 when viewed in the Z-axis direction. The virtual viewpoint PD1 is located on the left front side of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PD1 is further away from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q1. The virtual viewpoint PD1 is a viewpoint that looks down at the vehicle V obliquely from the left front.

  The virtual viewpoint PD2 will be described. In the present embodiment, the virtual viewpoint PD2 is located on the right front side of the vehicle V. The virtual viewpoint PD2 is located in front of the obstacle Q2 when viewed in the Z-axis direction. The virtual viewpoint PD2 is located in front of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PD2 is farther from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q2. The virtual viewpoint PD2 is a viewpoint that looks down at the vehicle V obliquely from the right front.

  The composition processing unit 453 synthesizes the video that has undergone the viewpoint conversion processing as the virtual viewpoint PD1 and the video that has undergone the viewpoint conversion processing as the virtual viewpoint PD2, and generates an overhead view video. For example, the composition processing unit 453 generates a bird's-eye view video by compositing so as to display the video subjected to the viewpoint conversion processing as the virtual viewpoint PD1 on the left side and to display the video subjected to the viewpoint conversion processing as the virtual viewpoint PD2 on the right side.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described. The bird's-eye view video generation device 40 performs processing according to the flowchart shown in FIG. In this embodiment, the process in step S16 is different from that in the first embodiment, and the processes in steps S11 to S15, step S17, and step S18 are the same as those in the first embodiment.

  The control unit 41 changes the virtual viewpoint position (step S16). More specifically, the control unit 41 is a bird's-eye view video generation unit 45 that uses the position information of the plurality of obstacles Q acquired by the obstacle information acquisition unit 43 to display the surrounding video acquired by the video data acquisition unit 42 as a vehicle. V and at least the obstacle Q1 are generated as a virtual viewpoint PD1 that is different from the virtual viewpoint P so that the viewpoint conversion process is performed. The control unit 41 is a bird's-eye view video generation unit 45 based on the positional information of the plurality of obstacles Q, as a virtual viewpoint PD2 different from the virtual viewpoint PD1 so that the surrounding video can be seen from the vehicle V and at least the obstacle Q2. Generate video that has undergone viewpoint conversion processing. Then, the control unit 41 generates a bird's-eye view video by combining the two videos in the bird's-eye view video generation unit 45 and displays it on the display panel 30. The control unit 41 proceeds to step S18.

  As described above, in the present embodiment, when the obstacle Q is detected on the left side and the right side in the traveling direction, an image obtained by performing viewpoint conversion processing with a plurality of virtual viewpoints is generated. Then, the overhead view video is generated by synthesizing the plurality of videos subjected to the viewpoint conversion processing. In the present embodiment, when obstacles Q are detected on the left side and the right side of the traveling direction of the traveling direction, a plurality of obstacles Q and the vehicle V are synthesized by synthesizing images obtained by performing viewpoint conversion processing as a plurality of virtual viewpoints. Can be displayed.

  When an obstacle Q is detected on the left and right sides of the traveling direction, if the viewpoint conversion process is performed from the left or right virtual viewpoint to generate a bird's eye view image, the obstacle Q and the vehicle on the opposite side of the virtual viewpoint are generated. There is a possibility that the relative positional relationship of V cannot be expressed.

  On the other hand, according to the present embodiment, even when the obstacle Q is detected on the left side and the right side in the traveling direction, for example, the vehicle V standing up behind each obstacle Q is guided. It is possible to display a bird's-eye view video obtained by synthesizing the video set as the virtual viewpoint PD1 and the video set as the virtual viewpoint PD2. Whether or not a plurality of obstacles Q are in contact with the vehicle V can be easily recognized from the overhead image. Thus, this embodiment can display the several obstruction Q appropriately in a bird's-eye view image | video. This embodiment can improve the visibility of a plurality of obstacles Q in a bird's-eye view video. In the present embodiment, the obstacle Q around the vehicle can be displayed so as to be appropriately confirmed.

[Fifth embodiment]
The overhead view video generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 15 is a schematic diagram illustrating an example of the position of the virtual viewpoint in the overhead view video generation system according to the fifth embodiment. The overhead view video generation system 1 of the present embodiment differs from the overhead view video generation system 1 of the first embodiment in the processing in the viewpoint conversion processing unit 451 of the overhead view video generation unit 45. In the present embodiment, the obstacle Q1 and the obstacle Q2 are positioned side by side in the vehicle width direction on the left rear side of the vehicle V.

  When the obstacle Q1 and the obstacle Q2 located at the left rear of the vehicle V are detected, the viewpoint conversion processing unit 451 detects the straight line L42 orthogonal to the extension line L41 from the center of the vehicle V to the obstacle Q1, and the vehicle V A bird's-eye view video may be generated as a virtual viewpoint PE located on a bisector L45 having an angle between a straight line L44 perpendicular to the extension line L43 from the center to the obstacle Q2. The position of the virtual viewpoint PE is represented by (xE, yE, zE).

  The virtual viewpoint PE will be described. In the present embodiment, the virtual viewpoint PE is located on the left front side of the vehicle V. The virtual viewpoint PE is located in front of the obstacle Q1 and the obstacle Q2 when viewed in the Z-axis direction. The virtual viewpoint PE is located on the left front side of the virtual viewpoint P in the Z-axis direction view. The virtual viewpoint PE is further away from the center of the vehicle V than the distance between the center of the vehicle V and the obstacle Q1 and the distance between the center of the vehicle V and the obstacle Q2. The virtual viewpoint PE is a viewpoint in which the vehicle V is looked down obliquely from the left front.

  Next, the flow of processing in the overhead view video generation device 40 of the overhead view video generation system 1 will be described. The bird's-eye view video generation device 40 performs processing according to the flowchart shown in FIG. In this embodiment, the process in step S16 is different from that in the first embodiment, and the processes in steps S11 to S15, step S17, and step S18 are the same as those in the first embodiment.

  The control unit 41 changes the virtual viewpoint position (step S16). More specifically, the control unit 41 is a bird's-eye view image generation unit 45 that includes a straight line L42 orthogonal to the extension line L41 from the center of the vehicle V to the obstacle Q1 and an extension line L43 from the center of the vehicle V to the obstacle Q2. A bird's-eye view video is generated as a virtual viewpoint PE located on a bisector L45 having an angle between the orthogonal straight line L44 and displayed on the display panel 30. The control unit 41 proceeds to step S18.

  As described above, in the present embodiment, when a plurality of obstacles Q are detected in the traveling direction, the vehicle V and the plurality of obstacles Q are separated from the virtual viewpoint P positioned directly above the vehicle V so that the vehicles V and the plurality of obstacles Q can be seen from above. A bird's-eye view video of the virtual viewpoint is generated. In the present embodiment, when a plurality of obstacles Q in the traveling direction are detected, the position of the virtual viewpoint is shifted so that the plurality of obstacles Q do not overlap, but the relative positions of the plurality of obstacles Q and the vehicle V An overhead video showing the relationship can be displayed. According to the present embodiment, whether or not a plurality of obstacles Q are in contact with the vehicle V can be easily recognized from an overhead video. Thus, this embodiment can display the several obstruction Q appropriately in a bird's-eye view image | video. This embodiment can improve the visibility of a plurality of obstacles Q in a bird's-eye view video. In the present embodiment, a plurality of obstacles Q around the vehicle can be displayed so as to be appropriately confirmed.

  In the present embodiment, for example, it is possible to display a bird's-eye view image as a virtual viewpoint PE that guides the approaching vehicle V while standing at a position where a plurality of obstacles Q can be visually recognized. Accordingly, it is possible to easily recognize whether or not a plurality of obstacles Q are in contact with the vehicle V as compared with the overhead view image 100 that is the virtual viewpoint P positioned directly above the vehicle V.

  Now, the overhead video generation system 1 according to the present invention has been described so far, but may be implemented in various different forms other than the above-described embodiment.

  Each component of the illustrated bird's-eye view video generation system 1 is functionally conceptual and may not necessarily be physically configured as illustrated. That is, the specific form of each device is not limited to the one shown in the figure, and all or a part of them is functionally or physically distributed or integrated in arbitrary units according to the processing load or usage status of each device. May be.

  The configuration of the overhead view video generation system 1 is realized by, for example, a program loaded in a memory as software. The above embodiment has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The components described above include those that can be easily assumed by those skilled in the art and those that are substantially the same. Furthermore, the structures described above can be appropriately combined. In addition, various omissions, substitutions, or changes in the configuration can be made without departing from the scope of the present invention.

  Although the predetermined condition has been described as the obstacle Q being positioned in the traveling direction of the vehicle V, the predetermined condition is not limited thereto. The predetermined condition may be, for example, the possibility of interference with the vehicle V. The possibility of interference with the vehicle V means, for example, that it is located in the traveling direction of the vehicle V and has a height from the ground where there is a possibility of coming into contact with the vehicle V.

  The predetermined condition is determined to satisfy the predetermined condition when the vehicle V is moving backward and the obstacle Q is positioned behind the front end of the vehicle V. However, the predetermined condition is not limited to this. For example, the predetermined condition is that when the vehicle V is moving backward, the obstacle Q is located behind the rear end of the vehicle V, or the obstacle Q is located behind the axle position of the driving wheel of the vehicle V. In this case, it may be determined that the predetermined condition is satisfied.

  In step S11, for example, the control unit 41 may determine whether or not to start the overhead view video display based on whether or not an operation for starting the overhead view video display on the operation unit is detected.

  The overhead view video in which the position of the virtual viewpoint is changed may be generated by shifting the center of the host vehicle icon from the center of the overhead view video so that the display area in the direction in which the obstacle Q is detected is widened.

  The virtual viewpoint P may be the position in the Z direction, for example, the height of the line of sight of the driver sitting in the driver's seat, the height of the line of sight when the driver stands, the height of the roof of the vehicle V, etc. .

DESCRIPTION OF SYMBOLS 1 Overhead image generation system 11 Front camera 12 Rear camera 13 Left side camera 14 Right side camera 20 Sensor group (obstacle sensor)
30 Display panel (display unit)
40 Overhead Video Generation Device 41 Control Unit 42 Video Data Acquisition Unit 43 Obstacle Information Acquisition Unit 44 Vehicle Information Acquisition Unit 45 Overhead Video Generation Unit 451 Viewpoint Conversion Processing Unit 452 Extraction Processing Unit 453 Compositing Processing Unit 48 Display Control Unit 49 Storage Unit P Virtual viewpoint Q Obstacle V Vehicle

Claims (7)

A video data acquisition unit for acquiring peripheral video data obtained by photographing the periphery of the vehicle;
An obstacle information acquisition unit for acquiring position information of obstacles detected in the vicinity of the vehicle;
A position directly above the vehicle so that the vehicle and the obstacle can overlook the peripheral video acquired by the video data acquisition unit based on the positional information of the obstacle acquired by the obstacle information acquisition unit. A bird's-eye view video generation unit that generates a bird's-eye view image synthesized by performing viewpoint conversion processing so that the virtual viewpoint is located at least in either the vehicle width direction or the front-rear direction;
A bird's-eye-view video generation device, comprising: a display control unit that displays the bird's-eye-view video generated by the bird's-eye-view video generation unit on a display unit. The overhead view image generation unit generates an overhead view image obtained by performing viewpoint conversion processing so that a virtual viewpoint is located on an orthogonal line orthogonal to an extension line from the vehicle to the obstacle.
The overhead image generation apparatus according to claim 1. When there are a plurality of obstacles, the overhead view video generation unit performs a viewpoint conversion process with a virtual viewpoint positioned based on orthogonal lines orthogonal to the extension lines of the vehicle and the obstacles. Generate overhead video,
The overhead image generation apparatus according to claim 1. The bird's-eye view image generation unit, when there are a plurality of obstacles, at least a part in the vehicle width direction or the front-rear direction from a position directly above the vehicle so that at least a part of the vehicle and the plurality of obstacles can be seen overhead From a position directly above the vehicle so that an image obtained by performing viewpoint conversion processing so that the virtual viewpoint is located apart from one another and at least the remaining part of the vehicle and the plurality of obstacles can be overlooked Generating a bird's-eye view image by synthesizing with the image that has undergone viewpoint conversion processing so that the virtual viewpoint is located apart from at least one of the vehicle width direction and the front-rear direction;
The overhead image generation apparatus according to claim 1. An overhead view video generation device according to any one of claims 1 to 4,
An overhead image generation system, comprising: at least one of a plurality of cameras that capture the periphery of the vehicle, the display unit, and a detection unit that detects an obstacle and outputs obstacle information. A video data acquisition step of acquiring peripheral video data obtained by photographing the periphery of the vehicle;
Obstacle information acquisition step for acquiring position information of obstacles detected in the vicinity of the vehicle;
Based on the position information of the obstacle acquired in the obstacle information acquisition step, the position directly above the vehicle so that the vehicle and the obstacle can overlook the peripheral video acquired in the video data acquisition step. An overhead view video generation step of generating a synthesized overhead view video by performing a viewpoint conversion process so that the virtual viewpoint is located at a distance from at least one of the vehicle width direction and the front-rear direction from
And a display control step for displaying the overhead view video generated in the overhead view video generation step on a display unit. A video data acquisition step of acquiring peripheral video data obtained by photographing the periphery of the vehicle;
Obstacle information acquisition step for acquiring position information of obstacles detected in the vicinity of the vehicle;
Based on the position information of the obstacle acquired in the obstacle information acquisition step, the position directly above the vehicle so that the vehicle and the obstacle can overlook the peripheral video acquired in the video data acquisition step. An overhead view video generation step of generating a synthesized overhead view video by performing a viewpoint conversion process so that the virtual viewpoint is located at a distance from at least one of the vehicle width direction and the front-rear direction from
A program for causing a computer operating as an overhead video generation apparatus to execute a display control step of displaying the overhead video generated in the overhead video generation step on a display unit.

Images