JP2019062414A - Overhead video generation device, overhead video generation system, overhead video generation method and program - Google Patents

Abstract

To display the surroundings of a vehicle in a checkable manner.SOLUTION: An overhead video generation device comprises: a video data acquisition part 42 that acquires peripheral video data photographed by each camera; a peripheral information acquisition part 43 that acquires peripheral information indicating a situation around a vehicle; a vehicle information acquisition part 44 that acquires vehicle information including the vehicle speed of the vehicle; an overhead video generation part 45 that performs viewpoint conversion processing on the peripheral video, performs composite processing of a plurality of peripheral video subjected to the viewpoint conversion processing, and generates an overhead video of a display range for displaying the periphery of the vehicle; and a display control part 49 that causes a display panel 31 to display the overhead video generated by the overhead video generation part 45. The display control part 49 causes the display panel 31 to display the overhead video when it is determined that the inter-vehicle distance between the vehicle and the surrounding vehicle running around the vehicle is preferred to be confirmed based on the peripheral information acquired by the peripheral information acquisition part 43 and the vehicle information acquired by the vehicle information acquisition part 44.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an overhead view video generation apparatus, an overhead view video generation system, an overhead view video generation method, and a program.

  There is known a technique related to a vehicle periphery display device that displays an overhead view image of a vehicle together with the vehicle image (see, for example, Patent Document 1). This technology widens the display area of the overhead image behind the vehicle image when the vehicle switches from forward to reverse.

JP, 2015-076645, A

  Vehicles may stop for traffic lights while driving or may stop due to traffic jams. If there is an entrance of a parking lot or an entrance of an alley or a branch of a branch lane of a road, for example, move to the entrance or branch lane depending on the stopping position of the vehicle. May interfere with the smooth passage of other vehicles. Also, for example, when it is desired to move the vehicle from the branch part to the right turn lane, it may be difficult to determine whether the vehicle can move to the right turn lane if the inter-vehicle distance between the vehicle and the preceding vehicle is unclear. Therefore, when it is desired to confirm the inter-vehicle distance between the vehicle and the preceding vehicle or the following vehicle, in order to support confirmation of the inter-vehicle distance between the vehicle and the following vehicle or the following vehicle, the vehicle periphery can be displayed in a checkable manner. desired.

  The present invention has been made in view of the above, and its object is to display the periphery of a vehicle in a checkable manner.

  In order to solve the problems described above and achieve the object, the overhead view video image generating apparatus according to the present invention is a video data acquisition unit for acquiring surrounding image data captured by an imaging unit for capturing an area around the vehicle; Performing a viewpoint conversion process on the peripheral image acquired by the peripheral information acquisition unit acquiring peripheral information indicating the peripheral situation, the vehicle information acquisition unit acquiring vehicle information including the vehicle speed of the vehicle, and the video data acquisition unit; The overhead image generation unit performs composition processing of a plurality of peripheral images subjected to viewpoint conversion processing to generate an overhead image of the display range for displaying the periphery of the vehicle, and the overhead image generated by the overhead image generation unit is displayed And the display control unit is configured to display the display control unit based on the surrounding information acquired by the surrounding information acquiring unit and the vehicle information acquired by the vehicle information acquiring unit. Surroundings If better to check the distance to the nearby vehicle traveling is determined to be, and wherein the displaying the overhead image on the display unit.

  The overhead view video generation system according to the present invention is characterized by including the above-described overhead view video generation device, and a photographing unit for photographing the periphery of the vehicle and supplying the peripheral video data to the video data acquisition unit.

  The overhead view video generation method according to the present invention includes a video data acquisition step of acquiring surrounding image data captured by an imaging unit that captures an area around the vehicle, and a periphery information acquiring step of acquiring periphery information indicating a situation around the vehicle. A vehicle information acquisition step of acquiring vehicle information including the vehicle speed of the vehicle, and viewpoint conversion processing on the peripheral image acquired in the video data acquisition step, and combining processing of a plurality of peripheral images subjected to viewpoint conversion processing The display control step of performing an overhead view video generation step of generating an overhead view video of a display range for displaying the periphery of the vehicle, and displaying the overhead view video generated in the overhead view video generation step on a display unit; The step is based on the peripheral information acquired in the peripheral information acquisition step and the vehicle information acquired in the vehicle information acquisition step. If better to check the distance to the surrounding vehicles traveling around the vehicle is determined to be, and displays the overhead image on the display unit.

  The program according to the present invention comprises a video data acquisition step of acquiring peripheral video data photographed by a photographing unit photographing a peripheral of the vehicle, a peripheral information acquisition step of acquiring peripheral information indicating a situation around the vehicle, and The vehicle information acquisition step of acquiring vehicle information including the vehicle speed of the vehicle and the peripheral image acquired in the image data acquisition step perform viewpoint conversion processing, and perform synthesis processing of a plurality of peripheral images subjected to viewpoint conversion processing, And a display control step of causing the display unit to display a bird's-eye view video generated in the bird's-eye view video generation step of generating a bird's-eye view video of a display range for displaying the periphery of the vehicle; The vehicle and the vehicle, based on the peripheral information acquired in the peripheral information acquisition step and the vehicle information acquired in the vehicle information acquisition step If better to check the distance to the surrounding vehicles traveling around the both is determined as good, it is executed by a computer that acts as overhead image generating apparatus that displays the overhead image on the display unit.

  According to the present invention, it is possible to display the periphery of the vehicle in a checkable manner.

FIG. 1 is a block diagram showing an example of the configuration of the overhead view video image generation system according to the first embodiment. FIG. 2 is a diagram for explaining the overhead view video image generation system according to the first embodiment, and shows a vehicle and its surroundings. FIG. 3 is a view showing an example of a bird's-eye view video generated by the bird's-eye view video generation system according to the first embodiment. FIG. 4 is a flow chart showing the flow of processing in the overhead view video generation device of the overhead view video generation system according to the first embodiment. FIG. 5 is a diagram for explaining the overhead view video image generation system according to the second embodiment, and shows a vehicle and its surroundings. FIG. 6 is a flow chart showing the flow of processing in the overhead view video generation device of the overhead view video generation system according to the second embodiment. FIG. 7 is a flow chart showing the flow of processing in the overhead view video generation device of the overhead view video generation system according to the third embodiment. FIG. 8 is a diagram for explaining the overhead view video image generation system according to the fourth embodiment, and shows a vehicle and its surroundings. FIG. 9 is a flow chart showing the flow of processing in the overhead view video generation device of the overhead view video generation system according to the fourth embodiment. FIG. 10 is a view showing another example of the overhead view video generated by the overhead view video generation system.

  DETAILED DESCRIPTION OF THE INVENTION Embodiments of the overhead view video generation device 40, the overhead view video generation system 1, the overhead view video generation method, and the program according to the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited by the following embodiments.

First Embodiment
FIG. 1 is a block diagram showing an example of the configuration of the overhead view video image generation system according to the first embodiment. FIG. 2 is a diagram for explaining the overhead view video image generation system according to the first embodiment, and shows a vehicle and its surroundings. The overhead view video generation system 1 generates and displays the overhead view video 100 (see FIG. 3) based on the peripheral information acquired by the peripheral information acquisition unit 43 and the vehicle information acquired by the vehicle information acquisition unit 44. More specifically, if it is determined that it is better to check the inter-vehicle distance between the vehicle V1 and the surrounding vehicles around the vehicle V1 based on the surrounding information and the vehicle information, the looking-down image generation system 1 Generate and display 100. The surrounding vehicle includes, for example, a preceding vehicle V2 or a following vehicle located in front of the vehicle V1.

  The overhead view video generation device 40 and the overhead view video generation system 1 are mounted on a vehicle V1. The overhead view video generation device 40 and the overhead view video generation system 1 may be portable and usable in the vehicle V1 in addition to those mounted on the vehicle V1.

  The overhead view video generation system 1 will be described with reference to FIG. The overhead view video generation system 1 includes a front camera (shooting unit) 11, a rear camera (shooting unit) 12, a left side camera (shooting unit) 13, a right side camera (shooting unit) 14, and a display panel (display unit) 31) and an overhead view video generation device 40.

  The front camera 11 is an overhead video camera. The front camera 11 is disposed in front of the vehicle V1 and captures an area around the front of the vehicle V1. As shown in FIG. 2, the front camera 11 captures an image of a first imaging range A1 of, for example, about 180 °. The first shooting range A1 includes a wider range in front of the vehicle V1 than the display range A of the overhead image 100. The front camera 11 outputs the captured video to the video data acquisition unit 42 of the overhead video generating device 40.

  The rear camera 12 is an overhead video camera. The rear camera 12 is disposed at the rear of the vehicle V1 and captures an image of the area around the rear of the vehicle V1. The rear camera 12 shoots a second shooting range A2 of about 180 °, for example. The second shooting range A2 includes a wider range behind the vehicle V1 than the display range A of the overhead image 100. The rear camera 12 outputs the captured video to the video data acquisition unit 42 of the overhead video generating device 40.

  The left side camera 13 is an overhead video camera. The left side camera 13 is disposed on the left side of the vehicle V1 and captures an image of the area around the left side of the vehicle V1. The left side camera 13 shoots a third shooting range A3 of about 180 °, for example. The third shooting range A3 includes a wider range on the left side of the vehicle V1 than the display range A of the overhead image 100. The left side camera 13 outputs the captured video to the video data acquisition unit 42 of the overhead video generating device 40.

  The right side camera 14 is a bird's eye view camera. The right side camera 14 is disposed on the right side of the vehicle V1 and captures an area around the right side of the vehicle V1. The right side camera 14 captures, for example, a fourth imaging range A4 of about 180 °. The fourth shooting range A4 includes a wider range on the right side of the vehicle V1 than the display range A of the overhead image 100. The right-side camera 14 outputs the captured video to the video data acquisition unit 42 of the overhead view video generation device 40.

  The forward camera 11, the rear camera 12, the left side camera 13 and the right side camera 14 capture images of all directions of the vehicle V <b> 1.

  Returning to FIG. 1, the display panel 31 is a display including, for example, a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminescence) display. The display panel 31 displays the overhead view video 100 based on the video signal output from the overhead view video generation device 40 of the overhead view video generation system 1. The display panel 31 may be dedicated to the overhead view video generation system 1 or may be used jointly with other systems including, for example, a navigation system. The display panel 31 is disposed at a position easily visible by the driver.

  The overhead view video generation device 40 includes a control unit 41 and a storage unit 50.

  The control unit 41 is, for example, an arithmetic processing unit configured by a CPU (Central Processing Unit) or the like. The control unit 41 loads the program stored in the storage unit 50 into the memory and executes an instruction included in the program. The control unit 41 includes a video data acquisition unit 42, a surrounding information acquisition unit 43, a vehicle information acquisition unit 44, an overhead video generation unit 45, and a display control unit 49. The control unit 41 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the control unit 41 and the like.

  The video data acquisition unit 42 acquires peripheral video data obtained by photographing the periphery of the vehicle V1. More specifically, the video data acquisition unit 42 acquires video data output from the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The video data acquisition unit 42 outputs the acquired video data to the peripheral information acquisition unit 43 and the overhead video generation unit 45. Video data acquired from each camera is, for example, a moving image composed of images at 30 frames per second.

  The surrounding information acquisition unit 43 acquires surrounding information indicating the situation around the vehicle. In the present embodiment, the peripheral information acquisition unit 43 detects the other vehicle V3 whose light is turned on from the peripheral image acquired by the peripheral vehicle information acquisition unit 431 from the peripheral image acquired by the video data acquisition unit 42, Do. In other words, the surrounding vehicle information acquisition unit 431 performs image processing on the surrounding image captured by at least one of the front camera 11, the rear camera 12, the left direction camera 13, and the right direction camera 14 to The other vehicle V3 which is lit is detected and used as surrounding information. A combination of existing technologies, such as image recognition that the direction indicator is lit, image recognition of the vehicle, extraction of the outline in the left-right direction, and judgment by acquiring luminance and chromaticity as numerical values in the outline Is possible. The peripheral information acquisition unit 43 outputs the acquired peripheral information to the overhead view video generation unit 45.

  The other vehicle V3 is another vehicle existing at a position close to the vehicle V1. Other vehicle V3 includes a following vehicle located behind vehicle V1, and a vehicle located on the side of vehicle V1 and trying to pass in front of and behind vehicle V1. For example, the following vehicle is a vehicle whose direction indicator is on to move to a branch lane. For example, a vehicle located on the side of the vehicle V1 turns on a traffic light before passing the vehicle V1 in order to enter the parking lot on the left side of the vehicle V1 from the opposite lane. It is. For example, a vehicle located on the side of the vehicle V1 has a turn indicator turned on in order to pass in front of and behind the vehicle V1 in order to make a right turn from the parking lot on the left side of the vehicle V1 to the oncoming lane. It is.

  Further, the surrounding vehicle information acquisition unit 431 detects one of the other vehicles V3 whose direction is to be turned by turning on the turn indicator, which is blocked by the stopping of the vehicle V1. It is also good. For example, the surrounding vehicle information acquisition unit 431 may detect only the other vehicle V3 which has stopped by lighting the turn signal. Furthermore, for example, the surrounding vehicle information acquisition unit 431 may perform image processing on the surrounding image to detect only the other vehicle V3 stopped by moving to the side where the turn indicator is lit. Furthermore, for example, the surrounding vehicle information acquisition unit 431 may perform image processing on the surrounding image to detect only the other vehicle V3 in which the inter-vehicle distance from the vehicle V1 is close to the threshold or less.

  The vehicle information acquisition unit 44 acquires vehicle information indicating the status of the vehicle V1 from CAN or various sensors that sense the status of the vehicle V1. The vehicle information acquisition unit 44 acquires, for example, vehicle speed information. The vehicle information acquisition unit 44 outputs the acquired vehicle information to the overhead view video generation unit 45.

  The overhead view video generation unit 45 performs viewpoint conversion processing on the peripheral video acquired by the video data acquisition unit 42, performs synthesis processing of a plurality of peripheral videos subjected to viewpoint conversion processing, and displays the periphery of the vehicle V1. An overhead video 100 is generated. More specifically, the overhead view video generation unit 45 generates the vehicle V1 and the periphery of the vehicle V1 based on the peripheral information indicating the peripheral state acquired by the peripheral information acquisition unit 43 and the vehicle information acquired by the vehicle information acquisition unit 44. If it is determined that it is preferable to confirm the inter-vehicle distance from the vehicle, the overhead view video 100 is generated. In the present embodiment, the overhead image generation unit 45 determines an inter-vehicle distance between the vehicle V1 and a vehicle adjacent to the vehicle V1 when there is another vehicle V3 whose light is turned on based on the surrounding information and the vehicle information. It is determined that it is better to confirm the, and an overhead video 100 is generated.

  Furthermore, the overhead view video generation unit 45 may add a determination as to whether the speed of the vehicle V1 is equal to or less than a threshold. More specifically, the overhead image generation unit 45 determines, based on the surrounding information and the vehicle information, when the speed of the vehicle V1 is equal to or less than the threshold and there is another vehicle V3 whose direction indicator is lit, the vehicle V1 and It may be determined that it is better to check the inter-vehicle distance between the vehicle V1 and the surrounding vehicles, and the overhead image 100 may be generated. In the present embodiment, the threshold value of the vehicle speed of the vehicle V1 is 20 km / h. This numerical value may be set according to, for example, the width of the road on which the vehicle V1 is traveling and the regulated speed, or may be set arbitrarily by the user.

  The overhead view video generation unit 45 outputs the generated overhead view video 100 to the display control unit 49. The overhead view video generation unit 45 includes a viewpoint conversion processing unit 451, a clipping processing unit 452, and a combining processing unit 453.

  The viewpoint conversion processing unit 451 performs viewpoint conversion processing on the surrounding video data acquired by the video data acquisition unit 42 so as to look down on the vehicle V1 from above. More specifically, the viewpoint conversion processing unit 451 generates a video subjected to viewpoint conversion processing based on peripheral video data captured by the front camera 11, the rear camera 12, the left side camera 13 and the right side camera 14. The method of viewpoint conversion processing may be any known method and is not limited. The viewpoint conversion processing unit 451 outputs the peripheral video data subjected to the viewpoint conversion processing to the clipping processing unit 452.

  The clipping processing unit 452 performs clipping processing on a video within a predetermined range from the peripheral video data subjected to the viewpoint conversion processing. Which range is to be cut out is registered and stored in advance. The clipping processing unit 452 outputs the video data of the video subjected to the clipping processing to the combining processing unit 453.

  The composition processing unit 453 performs composition processing for combining the video data subjected to the clipping processing. The composition processing unit 453 generates the overhead view image 100 in which the vehicle icon 110 is displayed on the combined image.

  The overhead view image 100 will be described with reference to FIG. FIG. 3 is a view showing an example of a bird's-eye view video generated by the bird's-eye view video generation system according to the first embodiment. The overhead image 100 displays a predetermined display range A from the vehicle V1. The overhead image 100 is located at a central portion surrounded by the front image 101, the rear image 102, the left side image 103, the right side image 104, the front image 101, the rear image 102, the left direction image 103, and the right side image 104. And a vehicle icon 110. The vehicle icon 110 indicates the position and the direction of the vehicle V1. The vehicle icon 110 is disposed at the center with the front-rear direction parallel to the front-rear direction of the overhead view video 100. The overhead image 100 can recognize the relative positional relationship between at least one of the front end of the vehicle V1 and the rear end of the preceding vehicle V2, and the rear end of the vehicle V1 and the front end of the following vehicle. In the present embodiment, the bird's-eye view image 100 displays a vehicle icon 110, a leading vehicle V2, and another vehicle V3. In the present embodiment, it is preferable that the overhead image 100 can recognize the relative positional relationship between the front end portion of the vehicle V1 and the rear end portion of the preceding vehicle V2. In the present embodiment, it is preferable that the overhead image 100 can recognize the relative positional relationship between the vehicle V1 and the other vehicle V3.

  In FIG. 3, diagonal broken lines indicating the boundaries between the front image 101, the rear image 102, the left side image 103, and the right side image 104 are illustrated for the sake of explanation, but they are actually displayed on the display panel 31. The broken line may not be displayed in the overhead image 100. The same applies to the other figures.

  The display control unit 49 causes the display panel 31 to display the overhead view video 100 generated by the overhead view video generation unit 45. More specifically, the display control unit 49 displays the vehicle V1 and the surrounding vehicles of the vehicle V1 based on the peripheral information indicating the peripheral situation acquired by the peripheral information acquiring unit 43 and the vehicle information acquired by the vehicle information acquiring unit 44. If it is determined that the inter-vehicle distance should be confirmed, the overhead image 100 is displayed on the display panel 31. In the present embodiment, the display control unit 49 determines the inter-vehicle distance between the vehicle V1 and the surrounding vehicles of the vehicle V1 when there is another vehicle V3 whose light is turned on based on the surrounding information and the vehicle information. It is determined that the confirmation is better, and the overhead image 100 is displayed on the display panel 31.

  Furthermore, the display control unit 49 may add a determination as to whether the speed of the vehicle V1 is equal to or less than a threshold. More specifically, the display control unit 49 determines the vehicle V1 and the vehicle based on the surrounding information and the vehicle information when the speed of the vehicle V1 is equal to or less than the threshold and there is another vehicle V3 whose direction indicator is lit. The overhead image 100 may be displayed on the display panel 31 by determining that it is better to check the inter-vehicle distance from the vehicle V1 to the surrounding vehicle.

  The storage unit 50 stores data required for various processes in the overhead view video generation apparatus 40 and various process results. The storage unit 50 is, for example, a semiconductor memory device such as a random access memory (RAM), a read only memory (ROM), a flash memory, or a storage device such as a hard disk or an optical disk.

  Next, the flow of processing in the overhead view video generation device 40 will be described using FIG. 4. FIG. 4 is a flow chart showing the flow of processing in the overhead view video generation device of the overhead view video generation system according to the first embodiment.

  The control unit 41 determines whether the vehicle speed of the vehicle V1 is equal to or less than a threshold (step S11). More specifically, if the control unit 41 determines that the vehicle speed is equal to or less than the threshold based on the vehicle information acquired by the vehicle information acquisition unit 44 (Yes in step S11), the control unit 41 proceeds to step S12. Control part 41 performs processing of Step S11 again, when it is judged based on the vehicle information which vehicle information acquisition part 44 acquired that the vehicle speed is not below a threshold (No in Step S11).

  Control unit 41 determines whether there is another vehicle V3 whose turn indicator is on (step S12). More specifically, the control unit 41 causes the surrounding information acquiring unit 43 to detect the other vehicle V3 whose light is turned on from the surrounding image acquired by the image data acquiring unit 42 and set it as surrounding information. If it is determined that there is another vehicle V3 whose light is turned on based on the peripheral information acquired from the peripheral information acquisition unit 43 (Yes in step S12), the control unit 41 proceeds to step S13. When it is determined that the other vehicle V3 whose light is turned on does not exist based on the peripheral information acquired from the peripheral information acquisition unit 43 (No in step S12), the control unit 41 performs the process in step S11 again. Run.

  The control unit 41 generates the overhead view image 100 (step S13). More specifically, the control unit 41 causes the overhead view video generation unit 45 to generate an overhead view video 100 obtained by performing viewpoint conversion so as to look down the vehicle V1 from above from the peripheral video acquired by the video data acquisition unit 42. The control unit 41 proceeds to step S14.

  The control unit 41 displays the overhead view image 100 (step S14). More specifically, the control unit 41 causes the display control unit 49 to display the overhead view image 100 on the display panel 31. The control unit 41 proceeds to step S15.

  The control unit 41 determines whether to end the overhead view video display (step S15). More specifically, the control unit 41 determines whether to end the display of the overhead view video 100 based on the presence or absence of the end trigger. The end trigger is, for example, a case where a predetermined time has elapsed after displaying the overhead view video 100, or a case where the vehicle speed of the vehicle V1 becomes equal to or more than a threshold. The end trigger may be, for example, a case where the overhead view image 100 is displayed, and a state in which the vehicle V1 is stopped has passed a predetermined time. In other words, the end trigger may be, for example, a case where the overhead image 100 is displayed, and a state in which the vehicle V1 is stopped has elapsed for a predetermined time, and the operation has not been performed. If there is an end trigger, the control unit 41 determines to end the display of the overhead view video 100 (Yes in step S15), and ends the process. When there is no end trigger, the control unit 41 determines that the display of the overhead view video 100 is not ended (No in step S15), and executes the process of step S13 again.

  Thus, the overhead view video image generation system 1 outputs a video signal for displaying the overhead view video 100 on the display panel 31 when it is determined that there is another vehicle V3 whose turn indicator is on. The display panel 31 displays the overhead view video 100 together with the navigation screen, for example, based on the video signal output from the overhead view video generation system 1.

  As described above, in the present embodiment, the overhead image 100 is displayed on the display panel 31 when it is determined that the other vehicle V3 whose turn indicator is lit is present. Thus, according to the present embodiment, the overhead video image 100 can allow the driver to easily recognize the relative positional relationship between the vehicle V1 and the surrounding vehicles of the vehicle V1. Thus, the present embodiment can display the vehicle periphery in a checkable manner.

  Unlike the present embodiment, even if the overhead image 100 is not displayed even if there is another vehicle V3 whose turn indicator is on, the driver recognizes the inter-vehicle distance between the vehicle V1 and the surrounding vehicles of the vehicle V1. It may be difficult. Thereby, as shown in FIG. 2, there is a possibility that the vehicle V1 may remain stopped at the stop position where the other vehicle V3 is blocked. In this case, for example, the other vehicle V3 attempting to pass behind the vehicle V1 from the oncoming lane can not advance until the vehicle V1 moves forward, and the following vehicle of the other vehicle V3 stays and smooth passing on the oncoming lane May be hampered.

  On the other hand, according to the present embodiment, the overhead image 100 as shown in FIG. 3 is displayed, so that the driver can confirm that the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 is long. . The driver can determine that the route of the other vehicle V3 can be secured by advancing the vehicle V1 and shortening the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 according to the overhead image 100. As described above, according to the present embodiment, the overhead image 100 enables the other vehicle V3 to pass smoothly. Furthermore, according to the present embodiment, the following vehicle of the other vehicle V3 can be suppressed from staying.

Second Embodiment
The overhead view image generation system 1 according to the present embodiment will be described with reference to FIGS. 5 and 6. FIG. 5 is a diagram for explaining the overhead view video image generation system according to the second embodiment, and shows a vehicle and its surroundings. FIG. 6 is a flow chart showing the flow of processing in the overhead view video generation device of the overhead view video generation system according to the second embodiment. The basic configuration of the overhead view video generation system 1 is the same as that of the overhead view video generation system 1 of the first embodiment. In the following description, the same reference numerals or corresponding reference numerals are given to the same components as the overhead view video generation system 1, and the detailed description thereof is omitted.

  In the present embodiment, the overhead view video generation system 1 is configured such that when there is an entrance / exit part through which the vehicle can enter / exit or a branch part Q of the road on the side of the vehicle V1 based on the surrounding information and the vehicle information. , Generate and display the overhead image 100. In the overhead view video generation system 1 of the present embodiment, the processing in the peripheral information acquisition unit 43, the overhead view video generation unit 45, and the display control unit 49 is the same as the peripheral information acquisition unit 43 and the overhead view video generation unit 45 of the first embodiment. It differs from the control unit 49.

  The entrance is a place where another vehicle V3 may move in and out of the vehicle V1. The entry / exit section is, for example, an entry / exit section of a parking lot or an entry / exit section of an alley. In such a place, the vehicle V1 may stop at a position where the entrance / exit section is closed, which may make it difficult to smoothly enter / exit the entrance / exit section of the other vehicle V3.

  The branch portion Q is a place where the lane branches. For example, the branch part Q is a place where a branch lane for right turn or left car branch branches.

  The peripheral information acquisition unit 43 performs image processing on the peripheral video acquired by the video data acquisition unit 42 to detect an entrance or branch Q existing on the side of the vehicle V1, and detects the detection result as peripheral information. I assume. For example, in the peripheral image captured by at least one of the rear camera 12, the left side camera 13 and the right side camera 14, the peripheral information acquisition unit 43 is a location where a guardrail or curb cut is present on the side of the vehicle V1. Detect as an entrance. For example, in the peripheral image captured by at least one of the rear camera 12, the left side camera 13 and the right side camera 14, the peripheral information acquisition unit 43 has a sign indicating the entrance of the parking lot on the side of the vehicle V1. We detect place as entrance. For example, in the peripheral image captured by at least one of the rear camera 12, the left side camera 13 and the right side camera 14, the peripheral information acquisition unit 43 may look like a left turn lane or a right turn lane existing to the side of the vehicle V1. The branch point Q of the branch lane is detected. The peripheral information acquisition unit 43 outputs the detection result of the entrance / exit unit or the branch unit Q to the overhead image generation unit 45 as peripheral information.

  The peripheral information acquisition unit 43 may detect an object to be photographed having a linearity equal to or greater than a predetermined length from the peripheral image as a guardrail or a curb by edge detection processing, and detect a cut of the object. The peripheral information acquisition unit 43 may detect a cut in the guardrail or the curb from the peripheral image by a recognition dictionary that stores the cut in the guardrail or the curb.

  Alternatively, the peripheral information acquisition unit 43 may detect the entrance / exit section Q from the map information of the navigation system (not shown) and the current position information of the vehicle V1.

  The overhead image generation unit 45 checks the inter-vehicle distance between the vehicle V1 and the surrounding vehicles of the vehicle V1 when the entrance / exit part or the branch part Q exists on the side of the vehicle V1 based on the surrounding information and the vehicle information. It is determined that it is preferable to generate the overhead view video 100.

  The display control unit 49 confirms the inter-vehicle distance between the vehicle V1 and the surrounding vehicles of the vehicle V1 when the entrance / exit part or the branch part Q exists on the side of the vehicle V1 based on the surrounding information and the vehicle information. It is determined that it is preferable to display the overhead view image 100 on the display panel 31.

  Next, the flow of processing in the overhead view video generation device 40 will be described using FIG. The processes of steps S21 and S23 to S25 of the flowchart shown in FIG. 6 are the same as the processes of steps S11 and S13 to S15 of the flowchart shown in FIG.

  The control unit 41 determines whether the entrance / exit part or the branch part Q exists on the side of the vehicle V1 (step S22). More specifically, the control unit 41 causes the surrounding information acquisition unit 43 to detect an entrance or branch part Q present on the side of the vehicle V1 from the surrounding video acquired by the video data acquisition unit 42 and use it as peripheral information. . When it is determined that the entrance / exit part or the branch part Q exists on the side of the vehicle V1 based on the peripheral information acquired from the peripheral information acquisition part 43 (Yes in step S22), the control part 41 proceeds to step S23. When it is determined that the entrance / exit part or the branching part Q does not exist on the side of the vehicle V1 based on the peripheral information acquired from the peripheral information acquisition part 43 (No in step S22), the control part 41 performs the process of step S21. Run again.

  As described above, the present embodiment can generate the overhead view image 100 when the entrance / exit part or the branch part Q is present on the side of the vehicle V1. According to the present embodiment, when the vehicle V1 stops to the side of the entrance / exit part or the branch part Q, the driver can easily recognize the relative positional relationship between the vehicle V1 and the surrounding vehicles of the vehicle V1. . Thus, the present embodiment can display the vehicle periphery in a checkable manner.

  Unlike in the present embodiment, even when the overhead image 100 is not displayed even if the entrance / exit part or the branch part Q is present on the side of the vehicle V1, the driver selects the vehicle V1 and the peripheral vehicles of the vehicle V1. It may be difficult to recognize the distance between vehicles. Thereby, as shown in FIG. 5, there is a possibility that the vehicle V1 may remain stopped at the stop position where the branch portion Q is closed. In this case, even if the following vehicle V4 shown by a broken line tries to enter the right turn lane from the branch point Q, it can not advance until the vehicle V1 advances, and the following vehicle of the following vehicle V4 stays and smooth traffic is impeded There is a fear.

  On the other hand, according to the present embodiment, the driver ensures the route of the following vehicle V4 by advancing the vehicle V1 with the overhead image 100 and shortening the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2. It can be judged that it is possible. As described above, according to the present embodiment, the overhead image 100 enables the succeeding vehicle V4 to pass smoothly. Furthermore, according to the present embodiment, the following vehicle of the following vehicle V4 can be suppressed from staying.

Third Embodiment
The overhead view image generation system 1 according to the present embodiment will be described with reference to FIG. 7. FIG. 7 is a flow chart showing the flow of processing in the overhead view video generation device of the overhead view video generation system according to the third embodiment.

  In the present embodiment, the overhead view video generation system 1 generates and displays the overhead view video 100 based on the surrounding information and the vehicle information, when the inter-vehicle distance between the vehicle V1 and the following vehicle is equal to or less than a threshold. In the overhead view video generation system 1 of the present embodiment, the processing in the peripheral vehicle information acquisition unit 431, the overhead view video generation unit 45, and the display control unit 49 of the peripheral information acquisition unit 43 is the same as the peripheral information acquisition unit 43 of the first embodiment. The overhead video generation unit 45 and the display control unit 49 are different. For example, the following vehicle is a vehicle in which the distance between the vehicle and the vehicle V1 is shortened to move to the branch lane, but the vehicle can not move ahead of the position and is stopped.

  The surrounding vehicle information acquisition unit 431 acquires the inter-vehicle distance between the vehicle V1 and the following vehicle as the surrounding vehicle information. The surrounding vehicle information acquisition unit 431 acquires the inter-vehicle distance between the vehicle V1 and the following vehicle from the rear camera 12 or various sensors for sensing.

  If the inter-vehicle distance between the vehicle V1 and the following vehicle is equal to or less than the threshold based on the surrounding information and the vehicle information, the overhead image generation unit 45 should check the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 The determination is made, and the overhead view video 100 is generated.

  When the inter-vehicle distance between the vehicle V1 and the following vehicle is equal to or less than the threshold based on the surrounding information and the vehicle information, the display control unit 49 determines that the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 should be confirmed. The overhead image 100 is displayed on the display panel 31.

  Next, the flow of processing in the overhead view video generation device 40 will be described using FIG. 7. The processes in steps S31 and S33 to S35 of the flowchart shown in FIG. 7 are the same as the processes in steps S11 and S13 to S15 of the flowchart shown in FIG.

  Control unit 41 determines whether the inter-vehicle distance between vehicle V1 and the following vehicle is equal to or less than a threshold (step S32). More specifically, the control unit 41 causes the surrounding vehicle information acquisition unit 431 to acquire surrounding vehicle information including the inter-vehicle distance between the vehicle V1 and the following vehicle. When it is determined that the inter-vehicle distance between the vehicle V1 and the following vehicle is equal to or less than the threshold based on the surrounding information acquired from the peripheral information acquiring unit 43 (Yes in Step S32), the control unit 41 proceeds to Step S33. When it is determined that the inter-vehicle distance between the vehicle V1 and the following vehicle is not equal to or less than the threshold (No in step S32), the control unit 41 executes the process of step S31 again.

  As described above, this embodiment can generate the overhead view image 100 when the inter-vehicle distance between the vehicle V1 and the following vehicle is equal to or less than the threshold. According to the present embodiment, when the inter-vehicle distance between the vehicle V1 and the following vehicle is equal to or less than the threshold value, the driver can easily recognize the relative positional relationship between the vehicle V1 and the preceding vehicle V2. Thus, the present embodiment can display the vehicle periphery in a checkable manner.

  According to the present embodiment, the driver determines that the route of the following vehicle V4 can be secured by advancing the vehicle V1 and shortening the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 by the overhead image 100. Can. As described above, according to the present embodiment, the overhead image 100 enables the succeeding vehicle V4 to pass smoothly. Furthermore, according to the present embodiment, the following vehicle of the following vehicle V4 can be suppressed from staying.

Fourth Embodiment
The overhead view image generation system 1 according to the present embodiment will be described with reference to FIGS. 8 and 9. FIG. 8 is a diagram for explaining the overhead view video image generation system according to the fourth embodiment, and shows a vehicle and its surroundings. FIG. 9 is a flow chart showing the flow of processing in the overhead view video generation device of the overhead view video generation system according to the fourth embodiment.

  In the present embodiment, the overhead view video generation system 1 generates and displays the overhead view video 100 when the vehicle V1 lights the turn indicator based on the vehicle information. The overhead image generation system 1 of this embodiment differs from the overhead image generation unit 45 and display control unit 49 of the first embodiment in the processing in the overhead image generation unit 45 and the display control unit 49.

  The overhead video image generation unit 45 determines that it is better to confirm the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 when the vehicle V1 operates the turn indicator based on the vehicle information, and the overhead video image 100 Generate

  The display control unit 49 determines that it is better to check the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 when the vehicle V1 operates the turn indicator based on the vehicle information, and displays the overhead image 100 Displayed on the display panel 31.

  Next, the flow of processing in the overhead view video generation device 40 will be described using FIG. 9. The processes of step S41 and steps S43 to S45 of the flowchart shown in FIG. 9 are the same as the processes of step S11 and steps S13 to S15 of the flowchart shown in FIG.

  The control unit 41 determines whether the vehicle V1 lights the turn signal (step S42). More specifically, when it is determined that the vehicle V1 lights the turn signal based on the vehicle information acquired by the vehicle information acquisition unit 44 (Yes in step S42), the control unit 41 proceeds to step S43. Control part 41 performs processing of Step S41 again, when it is judged that vehicles V1 do not turn on a turn signal based on vehicles information which vehicles information acquisition part 44 acquired (it is No at Step S42).

  As described above, in the present embodiment, the overhead view image 100 is displayed on the display panel 31 when it is determined that the vehicle V1 is in the state of turning on the turn signal. Thus, according to the present embodiment, the driver can easily recognize the relative positional relationship between the vehicle V1 and the preceding vehicle V2. Thus, the present embodiment can display the vehicle periphery in a checkable manner.

  According to the present embodiment, the driver can grasp the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 by the overhead image 100, advances the vehicle V1 and confirms the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2. It is then possible to secure the course of the vehicle V1 and determine that the lane can be changed at the branch point. Thus, according to the present embodiment, it is possible to allow the vehicle V1 to pass smoothly.

  Up to this point, the overhead-view image generation system 1 according to the present invention has been described, but may be implemented in various different modes other than the above-described embodiment.

  Each component of the illustrated overhead view video image generation system 1 is functionally conceptual and may not necessarily be physically configured as illustrated. That is, the specific form of each device is not limited to the illustrated one, and all or a part thereof is functionally or physically dispersed or integrated in an arbitrary unit according to the processing load and use condition of each device, etc. May be

  The configuration of the overhead view video generation system 1 is realized by, for example, a program loaded into a memory as software. The above embodiment has been described as a functional block realized by cooperation of these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The components described above include those which can be easily conceived by those skilled in the art, and substantially the same ones. Furthermore, the configurations described above can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the configuration are possible without departing from the scope of the present invention.

  The overhead view video generation system 1 may be configured to generate and display the overhead view video 100 when the vehicle V1 is located on the side of the bus stop based on the surrounding information and the vehicle information. Based on the peripheral information acquired by the peripheral information acquisition unit 43 and the vehicle information acquired by the vehicle information acquisition unit 44, the overhead image generation unit 45 determines that the bus stop is present on the side of the vehicle V1. If so, the overhead image 100 may be generated. The display control unit 49 may display the overhead image 100 on the display panel 31 when it is determined that the bus stop is present on the side of the vehicle V1 based on the surrounding information and the vehicle information. Thereby, the driver of the vehicle V1 can suppress the vehicle V1 from stopping by the side of the stop by the overhead image 100 to prevent the passage of the bus.

  In the first embodiment, based on the surrounding information and the vehicle information, based on the inter-vehicle distance between the vehicle V1 and the preceding vehicle V2 and the width of the road on which the other vehicle V3 is going to change its course, The overhead video generation unit 45 generates the overhead video 100 only when it is determined that the other vehicle V3 can pass when the V1 advances, and the display control unit 49 displays the overhead video 100 on the display panel 31. May be

  For example, the first embodiment and the third embodiment may be combined. If the following vehicle V4 turns on the turn indicator based on the surrounding information and the vehicle information, and the inter-vehicle distance between the vehicle V1 and the following vehicle V4 is equal to or less than the threshold, the overhead image generation unit 45 displays the overhead image 100 The display control unit 49 may generate the overhead view image 100 on the display panel 31.

  For example, the first to third embodiments may be combined. If it is determined based on the surrounding information and the vehicle information that there is an entrance / exit P on the side of the vehicle V1 and there is another vehicle V3 whose light is a turn indicator, the overhead view video generation unit 45 The overhead video 100 may be generated, and the display control unit 49 may display the overhead video 100 on the display panel 31.

  The overhead view image generation unit 45 may generate the overhead view image 100A in which the inter-vehicle distance between the vehicle V1 and a vehicle around the vehicle V1 is emphasized. FIG. 10 is a view showing another example of the overhead view video generated by the overhead view video generation system. The overhead view video generation unit 45 may generate the overhead view video 100A including the inter-vehicle emphasis icon 106A in which the inter-vehicle distance between the vehicle V1 and a vehicle around the vehicle V1 is highlighted. As a result, it is possible to make a display that allows easy confirmation of the inter-vehicle distance between the vehicle V1 and a vehicle surrounding the vehicle V1.

  When the road surface in the vicinity of the vehicle V1 is marked with the entry prohibited portion or the stop prohibited portion based on the surrounding information and the vehicle information, the overhead image generating unit 45 generates the overhead image 100, and the display control unit 49 May display the overhead view image 100 on the display panel 31. Thus, it is possible to suppress the vehicle V1 or the following vehicle V4 from stopping at the entry prohibited portion or the stop prohibited portion.

  Although the overhead view video generation unit 45 has been described as generating the overhead view video 100 when the predetermined condition is satisfied, the present invention is not limited to this. The overhead view video generation unit 45 may generate the overhead view video 100 while the overhead view video generation system 1 is activated. In this case, based on the peripheral information indicating the peripheral situation acquired by the peripheral information acquisition unit 43 and the vehicle information acquired by the vehicle information acquisition unit 44 by the display control unit 49, the vehicle V1 and the peripheral vehicles of the vehicle V1 are If it is determined that it is preferable to confirm the inter-vehicle distance, control may be performed to display the overhead image 100.

  The processing in the overhead-view video generation device 40 is executed along the flowchart shown in FIG. 4, FIG. 6, FIG. 7, and FIG. May be

DESCRIPTION OF SYMBOLS 1 bird's-eye view image generation system 11 front camera 12 rear camera 13 left-side camera 14 right-side camera 40 bird's-eye view video generator 41 control unit 42 image data acquisition unit 43 peripheral information acquisition unit 44 vehicle information acquisition unit 45 eye-view image generation unit 49 display control Department

Claims (9)

A video data acquisition unit for acquiring peripheral video data photographed by a photographing unit for photographing the vicinity of the vehicle;
A surrounding information acquisition unit that acquires surrounding information indicating a situation around the vehicle;
A vehicle information acquisition unit that acquires vehicle information including the vehicle speed of the vehicle;
An overhead view video for performing viewpoint conversion processing on the peripheral video acquired by the video data acquisition unit, combining processing of a plurality of peripheral videos subjected to viewpoint conversion processing, and generating a display video of a display range for displaying the periphery of the vehicle A generation unit,
A display control unit that causes the display unit to display the overhead view video generated by the overhead view video generation unit;
Equipped with
The display control unit determines an inter-vehicle distance between the vehicle and a peripheral vehicle traveling around the vehicle, based on the peripheral information acquired by the peripheral information acquisition unit and the vehicle information acquired by the vehicle information acquisition unit. If it is determined that it is better to confirm, the overhead video is displayed on the display unit,
An overhead view video generation device characterized by   The display control unit checks the inter-vehicle distance between the vehicle and the surrounding vehicle when there is another vehicle whose turn indicator is on the basis of the peripheral information acquired by the peripheral information acquisition unit The overhead view video generation apparatus according to claim 1, wherein the overhead view video is displayed on the display unit.   The display control unit is an entrance or exit unit through which the vehicle can enter and exit to the side of the vehicle based on the peripheral information acquired by the peripheral information acquisition unit and the vehicle information acquired by the vehicle information acquisition unit. It is determined that it is better to check an inter-vehicle distance between the vehicle and the surrounding vehicle when a branch of a road is present, and a bird's-eye view image is displayed on the display unit. Or the bird's-eye view image generation apparatus as described in 2. The peripheral information acquisition unit includes a peripheral vehicle information acquisition unit that acquires, as peripheral vehicle information, an inter-vehicle distance between the vehicle and a following vehicle traveling behind the vehicle.
When the inter-vehicle distance to the following vehicle is equal to or less than a threshold based on the surrounding vehicle information acquired by the surrounding vehicle information acquiring unit and the vehicle information acquired by the vehicle information acquiring unit, the display control unit is the vehicle It determines that it is better to confirm the inter-vehicle distance between the vehicle and the preceding vehicle traveling in front of the vehicle, and displays a bird's-eye-view image on the display unit. An overhead-view video generation apparatus as described in   It is better that the display control unit checks the inter-vehicle distance between the vehicle and the surrounding vehicle when the vehicle lights the turn indicator based on the vehicle information acquired by the vehicle information acquisition unit. The overhead view video generation apparatus according to claim 1, wherein the overhead view video is displayed on the display unit after the determination.   The display control unit determines that the inter-vehicle distance between the vehicle and the surrounding vehicle is better based on the vehicle information acquired by the vehicle information acquisition unit, and causes the display unit to display the overhead image The overhead view video generation device according to any one of claims 1 to 5, wherein the display of the overhead view video is ended when a state in which the operation of the vehicle is not performed after a predetermined time has elapsed. An overhead-view image generation apparatus according to any one of claims 1 to 6,
And an imaging unit for imaging the periphery of the vehicle and supplying the peripheral image data to the image data acquisition unit. A video data acquisition step of acquiring peripheral video data photographed by a photographing unit photographing the vicinity of the vehicle;
A surrounding information acquisition step of acquiring surrounding information indicating a situation around the vehicle;
A vehicle information acquisition step of acquiring vehicle information including the vehicle speed of the vehicle;
An overhead view video is generated by performing viewpoint conversion processing on the peripheral video acquired in the video data acquisition step, combining processing of a plurality of peripheral videos subjected to the viewpoint conversion processing, and generating a display image for displaying the periphery of the vehicle A generation step,
A display control step of causing the display unit to display the overhead view video generated in the overhead view video generation step;
Including
In the display control step, based on the peripheral information acquired in the peripheral information acquisition step and the vehicle information acquired in the vehicle information acquisition step, an inter-vehicle distance between the vehicle and a peripheral vehicle traveling around the vehicle is calculated. If it is determined that it is better to confirm, the overhead video is displayed on the display unit,
A method of generating an overhead image. A video data acquisition step of acquiring peripheral video data photographed by a photographing unit photographing the vicinity of the vehicle;
A surrounding information acquisition step of acquiring surrounding information indicating a situation around the vehicle;
A vehicle information acquisition step of acquiring vehicle information including the vehicle speed of the vehicle;
An overhead view video is generated by performing viewpoint conversion processing on the peripheral video acquired in the video data acquisition step, combining processing of a plurality of peripheral videos subjected to the viewpoint conversion processing, and generating a display image for displaying the periphery of the vehicle A generation step,
A display control step of causing the display unit to display the overhead view video generated in the overhead view video generation step;
Including
In the display control step, based on the peripheral information acquired in the peripheral information acquisition step and the vehicle information acquired in the vehicle information acquisition step, an inter-vehicle distance between the vehicle and a peripheral vehicle traveling around the vehicle is calculated. If it is determined that it is better to confirm, the overhead video is displayed on the display unit,
A program for causing a computer that operates as a video production device to execute things.

Images