JP6825512B2 - Bird's-eye view image generation device, bird's-eye view image generation system, bird's-eye view image generation method and program - Google Patents

Description

The present invention relates to a bird's-eye view image generation device, a bird's-eye view image generation system, a bird's-eye view image generation method, and a program.

A technique related to a vehicle peripheral display device for displaying a bird's-eye view image of a vehicle together with a vehicle image is known (see, for example, Patent Document 1). This technology expands the display area of the bird's-eye view image behind the vehicle image when the vehicle switches from forward to backward.

Japanese Unexamined Patent Publication No. 2015-0764645

The vehicle may stop due to traffic lights or traffic jams while driving. If, for example, there is an entrance / exit of a parking lot or an entrance / exit of an alley on the side of the vehicle, and the vehicle stops at a position that blocks the entrance / exit, smooth passage of other vehicles entering / exiting the entrance / exit. May interfere with. Therefore, when the vehicle is likely to stop near the doorway, it is desired to display the area around the vehicle in a recognizable manner in order to assist the driver in determining an appropriate stop position.

The present invention has been made in view of the above, and an object of the present invention is to display the periphery of a vehicle in a recognizable manner.

In order to solve the above-mentioned problems and achieve the object, the bird's-eye view image generation device according to the present invention includes an image data acquisition unit that acquires peripheral image data photographed by a photographing unit that photographs the periphery of the vehicle, and a video data acquisition unit of the vehicle. Peripheral information acquisition unit that acquires peripheral information indicating the surrounding situation, vehicle information acquisition unit that acquires vehicle information including the vehicle speed of the vehicle, and peripheral image acquired by the video data acquisition unit are subjected to viewpoint conversion processing. A bird's-eye view image generation unit that generates a bird's-eye view image of a display range that displays the periphery of the vehicle by synthesizing a plurality of peripheral images that have undergone viewpoint conversion processing, and a bird's-eye view image generation unit that displays the bird's-eye view image generated by the bird's-eye view image generation unit. The display control unit is provided with a display control unit to be displayed on the side of the vehicle, based on the peripheral information acquired by the peripheral information acquisition unit and the vehicle information acquired by the vehicle information acquisition unit. When it is determined that there is an entrance / exit where the vehicle can enter / exit and the vehicle may make it difficult for the vehicle to smoothly enter / exit the entrance / exit of another vehicle, a bird's-eye view image may be displayed on the display. It is a feature.

The bird's-eye view image generation system according to the present invention is characterized by including the above-mentioned bird's-eye view image generation device and a photographing unit that photographs the periphery of the vehicle and supplies peripheral image data to the image data acquisition unit.

The bird's-eye view image generation method according to the present invention includes a video data acquisition step of acquiring peripheral image data taken by a photographing unit that photographs the periphery of the vehicle, and a peripheral information acquisition step of acquiring peripheral information indicating the situation around the vehicle. Then, the vehicle information acquisition step of acquiring the vehicle information including the vehicle speed of the vehicle and the peripheral image acquired in the video data acquisition step are subjected to the viewpoint conversion process, and the composite process of the plurality of peripheral images subjected to the viewpoint conversion process is performed. The display control includes a bird's-eye view image generation step of generating a bird's-eye view image of a display range for displaying the periphery of the vehicle, and a display control step of displaying the bird's-eye view image generated in the bird's-eye view image generation step on a display unit. In the step, based on the peripheral information acquired in the peripheral information acquisition step and the vehicle information acquired in the vehicle information acquisition step, there is an entrance / exit portion on the side of the vehicle through which the vehicle can enter and exit, and the vehicle When it is determined that there is a possibility of making it difficult for the other vehicle to smoothly enter and exit the entrance / exit portion, a bird's-eye view image is displayed on the display unit.

The program according to the present invention includes a video data acquisition step of acquiring peripheral image data taken by a photographing unit that photographs the periphery of the vehicle, a peripheral information acquisition step of acquiring peripheral information indicating a situation around the vehicle, and the above. The vehicle information acquisition step for acquiring vehicle information including the vehicle speed of the vehicle and the peripheral image acquired in the video data acquisition step are subjected to viewpoint conversion processing, and a plurality of peripheral images subjected to the viewpoint conversion processing are combined. The display control step includes a bird's-eye view image generation step for generating a bird's-eye view image of a display range for displaying the periphery of the vehicle, and a display control step for displaying the bird's-eye view image generated in the bird's-eye view image generation step on a display unit. Based on the peripheral information acquired in the peripheral information acquisition step and the vehicle information acquired in the vehicle information acquisition step, there is an entrance / exit portion on the side of the vehicle through which the vehicle can enter and exit, and another vehicle is provided by the vehicle. When it is determined that there is a possibility of making it difficult to smoothly enter and exit the entrance / exit portion, the computer operating as the bird's-eye view image generator is made to display the bird's-eye view image on the display unit.

According to the present invention, there is an effect that the periphery of the vehicle can be displayed in a recognizable manner.

FIG. 1 is a block diagram showing a configuration example of a bird's-eye view video generation system according to the first embodiment. FIG. 2 is a diagram showing a vehicle and its surroundings. FIG. 3 is a diagram showing a vehicle and its surroundings. FIG. 4 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the first embodiment. FIG. 5 is a flowchart showing a processing flow in the bird's-eye view image generation device of the bird's-eye view image generation system according to the first embodiment. FIG. 6 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the second embodiment. FIG. 7 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the third embodiment. FIG. 8 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the fourth embodiment. FIG. 9 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the fifth embodiment.

The bird's-eye view image generation device 40, the bird's-eye view image generation system 1, the bird's-eye view image generation method, and the embodiment of the program according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the following embodiments.

[First Embodiment]
FIG. 1 is a block diagram showing a configuration example of a bird's-eye view video generation system according to the first embodiment. FIG. 2 is a diagram showing a vehicle and its surroundings. FIG. 3 is a diagram showing a vehicle and its surroundings. In the bird's-eye view image generation system 1, there is a possibility that the entrance / exit portion P through which the other vehicle V3 can enter / exit exists on the side of the vehicle V1 and that the vehicle V1 makes it difficult for the other vehicle V3 to smoothly enter / exit the entrance / exit portion P. If it is determined to be present, a bird's-eye view image 100 (see FIG. 4) is generated.

The entrance / exit portion P is a place where another vehicle V3 may enter / exit across the front / rear of the vehicle V1. In the present embodiment, the entrance / exit portion P is, for example, a place where another vehicle V3, which is a right-turning vehicle traveling in the oncoming lane, may enter / exit across the front of the vehicle V1. The doorway P is, for example, a parking lot doorway or an alley doorway. In such a place, the vehicle V1, in other words, the vehicle V1 stops at a position blocking the doorway P, which may make it difficult for the other vehicle V3 to smoothly enter and exit the doorway P.

The bird's-eye view image generation device 40 and the bird's-eye view image generation system 1 are mounted on the vehicle V1. The bird's-eye view image generation device 40 and the bird's-eye view image generation system 1 may be portable devices that can be used in the vehicle V1 in addition to those mounted on the vehicle V1.

The bird's-eye view image generation system 1 will be described with reference to FIG. The bird's-eye view image generation system 1 includes a front camera (shooting unit) 11, a rear camera (shooting unit) 12, a left side camera (shooting unit) 13, a right side camera (shooting unit) 14, and a display panel (display unit). ) 31 and a bird's-eye view image generator 40.

The front camera 11 is a camera for a bird's-eye view image. The front camera 11 is arranged in front of the vehicle V1 and photographs the surroundings centered on the front of the vehicle V1. As shown in FIG. 2, the front camera 11 captures, for example, a first photographing range A1 of about 180 °. The first shooting range A1 includes a wider range in front of the vehicle V1 than the display range A of the bird's-eye view image 100. The front camera 11 outputs the captured image to the image data acquisition unit 42 of the bird's-eye view image generation device 40.

The rear camera 12 is a bird's-eye view video camera. The rear camera 12 is arranged behind the vehicle V1 and photographs the surroundings centered on the rear of the vehicle V1. The rear camera 12 captures, for example, a second photographing range A2 of about 180 °. The second shooting range A2 includes a wider range behind the vehicle V1 than the display range A of the bird's-eye view image 100. The rear camera 12 outputs the captured image to the image data acquisition unit 42 of the bird's-eye view image generation device 40.

The left side camera 13 is a camera for a bird's-eye view image. The left-side camera 13 is arranged on the left side of the vehicle V1 and photographs the surroundings centered on the left side of the vehicle V1. The left-side camera 13 captures, for example, a third photographing range A3 of about 180 °. The third shooting range A3 includes a wider range on the left side of the vehicle V1 than the display range A of the bird's-eye view image 100. The left-side camera 13 outputs the captured image to the image data acquisition unit 42 of the bird's-eye view image generation device 40.

The right side camera 14 is a camera for a bird's-eye view image. The right-side camera 14 is arranged on the right side of the vehicle V1 and photographs the periphery centered on the right side of the vehicle V1. The right-side camera 14 captures, for example, a fourth photographing range A4 of about 180 °. The fourth shooting range A4 includes a wider range on the right side of the vehicle V1 than the display range A of the bird's-eye view image 100. The right-side camera 14 outputs the captured image to the image data acquisition unit 42 of the bird's-eye view image generation device 40.

The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 photograph the vehicle V1 in all directions.

Returning to FIG. 1, the display panel 31 is, for example, a display including a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminence) display. The display panel 31 displays the bird's-eye view image 100 based on the image signal output from the bird's-eye view image generation device 40 of the bird's-eye view image generation system 1. The display panel 31 may be dedicated to the bird's-eye view image generation system 1, or may be used jointly with other systems including a navigation system, for example. The display panel 31 is arranged at a position that is easily visible to the driver.

The bird's-eye view image generation device 40 has a control unit 41 and a storage unit 50.

The control unit 41 is, for example, an arithmetic processing unit composed of a CPU (Central Processing Unit) or the like. The control unit 41 loads the program stored in the storage unit 50 into the memory and executes the instruction included in the program. The control unit 41 includes a video data acquisition unit 42, a peripheral information acquisition unit 43, a vehicle information acquisition unit 44, a bird's-eye view image generation unit 45, and a display control unit 49. The control unit 41 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the control unit 41.

The video data acquisition unit 42 acquires peripheral video data obtained by photographing the periphery of the vehicle V1. More specifically, the video data acquisition unit 42 acquires the video data output by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The video data acquisition unit 42 outputs the acquired video data to the peripheral information acquisition unit 43 and the bird's-eye view image generation unit 45. The video data acquired from each camera is, for example, a moving image composed of an image of 30 frames per second.

The peripheral information acquisition unit 43 acquires peripheral information indicating the situation around the vehicle V1. In the present embodiment, the peripheral information acquisition unit 43 executes image processing on the peripheral image acquired by the video data acquisition unit 42, detects the entrance / exit unit P existing in front of the vehicle V1, and obtains the detection result. Peripheral information. For example, the peripheral information acquisition unit 43 uses a place where a guardrail or a curb cut exists on the left front side of the vehicle V1 as the entrance / exit unit P in the peripheral image taken by at least one of the front camera 11 and the left side camera 13. To detect. For example, the peripheral information acquisition unit 43 places a place where a sign indicating the entrance of the parking lot exists on the left front side of the vehicle V1 in the peripheral image taken by at least one of the front camera 11 and the left side camera 13. Detect as P. The peripheral information acquisition unit 43 outputs the detection result of the entrance / exit unit P to the bird's-eye view image generation unit 45 as peripheral information.

Peripheral information acquisition unit 43 may detect an object to be photographed having a linearity of a predetermined length or more as a guardrail or a curb from the peripheral image by edge detection processing, and detect a break thereof. The peripheral information acquisition unit 43 may detect the cut of the guardrail or the curb from the peripheral image by the recognition dictionary that stores the cut of the guardrail or the curb.

Alternatively, the peripheral information acquisition unit 43 may detect the entrance / exit unit P from the map information of the navigation system (not shown) and the current position information of the vehicle V1.

Further, the peripheral information acquisition unit 43 is at least one of the vehicle speed of the preceding vehicle V2 traveling in front of the vehicle V1, the inter-vehicle distance L1 between the vehicle V1 and the preceding vehicle V2, and the relative speed between the vehicle V1 and the preceding vehicle V2. The preceding vehicle information acquisition unit 431 that acquires the preceding vehicle information including the above may be provided. The preceding vehicle information acquisition unit 431 acquires the state of the preceding vehicle V2 from the front camera 11, various sensors that sense the state, vehicle information indicating the status of the vehicle V1 obtained from CAN (Control Area Network), and the like.

The vehicle information acquisition unit 44 acquires vehicle information indicating the status of the vehicle V1 from various sensors that sense the state of the CAN and the vehicle V1. The vehicle information acquisition unit 44 acquires vehicle speed information, for example. The vehicle information acquisition unit 44 outputs the acquired vehicle information to the bird's-eye view image generation unit 45.

The bird's-eye view image generation unit 45 allows another vehicle V3 to enter and exit the side of the vehicle V1 based on the peripheral information indicating the surrounding situation acquired by the peripheral information acquisition unit 43 and the vehicle information acquired by the vehicle information acquisition unit 44. A bird's-eye view image 100 is generated when there is an entrance / exit P and it is predicted that the vehicle V1 will stop or decelerate, which may make it difficult for the other vehicle V3 to enter / exit smoothly. To do. In the present embodiment, the bird's-eye view image generation unit 45 determines that the doorway P exists in front of the vehicle V1 and that the vehicle V1 may block at least a part of the doorway P and stop. In the case, the bird's-eye view image 100 is generated. More specifically, the bird's-eye view image generation unit 45 determines that the doorway P exists in front of the vehicle V1 and that the vehicle V1 may stop, and the predicted stop position based on the vehicle speed of the vehicle V1 is the doorway. When it is determined that at least a part of the part P is blocked, the bird's-eye view image 100 is generated.

The predicted stop position of the vehicle V1 will be described with reference to FIG. Blocking at least a part of the doorway P means that the vehicle V1 stopped at the predicted stop position calculated based on the vehicle speed is located on the side of the doorway P, and the front end of the vehicle V1 and the rear end of the preceding vehicle V2. It means that the inter-vehicle distance L1 with the portion is smaller than the width L2 of the entrance / exit portion P. The inter-vehicle distance L1 is such that another vehicle V3 cannot pass through.

Whether or not the stop or deceleration of the vehicle V1 is predicted, in other words, whether or not the vehicle V1 may stop is determined as follows, for example. Based on at least one of the vehicle information acquired by the vehicle information acquisition unit 44, the preceding vehicle information acquired by the preceding vehicle information acquisition unit 431, and the peripheral information indicating the surrounding situation acquired by the peripheral information acquisition unit 43. It is determined whether or not the vehicle V1 may stop. For example, when the vehicle speed or acceleration of the vehicle V1 is equal to or less than the threshold value, or when information such as the front signal being a red light is acquired, it is determined that the vehicle V1 may stop. Or, for example, when the preceding vehicle V2 exists, the vehicle speed of the preceding vehicle V2 is equal to or less than the threshold value, or the inter-vehicle distance L1 between the vehicle V1 and the preceding vehicle V2 is equal to or less than the threshold value, or with the vehicle V1. When the relative speed with the preceding vehicle V2 is equal to or less than the threshold value, it is determined that the vehicle V1 may stop. In the present embodiment, the threshold value of the vehicle speed of the vehicle V1 or the preceding vehicle V2 is 20 km / h. In the present embodiment, the threshold value of the inter-vehicle distance L1 is 10 m. These numerical values may be set according to, for example, the width of the road on which the vehicle V1 is traveling and the regulated speed, or may be arbitrarily set by the user.

The bird's-eye view image generation unit 45 outputs the generated bird's-eye view image 100 to the display control unit 49. The bird's-eye view image generation unit 45 includes a viewpoint conversion processing unit 451, a cutting processing unit 452, and a composition processing unit 453.

The viewpoint conversion processing unit 451 performs viewpoint conversion processing on the peripheral video data acquired by the video data acquisition unit 42 so as to look down on the vehicle V1 from above. More specifically, the viewpoint conversion processing unit 451 generates an image that has undergone the viewpoint conversion processing based on the peripheral image data captured by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method of the viewpoint conversion process may be any known method and is not limited. The viewpoint conversion processing unit 451 outputs the peripheral video data that has undergone the viewpoint conversion processing to the cutout processing unit 452.

The cutout processing unit 452 performs cutout processing for cutting out an image in a predetermined range from the peripheral image data that has undergone the viewpoint conversion process. Which range is to be the cutout range is registered and stored in advance. The cutout processing unit 452 outputs the video data of the cutout processed video to the synthesis processing unit 453.

The compositing processing unit 453 performs a compositing process for synthesizing the cutout processed video data. The compositing processing unit 453 generates a bird's-eye view video 100 in which the vehicle icon 110 is displayed on the composited video.

The bird's-eye view image 100 will be described with reference to FIG. FIG. 4 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the first embodiment. The bird's-eye view image 100 displays a predetermined display range A from the vehicle V1. The bird's-eye view image 100 is located in the central portion surrounded by the front image 101, the rear image 102, the left image 103, the right image 104, the front image 101, the rear image 102, the left image 103, and the right image 104. Includes the vehicle icon 110. The vehicle icon 110 indicates the position and orientation of the vehicle V1. The vehicle icon 110 is arranged at the center of the bird's-eye view image 100 with the front-rear direction parallel to the front-rear direction. The bird's-eye view image 100 can recognize the relative positional relationship between the vehicle V1 and the doorway P.

In FIG. 4, a diagonal broken line indicating the boundary between the front image 101, the rear image 102, the left image 103, and the right image 104 is shown for explanation, but it is actually displayed on the display panel 31. The broken line may not be displayed in the bird's-eye view image 100. The same applies to other figures.

The display control unit 49 allows another vehicle V3 to enter and exit the side of the vehicle V1 based on the peripheral information indicating the surrounding situation acquired by the peripheral information acquisition unit 43 and the vehicle information acquired by the vehicle information acquisition unit 44. When the doorway P exists and it is predicted that the vehicle V1 will stop or decelerate, which may make it difficult for the other vehicle V3 to smoothly enter and exit the doorway P, the bird's-eye view image generation unit 45 determines. The generated bird's-eye view image 100 is displayed on the display panel 31.

The storage unit 50 stores data required for various processes in the bird's-eye view image generation device 40 and various processing results. The storage unit 50 is, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

Next, the flow of processing in the bird's-eye view image generation device 40 will be described with reference to FIG. FIG. 5 is a flowchart showing a processing flow in the bird's-eye view image generation device of the bird's-eye view image generation system according to the first embodiment.

The control unit 41 determines whether or not there is an entrance / exit unit P on the side (step S11). More specifically, when the control unit 41 determines that the entrance / exit unit P has been detected based on the peripheral information acquired from the peripheral information acquisition unit 43 (Yes in step S11), the control unit 41 proceeds to step S12. When it is determined that the entrance / exit unit P has not been detected based on the peripheral information acquired from the peripheral information acquisition unit 43 (No in step S11), the control unit 41 re-executes the process of step S11.

The control unit 41 determines whether or not the vehicle V1 may stop near the entrance / exit portion P (step S12). More specifically, when the control unit 41 determines that the vehicle V1 may stop, and determines that the predicted stop position of the vehicle V1 may block at least a part of the doorway P and stop. (Yes in step S12), the process proceeds to step S13. When the control unit 41 does not determine that the vehicle V1 may stop, or when it determines that the predicted stop position of the vehicle V1 does not block at least a part of the doorway P and stop ( No) in step S12, the process of step S11 is executed again.

The control unit 41 generates a bird's-eye view image 100 (step S13). More specifically, the control unit 41 generates the bird's-eye view image 100 obtained by the bird's-eye view image generation unit 45, which has been subjected to viewpoint conversion so as to look down on the vehicle V1 from the peripheral image acquired by the image data acquisition unit 42. The control unit 41 proceeds to step S14.

The control unit 41 displays the bird's-eye view image 100 (step S14). More specifically, the control unit 41 displays the bird's-eye view image 100 on the display panel 31 by the display control unit 49. The control unit 41 proceeds to step S15.

The control unit 41 determines whether or not to end the bird's-eye view video display (step S15). More specifically, the control unit 41 determines whether or not to end the display of the bird's-eye view image 100 based on the presence or absence of the end trigger. The end trigger is, for example, a case where the vehicle V1 is no longer likely to stop, or a case where the vehicle V1 passes by the side of the doorway P. If there is an end trigger, the control unit 41 determines that the display of the bird's-eye view image 100 is finished (Yes in step S15), and ends the process. If there is no end trigger, the control unit 41 determines that the display of the bird's-eye view image 100 is not terminated (No in step S15), and executes the process of step S13 again.

In this way, the bird's-eye view image generation system 1 determines that the doorway P exists on the side of the vehicle V1 and that the vehicle V1 may block at least a part of the doorway P and stop. In this case, a video signal for displaying the bird's-eye view video 100 on the display panel 31 is output. The display panel 31 displays the bird's-eye view image 100 together with the navigation screen, for example, based on the image signal output from the bird's-eye view image generation system 1.

As described above, in the present embodiment, when it is determined that the doorway P exists on the side of the vehicle V1 and the vehicle V1 may block at least a part of the doorway P and stop. The bird's-eye view image 100 is displayed on the display panel 31. Thereby, according to the present embodiment, the driver can easily recognize that the entrance / exit portion P exists on the side of the vehicle V1. Further, according to the present embodiment, the bird's-eye view image 100 allows the driver to easily confirm the relative positional relationship between the vehicle V1 and the entrance / exit portion P. In this way, the present embodiment can display the periphery of the vehicle in a recognizable manner. The present embodiment can assist the driver in determining an appropriate stop position of the vehicle V1 by displaying the bird's-eye view image 100.

Unlike the present embodiment, there is a doorway P on the side of the vehicle V1, and even if the vehicle V1 may block at least a part of the doorway P and stop, the bird's-eye view image 100 is not displayed. , It may be difficult for the driver to recognize that the doorway portion P exists on the side of the vehicle V1. As a result, as shown in FIG. 3, the vehicle V1 may stop at a stop position that blocks the doorway P. In this case, the other vehicle V3 trying to enter the entrance / exit P from the oncoming lane across the front of the vehicle V1 cannot proceed until the vehicle V1 moves forward, and the following vehicle of the other vehicle V3 stays in the oncoming lane. There is a risk that smooth passage will be hindered.

On the other hand, according to the present embodiment, since the bird's-eye view image 100 as shown in FIG. 4 is displayed, the driver can easily confirm that when the vehicle V1 advances further, the entrance / exit portion P is blocked and stopped. be able to. The driver can determine that it is better to stop the vehicle V1 at the current position where the doorway P is not blocked. As a result, according to the present embodiment, it is possible to allow another vehicle V3 to pass smoothly. Further, according to the present embodiment, it is possible to prevent the following vehicle of the other vehicle V3 from staying.

[Second Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 6 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the second embodiment. The basic configuration of the bird's-eye view image generation system 1 is the same as that of the bird's-eye view image generation system 1 of the first embodiment. In the following description, the same components as those of the bird's-eye view video generation system 1 are designated by the same reference numerals or corresponding reference numerals, and detailed description thereof will be omitted. In the bird's-eye view image generation system 1 of the present embodiment, the processing in the bird's-eye view image generation unit 45 is different from that of the bird's-eye view image generation unit 45 of the first embodiment.

The bird's-eye view image generation unit 45 generates a bird's-eye view image 100A in which the display range of the peripheral image displaying the entrance / exit P side is widened. In the present embodiment, the bird's-eye view image generation unit 45 widens the display range of the left side image 103A displaying the entrance / exit P side, the left side of the display range of the front image 101A, and the left side of the display range of the rear image 102A. Generate a bird's-eye view image 100A. The right side image 104A is the same as the display range of the normal bird's-eye view image 100.

More specifically, even if the control unit 41 widens the imaging range of the camera that captures the direction of the doorway P, thereby generating a bird's-eye view image 100A in which the display range of the peripheral image displaying the doorway P side is widened. Good. In this case, the control unit 41 widens the third shooting range A3 of the left side camera 13, the first shooting range A1 of the front camera 11, and the second shooting range A2 of the rear camera 12 to the left to take an image. Then, the control unit 41 performs the viewpoint conversion processing of the peripheral video data captured by each camera by the viewpoint conversion processing unit 451. Then, the control unit 41 sets the left side image 103A, the front image 101A, and the rear image 102A from the peripheral image data that has been subjected to the viewpoint conversion process by the cutout processing unit 452 according to the shooting range in each direction. A cutting process is performed to cut out the right side image 104A in a normal cutting range in a wide cutting range. Then, the control unit 41 performs a synthesis process for synthesizing the video data that has been cut out by the synthesis processing unit 453. In this way, the bird's-eye view image 100A having a wide display range of the peripheral image displaying the entrance / exit P side is generated.

Alternatively, a bird's-eye view image 100A may be generated in which the display range of the peripheral image displaying the entrance / exit P side is widened by cutting out the peripheral image obtained by capturing the direction of the entrance / exit P in a wide cutout range. In this case, the control unit 41 cuts out the left side image 103A, the front image 101A, and the rear image 102A from the peripheral image data that has been subjected to the viewpoint conversion process by the cutout processing unit 452 in a wider cutout range than usual on the right side. A cutting process is performed to cut out the image 104A within a normal cutting range. Then, the control unit 41 performs a synthesis process for synthesizing the video data that has been cut out by the synthesis processing unit 453. In this way, the bird's-eye view image 100A having a wide display range of the peripheral image displaying the entrance / exit P side is generated.

The bird's-eye view image 100A will be described with reference to FIG. The bird's-eye view image 100A is widely generated on the left side with respect to the normal bird's-eye view image 100 indicated by the alternate long and short dash line. In the bird's-eye view image 100A, for example, a range of about 2 m from the vehicle V1 is displayed on the front, rear, and right sides, and a range of about 5 m from the vehicle V1 is displayed on the left side. The bird's-eye view image 100A can recognize the relative positional relationship between the vehicle V1 and the doorway P.

As described above, the present embodiment can generate a bird's-eye view image 100A in which the display range of the peripheral image displaying the entrance / exit P side is widened. According to the present embodiment, by displaying such a bird's-eye view image 100A, the entrance / exit portion P can be easily recognized. In this embodiment, the area around the vehicle can be displayed in a recognizable manner.

[Third Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the third embodiment. In the bird's-eye view image generation system 1 of the present embodiment, the processing in the bird's-eye view image generation unit 45 is different from that of the bird's-eye view image generation unit 45 of the first embodiment.

The bird's-eye view image generation unit 45 generates a bird's-eye view image 100B in which the display range A is shifted so that the entrance / exit P side is displayed relatively widely. In the present embodiment, the bird's-eye view image generation unit 45 generates a bird's-eye view image 100B in which the display range A is shifted to the left front so as to relatively widen the left side image 103B and the front image 101B that display the entrance / exit portion P. .. The rear image 102B and the right side image 104B are narrower than the normal bird's-eye view image 100.

The control unit 41 sets the rear image 102B and the right side in a cutout range in which the left side image 103B and the front image 101B are wider to the left front than usual from the peripheral image data that has been subjected to the viewpoint conversion process by the cutout processing unit 452. A cutting process is performed to cut out the square image 104B in a narrower cutting range than usual. Then, the control unit 41 performs a synthesis process for synthesizing the video data that has been cut out by the synthesis processing unit 453. In this way, the bird's-eye view image 100B in which the display range A is shifted so that the entrance / exit P side is displayed relatively widely is generated.

The bird's-eye view image 100B will be described with reference to FIG. 7. The bird's-eye view image 100B is generated by shifting the display range A of the normal bird's-eye view image 100 shown by the alternate long and short dash line in FIG. 7 to the left front. The bird's-eye view image 100B can recognize the relative positional relationship between the vehicle V1 and the entrance / exit portion P.

As described above, the present embodiment can generate the bird's-eye view image 100B in which the display range A is shifted to the entrance / exit portion P side. According to the present embodiment, by displaying such a bird's-eye view image 100B, the entrance / exit portion P can be easily recognized. In this embodiment, the area around the vehicle can be displayed in a recognizable manner.

[Fourth Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 8 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the fourth embodiment. In the bird's-eye view image generation system 1 of the present embodiment, the processing in the bird's-eye view image generation unit 45 is different from that of the bird's-eye view image generation unit 45 of the first embodiment.

The bird's-eye view image generation unit 45 generates a bird's-eye view image 100C in which the display range A is changed so that the entrance / exit P side is enlarged and displayed. In the present embodiment, the bird's-eye view image generation unit 45 generates a bird's-eye view image 100C in which the display range A is changed so that the front side of the left side image 103C displaying the entrance / exit portion P and the left side of the front image 101C are enlarged. To do.

The control unit 41 performs cutting processing by the cutting processing unit 452 to cut out the peripheral image data subjected to the viewpoint conversion processing in a cutting range that enlarges the left front. Then, the control unit 41 performs a synthesis process for synthesizing the video data that has been cut out by the synthesis processing unit 453.

The bird's-eye view image 100C will be described with reference to FIG. The bird's-eye view image 100C is generated by enlarging the front left. The bird's-eye view image 100C can recognize the relative positional relationship between the vehicle V1 and the entrance / exit portion P.

As described above, the present embodiment can generate a bird's-eye view image 100C in which the entrance / exit P side is enlarged. According to the present embodiment, by displaying such a bird's-eye view image 100C, the entrance / exit portion P can be easily recognized. In this embodiment, the area around the vehicle can be displayed in a recognizable manner.

[Fifth Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 9 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the fifth embodiment. In the bird's-eye view image generation system 1 of the present embodiment, the processing in the bird's-eye view image generation unit 45 is different from that of the bird's-eye view image generation unit 45 of the first embodiment.

The bird's-eye view image generation unit 45 generates a bird's-eye view image 100D in which the entrance / exit P side is highlighted. In the present embodiment, the bird's-eye view image generation unit 45 generates the bird's-eye view image 100D by synthesizing the entrance / exit icon 105D that highlights the entrance / exit unit P with the synthesized bird's-eye view image by the composition processing unit 453.

The bird's-eye view image 100D will be described with reference to FIG. The bird's-eye view image 100D includes an entrance / exit icon 105D that highlights the width of the entrance / exit portion P. The doorway icon 105D is a band-shaped icon corresponding to the width of the doorway portion P.

As described above, the present embodiment can generate a bird's-eye view image 100D in which the entrance / exit P side is highlighted. According to the present embodiment, by displaying such a bird's-eye view image 100D, the entrance / exit portion P can be easily recognized. In this embodiment, the area around the vehicle can be displayed in a recognizable manner.

Although the bird's-eye view image generation system 1 according to the present invention has been described so far, it may be implemented in various different forms other than the above-described embodiment.

Each component of the illustrated bird's-eye view image generation system 1 is a functional concept and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or part of each device is functionally or physically dispersed or integrated in an arbitrary unit according to the processing load and usage status of each device. You may.

The configuration of the bird's-eye view video generation system 1 is realized, for example, by a program loaded in a memory as software. In the above embodiment, it has been described as a functional block realized by linking these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The components described above include those that can be easily assumed by those skilled in the art and those that are substantially the same. Further, the configurations described above can be appropriately combined. Further, various omissions, substitutions or changes of the configuration can be made without departing from the gist of the present invention.

Peripheral information acquisition unit 43 may execute image processing on the peripheral image acquired by the video data acquisition unit 42 as peripheral information of the vehicle V1 to detect that the traffic light in front of the vehicle V1 is red. Peripheral information acquisition unit 43 may detect that the brake lamp of the preceding vehicle V2 is lit by image processing as peripheral information of the vehicle V1. Peripheral information acquisition unit 43 may detect that the inter-vehicle distance L1 between the vehicle V1 and the preceding vehicle V2 detected by a sensor (not shown) is equal to or less than the threshold value as the peripheral information of the vehicle V1. Then, the bird's-eye view image generation unit 45 may determine that the vehicle V1 may stop based on this information.

In the above embodiment, the other vehicle V3 has been described as a vehicle that is about to enter the doorway P by turning right, but the present invention is not limited to this. The other vehicle V3 may be a vehicle that is about to exit by turning right from the doorway P.

The bird's-eye view image generation unit 45 has been described as generating the bird's-eye view image 100 when a predetermined condition is satisfied, but the present invention is not limited to this. The bird's-eye view image generation unit 45 may generate the bird's-eye view image 100 while the bird's-eye view image generation system 1 is activated. In this case, the display control unit 49 predicts that the vehicle V1 has an entrance / exit P that allows the other vehicle V3 to enter / exit, and that the vehicle V1 is stopped or decelerated, so that the other vehicle V3 enters / exits P. When it is determined that smooth entry / exit may be difficult, the bird's-eye view image 100 may be controlled to be displayed.

1 Bird's-eye view image generation system 11 Front camera 12 Rear camera 13 Left-side camera 14 Right-side camera 40 Bird's-eye view image generation device 41 Control unit 42 Video data acquisition unit 43 Peripheral information acquisition unit 44 Vehicle information acquisition unit 45 Bird's-eye view image generation unit 49 Display control Department

Claims (11)

A video data acquisition unit that acquires peripheral video data taken by the shooting unit that shoots the surroundings of the vehicle,
Peripheral information acquisition unit that acquires peripheral information indicating the situation around the vehicle,
A vehicle information acquisition unit that acquires vehicle information including the vehicle speed of the vehicle,
The peripheral image acquired by the video data acquisition unit is subjected to viewpoint conversion processing, and a plurality of peripheral images subjected to the viewpoint conversion processing are combined to generate a bird's-eye view image of a display range for displaying the periphery of the vehicle. Generator and
A display control unit that displays the bird's-eye view image generated by the bird's-eye view image generation unit on the display unit,
With
The display control unit has an entrance / exit unit on the side of the vehicle on which the vehicle can enter and exit based on the peripheral information acquired by the peripheral information acquisition unit and the vehicle information acquired by the vehicle information acquisition unit. When it is determined that the vehicle may make it difficult for the vehicle to smoothly enter and exit the doorway, a bird's-eye view image is displayed on the display.
A bird's-eye view video generator characterized by this. The display control unit has the entrance / exit on the side of the vehicle based on the peripheral information acquired by the peripheral information acquisition unit and the vehicle information acquired by the vehicle information acquisition unit, and the vehicle is The bird's-eye view image generation device according to claim 1, wherein a bird's-eye view image is displayed on the display unit when it is determined that at least a part of the entrance / exit portion may be blocked and stopped. The peripheral information acquisition unit includes at least one of the vehicle speed of the preceding vehicle traveling in front of the vehicle, the distance between the vehicle and the preceding vehicle, and the relative speed between the vehicle and the preceding vehicle. Has a preceding vehicle information acquisition department, which acquires information,
The display control unit is based on the preceding vehicle information acquired by the preceding vehicle information acquisition unit, when the vehicle speed of the preceding vehicle is equal to or less than the threshold value, or when the distance between the vehicle and the preceding vehicle is equal to or less than the threshold value. The bird's-eye view image generation device according to claim 1 or 2, wherein a bird's-eye view image is generated when there is a case or when the relative speed between the vehicle and the preceding vehicle is equal to or less than a threshold value. Based on the preceding vehicle information acquired by the preceding vehicle information acquisition unit and the existence of the doorway, the display control unit determines that the vehicle may block at least a part of the doorway and stop. The bird's-eye view image generation device according to claim 3, wherein the bird's-eye view image is displayed on the display unit. The method according to any one of claims 1 to 4, wherein the bird's-eye view image generation unit generates the bird's-eye view image in which the display range of the peripheral image displaying the entrance / exit portion side is widened. A bird's-eye view video generator. Any one of claims 1 to 4, wherein the bird's-eye view image generation unit generates the bird's-eye view image in which the display range is shifted so that the entrance / exit side is displayed relatively widely. The bird's-eye view image generator described in. The bird's-eye view image generation unit is characterized in that it generates the bird's-eye view image in which the display range is changed so as to enlarge and display the entrance / exit portion side, according to any one of claims 1 to 4. The above-mentioned bird's-eye view image generator. The bird's-eye view image generation device according to any one of claims 1 to 7, wherein the bird's-eye view image generation unit generates the bird's-eye view image in which the entrance / exit portion is highlighted. The bird's-eye view image generator according to any one of claims 1 to 8.
A bird's-eye view image generation system characterized in that it includes a photographing unit that photographs the surroundings of the vehicle and supplies peripheral image data to the image data acquisition unit. A video data acquisition step to acquire peripheral video data taken by the shooting unit that shoots the surroundings of the vehicle, and
Peripheral information acquisition step to acquire peripheral information indicating the situation around the vehicle, and
A vehicle information acquisition step for acquiring vehicle information including the vehicle speed of the vehicle, and
The peripheral image acquired in the video data acquisition step is subjected to viewpoint conversion processing, and a plurality of peripheral images subjected to the viewpoint conversion processing are combined to generate a bird's-eye view image of a display range for displaying the periphery of the vehicle. Generation step and
A display control step for displaying the bird's-eye view image generated in the bird's-eye view image generation step on the display unit, and
Including
In the display control step, there is an entrance / exit portion on the side of the vehicle on which the vehicle can enter and exit based on the peripheral information acquired in the peripheral information acquisition step and the vehicle information acquired in the vehicle information acquisition step. When it is determined that the vehicle may make it difficult for the vehicle to smoothly enter and exit the entrance / exit of another vehicle, a bird's-eye view image is displayed on the display.
How to generate a bird's-eye view video. A video data acquisition step to acquire peripheral video data taken by the shooting unit that shoots the surroundings of the vehicle, and
Peripheral information acquisition step to acquire peripheral information indicating the situation around the vehicle, and
A vehicle information acquisition step for acquiring vehicle information including the vehicle speed of the vehicle, and
The peripheral image acquired in the video data acquisition step is subjected to viewpoint conversion processing, and a plurality of peripheral images subjected to the viewpoint conversion processing are combined to generate a bird's-eye view image of a display range for displaying the periphery of the vehicle. Generation step and
A display control step for displaying the bird's-eye view image generated in the bird's-eye view image generation step on the display unit, and
Including
In the display control step, there is an entrance / exit portion on the side of the vehicle on which the vehicle can enter and exit based on the peripheral information acquired in the peripheral information acquisition step and the vehicle information acquired in the vehicle information acquisition step. When it is determined that the vehicle may make it difficult for the vehicle to smoothly enter and exit the entrance / exit of another vehicle, a bird's-eye view image is displayed on the display.
A program that allows a computer that operates as a bird's-eye view video generator to execute this.

Images