JP2020182220A - Bird's-eye video creation device, bird's-eye video creation method, and program - Google Patents

Abstract

To make it possible to appropriately check an obstacle in the periphery of a vehicle.SOLUTION: A bird's-eye video creation device comprises: a video acquisition unit 42 that acquires a peripheral video obtained by photographing the periphery of a vehicle; an obstacle information acquisition unit 43 that acquires obstacle information on an obstacle detected in the periphery of the vehicle, the information including the distance from the vehicle to the obstacle; a bird's-eye video creation unit 45 that creates a bird's-eye video obtained by performing a viewpoint conversion processing on the peripheral video acquired by the video acquisition unit 42 so as to look down the vehicle from above; a superimposition video creation unit 46 that, based on the obstacle information acquired by the obstacle information acquisition unit 43, when the distance to the detected obstacle is equal to or more than a predetermined threshold, creates a bird's-eye video obtained by extending the information indicating the obstacle to be superimposed on the outside of a center part surrounded by the bird's-eye video, and when the distance to the detected obstacle is less than the predetermined threshold, creates a bird's-eye video obtained by superimposing the information indicating the obstacle on the center part; and a display control unit 47 that displays the bird's-eye video created by the superimposition video creation unit 46 on a display unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a bird's-eye view image generator, a bird's-eye view image generation method, and a program.

A technique related to a vehicle peripheral display device that displays a bird's-eye view image of a vehicle together with a vehicle image is known (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2015-0764645

In order to make it easier to check the surroundings of the vehicle, there is a technology for superimposing and displaying obstacle information of obstacles detected around the vehicle on a bird's-eye view image. However, if the obstacle information is superimposed on the bird's-eye view image, the visibility of the obstacle reflected in the bird's-eye view image may be impaired.

The present invention has been made in view of the above, and an object of the present invention is to make it possible to appropriately confirm an obstacle around a vehicle.

In order to solve the above-mentioned problems and achieve the purpose, a video acquisition unit that acquires a peripheral image of the surroundings of the vehicle and an obstacle detected in the vicinity of the vehicle, the distance from the vehicle to the obstacle. An obstacle information acquisition unit that acquires obstacle information including the above, a bird's-eye view image generation unit that generates a bird's-eye view image that has been subjected to viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit. If the distance to the detected obstacle is equal to or greater than a predetermined threshold value based on the obstacle information acquired by the obstacle information acquisition unit, the information indicating the obstacle is provided outside the central portion surrounded by the bird's-eye view image. If the distance to the detected obstacle is less than a predetermined threshold value, the bird's-eye view image that is stretched and superimposed on the obstacle is generated, and the information indicating the obstacle is superimposed on the central portion to generate the superimposed image. It is characterized by having a unit and a display control unit that displays a bird's-eye view image generated by the superimposed image generation unit on the display unit.

In another form of the bird's-eye view image generation device according to the present invention, the distance between the image acquisition unit that acquires the peripheral image of the surroundings of the vehicle and the obstacle detected in the vicinity of the vehicle from the vehicle to the obstacle. An obstacle information acquisition unit that acquires obstacle information including the above, a bird's-eye view image generation unit that generates a bird's-eye view image that has been subjected to viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit. Based on the obstacle information acquired by the obstacle information acquisition unit, if the detected distance to the obstacle is equal to or greater than a predetermined threshold value, a bird's-eye view image in which the information indicating the obstacle is superimposed on the bird's-eye view image is generated. If the detected distance to the obstacle is less than a predetermined threshold value, the superimposed image generation unit that generates the bird's-eye view image in which the information indicating the obstacle is superimposed on the central portion surrounded by the bird's-eye view image, and the superimposed image generation unit. It is characterized by having a display control unit for displaying a bird's-eye view image generated by the unit on the display unit.

The bird's-eye view image generation method according to the present invention includes an image acquisition step of acquiring a peripheral image of the periphery of the vehicle and an obstacle detected in the vicinity of the vehicle, including the distance from the vehicle to the obstacle. An obstacle information acquisition step for acquiring information, a bird's-eye view image generation step for generating a bird's-eye view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired in the image acquisition step, and the obstacle information. If the distance to the detected obstacle is equal to or greater than a predetermined threshold value based on the obstacle information acquired in the acquisition step, the information indicating the obstacle is extended to the outside of the central portion surrounded by the bird's-eye view image. If the superimposed bird's-eye view image is generated and the detected distance to the obstacle is less than a predetermined threshold value, the superimposed image generation step of generating the bird's-eye view image in which the information indicating the obstacle is superimposed on the central portion and the above-mentioned The bird's-eye view image generation device executes a display control step of displaying the bird's-eye view image generated in the superimposed image generation step on the display unit.

In another form of the bird's-eye view image generation method according to the present invention, there is an image acquisition step of acquiring a peripheral image of the periphery of the vehicle and a distance of an obstacle detected in the vicinity of the vehicle from the vehicle to the obstacle. An obstacle information acquisition step for acquiring obstacle information including the above, a bird's-eye view image generation step for generating a bird's-eye view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired in the image acquisition step. Based on the obstacle information acquired in the obstacle information acquisition step, if the detected distance to the obstacle is equal to or greater than a predetermined threshold value, a bird's-eye view image in which the information indicating the obstacle is superimposed on the bird's-eye view image is generated. If the detected distance to the obstacle is less than a predetermined threshold value, the superimposed image generation step of generating the bird's-eye view image in which the information indicating the obstacle is superimposed on the central portion surrounded by the bird's-eye view image, and the superimposed image generation The bird's-eye view image generation device executes a display control step of displaying the bird's-eye view image generated in the step on the display unit.

According to the present invention, there is an effect that obstacles around the vehicle can be appropriately confirmed.

FIG. 1 is a block diagram showing a configuration example of a bird's-eye view video generation system according to the first embodiment. FIG. 2 is a diagram showing a bird's-eye view image and an obstacle notification icon generated by the bird's-eye view image generation system according to the first embodiment. FIG. 3 is a flowchart showing a processing flow in the bird's-eye view image generation device of the bird's-eye view image generation system according to the first embodiment. FIG. 4 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the first embodiment. FIG. 5 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the first embodiment. FIG. 6 is a flowchart showing a processing flow in the bird's-eye view image generation device of the bird's-eye view image generation system according to the second embodiment. FIG. 7 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the second embodiment. FIG. 8 is a flowchart showing a processing flow in the bird's-eye view image generation device of the bird's-eye view image generation system according to the third embodiment. FIG. 9 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the fifth embodiment. FIG. 10 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the fifth embodiment. FIG. 11 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the fifth embodiment. FIG. 12 is a graph showing an example of the relationship between the distance between arcs and the distance to an obstacle in the obstacle notification icon. FIG. 13 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the sixth embodiment. FIG. 14 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the sixth embodiment. FIG. 15 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the sixth embodiment.

The bird's-eye view image generation device 40, the bird's-eye view image generation system 1, the bird's-eye view image generation method, and the embodiment of the program according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the following embodiments.
[First Embodiment]
FIG. 1 is a block diagram showing a configuration example of a bird's-eye view video generation system according to the first embodiment. The bird's-eye view image generation system 1 generates a bird's-eye view image 100 (see FIG. 2) of the vehicle. The bird's-eye view image generation device 40 and the bird's-eye view image generation system 1 are mounted on the vehicle. The bird's-eye view image generation device 40 and the bird's-eye view image generation system 1 may be portable devices that can be used in the vehicle, in addition to those mounted on the vehicle.

The bird's-eye view image generation system 1 will be described with reference to FIG. The bird's-eye view image generation system 1 includes a front camera (shooting unit) 11, a rear camera (shooting unit) 12, a left side camera (shooting unit) 13, a right side camera (shooting unit) 14, and a front left sensor (obstacle). Object detection unit) 21, front center sensor (obstacle detection unit) 22, front right sensor (obstacle detection unit) 23, rear left sensor (obstacle detection unit) 24, rear center sensor (obstacle detection unit) Section) 25, a rear right sensor (obstacle detection section) 26, a display panel 31, and a bird's-eye view image generation device 40.

The front camera 11 is arranged in front of the vehicle and photographs the surroundings centered on the front of the vehicle. The front camera 11 outputs the captured image to the image acquisition unit 42 of the bird's-eye view image generation device 40.

The rear camera 12 is arranged behind the vehicle and photographs the surroundings centered on the rear of the vehicle. The rear camera 12 outputs the captured image to the image acquisition unit 42 of the bird's-eye view image generation device 40.

The left-side camera 13 is arranged on the left side of the vehicle and photographs the surroundings centered on the left side of the vehicle. The left-side camera 13 outputs the captured image to the image acquisition unit 42 of the bird's-eye view image generation device 40.

The right-side camera 14 is arranged on the right side of the vehicle and photographs the surroundings centered on the right side of the vehicle. The right-side camera 14 outputs the captured image to the image acquisition unit 42 of the bird's-eye view image generator 40.

The front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14 capture all directions of the vehicle.

The front left sensor 21 is arranged on the front left side of the vehicle and detects an obstacle on the front left side of the vehicle. The front left sensor 21 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be composed of a combination thereof. The front left sensor 21 detects an object having a height above the ground that may come into contact with the vehicle. The front left sensor 21 detects an obstacle at a distance of, for example, about 5 m from the vehicle. The front left sensor 21 detects an obstacle in a range of, for example, about 40 ° centered on the central portion of the sensor in vertical viewing. The detection range of the front left sensor 21 may overlap with a part of the detection range of the front center sensor 22. The front left sensor 21 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the bird's-eye view image generation device 40. Examples of obstacle information include the presence / absence of an obstacle in the detection range of the front left sensor 21, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

The front center sensor 22 is arranged in the front center of the vehicle and detects an obstacle in the front center of the vehicle. The front center sensor 22 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be composed of a combination thereof. The front center sensor 22 detects an object having a height above the ground that may come into contact with the vehicle. The front center sensor 22 detects an obstacle at a distance of, for example, about 5 m from the vehicle. The front center sensor 22 detects an obstacle in a range of, for example, about 40 ° centered on the center of the sensor in vertical viewing. The detection range of the front center sensor 22 may overlap with a part of the detection range of the front left sensor 21 and the front right sensor 23. The front central sensor 22 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the bird's-eye view image generation device 40. Examples of obstacle information include the presence / absence of an obstacle in the detection range of the front central sensor 22, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

The front right sensor 23 is arranged on the front right side of the vehicle and detects an obstacle on the front right side of the vehicle. The front right sensor 23 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be composed of a combination thereof. The front right sensor 23 detects an object having a height from the ground that may come into contact with the vehicle. The front right sensor 23 detects an obstacle at a distance of, for example, about 5 m from the vehicle. The front right sensor 23 detects an obstacle in a range of, for example, about 40 ° centered on the central portion of the sensor in vertical viewing. The detection range of the front right sensor 23 may overlap with a part of the detection range of the front center sensor 22. The front right sensor 23 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the bird's-eye view image generation device 40. Examples of obstacle information include the presence / absence of an obstacle in the detection range of the front right sensor 23, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

The front left sensor 21, the front center sensor 22, and the front right sensor 23 detect obstacles in front of the vehicle.

The rear left sensor 24 is arranged on the rear left side of the vehicle and detects an obstacle on the rear left side of the vehicle. The rear left sensor 24 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be composed of a combination thereof. The rear left sensor 24 detects an object having a height above the ground that may come into contact with the vehicle. The rear left sensor 24 detects an obstacle at a distance of, for example, about 5 m from the vehicle. The rear left sensor 24 detects an obstacle in a range of, for example, about 40 ° centered on the central portion of the sensor in vertical viewing. The detection range of the rear left sensor 24 may overlap with a part of the detection range of the rear center sensor 25. The rear left sensor 24 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the bird's-eye view image generation device 40. Examples of obstacle information include the presence / absence of an obstacle in the detection range of the rear left sensor 24, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

The rear center sensor 25 is arranged in the rear center of the vehicle and detects an obstacle in the rear center of the vehicle. The rear center sensor 25 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be composed of a combination thereof. The rear center sensor 25 detects an object having a height above the ground that may come into contact with the vehicle. The rear center sensor 25 detects an obstacle at a distance of, for example, about 5 m from the vehicle. The rear center sensor 25 detects an obstacle in a range of, for example, about 40 ° centered on the center of the sensor in vertical viewing. The detection range of the rear center sensor 25 may overlap with a part of the detection range of the rear left sensor 24 and the rear right sensor 26. The rear central sensor 25 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the bird's-eye view image generation device 40. Examples of obstacle information include the presence / absence of an obstacle in the detection range of the rear central sensor 25, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

The rear right sensor 26 is arranged on the rear right side of the vehicle and detects an obstacle on the rear right side of the vehicle. The rear right sensor 26 is, for example, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or the like, and may be composed of a combination thereof. The rear right sensor 26 detects an object having a height above the ground that may come into contact with the vehicle. The rear right sensor 26 detects an obstacle at a distance of, for example, about 5 m from the vehicle. The rear right sensor 26 detects an obstacle in a range of, for example, about 40 ° centered on the central portion of the sensor in vertical viewing. The detection range of the rear right sensor 26 may overlap with a part of the detection range of the rear center sensor 25. The rear right sensor 26 outputs the detected obstacle information of the obstacle to the obstacle information acquisition unit 43 of the bird's-eye view image generation device 40. Examples of obstacle information include the presence / absence of an obstacle in the detection range of the rear right sensor 26, the distance to the obstacle, the existence range of the obstacle in the horizontal direction, and the like.

The rear left sensor 24, the rear center sensor 25, and the rear right sensor 26 detect obstacles behind the vehicle.

The display panel 31 is, for example, a display including a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminence) display. The display panel 31 displays the bird's-eye view image 100 based on the image signal output from the bird's-eye view image generation device 40 of the bird's-eye view image generation system 1. The display panel 31 may be dedicated to the bird's-eye view image generation system 1, or may be used jointly with other systems including, for example, a navigation system. The display panel 31 is arranged at a position that is easily visible to the driver.

The bird's-eye view image generation device 40 has a control unit 41 and a storage unit 49.

The control unit 41 is, for example, an arithmetic processing unit composed of a CPU (Central Processing Unit) or the like. The control unit 41 loads the program stored in the storage unit 49 into the memory and executes the instruction included in the program. The control unit 41 includes an image acquisition unit 42, an obstacle information acquisition unit 43, a vehicle information acquisition unit 44, a bird's-eye view image generation unit 45, a superimposed image generation unit 46, and a display control unit 47.

The image acquisition unit 42 acquires a peripheral image of the surroundings of the vehicle. More specifically, the image acquisition unit 42 acquires the images output by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The image acquisition unit 42 outputs the acquired image to the bird's-eye view image generation unit 45.

The obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in the vicinity of the vehicle. In the present embodiment, the obstacle information acquisition unit 43 acquires obstacle information including the distance to the detected obstacle. More specifically, the obstacle information acquisition unit 43 acquires the obstacle information output by the front left sensor 21, the front center sensor 22, the front right sensor 23, the rear left sensor 24, the rear center sensor 25, and the rear right sensor 26. To do. The obstacle information acquisition unit 43 outputs the acquired obstacle information to the superimposed video generation unit 46.

The vehicle information acquisition unit 44 acquires vehicle information that is a trigger for displaying a bird's-eye view image, such as vehicle gear operation information, from CAN (Control Area Network), various sensors that sense the state of the vehicle, and the like. The vehicle information acquisition unit 44 outputs the acquired vehicle information to the bird's-eye view image generation unit 45.

The bird's-eye view image generation unit 45 generates a bird's-eye view image 100 that has been subjected to viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit 42. More specifically, the bird's-eye view image generation unit 45 generates the bird's-eye view image 100 based on the images taken by the front camera 11, the rear camera 12, the left side camera 13, and the right side camera 14. The method for generating the bird's-eye view image 100 may be any known method and is not limited. The bird's-eye view image generation unit 45 outputs the generated bird's-eye view image 100 to the display control unit 47.

The bird's-eye view image 100 will be described with reference to FIG. FIG. 2 is a diagram showing a bird's-eye view image and an obstacle notification icon generated by the bird's-eye view image generation system according to the first embodiment. The bird's-eye view image 100 has a vertically long rectangular shape. The bird's-eye view image 100 is located in the central portion surrounded by the front image 101, the rear image 102, the left image 103, the right image 104, the front image 101, the rear image 102, the left image 103, and the right image 104. Includes the central image 105 to be The front image 101, the rear image 102, the left image 103, the right image 104, and the center image 105 may be separated by a frame-shaped boundary line.

The central image 105 is generated in a vertically long rectangular shape. In the central image 105, the boundary between the front image 101, the rear image 102, the left image 103, and the right image 104 is separated by a line. The central image 105 shows the position of the vehicle in the bird's-eye view image 100.

In FIG. 2, an oblique broken line indicating the boundary between the front image 101, the rear image 102, the left image 103, and the right image 104 is shown for explanation, but is actually displayed on the display panel 31. The broken line is not displayed in the bird's-eye view image 100. The same applies to other figures.

The superimposed image generation unit 46 generates a bird's-eye view image 100 in which information indicating an obstacle is superimposed on the central image 105 of the bird's-eye view image 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. In the present embodiment, the superimposed image generation unit 46 superimposes information indicating the direction in which the obstacle is detected on the central image 105 of the bird's-eye view image 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. Generate 100. In the present embodiment, the superimposed image generation unit 46 indicates information indicating the direction in which the obstacle is detected by the obstacle notification icon (information indicating the obstacle) 110. The obstacle notification icon 110 schematically shows the detection direction in the horizontal direction of each sensor that detects an obstacle, and the arrangement direction of the plurality of arcs is the detection direction starting from the mounting position of the sensor or the vehicle. It is a radial arrangement centered. The width of the arc constituting the obstacle notification icon 110 may indicate the detection range of each sensor that detects an obstacle, or may be a fixed width corresponding to the detection direction regardless of the detection range of each sensor. ..

The obstacle notification icon 110 is an icon for notifying an obstacle. The obstacle notification icon 110 indicates the distance and direction to the obstacle. The obstacle notification icon 110 includes a front left icon (information indicating an obstacle) 111, a front center icon (information indicating an obstacle) 112, a front right icon (information indicating an obstacle) 113, and a rear left icon (information indicating an obstacle). It includes (indicating information) 114, a rear center icon (information indicating an obstacle) 115, and a rear right icon (information indicating an obstacle) 116.

The front left icon 111 is an icon that notifies an obstacle on the front left side of the vehicle. More specifically, the front left icon 111 is an icon that notifies that an obstacle has been detected by the front left sensor 21. The front left icon 111 is superimposed on the upper left of the central image 105 of the bird's-eye view image 100 in FIG.

In the present embodiment, the front left icon 111 is composed of a triple arc-shaped curve. The arc-shaped curve bulges toward the outside of the bird's-eye view image 100. The triple arc-shaped curve has a smaller radius from the outside to the center of the bird's-eye view image 100. The triple arc-shaped curve becomes shorter from the outside to the center of the bird's-eye view image 100.

The front left icon 111 may change color depending on the distance to the obstacle. For example, the front left icon 111 is displayed in green when the distance to the obstacle is equal to or greater than the first predetermined distance. The front left icon 111 is displayed in yellow when the distance to the obstacle is less than the first predetermined distance and greater than or equal to the second predetermined distance smaller than the first predetermined distance. The front left icon 111 is displayed in red when the distance to the obstacle is less than the second predetermined distance.

The front left icon 111 may change the number of arcuate curves according to the distance to the obstacle. For example, the front left icon 111 displays the outermost arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is equal to or greater than the first predetermined distance. The front left icon 111 displays the outermost arc-shaped curve and the central arc-shaped curve among the triple arc-shaped curves when the distance to the obstacle is less than the first predetermined distance and greater than or equal to the second predetermined distance. To do. The front left icon 111 displays all triple arc curves when the distance to the obstacle is less than the second predetermined distance.

The front center icon 112 is an icon that notifies an obstacle in the front center of the vehicle. More specifically, the front center icon 112 is an icon that notifies that an obstacle has been detected by the front center sensor 22. The front center icon 112 is superimposed on the upper center of the center image 105 of the bird's-eye view image 100 in FIG. The front center icon 112 is configured in the same manner as the front left icon 111.

The front right icon 113 is an icon that notifies an obstacle on the front right side of the vehicle. More specifically, the front right icon 113 is an icon that notifies that an obstacle has been detected by the front right sensor 23. The front right icon 113 is superimposed on the upper right of the central image 105 of the bird's-eye view image 100 in FIG. The front right icon 113 is configured in the same manner as the front left icon 111.

The rear left icon 114 is an icon that notifies an obstacle on the rear left side of the vehicle. More specifically, the rear left icon 114 is an icon that notifies that an obstacle has been detected by the rear left sensor 24. The rear left icon 114 is superimposed on the lower left of the central image 105 of the bird's-eye view image 100 in FIG. The rear left icon 114 is configured in the same manner as the front left icon 111.

The rear center icon 115 is an icon that notifies an obstacle in the rear center of the vehicle. More specifically, the rear center icon 115 is an icon that notifies that an obstacle has been detected by the rear center sensor 25. The rear center icon 115 is superimposed on the lower center of the center image 105 of the bird's-eye view image 100 in FIG. The rear center icon 115 is configured in the same manner as the front left icon 111.

The rear right icon 116 is an icon that notifies an obstacle on the rear right side of the vehicle. More specifically, the rear right icon 116 is an icon that notifies that an obstacle has been detected by the rear right sensor 26. The rear right icon 116 is superimposed on the lower right of the center image 105 of the bird's-eye view image 100 in FIG. The rear right icon 116 is configured in the same manner as the front left icon 111.

The display control unit 47 causes the display panel 31 to display the bird's-eye view image 100 generated by the superimposed image generation unit 46.

The storage unit 49 stores data and various processing results required for various processing in the bird's-eye view image generation device 40. The storage unit 49 is, for example, a semiconductor memory element such as a RAM (Random Access Memory), a ROM (Read Only Memory), or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk.

Next, the flow of processing in the bird's-eye view image generation device 40 of the bird's-eye view image generation system 1 will be described with reference to FIG. FIG. 3 is a flowchart showing a processing flow in the bird's-eye view image generation device of the bird's-eye view image generation system according to the first embodiment.

The control unit 41 determines whether or not to start the bird's-eye view video display (step S11). As an example of the determination to start the bird's-eye view image display, the control unit 41 determines whether or not to disclose the bird's-eye view image display based on the presence or absence of the backward trigger. The backward trigger means, for example, that the shift position is set to "reverse". Alternatively, the reverse trigger means that the traveling direction of the vehicle is behind the vehicle in the front-rear direction. If there is no backward trigger, the control unit 41 determines that the bird's-eye view image display is not started (No in step S11), and executes the process of step S11 again. If there is a backward trigger, the control unit 41 determines that the bird's-eye view image display is started (Yes in step S11), and proceeds to step S12.

The control unit 41 generates and displays the bird's-eye view image 100 (step S12). More specifically, the control unit 41 causes the bird's-eye view image generation unit 45 to generate a bird's-eye view image 100 that has been subjected to viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit 42. Then, the control unit 41 causes the display control unit 47 to display the generated bird's-eye view image 100 on the display panel 31.

The control unit 41 determines whether or not an obstacle has been detected (step S13). More specifically, the control unit 41 determines whether or not the obstacle information has been acquired by the obstacle information acquisition unit 43. When the control unit 41 determines that the obstacle information has been acquired by the obstacle information acquisition unit 43 (Yes in step S13), the control unit 41 proceeds to step S14. When the control unit 41 determines that the obstacle information has not been acquired by the obstacle information acquisition unit 43 (No in step S13), the control unit 41 proceeds to step S15.

The control unit 41 superimposes and displays an obstacle notification icon 110 indicating an obstacle on the central portion of the bird's-eye view image 100 (step S14). More specifically, the control unit 41 indicates the direction in which the obstacle is detected in the central image 105 of the bird's-eye view image 100 based on the obstacle information acquired by the obstacle information acquisition unit 43 in the superimposed image generation unit 46. A bird's-eye view image 100 on which an obstacle notification icon 110 is superimposed is generated. Then, the control unit 41 causes the display control unit 47 to display the generated bird's-eye view image 100 on the display panel 31.

When the control unit 41 acquires obstacle information from a plurality of sensors of the front left sensor 21, the front center sensor 22, the front right sensor 23, the rear left sensor 24, the rear center sensor 25, and the rear right sensor 26, the superimposed image is superimposed. The generation unit 46 generates the bird's-eye view image 100 in which a plurality of obstacle notification icons 110 are superimposed on the central image 105 of the bird's-eye view image 100.

The control unit 41 determines whether or not to end the bird's-eye view video display (step S15). More specifically, the control unit 41 determines whether or not to end the bird's-eye view image display based on the presence or absence of the backward trigger. The control unit 41 determines that the bird's-eye view image display is terminated when there is no backward trigger, in other words, when the backward trigger is released (Yes in step S15), and the process is terminated. When there is a backward trigger, the control unit 41 determines that the bird's-eye view image display is not finished (No in step S15), returns to step S13, and continues the process.

In this way, when an obstacle is detected, the bird's-eye view image generation system 1 superimposes an obstacle notification icon 110 on the central portion of the bird's-eye view image 100 so as to indicate the direction in which the obstacle is detected, and displays the display panel. A video signal is output to 31. The display panel 31 displays the bird's-eye view image 100 together with the navigation screen, for example, based on the image signal output from the bird's-eye view image generation system 1.

For example, an example in which an obstacle is detected on the left rear side when the vehicle is moving backward will be described with reference to FIG. FIG. 4 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the first embodiment. In FIG. 4, the rear image 102 of the bird's-eye view image 100 includes an obstacle image 200 of an obstacle on the rear left side.

The control unit 41 determines that an obstacle has been detected in step S13. Then, in step S14, the control unit 41 detects an obstacle in the central image 105 of the bird's-eye view image 100 based on the obstacle information acquired by the obstacle information acquisition unit 43 in the superimposed image generation unit 46, and then left. A bird's-eye view image 100 on which the obstacle notification icon 110 is superimposed so as to indicate the direction is generated. Then, the control unit 41 causes the display control unit 47 to display the generated bird's-eye view image 100 on the display panel 31.

Another example in which an obstacle is detected on the left rear side when the vehicle is moving backward will be described with reference to FIG. FIG. 5 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the first embodiment.

In FIG. 5, the vehicle icon 120, which is the vehicle icon indicating the vehicle when the vehicle is viewed from above, is combined with the central image 105 of the bird's-eye view image 100. In step S14, the control unit 41 causes the superimposed image generation unit 46 to generate a bird's-eye view image 100 on which the obstacle notification icon 110 is superimposed on the own vehicle icon 120 so as to indicate the left direction after detecting the obstacle.

As described above, in the present embodiment, when an obstacle is detected around the vehicle, the obstacle notification icon 110 is superimposed on the central image 105 of the bird's-eye view image 100 so as to indicate the direction in which the obstacle is detected. The image 100 is displayed on the display panel 31. In the present embodiment, since the obstacle notification icon 110 is superimposed on the central image 105 of the bird's-eye view image 100, the obstacle notification icon 110 and the obstacle reflected in the bird's-eye view image 100 are not superimposed. In other words, the present embodiment can clearly display the obstacle without impairing the visibility of the obstacle reflected in the bird's-eye view image 100. In this way, the present embodiment can appropriately confirm obstacles around the vehicle.

In this embodiment, the obstacle notification icon 110 can notify the direction and distance at which the obstacle is detected.

In this embodiment, as shown in FIG. 4, when the own vehicle icon 120 is not displayed on the central image 105 of the bird's-eye view image 100, the visibility of the obstacle notification icon 110 can be further improved. In this way, the present embodiment can more appropriately confirm obstacles around the vehicle.

In the present embodiment, as shown in FIG. 5, the own vehicle icon 120 may be displayed on the central image 105 of the bird's-eye view image 100. Thereby, the present embodiment can display the bird's-eye view image 100 that makes it easier to recognize the direction of the obstacle with respect to the vehicle. In this way, the present embodiment can appropriately confirm obstacles around the vehicle.
[Second Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIGS. 6 and 7. FIG. 6 is a flowchart showing a processing flow in the bird's-eye view image generation device of the bird's-eye view image generation system according to the second embodiment. FIG. 7 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the second embodiment. The basic configuration of the bird's-eye view image generation system 1 is the same as that of the bird's-eye view image generation system 1 of the first embodiment. In the following description, the same components as those of the bird's-eye view video generation system 1 are designated by the same reference numerals or corresponding reference numerals, and detailed description thereof will be omitted. In the bird's-eye view image generation system 1 of the present embodiment, the processing in the obstacle information acquisition unit 43, the superimposed image generation unit 46, and the control unit 41 of the bird's-eye view image generation device 40 is different from the bird's-eye view image generation system 1 of the first embodiment. ..

The obstacle notification icon 110 includes the front left icon 111, the front center icon 112, the front right icon 113, the rear left icon 114, the rear center icon 115, and the rear right icon 116 of the first embodiment, and the second front left icon 110. Icon (information indicating an obstacle), second front center icon (information indicating an obstacle), second front right icon (information indicating an obstacle), second rear left icon (information indicating an obstacle), and second Includes a rear center icon (information indicating an obstacle) and a second rear right icon (information indicating an obstacle). In FIG. 7, the second front left icon, the second front center icon, the second front right icon, the second rear center icon, and the second rear right icon are not shown, and the second rear left icon (obstacles). Information to be shown) is shown.

The front left icon 111 is an icon that notifies an obstacle on the front left side of the vehicle when the distance to the obstacle is less than the threshold value. More specifically, the front left icon 111 is an icon that notifies that the front left sensor 21 has detected an obstacle whose distance to the obstacle is less than the threshold value.

The front center icon 112 is an icon that notifies an obstacle in the front center of the vehicle when the distance to the obstacle is less than the threshold value. More specifically, the front center icon 112 is an icon that notifies that the front center sensor 22 has detected an obstacle whose distance to the obstacle is less than the threshold value.

The front right icon 113 is an icon that notifies an obstacle on the front right side of the vehicle when the distance to the obstacle is less than the threshold value. More specifically, the front right icon 113 is an icon that notifies that the front right sensor 23 has detected an obstacle whose distance to the obstacle is less than the threshold value.

The rear left icon 114 is an icon that notifies an obstacle on the rear left side of the vehicle when the distance to the obstacle is less than the threshold value. More specifically, the rear left icon 114 is an icon that notifies that the rear left sensor 24 has detected an obstacle whose distance to the obstacle is less than the threshold value.

The rear center icon 115 is an icon that notifies an obstacle in the rear center of the vehicle when the distance to the obstacle is less than the threshold value. More specifically, the rear center icon 115 is an icon that notifies that the rear center sensor 25 has detected an obstacle whose distance to the obstacle is less than the threshold value.

The rear right icon 116 is an icon for notifying an obstacle on the rear right side of the vehicle when the distance to the obstacle is less than the threshold value. More specifically, the rear right icon 116 is an icon that notifies that the rear right sensor 26 has detected an obstacle whose distance to the obstacle is less than the threshold value.

The second front left icon is an icon that notifies an obstacle on the front left side of the vehicle when the distance to the obstacle is equal to or greater than the threshold value. More specifically, the second front left icon is an icon that notifies that the front left sensor 21 has detected an obstacle whose distance to the obstacle is equal to or greater than the threshold value. The second front left icon is superimposed on the upper left outside the central image 105 of the bird's-eye view image 100. The second front left icon is configured in the same manner as the front left icon 111.

The second front center icon is an icon that notifies the obstacle in the front center of the vehicle when the distance to the obstacle is equal to or greater than the threshold value. More specifically, the second front center icon is an icon that notifies that the front center sensor 22 has detected an obstacle whose distance to the obstacle is equal to or greater than the threshold value. The second front center icon is superimposed on the upper center outside the center image 105 of the bird's-eye view image 100. The second front center icon is configured in the same manner as the front left icon 111.

The second front right icon is an icon that notifies an obstacle on the front right side of the vehicle when the distance to the obstacle is equal to or greater than the threshold value. More specifically, the second front right icon is an icon that notifies that the front right sensor 23 has detected an obstacle whose distance to the obstacle is equal to or greater than the threshold value. The second front right icon is superimposed on the upper right outside the center image 105 of the bird's-eye view image 100. The second front right icon is configured in the same manner as the front left icon 111.

The second rear left icon is an icon that notifies an obstacle on the rear left side of the vehicle when the distance to the obstacle is equal to or greater than the threshold value. More specifically, the second rear left icon is an icon that notifies that the rear left sensor 24 has detected an obstacle whose distance to the obstacle is equal to or greater than the threshold value. The second rear left icon is superimposed on the lower left outside the center image 105 of the bird's-eye view image 100. The second rear left icon is configured in the same manner as the front left icon 111.

The second rear center icon is an icon that notifies the obstacle in the rear center of the vehicle when the distance to the obstacle is equal to or greater than the threshold value. More specifically, the second rear center icon is an icon that notifies that the rear center sensor 25 has detected an obstacle whose distance to the obstacle is equal to or greater than the threshold value. The second rear center icon is superimposed on the lower center outside the center image 105 of the bird's-eye view image 100. The second rear center icon is configured in the same manner as the front left icon 111.

The second rear right icon is an icon that notifies an obstacle on the rear right side of the vehicle when the distance to the obstacle is equal to or greater than the threshold value. More specifically, the second rear right icon is an icon that notifies that the rear right sensor 26 has detected an obstacle whose distance to the obstacle is equal to or greater than the threshold value. The second rear right icon is superimposed on the lower right outside the center image 105 of the bird's-eye view image 100. The second rear right icon is configured in the same manner as the front left icon 111.

The obstacle information acquisition unit 43 acquires information including the distance to the detected obstacle. The obstacle information acquisition unit 43 outputs information including the acquired distance to the obstacle to the superimposed video generation unit 46.

Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 46 displays the own vehicle icon 120 on the central image 105 of the bird's-eye view image 100 if the distance to the detected obstacle is equal to or greater than a predetermined threshold value. Is displayed, and an obstacle notification icon 110, which is information indicating an obstacle, is superimposed on the outside of the central portion of the bird's-eye view image 100 to generate a bird's-eye view image 100.

If the distance to the detected obstacle is less than a predetermined threshold based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 46 displays the obstacle notification icon 110 at the center of the bird's-eye view image 100. A bird's-eye view image 100 superimposed on the image 105 is generated.

Next, the flow of processing in the bird's-eye view image generation device 40 of the bird's-eye view image generation system 1 will be described with reference to FIG. The processing of step S21, step S23, step S26, and step S27 of the flowchart shown in FIG. 6 is the same as the processing of steps S11, S13, step S14, and step S15 of the flowchart shown in FIG.

The control unit 41 generates and displays a bird's-eye view image 100 on which the own vehicle icon 120 is superimposed (step S22). More specifically, the control unit 41 causes the bird's-eye view image generation unit 45 to generate a bird's-eye view image 100 that has been subjected to viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit 42. The control unit 41 synthesizes the own vehicle icon 120 with the central portion of the generated bird's-eye view image 100 by the display control unit 47. Then, the control unit 41 causes the display control unit 47 to display the generated bird's-eye view image 100 on the display panel 31.

The control unit 41 determines whether or not the distance to the obstacle is equal to or greater than the threshold value (step S24). More specifically, the control unit 41 proceeds to step S25 when the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold value (Yes in step S24). When the distance to the detected obstacle, which is acquired by the obstacle information acquisition unit 43, is less than a predetermined threshold value (No in step S24), the control unit 41 proceeds to step S26.

The predetermined threshold value is the position where the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon are displayed. It is set to a value that does not overlap with the image of the obstacle in. More specifically, the predetermined threshold is the outermost arc shape of the second front left icon, the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon. The value may be greater than or equal to the distance from the vehicle corresponding to the position of the curve. For example, the predetermined threshold value may be about 2 m.

The control unit 41 superimposes and displays the obstacle notification icon 110 on the bird's-eye view image 100 outside the own vehicle icon 120 (step S25). More specifically, the control unit 41 is a superposed image generation unit 46, and based on the obstacle information acquired by the obstacle information acquisition unit 43, the second front left icon is displayed on the bird's-eye view image 100 outside the own vehicle icon 120. A bird's-eye view image 100 is generated in which at least one of the second front center icon, the second front right icon, the second rear left icon, the second rear center icon, and the second rear right icon is superimposed. Then, the control unit 41 causes the display control unit 47 to display the generated bird's-eye view image 100 on the display panel 31.

In this way, when the distance to the obstacle is equal to or greater than a predetermined threshold value, the bird's-eye view image generation system 1 displays the second front left icon, the second front center icon, and the second front left icon on the bird's-eye view image 100 outside the own vehicle icon 120. (Ii) A bird's-eye view image 100 in which at least one of the front right icon, the second rear left icon, the second rear center icon, and the second rear right icon is superimposed is generated, and the image signal is output to the display panel 31. When the distance to the obstacle is less than a predetermined threshold value, the bird's-eye view image generation system 1 displays the front left icon 111, the front center icon 112, the front right icon 113, the rear left icon 114, and the rear in the center image 105 of the bird's-eye view image 100. A bird's-eye view image 100 in which at least one of the center icon 115 and the rear right icon 116 is superimposed is generated, and the image signal is output to the display panel 31.

For example, an example in which an obstacle is detected at a distance above a predetermined threshold value on the left rear side when the vehicle is moving backward will be described with reference to FIG. 7.

The control unit 41 determines that an obstacle has been detected in step S23. Then, in step S24, the control unit 41 determines that the distance to the detected obstacle acquired by the obstacle information acquisition unit 43 is equal to or greater than a predetermined threshold value (Yes in step S24). Then, in step S25, the control unit 41 superimposes the second rear left icon 117 indicating the left direction on the bird's-eye view image 100 outside the own vehicle icon 120 after detecting an obstacle, and the bird's-eye view image 100 shown in FIG. To generate. Then, the control unit 41 displays the generated bird's-eye view image 100 on the display panel 31.

As described above, in the present embodiment, when the distance to the obstacle is equal to or greater than a predetermined threshold value, the bird's-eye view image 100 in which the obstacle notification icon 110 is superimposed on the bird's-eye view image 100 outside the own vehicle icon 120 is generated and displayed. It is displayed on the panel 31. In the present embodiment, when the distance to the obstacle is equal to or greater than a predetermined threshold value, the obstacle notification icon 110 is superimposed on the bird's-eye view image 100 outside the own vehicle icon 120, so that the obstacle notification icon 110 and the bird's-eye view image 100 are displayed. It does not overlap with the obstacles. In other words, the present embodiment can clearly display the obstacle without impairing the visibility of the obstacle reflected in the bird's-eye view image 100. In this way, the present embodiment can appropriately confirm obstacles around the vehicle.

In the present embodiment, when the distance to the obstacle is less than a predetermined threshold value, the bird's-eye view image 100 on which the obstacle notification icon 110 is superimposed is generated on the central image 105 of the bird's-eye view image 100 and displayed on the display panel 31. In the present embodiment, when the distance to the obstacle is less than a predetermined threshold value, the obstacle notification icon 110 is superimposed on the central image 105 of the bird's-eye view image 100, so that the obstacles displayed on the obstacle notification icon 110 and the bird's-eye view image 100 And do not overlap. In other words, the present embodiment can clearly display the obstacle without impairing the visibility of the obstacle reflected in the bird's-eye view image 100.

As described above, in this embodiment, obstacles around the vehicle can be appropriately confirmed according to the distance to the obstacle.

In this embodiment, the display position of the obstacle notification icon 110 in the bird's-eye view image 100 changes according to the distance to the obstacle. Therefore, in this embodiment, it is possible to make it easier to grasp the distance to the obstacle.
[Third Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing a processing flow in the bird's-eye view image generation device of the bird's-eye view image generation system according to the third embodiment. In the bird's-eye view image generation system 1 of the present embodiment, the processing in the obstacle information acquisition unit 43, the superimposed image generation unit 46, and the control unit 41 of the bird's-eye view image generation device 40 is different from the bird's-eye view image generation system 1 of the second embodiment. ..

The front left sensor 21, the front center sensor 22, the front right sensor 23, the rear left sensor 24, the rear center sensor 25, and the rear right sensor 26 have a detection range farther than the range displayed as the bird's-eye view image 100.

The obstacle information acquisition unit 43 acquires obstacle information of obstacles detected in a range farther than the range displayed as the bird's-eye view image 100.

The superimposed image generation unit 46 is the central portion of the bird's-eye view image 100 if the distance to the detected obstacle is farther than the range displayed as the bird's-eye view image 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. Display the own vehicle icon 120, and generate a bird's-eye view image 100 in which the obstacle notification icon 110 is superimposed on the bird's-eye view image 100 outside the own vehicle icon 120.

The superimposed image generation unit 46 is an obstacle notification icon if the distance to the detected obstacle is within the range to be displayed as the bird's-eye view image 100 based on the obstacle information acquired by the obstacle information acquisition unit 43. The bird's-eye view image 100 is generated by superimposing the 110 on the central image 105 of the bird's-eye view image 100 so as to indicate the direction in which the obstacle is detected.

Next, the flow of processing in the bird's-eye view image generation device 40 of the bird's-eye view image generation system 1 will be described with reference to FIG. The processing of steps S31 to S33 and steps S35 to S37 of the flowchart shown in FIG. 8 is the same as the processing of steps S21 to S23 and steps S25 to S27 of the flowchart shown in FIG.

The control unit 41 determines whether or not the distance to the obstacle is equal to or greater than the display range of the bird's-eye view image 100 (step S34). When the distance to the detected obstacle, which is acquired by the obstacle information acquisition unit 43, is farther than the display range of the bird's-eye view image 100 (Yes in step S34), the control unit 41 proceeds to step S35. When the distance to the detected obstacle, which is acquired by the obstacle information acquisition unit 43, is included in the display range of the bird's-eye view image 100 (No in step S34), the control unit 41 proceeds to step S36.

In this way, when the bird's-eye view image generation system 1 has a distance to an obstacle farther than the display range of the bird's-eye view image 100, the obstacle notification icon 110 is displayed on the bird's-eye view image 100 outside the own vehicle icon 120. The superimposed bird's-eye view image 100 is generated, and the image signal is output to the display panel 31. When the distance to the obstacle is included in the display range of the bird's-eye view image 100, the bird's-eye view image generation system 1 generates the bird's-eye view image 100 in which the obstacle notification icon 110 is superimposed on the central image 105 of the bird's-eye view image 100, and the display panel. A video signal is output to 31.

As described above, in the present embodiment, when the distance to the obstacle is in a range farther than the display range of the bird's-eye view image 100, the obstacle notification icon 110 is superimposed on the bird's-eye view image 100 outside the own vehicle icon 120. The bird's-eye view image 100 is generated and displayed on the display panel 31. In this embodiment, when the distance to the obstacle is farther than the display range of the bird's-eye view image 100, the obstacle notification icon 110 is superimposed on the outside of the own vehicle icon 120, so that the obstacle notification icon 110 and the bird's-eye view image The obstacle reflected in 100 does not overlap. In other words, the present embodiment can clearly display the obstacle without impairing the visibility of the obstacle reflected in the bird's-eye view image 100. In this way, the present embodiment can appropriately confirm obstacles around the vehicle.

In this embodiment, when the distance to the obstacle is included in the display range of the bird's-eye view image 100, the bird's-eye view image 100 in which the obstacle notification icon 110 is superimposed on the central image 105 of the bird's-eye view image 100 is generated and displayed on the display panel 31. Let me. In the present embodiment, when the distance to the obstacle is included in the display range of the bird's-eye view image 100, the obstacle notification icon 110 is superimposed on the central image 105 of the bird's-eye view image 100, so that the obstacle notification icon 110 and the bird's-eye view image 100 The obstacles reflected in the image do not overlap. In other words, the present embodiment can clearly display the obstacle without impairing the visibility of the obstacle reflected in the bird's-eye view image 100.

As described above, in this embodiment, obstacles around the vehicle can be appropriately confirmed according to the distance to the obstacle.

In this embodiment, the obstacle notification icon 110 can be displayed even when there is an obstacle in a range far from the display range of the bird's-eye view image 100. Therefore, in this embodiment, obstacles in a range far from the display range of the bird's-eye view image 100 can be notified.
[Fourth Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described. The bird's-eye view image generation system 1 of the present embodiment is different from the bird's-eye view image generation system 1 of the first embodiment in the processing in the superimposed image generation unit 46 and the control unit 41 of the bird's-eye view image generation device 40. More specifically, it differs from the bird's-eye view image generation system 1 of the first embodiment in that the control unit 41 detects an obstacle located in the traveling direction of the vehicle in determining whether or not an obstacle has been detected.

Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 46 superimposes the obstacle notification icon 110 on the central image 105 of the bird's-eye view image 100 for obstacles located in the traveling direction of the vehicle. A bird's-eye view image 100 is generated.

Next, the processing flow in the bird's-eye view image generation device 40 of the bird's-eye view image generation system 1 will be described.

In step S13, the control unit 41 determines whether or not an obstacle has been detected in the traveling direction of the vehicle based on the obstacle information acquired by the obstacle information acquisition unit 43. When the control unit 41 determines that an obstacle has been detected in the traveling direction of the vehicle (Yes in step S13), the control unit 41 proceeds to step S14. When the control unit 41 determines that no obstacle is detected in the traveling direction of the vehicle (No in step S13), the control unit 41 proceeds to step S15.

In this way, the control unit 41 displays the obstacle notification icon 110 for obstacles approaching the vehicle, and hides the obstacle notification icon 110 for obstacles moving away from the vehicle.

As described above, the present embodiment can more appropriately confirm obstacles around the vehicle.
[Fifth Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIGS. 9 to 12. FIG. 9 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the fifth embodiment. FIG. 10 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the fifth embodiment. FIG. 11 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the fifth embodiment. FIG. 12 is a graph showing an example of the relationship between the distance between arcs and the distance to an obstacle in the obstacle notification icon. In the bird's-eye view image generation system 1 of the present embodiment, the processing in the superimposed image generation unit 46 of the bird's-eye view image generation device 40 is different from that of the bird's-eye view image generation system 1 of the first embodiment.

As shown in FIGS. 9 to 11, the obstacle notification icon 110 has a plurality of components facing the direction in which the obstacle is detected, and indicates the distance to the obstacle by the distance between the plurality of components. In the present embodiment, the obstacle notification icon 110 has a triple arc as a component. In the present embodiment, the obstacle notification icon 110 indicates the distance to the obstacle at intervals of arcs. In the present embodiment, the obstacle notification icon 110 indicates that the larger the distance to the obstacle, the wider the distance between the arcs, and the smaller the distance to the obstacle, the narrower the distance between the arcs.

As shown by the solid line in FIG. 12, the distance between the arcs of the obstacle notification icon 110 may be set linearly so that the larger the distance to the obstacle, the wider the distance.

As shown by the broken line in FIG. 12, the distance between the arcs of the obstacle notification icon 110 may be set in a stepped manner so that the larger the distance to the obstacle, the wider the distance. More specifically, when the distance to the obstacle is equal to or greater than the first threshold value, for example, when it is 2 m or more, the interval between the arcs is set to the first interval d1. The first threshold value may be the distance of the boundary included in the range displayed as the bird's-eye view image 100. When the distance to the obstacle is equal to or greater than the first threshold value, the outermost arcuate curve may be displayed outside the central image 105 of the bird's-eye view image 100. When the distance to the obstacle is less than the first threshold value and more than the second threshold value, for example, when it is less than 2 m and 1 m or more, the distance between the arcs is set to the second distance d2 which is smaller than the first distance d1. When the distance to the obstacle is less than the second threshold value, for example, less than 1 m, the distance between the arcs is set to the third distance d3 which is smaller than the second distance d2. Further, when the distance to the obstacle becomes smaller, the arcs may be overlapped with the interval between the arcs set to zero.

The distance to the obstacle is the distance from each sensor that detects the obstacle to the obstacle. The distance to the obstacle is approximately the same as the distance from the end of the vehicle in the direction of the obstacle to the obstacle.

The superimposed image generation unit 46 displays the obstacle notification icon 110 whose arc spacing is changed based on the detected distance to the obstacle acquired by the obstacle information acquisition unit 43 on the central image 105 of the bird's-eye view image 100. The superimposed bird's-eye view image 100 is generated.

An example in which an obstacle is detected on the left rear side when the vehicle is moving backward will be described with reference to FIGS. 9 to 11.

For example, when the vehicle is moving backward, an obstacle is detected on the left rear side, and an example in which the distance to the obstacle is 2 m will be described with reference to FIG. In step S14, the control unit 41 indicates the left direction after detecting an obstacle in the bulging direction of the arc in the central image 105 of the bird's-eye view image 100 by the superimposed image generation unit 46, and reaches the obstacle at intervals of the arc. A bird's-eye view image 100 on which an obstacle notification icon 110 indicating a distance is superimposed is generated. In the obstacle notification icon 110, the distance between the arcs is the first distance d1. In the obstacle notification icon 110, the outermost arc-shaped curve is located outside the central image 105. The control unit 41 causes the superimposed image generation unit 46 to display the generated bird's-eye view image 100 shown in FIG. 9 on the display panel 31. Since the distance between the arcs of the obstacle notification icon 110 and the outermost arc-shaped curve are located outside the central image 105, the obstacle is located near the boundary of the range displayed as the bird's-eye view image 100. You can see that.

For example, an example in which an obstacle is detected on the left rear side when the vehicle is moving backward and the distance to the obstacle is 1.5 m will be described with reference to FIG. In step S14, the control unit 41 indicates the left direction after detecting an obstacle in the bulging direction of the arc in the central image 105 of the bird's-eye view image 100 by the superimposed image generation unit 46, and reaches the obstacle at intervals of the arc. A bird's-eye view image 100 on which an obstacle notification icon 110 indicating a distance is superimposed is generated. In the obstacle notification icon 110, the interval between the arcs is the second interval d2. The control unit 41 causes the superimposed image generation unit 46 to display the generated bird's-eye view image 100 shown in FIG. 10 on the display panel 31. From the distance between the arcs of the obstacle notification icon 110, it can be seen that the obstacle is located within the range displayed as the bird's-eye view image 100.

For example, when the vehicle is moving backward, an obstacle is detected on the left rear side, and an example in which the distance to the obstacle is 1 m will be described with reference to FIG. In step S14, the control unit 41 indicates the left direction after detecting an obstacle in the bulging direction of the arc in the central image 105 of the bird's-eye view image 100 by the superimposed image generation unit 46, and reaches the obstacle at intervals of the arc. A bird's-eye view image 100 on which an obstacle notification icon 110 indicating a distance is superimposed is generated. In the obstacle notification icon 110, the distance between the arcs is the third distance d3. The control unit 41 causes the superimposed image generation unit 46 to display the generated bird's-eye view image 100 shown in FIG. 11 on the display panel 31. Since the distance between the arcs of the obstacle notification icon 110 is narrow, it can be seen that the obstacle is located near the rear end of the vehicle.

In this way, the control unit 41 generates the bird's-eye view image 100 in which the obstacle notification icon 110 whose arc spacing is changed is superimposed on the central image 105 of the bird's-eye view image 100 based on the detected distance to the obstacle. ..

As described above, in the present embodiment, when an obstacle is detected around the vehicle, the central image 105 of the bird's-eye view image 100 shows the direction in which the obstacle is detected, and the distance to the obstacle is determined by the interval of the arc. As shown, the bird's-eye view image 100 on which the obstacle notification icon 110 is superimposed is displayed on the display panel 31. In this embodiment, the direction of the obstacle and the distance to the obstacle can be displayed by the obstacle notification icon 110. In this way, the present embodiment can appropriately confirm obstacles around the vehicle.
[Sixth Embodiment]
The bird's-eye view image generation system 1 according to the present embodiment will be described with reference to FIGS. 13 to 15. FIG. 13 is a diagram showing an example of a bird's-eye view image generated by the bird's-eye view image generation system according to the sixth embodiment. FIG. 14 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the sixth embodiment. FIG. 15 is a diagram showing another example of the bird's-eye view image generated by the bird's-eye view image generation system according to the sixth embodiment. In the bird's-eye view image generation system 1 of the present embodiment, the processing in the superimposed image generation unit 46 of the bird's-eye view image generation device 40 is different from that of the bird's-eye view image generation system 1 of the first embodiment.

As shown in FIGS. 13 to 15, the obstacle notification icon 110 has a horizontal detection range of the sensor detecting an obstacle and a horizontal width of the detected obstacle depending on the horizontal width of the icon itself. Indicates the width. In the present embodiment, the obstacle notification icon 110 has a larger arc length as the detection range in the horizontal direction of the sensor detecting the obstacle is wider, and the detection range in the horizontal direction of the sensor detecting the obstacle. The narrower the arc, the smaller the arc length. In the present embodiment, the obstacle notification icon 110 indicates that the wider the horizontal width of the obstacle is, the larger the arc length is, and the narrower the horizontal width of the obstacle is, the smaller the arc length is.

More specifically, for example, the obstacle notification icon 110 may have the projection width from the central portion of the bird's-eye view image 100 to both ends of the obstacle in the horizontal direction as the length of the arc.

Based on the obstacle information acquired by the obstacle information acquisition unit 43, the superimposed image generation unit 46 sets the width of the information indicating the obstacle as the width corresponding to the horizontal width of the detected obstacle of the bird's-eye view image 100. A bird's-eye view image 100 superimposed on the central image 105 is generated.

An example in which an obstacle is detected on the left rear side when the vehicle is moving backward will be described with reference to FIGS. 13 to 15.

For example, when the vehicle is moving backward, an obstacle is detected on the left rear side, and an example in which the distance to the obstacle is 2 m will be described with reference to FIG. In step S14, the control unit 41 displays the horizontal detection range of the sensor detecting an obstacle in the central image 105 of the bird's-eye view image 100 according to the horizontal width of the icon itself in the superimposed image generation unit 46. A bird's-eye view image 100 is generated on which an obstacle notification icon 110 indicating the width of the detected obstacle in the horizontal direction is superimposed. The horizontal width of the obstacle referred to here is the apparent width detected by the sensor, that is, the projected width. The control unit 41 causes the superimposed image generation unit 46 to display the generated bird's-eye view image 100 shown in FIG. 13 on the display panel 31. It can be seen from the horizontal width of the obstacle notification icon 110 that the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrow.

For example, an example in the case where the distance from the state shown in FIG. 13 to the obstacle is 1 m will be described with reference to FIG. In step S14, the control unit 41 causes the superimposed image generation unit 46 to generate the bird's-eye view image 100 on which the obstacle notification icon 110 is superimposed on the central image 105 of the bird's-eye view image 100. Since the projection width from the central portion of the bird's-eye view image 100 to both ends of the obstacle in the horizontal direction is widened, the width of the obstacle notification icon 110 in the horizontal direction is the width in the horizontal direction of the obstacle notification icon 110 in FIG. Wider. The control unit 41 causes the superimposed image generation unit 46 to display the generated bird's-eye view image 100 shown in FIG. 14 on the display panel 31. Since the horizontal width of the obstacle notification icon 110 is wider than that in FIG. 13, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are widened. That is, it can be seen that the obstacle is approaching the rear end of the vehicle.

In this way, the superimposed image generation unit 46 changes the width of the obstacle notification icon 110 in the horizontal direction and displays it as shown in FIGS. 13 to 14, depending on the proximity situation of the obstacle. The change in the horizontal width of the obstacle notification icon 110 indicates whether the obstacle is approaching or moving away from the vehicle.

For example, when the vehicle is reversing, an example in which an obstacle having a width narrower in the horizontal direction than the obstacle shown in FIG. 14 is detected on the rear left side will be described with reference to FIG. In step S14, the control unit 41 causes the superimposed image generation unit 46 to generate the bird's-eye view image 100 on which the obstacle notification icon 110 is superimposed on the central image 105 of the bird's-eye view image 100. The width of the obstacle notification icon 110 in the horizontal direction is narrower than the width of the obstacle notification icon 110 in FIG. 14 in the horizontal direction. The control unit 41 causes the superimposed image generation unit 46 to display the generated bird's-eye view image 100 shown in FIG. 15 on the display panel 31. Since the horizontal width of the obstacle notification icon 110 is narrower than that in FIG. 14, the horizontal detection range of the sensor detecting the obstacle and the horizontal width of the detected obstacle are narrowed. That is, it can be seen that the horizontal width of the obstacle is narrow.

As described above, in the present embodiment, when an obstacle is detected around the vehicle, the central image 105 of the bird's-eye view image 100 indicates the direction in which the obstacle is detected, and the width of the horizontal arc indicates the obstacle. A bird's-eye view image 100 on which the obstacle notification icon 110 is superimposed so as to indicate the width in the horizontal direction is displayed on the display panel 31. In this embodiment, the direction of the obstacle and the horizontal width of the obstacle can be displayed by the obstacle notification icon 110. In this way, the present embodiment can appropriately confirm obstacles around the vehicle.

Further, in the present embodiment, when the distance to the obstacle changes, the horizontal width of the obstacle notification icon 110 is changed and displayed as shown in FIGS. 13 to 14, for example. As a result, according to the present embodiment, it is possible to confirm the proximity situation of the obstacle by changing the width of the obstacle notification icon 110 in the horizontal direction.

Although the bird's-eye view image generation system 1 according to the present invention has been described so far, it may be implemented in various different forms other than the above-described embodiment.

Each component of the illustrated bird's-eye view image generation system 1 is a functional concept and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or part of each device is functionally or physically dispersed or integrated in an arbitrary unit according to the processing load and usage status of each device. You may.

The configuration of the bird's-eye view video generation system 1 is realized, for example, by a program loaded in a memory as software. In the above embodiment, it has been described as a functional block realized by linking these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The above-mentioned components include those that can be easily assumed by those skilled in the art and those that are substantially the same. Further, the above configurations can be combined as appropriate. Further, various omissions, substitutions or changes of the configuration can be made without departing from the gist of the present invention.

For example, when the distance to the nearest obstacle is less than a predetermined threshold value, the superimposed image generation unit 46 may display the distance to the obstacle numerically together with the obstacle notification icon 110 on the central image 105 of the bird's-eye view image 100. Good. Alternatively, for example, when the distance to the nearest obstacle is less than a predetermined threshold value, the superimposed image generation unit 46 may be formed on the central image 105 of the bird's-eye view image 100 together with the obstacle notification icon 110 according to the distance to the obstacle. A figure whose color changes may be displayed. The bird's-eye view image generation system 1 can notify the distance to the obstacle in addition to the change of the obstacle notification icon 110 according to the distance to the obstacle. As a result, the bird's-eye view image generation system 1 can more appropriately confirm obstacles around the vehicle.

When there are a plurality of obstacles, the control unit 41 may display the obstacle notification icon 110 for the obstacle having a high priority. For example, the priority may be the one with the shortest distance to the obstacle as the highest priority. Alternatively, for example, the priority may be the one in which the obstacle is a moving object as the highest priority. Alternatively, for example, the priority may be the one in which the obstacle is a moving object and is approaching the vehicle as the highest priority.

When it is detected that the obstacle is a moving object, an icon indicating that the obstacle is a moving object may be displayed on the central image 105 of the bird's-eye view image 100. The icon indicating that the object is a moving object may be, for example, an icon indicating a pedestrian or an icon indicating that the vehicle is a vehicle. As a result, the bird's-eye view image generation system 1 can more appropriately confirm obstacles around the vehicle.

The obstacle notification icon 110 has been described as being composed of a triple arc-shaped curve, but the present invention is not limited to this. The obstacle notification icon 110 may be, for example, an arrow-shaped figure indicating the direction of the obstacle. In this case, the obstacle notification icon 110 may be an arrow-shaped figure whose thickness and size are changed according to the distance to the obstacle.

In the present embodiment, six sensors, a front left sensor 21, a front center sensor 22, a front right sensor 23, a rear left sensor 24, a rear center sensor 25, and a rear right sensor 26, are provided as obstacle detection units. However, the number of sensors is not limited. If you want to detect the direction of obstacles in more detail, you may increase the number of sensors.

In the present embodiment, the obstacle notification icon 110 includes a front left icon 111, a front center icon 112, a front right icon 113, a rear left icon 114, a rear center icon 115, and a rear right icon 116, depending on the number of sensors. Is included, but is not limited to this. In other words, the obstacle notification icon 110 may be associated with the detection result of each sensor and the icon to be displayed.

1 Bird's-eye view image generation system 11 Front camera (shooting section)
12 Rear camera (shooting section)
13 Left side camera (shooting section)
14 Right side camera (shooting section)
21 Front left sensor (obstacle detection unit)
22 Front central sensor (obstacle detection unit)
23 Front right sensor (obstacle detection unit)
24 Rear left sensor (obstacle detection unit)
25 Rear central sensor (obstacle detection unit)
26 Rear right sensor (obstacle detection unit)
31 Display panel 40 Bird's-eye view image generator 41 Control unit 42 Image acquisition unit 43 Obstacle information acquisition unit 44 Vehicle information acquisition unit 45 Bird's-eye view image generation unit 46 Superimposed image generation unit 47 Display control unit 49 Storage unit 100 Overlook video 110 Obstacle notification Icon (information indicating an obstacle)
111 Front left icon (information indicating obstacles)
112 Front center icon (information indicating obstacles)
113 Front right icon (information indicating obstacles)
114 Rear left icon (information indicating obstacles)
115 Rear center icon (information indicating obstacles)
116 Rear right icon (information indicating obstacles)
120 own car icon

Claims (5)

An image acquisition unit that acquires peripheral images of the surroundings of the vehicle,
An obstacle information acquisition unit that acquires obstacle information including the distance from the vehicle to the obstacle of an obstacle detected in the vicinity of the vehicle.
A bird's-eye view image generation unit that generates a bird's-eye view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit.
If the distance to the detected obstacle is equal to or greater than a predetermined threshold value based on the obstacle information acquired by the obstacle information acquisition unit, the information indicating the obstacle is provided outside the central portion surrounded by the bird's-eye view image. Generates a bird's-eye view image that is stretched and superimposed on the above, and if the distance to the detected obstacle is less than a predetermined threshold value, the information indicating the obstacle is superimposed on the central portion to generate a superimposed image. Department and
A bird's-eye view image generation device characterized by having a display control unit for displaying a bird's-eye view image generated by the superimposed image generation unit on the display unit. The information indicating the obstacle has a plurality of components facing the direction in which the obstacle is detected, and indicates the distance to the obstacle by the distance between the plurality of components.
The superimposed image generation unit is extended to the outside of the central portion by changing the spacing between the plurality of components based on the detected distance to the obstacle.
The bird's-eye view image generator according to claim 1. An image acquisition unit that acquires peripheral images of the surroundings of the vehicle,
An obstacle information acquisition unit that acquires obstacle information including the distance from the vehicle to the obstacle of an obstacle detected in the vicinity of the vehicle.
A bird's-eye view image generation unit that generates a bird's-eye view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired by the image acquisition unit.
Based on the obstacle information acquired by the obstacle information acquisition unit, if the detected distance to the obstacle is equal to or greater than a predetermined threshold value, a bird's-eye view image in which the information indicating the obstacle is superimposed on the bird's-eye view image is generated. If the detected distance to the obstacle is less than a predetermined threshold value, the superimposed image generation unit that generates the bird's-eye view image in which the information indicating the obstacle is superimposed on the central portion surrounded by the bird's-eye view image.
A bird's-eye view image generation device characterized by having a display control unit for displaying a bird's-eye view image generated by the superimposed image generation unit on the display unit. A video acquisition step to acquire a peripheral image of the surroundings of the vehicle,
An obstacle information acquisition step for acquiring obstacle information including a distance from the vehicle to the obstacle of an obstacle detected in the vicinity of the vehicle.
A bird's-eye view image generation step of generating a bird's-eye view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired in the image acquisition step.
If the distance to the detected obstacle is equal to or greater than a predetermined threshold value based on the obstacle information acquired in the obstacle information acquisition step, the information indicating the obstacle is provided outside the central portion surrounded by the bird's-eye view image. Generates a bird's-eye view image that is stretched and superimposed on the above, and if the distance to the detected obstacle is less than a predetermined threshold value, the information indicating the obstacle is superimposed on the central portion to generate a superimposed image. Steps and
A bird's-eye view image generation method in which a bird's-eye view image generator executes a display control step of displaying a bird's-eye view image generated in the superimposed image generation step on a display unit. A video acquisition step to acquire a peripheral image of the surroundings of the vehicle,
An obstacle information acquisition step for acquiring obstacle information including a distance from the vehicle to the obstacle of an obstacle detected in the vicinity of the vehicle.
A bird's-eye view image generation step of generating a bird's-eye view image obtained by performing viewpoint conversion processing so as to look down on the vehicle from above from the peripheral image acquired in the image acquisition step.
Based on the obstacle information acquired in the obstacle information acquisition step, if the detected distance to the obstacle is equal to or greater than a predetermined threshold value, a bird's-eye view image in which the information indicating the obstacle is superimposed on the bird's-eye view image is generated. If the detected distance to the obstacle is less than a predetermined threshold value, the superimposed image generation step of generating the bird's-eye view image in which the information indicating the obstacle is superimposed on the central portion surrounded by the bird's-eye view image, and
A bird's-eye view image generation method in which a bird's-eye view image generator executes a display control step of displaying a bird's-eye view image generated in the superimposed image generation step on a display unit.

Images