JP2021140689A - Drive assistance system, drive assistance device, drive assistance method, and drive assistance program - Google Patents

Abstract

To obtain a drive assistance system that can provide a captured image of a traveling area including a junction area to a driver of a vehicle, while reducing difficulty in installation of a camera.SOLUTION: A drive assistance system 100 comprises: a plurality of cameras 21, 22, and 23; and a drive assistance device 1. Each of the plurality of cameras 21, 22, and 23 images a traveling area including a first traveling route area, a second traveling route area, and an area where the first traveling route area and the second traveling route area merge. The drive assistance device 1 comprises: a conversion unit 21; a synthesis unit 22; and a display processing unit 23. The conversion unit 21 converts each of a plurality of images into an overhead view image. The synthesis unit 22 generates a synthesis image by synthesizing a plurality of overhead view images converted by the conversion unit 21. The display processing unit 23 causes each of display devices 31, 32, 33, and 34 to display the synthesis image generated by the synthesis unit 22 as an assisting image.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to a driving support system, a driving support device, a driving support method, and a driving support program that support the driving of a vehicle in a driving area including a confluence area of two driving paths.

Conventionally, there is known a technique for supporting the driving of a vehicle in a traveling area including a merging area where a plurality of traveling paths merge. For example, Patent Document 1 discloses a technique of capturing an image of each vehicle traveling on a road toward a confluence region with a camera and displaying the captured image of the camera on a display device. In the technique described in Patent Document 1, the future position of each vehicle is predicted, and based on the predicted result, the instruction information for causing the vehicle to execute the recommended driving method and the vehicle subject to the instruction information are emphasized. An image in which the image and the image are arranged on the captured image of the camera is displayed on the display device.

Japanese Unexamined Patent Publication No. 2015-52902

However, in the technique described in Patent Document 1, the traveling region is photographed vertically downward by the camera. Therefore, for example, on an expressway, when trying to capture an image of a traveling area including a main line, a side road, and a confluence of the main line and a side road, it is necessary to place the camera at a high position in the sky above the traveling area. , Camera installation may be difficult.

The present disclosure has been made in view of the above, and obtains a driving support system capable of providing a captured image of a traveling region including a merging region to a vehicle driver while reducing the difficulty of installing a camera. The purpose is.

In order to solve the above-mentioned problems and achieve the object, the driving support system of the present disclosure includes a plurality of cameras and a driving support device. The plurality of cameras cover different regions of the travel region including the region of the first travel path, the region of the second travel path, and the region where the first travel path and the second travel path meet. Take an image. The driving support device causes the display device to display a support image for supporting the driving of the vehicle in the traveling area based on a plurality of images captured by the plurality of cameras. The driving support device includes a conversion unit, a synthesis unit, and a display processing unit. The conversion unit converts each of the plurality of images into a bird's-eye view image. The compositing unit generates a composite image in which a plurality of bird's-eye view images converted by the conversion unit are combined. The display processing unit causes the display device to display the composite image generated by the composite unit as a support image.

According to the present disclosure, it is possible to provide the driver of the vehicle with an image of the traveling region including the confluence region while reducing the difficulty of installing the camera.

The figure for demonstrating the driving support method which concerns on Embodiment 1. The figure which shows an example of the structure of the driving support system which concerns on Embodiment 1. Top view of the first travel path, the second travel path, and the travel region including the confluence region according to the first embodiment. The figure which shows an example of each arrangement of a plurality of cameras and a plurality of display devices which concerns on Embodiment 1. The figure which shows an example of the structure of the camera which concerns on Embodiment 1. The figure which shows an example of the structure of the display device which concerns on Embodiment 1. The figure which shows the positional relationship of a plurality of bird's-eye view images with respect to the composite image which concerns on Embodiment 1. The figure which shows an example of the composite image displayed on the display device which concerns on Embodiment 1. The figure which shows an example of the composite image when the number and arrangement of the cameras concerning Embodiment 1 are different. The figure which shows an example of the composite image when the number and arrangement of the cameras concerning Embodiment 1 are further different. The figure which shows an example of coloring of the traveling area which concerns on Embodiment 1. The figure which shows an example of the image which looked forward from the driver of the vehicle traveling in the traveling area shown in FIG. A flowchart showing an example of processing by the processing unit of the driving support device according to the first embodiment. The figure which shows an example of the hardware configuration of the driving support device which concerns on Embodiment 1. The figure for demonstrating the driving support system which concerns on Embodiment 2. The figure which shows an example of the partial structure of the vehicle which concerns on Embodiment 2. The figure which shows an example of the structure of the operation support device which concerns on Embodiment 2. A flowchart showing an example of processing by the processing unit of the driving support device according to the second embodiment.

Hereinafter, the driving support system, the driving support device, the driving support method, and the driving support program according to the embodiment will be described in detail based on the drawings.

Embodiment 1.
FIG. 1 is a diagram for explaining a driving support method according to the first embodiment. As shown in FIG. 1, in the driving support method according to the first embodiment, the region of the first travel path, the region of the second travel path, and the first travel path and the second travel path merge. The traveling area including the confluence area, which is the area to be used, is imaged by a plurality of cameras. If the driving area is a highway, the first driving road is a part of the main lane, the second driving road is a frontage road, and the merging area includes the entire acceleration lane and a part of the main lane. Is done. The traveling area is not limited to the expressway.

In the example shown in FIG. 1, the first camera of the plurality of cameras images the area of the first travel path, and the second camera images the region of the second travel path, and the third camera. The confluence area is imaged by the camera of. The captured image, which is an image captured by each camera, also includes an image of a traveling vehicle, which is a vehicle traveling in the traveling region. The information of the captured images output from these plurality of cameras is the information of the moving image having a frame rate of 30 FPS (Flames Per Second), but the information of the moving image having a frame rate of less than 30 FPS may be used, and the frame may be used. It may be moving image information having a rate exceeding 30 FPS. In the following, a moving image may be described as an image, and image information may be described as an image.

In this driving support method, a conversion process for converting an image captured by each camera into a bird's-eye view image is performed by a driving support device (not shown). The bird's-eye view image is an image that has undergone viewpoint conversion processing so as to look down from above in the vertical direction. Then, in this driving support method, a driving support device (not shown) performs a compositing process of synthesizing a plurality of bird's-eye views images generated by the conversion process, and the generated composite image is used in the traveling region. As a support image for supporting the driving of the vehicle, a display process for displaying on a plurality of display devices is performed.

As described above, in this driving support method, the captured image, which is an image captured by each of the plurality of cameras, is converted into a bird's-eye view image by the driving support device (not shown), and the converted plurality of bird's-eye view images are combined and combined. The image is displayed on the display device. As a result, since it is not necessary to orient the camera vertically downward, it is possible to provide the driver of the vehicle with a captured image of the traveling region including the confluence region while reducing the difficulty of installing the camera. Therefore, for example, the driver of the vehicle traveling on the second traveling path can grasp the situation of the vehicle on the first traveling path, which is not actually visible, with a sense of reality.

In addition, when the traveling area is an expressway, surveillance cameras are often installed on each of the main road and the frontage road, and these existing surveillance cameras can also be used in this driving support method. .. In this case, since it is not necessary to newly install a camera, this driving support method can be easily applied.

Hereinafter, the configuration of the driving support system that executes this driving support method will be specifically described. FIG. 2 is a diagram showing an example of the configuration of the driving support system according to the first embodiment. As shown in FIG. 2, the driving support system 100 according to the first embodiment includes a driving support device 1, a plurality of cameras 2 ₁ , 2 ₂ , _23, and a plurality of display devices 3 ₁ , 3 ₂ , 3 _3. , ₃₄ , a server 4, and a terminal device 5.

The driving support device 1 is wirelessly or wiredly connected to each of the plurality of cameras 2 ₁ , 2 ₂ , 2 ₃ and the plurality of display devices 3 ₁ , 3 ₂ , 3 ₃ , 3 _{4 so} as to be able to communicate with each other. Driving support device 1 includes a plurality of cameras 2 _1, 2 _2, 2 _3, based on the plurality of images captured by the display device support image for supporting the operation of the vehicle by the driver in the running region 3 _1, 3 Display on ₂ , 3 ₃ , 3 _4.

The driving support device 1 is communicably connected to the server 4 via the gateway 6 and the network 7. In the example shown in FIG. 2, the server 4 is a cloud server provided on the cloud. The server 4 is communicably connected to the terminal device 5 of the road management company via the network 8.

FIG. 3 is a plan view of a traveling area including the first traveling path, the second traveling path, and the confluence region according to the first embodiment. As shown in FIG. 3, drive region 9 according to the first embodiment, ₁ and the first travel path 9, and the second running path 9 _2, first running path 9 ₁ and the second of the running path 9 and a confluence region 9 ₃ is an _{area 2} and are joined. If drive region 9 is the highway, the first running path 9 ₁ is part of the main line, the second running path 9 ₂ is a side road, the confluence region 9 _3, the acceleration lane Includes the entire and part of the main line.

FIG. 4 is a diagram showing an example of the arrangement of each of the plurality of cameras and the plurality of display devices according to the first embodiment. In the example shown in FIG. 4, the camera 2 ₁ is attached to the gate-shaped struts straddling the first running path 9 _1, in a position for imaging the first running path 9 ₁ from a first upper travel path 9 ₁ Be placed. Camera 2 ₂ is mounted on a gate-shaped struts straddling the second running path 9 _2, it is arranged at a position for imaging the second running path 9 ₂ from above the second traveling path 9 _2.

The camera 2 ₃ is attached to the gate-shaped struts straddling the confluence region 9 _3, it is disposed at a position for imaging the confluence region 9 ₃ from above the confluence region 9 _3. The method using a gantry support is also called an overhead type or a gantry type. Hereinafter, when each of the plurality of cameras 2 ₁ , 2 ₂ , and 2 ₃ is shown without distinction, it may be referred to as camera 2.

FIG. 5 is a diagram showing an example of the configuration of the camera according to the first embodiment. As shown in FIG. 5, the camera 2 includes an imaging unit 30, a communication unit 31, and a processing unit 32. The imaging unit 30 is, for example, an image sensor such as a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. The communication unit 31 transmits / receives information to / from the driving support device 1 wirelessly or by wire. The processing unit 32 causes the communication unit 31 to transmit information on the moving image captured by the imaging unit 30.

In the example shown in FIG. 4, the display device 3 ₁ is attached to the gate-shaped struts straddling the first running path 9 _1, arranged in a position visible from the driver of the vehicle traveling the first travel path 9 ₁ NS. Display 3 ₂ is attached to the gate-shaped struts straddling the second running path 9 _2, is arranged in a position visible from the driver of the vehicle traveling the second running path 9 _2.

Display device 3 ₃ is attached to the gate-shaped struts straddling the confluence region 9 _3, it is arranged in a position visible from the driver of the vehicle traveling on the confluence region 9 _3. Display device 3 ₄ is attached to the gate-shaped struts are provided at the inlet of the second running path 9 _2, the second running path 9 of a vehicle that is about to enter the _second inlet driver or the second running path 9 It is arranged at a position that can be visually recognized by a driver or the like of a vehicle traveling on the traveling road connected to _2. In this way, each of the display devices 3 ₁ , 3 ₂ , 3 ₃ , 3 ₄ is provided above the traveling region 9 at a position where the driver of the vehicle traveling in the traveling region 9 can see. Hereinafter, when each of the plurality of display devices 3 ₁ , 3 ₂ , 3 ₃ , 3 ₄ is shown without distinction, it may be referred to as display device 3.

FIG. 6 is a diagram showing an example of the configuration of the display device according to the first embodiment. As shown in FIG. 6, the display device 3 includes a communication unit 40, a display unit 41, and a processing unit 42. The communication unit 40 transmits / receives information to / from the driving support device 1 wirelessly or by wire. The display unit 41 is a large display unit, and is, for example, an LED vision that displays an image with a plurality of LEDs (Light Emitting Diodes). The processing unit 42 causes the display unit 41 to display the information of the support image, which is the information of the moving image from the driving support device 1 received by the communication unit 40.

Returning to FIG. 2, the description of the driving support device 1 will be continued. As shown in FIG. 2, the driving support device 1 includes a first communication unit 10, a second communication unit 11, and a processing unit 12. The first communication unit 10 is wirelessly or wiredly connected to each of a plurality of cameras 2 ₁ , 2 ₂ , 2 ₃ and display devices 3 ₁ , 3 ₂ , 3 ₃ , 3 _{4 so} as to be able to communicate with each other. The second communication unit 11 is communicably connected to the server 4 via the gateway 6 and the network 7.

The processing unit 12 includes an acquisition unit 20, a conversion unit 21, a synthesis unit 22, and a display processing unit 23. The acquisition unit 20 acquires the information of the captured image from each camera 2. The acquisition unit 20 causes the second communication unit 11 to transmit the acquired image information as monitoring image information to the server 4. The server 4 transmits the information of the captured image from each camera 2 acquired from the driving support device 1 to the terminal device 5 via the network 8. As a result, the captured image of each camera 2 is displayed on a display unit (not shown) in the terminal device 5, and the employee of the road management company can travel area based on the captured image of each camera 2 displayed on the terminal device 5. The state of 9 can be monitored.

Further, the acquisition unit 20 outputs the information of the acquired plurality of captured images to the conversion unit 21. The conversion unit 21 converts the captured image of each camera 2 into a bird's-eye view image based on the information of the plurality of captured images output from the acquisition unit 20. As described above, the bird's-eye view image is an image generated by performing a viewpoint conversion process so as to look down from above in the vertical direction. That is, the bird's-eye view image is an image obtained by setting the viewpoint position above the traveling region 9 and setting the viewpoint direction downward in the vertical direction and transforming the coordinates of the image of the camera 2. Incidentally, when the second running path 9 ₂ is slope, the second running path 9 ₂ of the overhead image may be an image looking down from a direction perpendicular to the second running path 9 _2.

The compositing unit 22 generates a composite image in which a plurality of bird's-eye view images converted by the conversion unit 21 are combined. FIG. 7 is a diagram showing the positional relationship of a plurality of bird's-eye view images with respect to the composite image according to the first embodiment. In the composite image 50 shown in FIG. 7, the first image 51 is a bird's-eye view image obtained by the conversion unit 21 from the camera 2 ₁ captured image, the second image 52, the conversion unit from the camera 2 ₂ captured image a bird's-eye view image obtained at 21, the third image 53 is an overhead image obtained by the conversion unit 21 from the camera 2 ₃ captured images.

Each of the first image 51, the second image 52, and the third image 53 is an image of different regions of the traveling region 9. Specifically, the first image 51 is _{an image of the region of the first} travel path 91, and the second image 52 is _{an image of the region of the second} travel path 92, which is a third image. image 53 is an image of the confluence region _{9 3.}

When the camera _{2 3} captures an image region overlapping the camera ₂ 1, _{2 2,} the overhead view image from the camera _{2 3} captured images obtained by the conversion unit 21, the camera ₂ 1, _{2 2} of the converter 21 from the captured image The overlapping part with the obtained bird's-eye view image is deleted. Also, when the camera 2 ₁ images the region overlapping the camera 2 _2, for example, from the bird's-eye view image from the camera 2 ₁ of the captured image obtained by the conversion unit 21, obtained by the conversion unit 21 from the camera 2 ₂ captured image The overlapping part with the bird's-eye view image is deleted. The deletion of the overlapping portion is executed by, for example, the conversion unit 21 or the synthesis unit 22.

The compositing unit 22 can also generate, for example, a composite image obtained by compressing a composite image obtained by combining a plurality of bird's-eye view images in the traveling direction of the vehicle as a composite image.

The display processing unit 23 causes a plurality of display devices 3 ₁ , 3 ₂ , 3 ₃ , 3 ₄ to transmit the information of the composite image generated by the composite unit 22 to the first communication unit 10, and is generated by the composite unit 22. The composite image is displayed as a support image on the display devices 3 ₁ , 3 ₂ , 3 ₃ , 3 ₄ .

FIG. 8 is a diagram showing an example of a composite image displayed on the display device according to the first embodiment. The composite image 50 shown in FIG. 8 is an image of a plurality of vehicles traveling in the traveling region 9 viewed from above. Such composite image 50, display device 3 _1, 3 2, _{3 3,} 3 ₄ be displayed by the vehicle driver traveling on the travel area 9, the running state of other vehicles in the travel area 9 composite image It can be easily grasped from 50.

For example, if the second vehicle running path 9 ₂ is traveling, from where confluence region 9 of what to put merge into the _third main line more reasonable judgment of the acceleration lane in the combined area 9 _3, the second it is possible to prompt the vehicle driver traveling on the traveling path 9 _2. Further, the driver of the vehicle traveling the first travel path 9 ₁ can grasp a vehicle approaching from the second running path 9 ₂ the joining regions 9 ₃ in advance, from the second of the running path 9 ₂ vehicle confluence region 9 ₃ can travel first traveling path 9 ₁ for easy entry. The display device 3 _4, for example, to be placed at the entrance to the side road leading to the highway, it is possible to reduce the vehicle is stuck in traffic. Therefore, in the driving support system 100, it is possible to expect early elimination of traffic congestion.

The number and arrangement of the cameras 2 are not limited to the above-mentioned example. FIG. 9 is a diagram showing an example of a composite image when the number and arrangement of cameras according to the first embodiment are different. FIG. 10 is a diagram showing an example of a composite image when the number and arrangement of cameras according to the first embodiment are further different.

In the example shown in FIG. 9, the composite image 50 is generated from the captured images of the four cameras 2. Specifically, the composite image 50 is composed of a first image, a second image, a third image, and a fourth image. The first image is an image obtained from the camera 2 _1, second image is an image obtained from the camera 2 _2. Also, each of the third image and the fourth image is obtained from an image captured by the two different camera 2 and camera 2 _1, 2 _2.

In the example shown in FIG. 10, the composite image 50 is generated from the captured images of the two cameras 2. Specifically, the composite image 50 is composed of a first image and a second image. The first image is an image obtained from the first camera 2, and the second image is an image obtained from the second camera 2. The first camera 2 images an area of the traveling area 9 including the first traveling path 9 ₁ and a part of the merging area 93 ₃ , and the second camera 2 is the third traveling area 9. An image is taken of a region including a traveling path 9 _{2 of 2} and a part of the merging region 93 _3. The number of cameras 2 may be five or more, and the arrangement of the cameras 2 is not limited to the above-mentioned example.

Each of the plurality of regions divided in the direction in which the vehicle travels is colored differently in the traveling region 9, so that the driver of the vehicle traveling in the traveling region 9 can use the region in which the vehicle is traveling. From the color of the above and the composite image 50 displayed on the display device 3, it is possible to easily grasp the position where the vehicle on board is traveling among the plurality of vehicles included in the composite image 50.

FIG. 11 is a diagram showing an example of coloring the traveling region according to the first embodiment. In the example shown in FIG. 11, each of a plurality of regions of the traveling regions 9 that are colored differently from each other is indicated by different hatching. That is, in FIG. 11, the difference in color is expressed by hatching.

In the example shown in FIG. 11, each of the plurality of regions of the traveling region 9 is colored differently. For example, the first running path 9 ₁ area 91 and the second running path 9 ₂ region 93 is painted in yellow, and the first running path 9 ₁ region 92 second running path 9 ₂ regions 94 and is painted with orange confluence region 9 ₃ are painted blue.

FIG. 12 is a diagram showing an example of an image looking forward from the driver of the vehicle traveling in the traveling region shown in FIG. Example shown in FIG. 12 is an image viewed from a vehicle traveling in the first travel path 9 ₁ area 91, the other vehicle is present in the first running path 9 ₁ area 92. First running path 9 ₁ of the vehicle traveling on the region 91 driver, the color of the area where the vehicle is traveling, from the displayed synthesized image 50. on the display unit 3 is included in the composite image 50 It is possible to easily grasp the position where the vehicle on board is traveling among a plurality of vehicles.

Subsequently, the processing by the processing unit 12 of the driving support device 1 will be described with reference to the flowchart. FIG. 13 is a flowchart showing an example of processing by the processing unit of the driving support device according to the first embodiment. The process shown in FIG. 13 is repeatedly executed by the processing unit 12 of the driving support device 1. For example, the processing unit 12 executes the process shown in FIG. 13 every time the information of the image of one frame is acquired from each camera 2.

As shown in FIG. 13, the processing unit 12 of the driving support device 1 acquires captured images from each of the plurality of cameras 2 (step S10). The processing unit 12 transmits the plurality of captured images acquired in step S10 to the server 4 (step S11), and converts each of the plurality of captured images acquired in step S10 into a bird's-eye view image (step S12).

Next, the processing unit 12 synthesizes a plurality of bird's-eye view images converted in step S12 to generate a composite image (step S13). The processing unit 12 displays the composite image generated in step S13 as a support image on each display device 3 (step S14), and causes the second communication unit 11 to transmit the composite image generated in step S13 to the server 4 (step S14). Step S15), the process shown in FIG. 13 is terminated. The processing unit 12 may not perform the processing of step S15.

FIG. 14 is a diagram showing an example of the hardware configuration of the driving support device according to the first embodiment. As shown in FIG. 14, the driving support device 1 includes a computer including a processor 101, a memory 102, a first communication device 103, and a second communication device 104. The processor 101, the memory 102, the first communication device 103, and the second communication device 104 can send and receive data to and from each other by, for example, the bus 105. The first communication unit 10 is realized by the first communication device 103, and the second communication unit 11 is realized by the second communication device 104. The processor 101 executes the functions of the acquisition unit 20, the conversion unit 21, the synthesis unit 22, and the display processing unit 23 by reading and executing the program stored in the memory 102.

The processor 101 is, for example, an example of a processing circuit, and includes one or more of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a system LSI (Large Scale Integration). The memory 102 includes one or more of RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (registered trademark) (Electrically Erasable Programmable Read Only Memory). include. Further, the memory 102 includes a recording medium on which the above-mentioned program that can be read by a computer is recorded. Such recording media include one or more of non-volatile or volatile semiconductor memories, magnetic disks, flexible memories, optical disks, compact discs, and DVDs (Digital Versatile Discs). The processing unit 12 of the driving support device 1 may include integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array).

As described above, the driving support system 100 according to the first embodiment includes a plurality of cameras 2 and a driving support device 1. A plurality of cameras 2 is the region where the first running path 9 ₁ region, and the second running path 9 ₂ regions, the first running path 9 ₁ and the ₂ second running path 9 are merged Different regions of the traveling region 9 including the confluence region 9 _{3 are imaged.} The driving support device 1 causes the display device 3 to display a support image for supporting the driving of the vehicle in the traveling area 9 based on a plurality of images captured by the plurality of cameras 2. The driving support device 1 includes a conversion unit 21, a synthesis unit 22, and a display processing unit 23. The conversion unit 21 converts each of the plurality of images captured by the plurality of cameras 2 into a bird's-eye view image. The compositing unit 22 generates a composite image in which a plurality of bird's-eye view images converted by the conversion unit 21 are combined. The display processing unit 23 causes the display device 3 to display the composite image generated by the composite unit 22 as a support image. Thus, it is not necessary to the orientation of the camera 2 vertically downward, while reducing the difficulty of the installation the camera 2, to provide a captured image of the travel area 9 including a confluence region 9 ₃ to the driver of the vehicle Can be done. Therefore, for example, the driver of the vehicle traveling on the _second travel path 92 can grasp the situation of the vehicle on the _{first travel path 91, which is not actually visible, with a sense of reality.}

Further, in the traveling region 9, each of the plurality of regions divided in the traveling direction of the vehicle is colored differently. As a result, the driver of the vehicle traveling in the traveling region 9 can board the plurality of vehicles included in the composite image 50 from the color of the region in which the vehicle is traveling and the composite image 50 displayed on the display device 3. It is possible to easily grasp the position where the vehicle is traveling.

Further, the display device 3 is provided at a position above the traveling region 9 and visible to the driver of the vehicle traveling in the traveling region 9. As a result, the driver of the vehicle traveling in the traveling region 9 can easily grasp the state of the traveling region 9.

Further, the driving support device 1 includes a second communication unit 11 that transmits the information of the image captured by each of the plurality of cameras 2 to the server 4 as the information of the monitoring image. The server 4 is an example of an external device, and the second communication unit 11 is an example of a communication unit. In the driving support system 100, the existing surveillance camera can be used as the camera 2, and it is not necessary to newly install the camera, so that the system construction is easy.

Embodiment 2.
In the driving support system according to the second embodiment, a composite image is also displayed on the display device of the vehicle, the traveling area is colored by the projection device, and the road condition is determined from the image captured by the camera and displayed on the display device. In that respect, it differs from the driving support system 100 according to the first embodiment. In the following, the components having the same functions as those of the first embodiment will be designated by the same reference numerals and the description thereof will be omitted, and the differences from the driving support system 100 of the first embodiment will be mainly described.

FIG. 15 is a diagram for explaining the driving support system according to the second embodiment. As shown in FIG. 15, the driving support system 100A according to the second embodiment includes a driving support device 1A, a plurality of cameras 2 ₁ , 2 ₂ , _23, and a plurality of display devices 3 ₁ , 3 ₂ , 3 ₃ comprises a _{3 4,} and the projection device _{70 1,} 70 2, 70 _3, and the vehicle 60. Note that in FIG. 15, the server 4 and the terminal device 5 and the like are not shown. Although only two vehicles 60 are shown in FIG. 15, the driving support system 100A includes three or more vehicles 60.

The vehicle 60 displays the composite image based on the information of the composite image transmitted from the driving support device 1A. FIG. 16 is a diagram showing an example of a partial configuration of the vehicle according to the second embodiment. As shown in FIG. 16, the vehicle 60 includes a communication unit 61, a display unit 62, and a processing unit 63.

The communication unit 61 wirelessly transmits and receives information to and from the driving support device 1A via the network 7 and the like. The display unit 62 is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The processing unit 63 causes the display unit 62 to display the information of the composite image transmitted from the driving support device 1A and received by the communication unit 61. As a result, the driver of the vehicle 60 can also drive the vehicle 60 by the composite image displayed in the vehicle 60.

The projection device 70 _1, 70 _2, 70 _3, based on a control command transmitted from the driving support device 1A, different to each of the plurality of regions divided in the direction in which the vehicle 60 travels out of the drive area 9 Project color. The control command, for example, includes information of an image to be respectively projected onto the projection device _{70 1,} 70 2, 70 _3.

Projection device 70 _1, for example, by projecting the yellow image of the first traveling path 9 _first region 91 shown in FIG. 11, for projecting an image of the orange first running path 9 ₁ area 92. The projection device 70 _2, the yellow image is projected onto the second traveling path 9 _second region 93 shown in FIG. 11, for projecting an image of the orange to the second running path 9 ₂ region 94. The projection device 70 ₃ projects the blue image in the region of the joining region 9 ₃ shown in FIG. 11. Hereinafter, the case shown each projection device _{70 1,} 70 2, 70 ₃ to not separately distinguished, may be referred to as the projection device 70.

As a result, for example, the driver of the vehicle 60 is traveling on the vehicle 60 among the plurality of vehicles 60 included in the composite image 50 even when the traveling area 9 is traveled at night. The position can be easily grasped.

FIG. 17 is a diagram showing an example of the configuration of the driving support device according to the second embodiment. As shown in FIG. 17, the operation support device 1A according to the second embodiment includes a first communication unit 10A, a second communication unit 11A, and a processing unit 12A.

The first communication unit 10A, a plurality of cameras _{2 1,} 2 2, _{2 3} and a plurality of display devices _{3 1,} 3 _2, 3 3, ₃ in addition to the _4, the projection device ₇₀ 1 via wireless or wired, ₇₀ 2, 70 in each of the ₃ points that are communicatively connected to each other, different from the first communication unit 10.

The second communication unit 11A is communicably connected to the server 4 via the gateway 6 and the network 7, and is communicably connected to the communication unit 61 of the vehicle 60 via the gateway 6 and the network 7. , Different from the second communication unit 11.

The processing unit 12A is different from the processing unit 12 in that the display processing unit 23A is provided in place of the display processing unit 23, and the analysis unit 24 and the control processing unit 25 are further provided. The display processing unit 23A further has a function of causing the first communication unit 10A to transmit the composite image generated by the synthesis unit 22 to the vehicle 60 and display the information of the composite image on the display unit 62 of the vehicle 60. It is different from the display processing unit 23.

The analysis unit 24 analyzes the traveling state of the vehicle 60 in the traveling region 9 based on the plurality of captured images acquired by the acquisition unit 20. The analysis unit 24 extracts each vehicle 60 from a plurality of captured images, assigns a vehicle ID (Identifier) to each extracted vehicle 60, and manages the state of each vehicle 60 by the vehicle ID. The analysis unit 24 calculates the speed of each of the extracted vehicles 60, and calculates the average speed of the plurality of vehicles 60. Further, the analysis unit 24 counts the number of vehicles 60 in the traveling region 9 and calculates the counting result as the number of passing vehicles. The analysis unit 24 determines whether or not the traveling area 9 is congested based on the calculated average speed and the number of passing vehicles.

Further, the analysis unit 24 determines whether or not there is a vehicle 60 traveling in the reverse direction in the traveling region 9 based on the plurality of captured images acquired by the acquisition unit 20, and determines whether or not there is a vehicle 60 traveling in the reverse direction, or the first traveling path 9 ₁ It is possible to determine whether or not the vehicle 60 traveling in the vehicle is evading. The analysis unit 24 causes the second communication unit 11A to transmit the analysis information indicating the analysis result to the server 4. The analysis information includes, for example, information indicating average speed, congestion, escape, reverse driving, and number of vehicles passing.

The display processing unit 23A can cause the first communication unit 10A to transmit the highly urgent information among the analysis results of the analysis unit 24 to the display device 3, and cause the display device 3 to display the highly urgent information. Information with high urgency is, for example, information such as reverse driving. In this way, the display processing unit 23A displays the highly urgent information among the analysis information determined in real time on the display device 3, so that the driver of the vehicle 60 can feel the current road condition with a stronger sense of presence. It can be grasped.

The display processing unit 23A causes the display device 3 to display an image in which highly urgent information among the analysis results of the analysis unit 24 is superimposed on the composite image generated by the synthesis unit 22 as a support image. The analysis unit 24 can also cause the first communication unit 10A to transmit the highly urgent information among the analysis information to the vehicle 60 and display it on the display unit 62 of the vehicle 60. Further, the display processing unit 23A can cause the first communication unit 10A to transmit all the analysis results of the analysis unit 24 to the display device 3, and cause the display device 3 to display all the analysis results of the analysis unit 24. In this case as well, the display processing unit 23A can cause the display device 3 to display an image in which all the analysis results of the analysis unit 24 are superimposed on the composite image generated by the synthesis unit 22.

Control processing unit 25, a control instruction of the control command including information of an image to be projected by the projection device 70 ₁ is transmitted first to the projection device 70 ₁ to the communication unit 10A, including the information of an image to be projected by the projection device 70 ₂ It is allowed to transmit to the projection device 70 ₂ to the first communication unit 10A. The control processor 25 to transmit the control command including information of an image to be projected by the projection device 70 ₃ to the projection device 70 ₃ to the first communication unit 10A. Projection device _{70 1,} 70 2, 70 _3, projects an image included in the control command transmitted from the driving support device 1A to the traveling region 9.

Image to be projected by the projection device _{70 1,} 70 2, 70 ₃ may be varied. For example, the control processor 25, based on the running condition of the vehicle 60 in the traveling region 9 analyzed by the analysis section 24 generates a control command for changing the image to be projected by the projection device 70 _1, 70 _2, 70 ₃ be able to.

For example, the control processing unit 25 can shorten the interval at which images of different colors are projected in the direction in which the vehicle 60 travels, as the traveling speed of the vehicle 60 analyzed by the analysis unit 24 is slower. Further, the control processing unit 25 can shorten the interval at which images of different colors are projected in the direction in which the vehicle 60 travels as the degree of congestion of the vehicle 60 in the traveling region 9 analyzed by the analysis unit 24 increases. ..

Further, the control processing unit 25 sends a control command including an image that does not project to the region where each vehicle 60 exists to the first communication unit 10A based on the position of each vehicle 60 analyzed by the analysis unit 24. It can be transmitted to the projection device 70. This makes it possible to prevent the light from the projection device 70 from directly hitting the driver of the vehicle 60. The projection device 70 can also project an image from a position lower than the height of the driver's head of the vehicle 60 toward the traveling region 9. This also prevents the light from the projection device 70 from directly hitting the driver of the vehicle 60. The projection device 70 may be arranged so that the projection direction is directed in the traveling direction of the vehicle 60 so as to project from the rear side of the vehicle 60. By projecting from the rear side of the vehicle 60 by the projection device 70 in this way, it is possible to prevent the light from the projection device 70 from hitting from the front of the driver of the vehicle 60.

The control processor 25, for example, in the case the number of vehicles 60 is preset number or more in the running area 9, the control command the projection apparatus to the first communication unit _{10A 70 1, 70 2, 70} 3 Can be sent to. Thus, the driving support system 100A, when the number of vehicles 60 is preset number or more in the running area 9, that projects an image in the drive area 9 from the projection device _{70 1,} 70 2, 70 ₃ can.

The control processor 25 may be, for example, current time to transmit the control command when a predetermined time period to the first projection device 70 1 to the communication unit 10A _of, 70 _2, 70 _3. Accordingly, the driving support system 100A, for example, can be restricted to the time zone such as night, to project an image to the running area 9 from the projection device _{70 1,} 70 2, 70 _3.

Subsequently, the processing by the processing unit 12A of the driving support device 1A will be described with reference to the flowchart. FIG. 18 is a flowchart showing an example of processing by the processing unit of the driving support device according to the second embodiment. The process shown in FIG. 18 is repeatedly executed by the processing unit 12A of the driving support device 1A. Since the processing of steps S20 to S25 shown in FIG. 18 is the same as that of steps S10 to S15 shown in FIG. 13, the description thereof will be omitted.

As shown in FIG. 18, the processing unit 12A of the driving support device 1A extracts each vehicle 60 from the captured images output from each camera 2 (step S26). The processing unit 12A assigns a vehicle ID to each vehicle 60 extracted in step S26 (step S27). As a result, the processing unit 12A can manage each of the plurality of vehicles 60 whose captured image includes the image.

The processing unit 12A analyzes the traveling state of the vehicle 60 in the traveling region 9 based on the speed and position of each of the extracted vehicles 60 in the traveling region 9 (step S28). In the process of step S28, the processing unit 12A analyzes, for example, the average speed, the traffic jam, the avoidance, the reverse driving, and the number of passing vehicles in the traveling area 9.

The processing unit 12A causes the first communication unit 10A to transmit the analyzed information indicating the traveling status of the vehicle 60 to each display device 3, and causes each display device 3 to display the information indicating the traveling status of the vehicle 60 (step S29). ). For example, the processing unit 12A causes each display device 3 to display an image in which information indicating the traveling status of the vehicle 60 is superimposed on the composite image generated by the synthesis unit 22 as a support image. Further, the processing unit 12A causes the server 4 to transmit the information indicating the traveling status of the analyzed vehicle 60 to the second communication unit 11A (step S30).

Next, the processing unit 12A colors the traveling region 9 based on the traveling condition of the analyzed vehicle 60 (step S31). When the process of step S31 is completed, the processing unit 12A ends the process shown in FIG.

The hardware configuration example of the driving support device 1A according to the second embodiment is the same as the hardware configuration of the driving support device 1 shown in FIG. The processor 101 executes the functions of the acquisition unit 20, the conversion unit 21, the synthesis unit 22, the display processing unit 23A, the analysis unit 24, and the control processing unit 25 by reading and executing the program stored in the memory 102. be able to.

As described above, the driving support device 1A according to the second embodiment, a plurality of projection device 70 ₁ for projecting a different color for each of a plurality of regions divided in the direction in which the vehicle 60 is traveling within the travel area 9, It includes 70 ₂ , 70 ₃ . As a result, for example, the driver of the vehicle 60 is traveling on the vehicle 60 among the plurality of vehicles 60 included in the composite image 50 even when the traveling area 9 is traveled at night. The position can be easily grasped.

Further, the display unit 62 is provided on the vehicle 60. The display unit 62 is an example of a display device. The display processing unit 23A causes the display unit 62 to display the composite image generated by the composite unit 22. As a result, the driver of the vehicle 60 traveling in the traveling region 9 can easily grasp the state of the traveling region 9 based on the image displayed on the display unit 62.

Further, the driving support device 1A includes an analysis unit 24 that analyzes the traveling state of the vehicle 60 in the traveling region 9 based on a plurality of images captured by the plurality of cameras 2. The display processing unit 23A, the display device ₃ 1 the information indicating the running condition of the vehicle 60 that has been analyzed by the analyzer _24, 3 _2, 3 3, _{3 4} To display. As a result, the driver of the vehicle 60 traveling in the traveling region 9 can more easily grasp the traveling condition of the vehicle 60.

Further, the display processing unit 23A causes the display device 3 to display an image obtained by superimposing the information indicating the traveling state of the vehicle 60 analyzed by the analysis unit 24 on the composite image generated by the composition unit 22 as a support image. As a result, the driver of the vehicle 60 can more specifically grasp the state of the traveling region 9 from the image of the display device 3.

The configuration shown in the above embodiments is an example, and can be combined with another known technique, can be combined with each other, and does not deviate from the gist. It is also possible to omit or change a part of the configuration.

1,1A driving support _device, 2,2 _1, 2 2, _{2 3 camera,} 3,3 _1, 3 _2, 3 3, _{3 4} display device, 4 server, 5 terminals, 6 gateways, 7,8 network, 9 Traveling area, 9 ₁ 1st driving path, 9 ₂ 2nd driving path, 9 ₃ Confluence area, 10, 10A 1st communication unit, 11, 11A 2nd communication unit, 12, 12A, 32, 42 , 63 Processing unit, 20 Acquisition unit, 21 Conversion unit, 22 Synthesis unit, 23, 23A Display processing unit, 24 Analysis unit, 25 Control processing unit, 30 Imaging unit, 31, 40, 61 Communication unit, 41, 62 Display unit , 50 the composite image, 51 the first image, 52 the second image, 53 the third image, 60 _vehicle, 70, 70 _1, 70 2, 70 ₃ projecting device, 91 to 94 regions, 100, 100A driving support system.

Claims (11)

A plurality of images of different regions of the travel region including the region of the first travel path, the region of the second travel path, and the region where the first travel path and the second travel path meet. Camera and
A driving support device for displaying a support image for supporting the driving of a vehicle in the traveling area on a display device based on a plurality of images captured by the plurality of cameras is provided.
The driving support device is
A conversion unit that converts each of the plurality of images into a bird's-eye view image,
A compositing unit that generates a composite image that combines a plurality of the bird's-eye view images converted by the conversion unit, and a compositing unit.
A driving support system including a display processing unit that displays the composite image generated by the synthesis unit on the display device as the support image. In the traveling area,
The driver assistance system according to claim 1, wherein each of the plurality of regions divided in the direction in which the vehicle travels is colored differently. The driving support system according to claim 1, further comprising a plurality of projection devices that project different colors on each of the plurality of regions of the traveling region divided in the direction in which the vehicle travels. The display device is
The driving support system according to any one of claims 1 to 3, wherein the driver of the vehicle traveling in the traveling area is provided at a position visible above the traveling area. The display device is
Provided on the vehicle
The display processing unit
The driver assistance system according to any one of claims 1 to 3, wherein the composite image generated by the synthesis unit is displayed on the display device. An analysis unit for analyzing the traveling state of the vehicle in the traveling region based on the plurality of images is provided.
The display processing unit
The driving support system according to any one of claims 1 to 5, wherein the display device displays information indicating the traveling situation analyzed by the analysis unit. The display processing unit
The sixth aspect of claim 6, wherein an image in which information indicating the traveling situation analyzed by the analysis unit is superimposed on the composite image generated by the synthesis unit is displayed as the support image on the display device. Driving support system. The driving support device is
The driver assistance system according to any one of claims 1 to 7, further comprising a communication unit that transmits information of an image captured by each of the plurality of cameras to an external device as information of a monitoring image. .. A plurality of images of different regions of the travel region including the region of the first travel path, the region of the second travel path, and the region where the first travel path and the second travel path meet. A conversion unit that converts multiple images captured by the camera into a bird's-eye view image,
A compositing unit that generates a composite image that combines a plurality of the bird's-eye view images converted by the conversion unit, and a compositing unit.
A driving support device including a display processing unit that displays the composite image generated by the synthesis unit on a display device as a support image for supporting the driving of a vehicle in the traveling region. It is a driving support method executed by a computer.
A plurality of images of different regions of the travel region including the region of the first travel path, the region of the second travel path, and the region where the first travel path and the second travel path meet. The first step of converting each of the multiple images captured by the camera into a bird's-eye view image,
A second step of generating a composite image by synthesizing the plurality of the bird's-eye view images converted by the first step, and
A driving support method comprising a third step of displaying the composite image generated by the second step on a display device as a support image for supporting the driving of a vehicle in the traveling region. A plurality of images of different regions of the travel region including the region of the first travel path, the region of the second travel path, and the region where the first travel path and the second travel path meet. The first step of converting each of the multiple images captured by the camera into a bird's-eye view image,
A second step of generating a composite image by synthesizing the plurality of the bird's-eye view images converted by the first step, and
The operation is characterized in that the computer executes the third step of displaying the composite image generated by the second step on the display device as a support image for supporting the driving of the vehicle in the traveling region. Support program.

Images