JP2023102660A - Drive support device, drive support method and drive support program - Google Patents

Abstract

To guide a return path without using a camera (back camera) that images the rear side in the travel direction.SOLUTION: There is provided a drive support device which guides a return path on the basis of a forward path image captured in a forward path. The drive support device comprises: a position information acquisition unit which acquires position information including the position and the travel direction of a vehicle; an imaging unit which images the front side of the vehicle to acquire a forward path image and a return path image; an image generation unit which generates an image for return path guide used for the guide of the return path on the basis of the inverted image obtained by inverting left and right of the forward path image; a storage unit which stores the position information, the forward path image and the image for return path guide in association with each other; a determination unit which determines whether or not the vehicle is traveling in the correct path in the return path on the basis of the position information, the image for return path guide and the return path image; and a notification unit which outputs the notification when it is determined that the vehicle is not traveling in the correct path in the return path.SELECTED DRAWING: Figure 3

Description

The present invention relates to a driving assistance device, a driving assistance method, and a driving assistance program.

Japanese Patent Application Laid-Open No. 2002-201002 discloses a navigation device that can use guidance image data captured by a capturing device that captures an image of the rear side in the direction of travel when providing guidance on a return route.

JP 2011-169651 A

However, the device described in Patent Literature 1 requires a camera (rear camera) for photographing the rear in the direction of travel, and cannot be applied to vehicles that do not have a rear camera.

A four-wheeled vehicle is generally equipped with a camera for photographing the forward direction in the direction of travel, but is often not equipped with a back camera. In particular, most motorcycles are not provided with a back camera. Therefore, the invention described in Patent Document 1 cannot be applied to these vehicles.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a driving assistance device, a driving assistance method, and a driving assistance program that can provide guidance on a return route without using a back camera.

In order to solve the above-described problems, a driving assistance device according to the present invention is, for example, a driving assistance device that performs return route guidance based on an outward route image captured on an outward route, and includes a position information acquisition unit that acquires position information including the position and traveling direction of a vehicle, an imaging unit that acquires an image in front of the vehicle and acquires the outward route image and the return route image that is captured during the return route, and generates a return route guidance image used for the return route guidance based on the inverted image obtained by horizontally reversing the outward route image. a storage unit that associates and stores the position information, the forward route image, and the return route guidance image; a determination unit that determines whether or not the vehicle is traveling the correct route on the return route based on the position information, the return route guidance image, and the return route image;

A driving assistance method according to another aspect of the present invention is, for example, a driving assistance method for providing return route guidance based on an outward route image captured on an outward route by an imaging unit that captures an image of the front of the vehicle. obtaining a return route image captured by the imaging unit on the return route; determining whether or not the vehicle is traveling on the correct route on the return route based on the position information, the return route guidance image, and the return route image; and outputting a notification when it is determined that the vehicle is not traveling on the correct route on the return route.

A driving assistance program according to another aspect of the present invention is, for example, a driving assistance program that performs return route guidance based on an outward route image captured by an imaging unit that captures an image of the front of the vehicle on an outward route, the computer being a position information acquisition unit that acquires position information including the position and traveling direction of the vehicle; a determination unit that determines whether the vehicle is traveling the correct route on the return trip based on the position information, the return route guidance image, and the return route image captured by the imaging unit during the return route; and a notification unit that outputs a notification when it is determined that the vehicle is not traveling on the correct route on the return route.
The computer program can be provided by downloading via a network such as the Internet, or can be provided by being recorded on various computer-readable recording media such as a CD-ROM.

In any one of the above aspects of the present invention, it is determined whether or not the vehicle is traveling the correct route on the return trip based on the return route guidance image created based on the outward route image and the return route image. As a result, it is possible to guide the return route without using the back camera.

The image generation unit may extract structures from the reversed image, reduce a first structure located near the vehicle in the reversed image among the extracted structures and place it near the center of the reversed image, and enlarge a second structure farther from the vehicle in the reversed image among the extracted structures and place it near the edge of the reversed image to generate the return route guidance image. As a result, the returning route guidance image can be made closer to the returning route image, and the accuracy of determining whether or not the vehicle is traveling on the correct route on the returning route can be improved.

The determination unit may extract structures from the return route guidance image and the return route image, and determine that the vehicle is traveling on the correct route when a third structure is extracted from the return route guidance image and the return route image, respectively. This makes it possible to easily determine whether or not the return route image and the return route guidance image are of the same scenery.

The determination unit may determine that the vehicle is traveling on the correct route based on the position of the third structure in the return route guidance image and the position of the third structure in the return route image. As a result, it is possible to improve the accuracy of determining whether or not the vehicle is traveling on the correct route on the return trip.

When the position on the return route is the first position, the determination unit may make the determination using a return route guidance image associated with a second time that is a predetermined time earlier than a first time at which the first position is passed on the outbound route, among the return route guidance images stored in the storage unit. As a result, the same structure can be included in the outward route image and the return route image, which are the basis of the return route guidance image. Therefore, it is possible to improve the accuracy of determining whether the vehicle is traveling on the correct route on the return trip.

An output unit including a speaker or a display unit provided in the vehicle may be provided, and the notification unit may output a notification by sound or text information from the output unit when the determination unit determines that the return route is not correct. As a result, it is possible to provide driving assistance for the correct route to the driver on the return trip.

A speed acquisition unit that acquires a vehicle speed that is the speed of the vehicle may be provided, and the position information acquisition unit may calculate the position based on the vehicle speed. As a result, even if the position information cannot be acquired, it is possible to guide the return route using only the image captured in the forward direction.

According to the present invention, return route guidance can be performed without using a camera that captures the rear in the direction of travel.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram schematically showing an outline of a vehicle C to be assisted by a driving assistance device 1; 1 is a block diagram schematically showing an example of a hardware configuration of a driving assistance device 1; FIG. 2 is a diagram showing an outline of electrical functional blocks of the driving assistance device 1; FIG. It is a figure which shows an example of data for return navigation. FIG. 10 is a flow chart showing a flow of processing for acquiring an outward route image and generating a return route navigation image; FIG. (A) is an example of an outward route image, (B) is an outward route image in which only the road portion is deleted, and (C) is an example of an inverted image. (A) is an example of dividing a reversed image into band-shaped images, (B) is an example of rearranging the band-shaped images, and (C) is an example of a return navigation image. 10 is a flowchart showing a flow of processing in which the driving assistance device 1 provides guidance on a return trip using a return navigation image. It is a figure explaining acquisition of the image for return navigation. (A) is an example of an inward travel image, and (B) is an example of an inward travel image for comparison. (A) is an example of a return route navigation image, and (B) is an example of a return route image for comparison. 2 is a diagram showing an outline of electrical functional blocks of the driving assistance device 2; FIG. (A) is an example of a return navigation image (reversed image), and (B) is an example of a return route image for comparison. 3 is a diagram showing an outline of electrical functional blocks of the driving support device 3; FIG. FIG. 4 is a diagram showing an example of return route navigation data in which position information, an outward route image, and a return route navigation image are associated; FIG. 10 is a flow chart showing a flow of processing for acquiring an outward route image and generating a return route navigation image; FIG. 10 is a flowchart showing a flow of processing in which the driving support device 3 performs guidance on a return trip using a return navigation image.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of a driving assistance device according to the present invention will be described in detail below with reference to the drawings. INDUSTRIAL APPLICABILITY The present invention is a driving support device capable of taking an image of the forward direction in the direction of travel on an outward trip and using the image to provide guidance on the return trip. Vehicles in the present invention include four-wheeled vehicles, motorcycles, bicycles, and the like.
<First embodiment>
FIG. 1 is a diagram schematically showing an outline of a vehicle C to be assisted by a driving assistance device 1 according to the first embodiment. A driving support device 1 is mounted on a vehicle C. As shown in FIG. The vehicle C is provided with an imaging unit 14 that captures an image of the front of the vehicle C (see the two-dot chain line in FIG. 2).

FIG. 2 is a block diagram schematically showing an example of the hardware configuration of the driving assistance device 1. As shown in FIG. The driving assistance device 1 includes a control device 50 , a communication device 52 and a storage device 54 . The control device 50 mainly includes a CPU (Central Processing Unit) 56 and a memory 58 .

In the control device 50, the CPU executes predetermined programs stored in the storage device 54, the memory 58, or the like, thereby functioning as various functional units.

The communication device 52 is composed of a communication interface or the like for communicating with an external device. The communication device 52 transmits and receives various information to and from a terminal device or the like.

The storage device 54 is composed of a hard disk or the like. The storage device 54 stores various programs and information necessary for execution of processing in the control device 50 and information on processing results.

The driving assistance device 1 can be realized using an information processing device such as a dedicated or general-purpose server computer. Further, the driving support device 1 may be configured by a single information processing device, or may be configured by a plurality of information processing devices distributed over a communication network. Further, FIG. 2 only shows a part of the main hardware configuration of the driving assistance device 1, and the driving assistance device 1 can have other configurations that information processing devices generally have.

FIG. 3 is a diagram showing an outline of electrical functional blocks of the driving assistance device 1. As shown in FIG. The driving assistance device 1 mainly includes a communication unit 11 , a direction sensor 12 , an altitude sensor 13 , an imaging unit 14 , an output unit 15 , a control unit 20 and a storage unit 30 .

The communication unit 11 is a functional unit that acquires information on the position of the vehicle C from a positioning system (not shown). Note that the communication unit 11 may be a functional unit that communicates with an information transmission device provided on the road, an external information processing device, a driving support device of another vehicle, or the like.

The azimuth sensor 12 is a functional unit that acquires the traveling direction of the vehicle C by detecting the azimuth. The direction sensor 12 can use, for example, a three-axis geomagnetic sensor.

The altitude sensor 13 is a functional unit that acquires the altitude of the vehicle C. As shown in FIG. An atmospheric pressure sensor, for example, can be used as the altitude sensor 13 . The altitude sensor 13 is not essential, and the communication unit 11 may acquire altitude information of the vehicle C from a positioning system (not shown).

The imaging unit 14 is a functional unit that images the front of the vehicle C. As shown in FIG. An image captured by the imaging unit 14 on the outward trip is defined as an outward image, and an image captured by the imaging unit 14 on the return trip is defined as a backward image. The image capturing unit 14 continuously captures an outward travel image and an inward travel image.

The output unit 15 is a functional unit that outputs notifications such as sound and character information. The output unit 15 is provided in the vehicle C and includes a speaker 15a and a display unit 15b. The display unit 15b includes a display device such as liquid crystal, a lamp, and the like. Note that the output unit 15 may have a configuration other than the speaker 15a and the display unit 15b, or may have either one of the speaker 15a and the display unit 15b.

The control unit 20 has at least a position information acquisition unit 21, an image acquisition unit 22, an image generation unit 23, a determination unit 24, and a notification unit 25 as software resources by an arithmetic device such as a CPU (Central Processing Unit) 56 and a storage device 54. The control unit 20 may be realized by an ECU (Electronic Control Unit).

The position information acquisition unit 21 is a functional unit that acquires the position information of the vehicle C acquired by the communication unit 11, direction sensor 12, altitude sensor 13, and the like. The position information of the vehicle C includes the position of the vehicle C, traveling direction, altitude, and the like. However, altitude is not required.

The image acquisition unit 22 is a functional unit that acquires an outward travel image and a return travel image from the imaging unit 14 . Further, the image acquisition unit 22 acquires the position information from the position information acquisition unit 21 , associates the outbound route image with the position information of the point where the outbound route image was captured, and stores them in the storage unit 30 .

The image generation unit 23 is a functional unit that generates a return route navigation image (corresponding to a return route guidance image of the present invention) used for return route guidance based on the outward route image stored in the storage unit 30 . The processing by which the image generator 23 generates the return route navigation image will be described in detail later.

In addition, the image generation unit 23 stores the generated return route navigation image in the storage unit 30 in association with the position information and the outward route image. Hereinafter, the data in which the position information, the outward route image, and the return route navigation image are associated will be referred to as return route navigation data.

FIG. 4 is a diagram showing an example of return route navigation data. The return navigation data is associated with time, position information at that time, an outward route image (in FIG. 4, represented by a file name (captured file name) attached to the outward route image), and a return route navigation image (in FIG. 4, represented by a file name (return route navigation image name) attached to the return route navigation image). The position information includes position, azimuth, altitude, traveling direction (azimuth), and traveling direction (altitude).

Each item in FIG. 4 will be described. The position is information about the position acquired via the communication unit 11, and includes latitude and longitude information. The azimuth is information acquired by the azimuth sensor 12 and is represented by a true azimuth (an angle based on the north). The altitude is information acquired by the altitude sensor 13 and is represented by a geoid height (height from the surface of an ellipsoid representing the earth virtually).

The travel direction (azimuth) is a change in the travel direction of the vehicle C, and is calculated from the true azimuth acquired by the azimuth sensor 12 . The traveling direction (height) is the change in altitude of the vehicle C, and is calculated from the geoid height acquired by the altitude sensor 13 . For example, at time t2, the difference between the azimuth and altitude at time t2 and time t1 is the traveling direction (azimuth) and the traveling direction (height).

Returning to the description of FIG. The determination unit 24 is a functional unit that determines whether or not the vehicle C is traveling the correct route on the return trip based on the position information, the image for the return route navigation, and the return route image. The determination unit 24 compares the return route image and the return route navigation image, and determines whether or not the vehicle C is traveling on the correct route based on the result of the comparison. The processing of the determination unit 24 will be detailed later.

The notification unit 25 is a functional unit that outputs a notification when it is determined that the vehicle C does not travel the correct route on the return trip. For example, the notification unit 25 performs notification by sound or character information from the output unit 15 . The notification unit 25 notifies by outputting sound (including beep sound and voice) from the speaker 15a, and outputs character information to the liquid crystal display device, which is the display unit 15b.

The control unit 20 is connected to a vehicle control unit (not shown) mounted on the vehicle C. As shown in FIG. The vehicle control unit is connected to various sensors and the engine provided in the vehicle C, and performs various controls including driving and controlling the engine. The control unit 20 is also connected to an in-vehicle device including a car navigation system.

The storage unit 30 is a functional unit that stores information necessary for processing of the control unit 20 . The information stored in the storage unit 30 includes forward route images, image data including return route navigation images, return route navigation data, and the like.

FIG. 5 is a flow chart showing the flow of processing for obtaining an outbound route image and generating a return route navigation image. The process shown in FIG. 5 is started by the control unit 20 when the vehicle C starts driving. Further, in the present embodiment, the control unit 20 acquires the start and end of the outward trip from the in-vehicle device.

The position information acquisition unit 21 acquires position information of the vehicle C from the communication unit 11, the direction sensor 12 and the altitude sensor 13, and outputs the position information to the image acquisition unit 22 (step SP11). The image acquisition unit 22 acquires an outward travel image from the imaging unit 14 (step SP12). Then, the image acquiring unit 22 associates the position information acquired in step SP11 with the outward travel image acquired in step SP12 and stores them in the storage unit 30 (step SP13).

The image acquisition unit 22 acquires from the in-vehicle device or the like whether or not the forward driving has ended (step SP14). If the forward driving has not ended (NO in step SP14), the process returns to step SP11.

When the forward driving is completed (YES in step SP14), the image generation unit 23 performs a process of generating a return navigation image (steps SP15 to SP17).

First, the image generator 23 generates a return route navigation image from an arbitrary outward route image (for example, an outward route image associated with time t1) stored in the storage unit 30 (step SP15). 6 and 7 are diagrams schematically showing the process of generating a return navigation image.

FIG. 6A is an example of an outward travel image. The image generation unit 23 acquires an outward route image and extracts roads and structures (buildings, signboards, etc.). Known image processing techniques can be used to extract roads and structures.

Next, the image generation unit 23 deletes only the road portion from the forward route image. Since the road is information that cannot be used for the return route guidance, only the road portion is deleted from the outward route image to generate an image in which the structures used for the return route guidance remain. FIG. 6B is an outbound route image from which only the road portion is deleted. Then, the image generation unit 23 reverses the forward route image from which only the road portion is deleted to generate a reversed image. FIG. 6C is an example of a reversed image.

Next, the image generator 23 generates a return navigation image based on the reversed image. First, as shown in FIG. 7A, the reversed image is divided into a plurality of band-like images (strip-like images) so that one structure is included in one band-like image. A band-shaped image is generated by cutting the reversed image along the vertical direction of the reversed image.

Next, the image generation unit 23 generates an image for homeward navigation by reducing the size of a structure near the vehicle C in the reversed image and arranging it near the center of the reversed image, and increasing the size of the structure farther from the vehicle C in the reversed image and arranging it near the edge of the reversed image.

First, the image generation unit 23 rearranges the strip images, and arranges the strip image including the structure near the vehicle C in the reverse image (that is, the structure that appears large near the edge of the reverse image) near the center, and places the strip image that includes the structure far from the vehicle C in the reverse image (that is, the structure that appears small near the center of the reverse image) near the end, as shown in FIG. 7B. Arrows between FIGS. 7A and 7B indicate how the strip images are rearranged.

Next, after the rearrangement shown in FIG. 7B, the image generation unit 23 reduces the band-like image near the center and enlarges the band-like image near the ends. At this time, the image generation unit 23 enlarges or reduces the strip image so that the structures appearing in the strip image near the center are small and the structures appearing in the strip image become larger toward the edges. As a result, a return navigation image as shown in FIG. 7(C) is generated.

Returning to the description of FIG. The image generation unit 23 stores the return navigation image generated in step SP15 in the storage unit 30 in association with the original outbound route image and position information (step SP16).

The image generation unit 23 determines whether or not images for return route navigation have been generated and stored in the storage unit 30 for all outward route images stored in the storage unit 30 (step SP17). If the processes of steps SP15 and SP16 have not been performed on all of the outward travel images (NO in step SP17), the image generator 23 returns the process to step SP15. When the processes of steps SP15 and SP16 have been performed on all the outward travel images (YES in step SP17), the image generator 23 ends the series of processes.

FIG. 8 is a flow chart showing a flow of processing in which the driving support device 1 performs guidance using the return route navigation image on the return route. The process shown in FIG. 8 is started by the control unit 20 when the vehicle C starts driving on the return trip. Further, in the present embodiment, the control unit 20 acquires the start of the return trip from the in-vehicle device.

The determination unit 24 determines whether or not a certain period of time has elapsed since the previous determination (step SP26, which will be described in detail later) (step SP21). If the predetermined time has not elapsed since the previous determination (NO in step SP21), the determination unit 24 repeats the process of step SP21.

If a certain period of time has passed since the previous determination (including the case where step SP21 is YES and step SP21 is performed first), the position information acquisition unit 21 acquires the position information of the vehicle C from the communication unit 11, direction sensor 12, and altitude sensor 13, and outputs it to the image acquisition unit 22 (step SP22). The image acquisition unit 22 acquires the backward image from the imaging unit 14 (step SP23). Then, the image acquisition section 22 outputs the position information and the return image to the determination section 24 .

The determination unit 24 acquires a return navigation image from the storage unit 30 based on the position information acquired in step SP21 (step SP24). FIG. 9 is a diagram for explaining acquisition of an image for return navigation. Hereinafter, the process of step SP24 will be described by taking as an example a case where the position of the vehicle C on the return trip is a point A, and an image for return navigation to be compared with the image of the return trip at the point A is acquired.

The field of view of the imaging unit 14 when the position of the vehicle C is at the point A is different between the forward route and the return route, and the field of view of the imaging unit 14 when the forward route image is taken is the field of view a1, and the field of view of the imaging unit 14 when the backward route image is taken is the field of view a2. In other words, the structures included in the return route navigation image generated based on the outward route image captured at point A are different from the structures included in the return route image, and driving support cannot be provided by comparing the return route image captured at point A with the return route navigation image generated based on the outward route image captured at point A.

Therefore, assuming that a structure in area D is captured in an inbound route image captured at point A, the determining unit 24 needs to acquire an outbound route image in which the structure in region D is captured. An outbound route image including structures in area D can be captured at point B facing point A with area D interposed therebetween. Since the field of view of the imaging unit 14 when capturing the outward route image at the point B is the field of view b1, the structure in the area D is included in the outward route image captured at the point B. FIG.

On the outward trip, vehicle C passes point B a predetermined time (for example, 30 seconds) earlier than the time it passed point A. Therefore, the determination unit 24 extracts the time (time ta) at which point A is passed on the outward route from the return navigation data, and also extracts data at a time (time tb) earlier than the time ta by a predetermined time from the return route navigation data. Then, the determination unit 24 acquires from the storage unit 30 the file (return navigation image) having the name of the return navigation image associated with the time tb in the return navigation data.

Returning to the description of FIG. The determination unit 24 compares the return route image acquired in step SP24 with the return route navigation image acquired in step SP25 (step SP25), and determines whether the return route navigation image and the return route image have the same scenery, that is, whether the vehicle C is traveling the correct route on the return route (step SP26). The processing of steps SP25 and SP26 will be described with reference to FIGS.

FIG. 10(A) is an example of a return route image, and FIG. 10(B) is an example of a return route image (hereinafter referred to as a comparative return route image) used for comparison with the return route navigation image. First, the determination unit 24 outputs the return travel image to the image generation unit 23, and the image generation unit 23 processes the return travel image to generate a return travel image for comparison. Specifically, the image generation unit 23 acquires a return route image, extracts roads and structures (buildings, signboards, etc.), deletes only the road portion from the return route image, and generates a return route image for comparison. Then, the image generation unit 23 outputs the comparison backward image to the determination unit 24 .

FIG. 11(A) is an example of a return navigation image, and FIG. 11(B) is an example of a return route image for comparison. The determination unit 24 extracts a component from the comparison areas of the return route navigation image and the comparison return route image. In FIG. 11, the center line passing through the centers of the return route navigation image and the return route image for comparison is indicated by a dashed line, and the comparison area is surrounded by a dotted line. The comparison area is an area that does not include the center of the homeward navigation image and the homeward route image for comparison, and is preferably set at a position closer to both ends from the center. Various known methods can be used as a method for extracting the constituent from the comparison region.

Then, the determination unit 24 compares the structures extracted from the comparison areas of the return navigation image and the comparison return route image, and determines whether or not the same structure can be extracted from the return route navigation image and the comparison return route image. Whether or not the structure extracted from the return navigation image and the structure extracted from the comparison return route image are the same is determined based on the color, shape, and the like of the structure. In the example shown in FIG. 11, the determination unit 24 can determine that the structure S1 in the return navigation image and the structure S3 in the comparison return route image are the same based on the shape, and can determine that the structure S2 in the return route navigation image and the structure S4 in the comparison return route image are the same based on the shape and color.

Then, when the same structure can be extracted from the return route navigation image and the return route image for comparison, the determination unit 24 determines that the return route image for comparison and the return route navigation image have the same scenery, that is, that the vehicle C is traveling the correct route on the return route.

Returning to the description of FIG. If the return route image for comparison and the return route navigation image have the same scenery, that is, if the vehicle C is traveling on the correct route on the return route (YES in step SP26), the determination unit 24 returns the process to step SP21.

If the return route image for comparison and the return route navigation image do not have the same scenery, that is, if the vehicle C does not travel the correct route on the return route (NO in step SP26), the determination unit 24 outputs to the notification unit 25 that there is a possibility that the vehicle C does not travel the correct route on the return route, and the notification unit 25 outputs, via the output unit 15, a notification indicating that the vehicle C may not travel the correct route on the return route (step SP27). In the present embodiment, the notification unit 25 outputs the voice information "There is a possibility that the route is incorrect" from the speaker 15a, but the form of notification is not limited to this.

According to the present embodiment, the return route navigation image created based on the forward route image and the comparative return route image created based on the return route image are compared to determine whether or not the vehicle C is traveling on the correct route on the return route. Thus, the return route guidance can be provided using only the image captured in the forward direction without using the back camera. For example, when a person gets lost, guidance based only on position information may not provide sufficient guidance. However, guidance using images enables detailed guidance.

When it is determined that the return route is not correct, the output unit 15 outputs a notification using sound or text information, so that the driver can be assisted in driving the correct route on the return route.

Further, according to the present embodiment, the structure near the vehicle C is made smaller and placed near the center of the reversed image, and the structure farther from the vehicle C in the reversed image is made larger and placed near the edge of the reversed image to form the return route navigation image, thereby making the return route navigation image closer to the return route image. As a result, it is possible to improve the accuracy of determining whether or not the vehicle C is traveling on the correct route on the return trip.

Further, according to the present embodiment, when the same structure can be extracted from the return route navigation image and the comparison return route image, it is possible to easily determine whether or not the comparative return route image and the return route navigation image have the same scenery by determining that the vehicle C is traveling on the correct route.

Further, according to the present embodiment, when the position on the return route is position A, the determination unit 24 makes a determination using the return route navigation image associated with the time tb, which is earlier than the time ta at which the position A is passed on the return route, from among the return route navigation images stored in the storage unit 30. Therefore, the same structure can be included in the outward route image and the return route image, which are the basis of the return route navigation image. Therefore, the determination unit 24 can improve the determination accuracy of whether or not the vehicle C is traveling on the correct route on the return trip.

In the present embodiment, the image generation unit 23 extracts roads and structures from the outbound and return images, deletes only roads from the outbound and return images, and generates reversed images and comparison return images. The image generation unit 23 may use the forward route image in which the road is not deleted as it is to generate the reversed image. If the road is not deleted, the return route image for comparison is the same as the return route image, and the determination unit 24 may determine whether or not the vehicle C is traveling the correct route based on the return route image.

Further, in the present embodiment, the image generation unit 23 extracts the road and the structure from the outbound route image and the return route image, but the image generation unit 23 may extract a characteristic structure located near the corner, and the determination unit 24 may determine whether or not the vehicle C is traveling on the correct route using the structure. For example, the image generating unit 23 may acquire the position of the turn on the return trip from the position information of the outbound trip included in the return navigation data, and extract the characteristic structures located near the turn corner by extracting the structures with characteristic colors, shapes, etc., from the structures located at the turn corner.

In the present embodiment, when the same structure can be extracted from the return navigation image and the comparison return route image, the determination unit 24 determines that the comparison return image and the return route navigation image have the same scenery. and the return route navigation image may be determined to have the same scenery. This makes it possible to determine with high accuracy whether or not the return route image for comparison and the return route navigation image have the same scenery.

Further, in the present embodiment, when the inbound route image for comparison and the inbound route navigation image do not have the same scenery (NO in step SP26), the notification unit 25 notifies that the vehicle C does not travel the correct route on the return route (step SP27), but the method for determining that the vehicle C does not travel on the correct route on the return route is not limited to this. If the return route image for comparison and the return route navigation image do not have the same scenery (NO in step SP26), the determination unit 24 may refer to the return route navigation data to determine whether or not the vehicle C is traveling on the correct route on the return route. For example, the determination unit 24 may extract data having the same position information as the current position of the vehicle C on the return trip from the return navigation data, and may determine whether or not the vehicle C is traveling the correct route on the return trip based on the position information (azimuth, altitude) of the vehicle C on the return trip and the orientation, altitude, traveling direction (azimuth), and traveling direction (altitude) of the extracted return navigation data. This can reduce erroneous determinations.

<Second Embodiment>
In the first embodiment of the present invention, the reversed image is divided into a plurality of belt-shaped images, structures located close to the vehicle C are made smaller and placed near the center of the reversed image, and structures farther from the vehicle C in the reversed image are enlarged and placed near the edge of the reversed image. The method of determining whether or not the vehicle C is traveling on the correct route is not limited to this.

A driving support device 2 according to a second embodiment of the present invention will be described below. The same reference numerals are assigned to the same parts as in the first embodiment, and the description thereof is omitted.

FIG. 12 is a diagram showing an outline of electrical functional blocks of the driving assistance device 2. As shown in FIG. The driving support device 2 mainly includes a communication unit 11, a direction sensor 12, an altitude sensor 13, an imaging unit 14, an output unit 15, a control unit 20A, and a storage unit 30. The control unit 20A has at least a position information acquisition unit 21, an image acquisition unit 22, an image generation unit 23A, a determination unit 24A, and a notification unit 25 as software resources by an arithmetic device such as a CPU (Central Processing Unit) 56 and a storage device 54.

The image generation unit 23A is a functional unit that generates a return route navigation image from the outbound route image stored in the storage unit 30 . Specifically, the image generation unit 23A acquires an outward route image, extracts roads and structures, and deletes only the road portion from the outward route image. Further, the image generator 23A reverses the forward route image from which only the road portion is deleted to generate a reversed image, and uses this reversed image as the reverse route navigation image.

The determination unit 24A is a functional unit that determines whether or not the vehicle C is traveling the correct route on the return trip based on the position information, the return navigation image, and the return route image. FIG. 13(A) is an example of an image for return navigation (inverted image), and FIG. 13(B) is an example of a return route image for comparison. The comparison backward image used by the determination unit 24A is the same as that used by the determination unit 24A.

The determination unit 24A extracts color information from each of the comparison regions that do not include the centers of the return route navigation image and the comparison return route image. The reason why the center is deleted from the comparison area is that the center of the reversed image is located on the back side of the vehicle C, and there is a possibility that it will not be captured in the backward travel image for comparison.

Based on the extracted color information, the determination unit 24A compares the color type and arrangement of the return route navigation image with the color type and arrangement of the comparison return route image. Note that since the color arrangement is reversed in the return navigation image (reversed image), the determination unit 24A performs the comparison in consideration of the color reversal. Then, the determination unit 24A determines whether or not the vehicle C is traveling the correct route on the return trip based on the result of the comparison.

For example, the determination unit 24A determines that the vehicle C is traveling the correct route on the return trip when the characteristic color included in the left half region L of the comparative return route image is included in the left half region L of the return route navigation image, and the characteristic color included in the right half region R of the comparison return route image is included in the right half region R of the return route navigation image.

According to the present embodiment, the processing of the image generator 23A can be lightened, and the time and load required for the processing can be shortened.

<Third Embodiment>
In the first embodiment of the present invention, the position information of the vehicle C is obtained from the positioning system via the communication unit 11, but the position of the vehicle C may be calculated when the information cannot be obtained from the positioning system.

A driving support device 3 according to a third embodiment of the present invention will be described below. The same reference numerals are assigned to the same parts as in the first embodiment, and the description thereof is omitted.

FIG. 14 is a diagram showing an outline of electrical functional blocks of the driving assistance device 3. As shown in FIG. The driving support device 3 mainly includes a communication unit 11, a direction sensor 12, an altitude sensor 13, an imaging unit 14, an output unit 15, a vehicle speed sensor 16, a control unit 20B, and a storage unit 30. The control unit 20B has at least a position information acquisition unit 21A, an image acquisition unit 22, an image generation unit 23, a determination unit 24, and a notification unit 25 as software resources by means of an arithmetic device such as a CPU (Central Processing Unit) 56 and a storage device 54.

The vehicle speed sensor 16 (corresponding to the speed acquisition unit of the present invention) is a functional unit that acquires the speed of the vehicle C (hereinafter referred to as vehicle speed). The vehicle speed sensor 16 generates an electric signal from a sensor provided on the wheel, and obtains the speed based on the electric signal. The form of the vehicle speed sensor 16 is not limited to this. For example, the speed may be calculated based on the acceleration acquired by the acceleration sensor.

21 A of position information acquisition parts are functional parts which acquire the position information of the vehicle C acquired by the communication part 11, the direction sensor 12, the altitude sensor 13 grade|etc.,. Further, the positional information acquisition section 21A is a functional section that calculates the positional information of the vehicle C based on the vehicle speed acquired by the vehicle speed sensor 16 . The details of the position information acquisition unit 21A will be described later.

The image acquisition unit 22 acquires the position information from the position information acquisition unit 21A, associates the outbound route image and the position information of the point where the outbound route image was captured, and stores them in the storage unit 30 . The image generation unit 23 stores the generated return route navigation image in the storage unit 30 in association with the position information and the outward route image.

FIG. 15 is a diagram showing an example of data for return route navigation in which position information, an outward route image, and an image for return route navigation are associated. The return route navigation data is associated with time, position information at that time, an outward route image, and a return route navigation image. The position information includes position, direction, altitude, vehicle speed, traveling direction (azimuth), and traveling direction (altitude). The vehicle speed in FIG. 15 is information about the vehicle speed obtained from the vehicle speed sensor 16 and is expressed in terms of speed per hour.

FIG. 16 is a flow chart showing the flow of processing for obtaining an outbound route image and generating a return route navigation image. The process shown in FIG. 16 is started by the control unit 20B when the vehicle C starts driving. Further, in the present embodiment, the control unit 20B acquires the start and end of the outbound route from the in-vehicle device.

The position information acquisition unit 21A acquires the position information of the vehicle C from the communication unit 11, the azimuth sensor 12, and the altitude sensor 13, and outputs it to the image acquisition unit 22 (step SP31), and the image acquisition unit 22 acquires the outbound route image from the imaging unit 14 (step SP32). Step SP11 is the same as steps SP11 and SP12.

21 A of position information acquisition parts determine whether the position was able to be acquired from the positioning system via the communication part 11 in step SP31 (step SP33). If the position can be acquired from the positioning system (YES in step SP33), the image acquisition unit 22 associates the position information acquired in step SP31 with the outbound image acquired in step SP12 and stores them in the storage unit 30 (step SP35).

If the position cannot be acquired from the positioning system (NO in step SP33), the position information acquisition unit 21A acquires the vehicle speed from the vehicle speed sensor 16, calculates the position of the vehicle C based on the vehicle speed, and outputs the calculated position to the image acquisition unit 22 as the position information of the vehicle C (step SP34). For example, the position information acquisition unit 21A can calculate the current position of the vehicle C by calculating the traveled distance from when the position was finally acquired from the positioning system based on the elapsed time when the position was last acquired from the positioning system and the vehicle speed acquired from the vehicle speed sensor 16, and adding the traveled distance to the position information when the position was last acquired from the positioning system.

Then, the image acquisition unit 22 associates the position information acquired from the position information acquisition unit 21A in steps SP31 and SP34 with the outbound route image acquired in step SP12, and stores them in the storage unit 30 (step SP35).

The image acquisition unit 22 acquires from the in-vehicle device or the like whether or not the forward driving has ended (step SP36). If the forward driving has not ended (NO in step SP36), the process returns to step SP11. When the forward driving is finished (YES in step SP36), the image generator 23 performs processing for generating a return navigation image (steps SP37 to SP39). Steps SP36-SP39 are the same as steps SP14-SP17.

FIG. 17 is a flow chart showing the flow of processing in which the driving support device 3 performs guidance using the return route navigation image on the return route. The process shown in FIG. 17 is started by the control unit 20B when the vehicle C starts driving on the return trip. Further, in the present embodiment, the control unit 20B acquires the start of the return trip from the in-vehicle device.

The determination unit 24 determines whether or not a certain period of time has elapsed since the previous determination (step SP49, which will be described in detail later) (step SP41). If the predetermined time has not elapsed since the previous determination (NO in step SP41), the determination unit 24 repeats the process of step SP41. If a certain period of time has elapsed since the previous determination (including the case where step SP41 is YES and the process of step SP41 is performed first), the position information acquisition unit 21A acquires the position information of the vehicle C from the communication unit 11, direction sensor 12, and altitude sensor 13, and outputs the position information to the image acquisition unit 22 (step SP42). The image acquisition unit 22 acquires the backward travel image from the imaging unit 14, and outputs the position information and the backward travel image to the determination unit 24 (step SP43). Steps SP41 to SP43 are the same as steps SP21 to SP23.

21 A of position information acquisition parts determine whether the position was able to be acquired from the positioning system via the communication part 11 in step SP42 (step SP44). If the position can be acquired from the positioning system (YES in step SP44), the determination unit 24 acquires the return navigation image from the storage unit 30 based on the position information acquired in step SP41 (step SP45). Step SP45 is the same as step SP24.

If the position cannot be acquired from the positioning system (NO in step SP44), the position information acquisition unit 21A acquires the vehicle speed from the vehicle speed sensor 16, calculates the position of the vehicle C based on the vehicle speed, and outputs the calculated position to the image acquisition unit 22 as the position information of the vehicle C together with the information acquired from the azimuth sensor 12 and the altitude sensor 13 (step SP46). The processing of step SP46 is the same as that of step SP34.

Then, the determination unit 24 acquires the return route navigation image from the storage unit 30 based on the position information acquired from the position information acquisition unit 21A in step SP46 (step SP47). In step SP47, the method of acquiring the return navigation image based on the position information is the same as in step SP24.

The determination unit 24 compares the return route image acquired in step SP43 with the return route navigation image acquired in step SP45 or SP47 (step SP48), and determines whether or not the vehicle C is traveling the correct route on the return route (step SP49). If the vehicle C is traveling on the correct route on the return trip (YES in step SP49), the process returns to step SP41. When the vehicle C does not travel the correct route on the return trip (NO in step SP49), the determination unit 24 causes the notification unit 25 to output a notification indicating that the vehicle C may not travel the correct route on the return trip through the output unit 15 (step SP50). Steps SP48-SP50 are the same as steps SP25-SP27.

According to the present embodiment, even if the position information cannot be acquired, it is possible to provide guidance on the return route using only an image of the forward direction in the direction of travel. For example, it is difficult to obtain information from the positioning system inside a tunnel or in a valley, but by obtaining the position of the vehicle C from the vehicle speed, appropriate guidance can be provided even at a branch inside the tunnel.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are included within the scope of the present invention.

1, 2, 3: Driving support device 11: Communication unit 12: Direction sensor 13: Altitude sensor 14: Imaging unit 15: Output unit 15a: Speaker 15b: Display units 20, 20A, 20B: Control units 21, 21A: Position information acquisition unit 22: Image acquisition units 23, 23A: Image generation units 24, 24A: Judgment unit 25: Notification unit 30: Storage unit 50: Control Device 52: communication device 54: storage device 58: memory

Claims (9)

A driving support device that guides a return route based on an outward route image captured on the outward route,
a position information acquisition unit that acquires position information including the position and traveling direction of the vehicle;
an imaging unit that captures an image in front of the vehicle and acquires the outbound image and the inbound image captured in the inbound route;
an image generation unit that generates a return route guidance image used for guidance on the return route based on the inverted image obtained by horizontally reversing the forward route image;
a storage unit that associates and stores the position information, the outbound route image, and the return route guidance image;
a determination unit that determines whether the vehicle is traveling on the correct route on the return trip based on the position information, the return route guidance image, and the return route image;
a notification unit that outputs a notification when it is determined that the vehicle does not travel on the correct route on the return trip;
A driving support device comprising: The driving support device according to claim 1, wherein the image generation unit extracts structures from the reversed image, reduces a first structure located near the vehicle in the reversed image among the extracted structures and places it near the center of the reversed image, and enlarges a second structure farther from the vehicle in the reversed image among the extracted structures and places it near an edge of the reversed image to generate the return route guidance image. The driving support device according to claim 1 or 2, wherein the determination unit extracts structures from the return route guidance image and the return route image, and determines that the vehicle is traveling on a correct route when a third structure can be extracted from the return route guidance image and the return route image. The driving support device according to claim 3, wherein the determination unit determines that the vehicle is traveling on a correct route based on the position of the third structure in the return route guidance image and the position of the third structure in the return route image. The driving support device according to any one of claims 1 to 4, wherein, when the position on the return route is the first position, the determination unit performs the determination using a return route guidance image associated with a second time that is a predetermined time earlier than a first time, which is a time at which the first position is passed on the outward route, among the return route guidance images stored in the storage unit. An output unit including a speaker or a display unit provided in the vehicle,
The driving support device according to any one of claims 1 to 5, wherein the notification unit outputs a notification by sound or text information from the output unit when the determination unit determines that the return route is not correct. A speed acquisition unit that acquires a vehicle speed that is the speed of the vehicle,
The driving support device according to any one of claims 1 to 6, wherein the position information acquisition unit calculates the position based on the vehicle speed. A driving support method for providing guidance for a return route based on an outward route image captured on the outward route by an imaging unit that images the front of the vehicle,
a step of acquiring position information including the position and traveling direction of the vehicle and the outbound route image;
a step of generating a return route guidance image to be used for guidance on the return route based on the inverted image obtained by horizontally reversing the forward route image;
a step of associating and storing the position information, the outbound route image, and the return route guidance image;
a step of acquiring a backward image taken by the imaging unit on the backward trip;
a step of determining whether or not the vehicle is traveling the correct route on the return trip based on the position information, the return route guidance image, and the return route image;
a step of outputting a notification when it is determined that the vehicle is not traveling on the correct route on the return trip;
A driving support method comprising: A driving support program for guiding a return route based on an outward route image captured on the outward route by an imaging unit that captures an image of the front of the vehicle,
the computer,
a position information acquisition unit that acquires position information including the position and traveling direction of the vehicle;
an image generation unit that generates a return route guidance image used for guidance on the return route based on the inverted image obtained by horizontally reversing the forward route image, and stores the position information, the outward route image, and the return route guidance image in a storage unit in association with each other;
A determination unit that determines whether the vehicle is traveling on the correct route on the return trip based on the position information, the return route guidance image, and the return route image captured by the imaging unit on the return route;
A notification unit that outputs a notification when it is determined that the vehicle does not travel the correct route on the return trip;
A driving support program characterized by functioning as

Images