JP2008305283A - Drive support apparatus and drive support method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that it is not possible to carry out a drive support based on an object that is not registered to a database, and especially it is not possible to issue a warning about a mobile body in the periphery of one's own vehicle. <P>SOLUTION: The direction of the line of sight of a driver who drives the own vehicle is detected, image information that indicates the image around the own vehicle is obtained, a mobile body that exists in the direction of the line of sight is detected based on the image information, the motion of the mobile body is detected, it is detected that the mobile body is present in the warning region which is set around the own vehicle based on the motion of the mobile body, and the control for warning is carried out. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a driving support apparatus, method, and program for warning a moving body that moves around a host vehicle.

Conventionally, as a technique for guiding an object that a driver is paying attention to, an image is captured and stored in time series, and the driver's line-of-sight direction and image before the timing when the driver issues a question A technique is known in which an object in the line-of-sight direction is extracted based on the image, and information on the object is guided with reference to a database (Patent Document 1).
JP 2006-90790 A

In the prior art, driving support based on an object not recorded in the database could not be performed. In particular, warnings could not be given for moving objects around the host vehicle.
That is, in the technique disclosed in Patent Literature 1, the driver's desired object is guided without disturbing driving by detecting the driver's line-of-sight direction. Only pre-stored objects. Therefore, it is impossible to provide guidance for objects that cannot be stored in the database in advance, such as a moving object, and it is impossible to issue a warning regarding the moving object.
The present invention has been made in view of the above problems, and aims to provide driving support based on an object not recorded in a database, and in particular to provide a technique for warning a moving body around the host vehicle. To do.

  In order to achieve the above object, in the present invention, a moving body that exists in the driver's line-of-sight direction is detected, the movement of the moving body is detected, and a warning is issued when the moving body is present in the warning area. Control for. In other words, the movement of the moving body focused by the driver is tracked based on the image, and it is determined by the tracking whether or not the moving body exists in a warning area where a warning related to the moving body should be issued. A warning is issued when the moving object is present in the warning area. Therefore, when the driver pays attention to the moving body existing around the host vehicle, a warning is given to the moving body, and the driver's attention can be drawn.

  The line-of-sight direction detection means only needs to be able to detect the driver's line-of-sight direction, and can be realized by various techniques for detecting the driver's line-of-sight direction. For example, the driver's face direction, eye position, eyeball direction, etc. are detected by a camera or various sensors in the vehicle interior of the host vehicle, and the line of sight is determined based on the face direction, eye position, eyeball direction, etc. What is necessary is just to detect a direction. Of course, when detecting the gaze direction of the driver, when the gaze is maintained within a certain range for a certain time, the direction may be adopted as the gaze direction.

  The line-of-sight direction only needs to be defined in a predetermined coordinate system. For example, a coordinate system in which a fixed origin is defined at the reference position of the host vehicle can be employed. In this case, since the line-of-sight direction is detected with reference to the own vehicle, if the position and orientation of the own vehicle are specified on the road surface in addition to the line-of-sight direction, the line-of-sight direction on the road surface can be detected. Of course, in addition to the above-described configuration, the coordinate system may be a coordinate based on the reference set on the road surface, a coordinate using latitude and longitude, and various configurations can be adopted.

  The image information acquisition means only needs to be able to acquire an image including a moving body that should alert the driver, and adopts various configurations such as a configuration in which the surroundings of the host vehicle are photographed by a camera mounted on the host vehicle. Is possible. The moving body motion detecting means only needs to detect a moving body that exists in the line-of-sight direction based on the image information and can detect subsequent movements in the moving body. In the image indicated by the image information, each position in the image is associated with the line-of-sight direction in advance, and the moving body exists in the line-of-sight direction when the image of the moving body exists at the position associated with the line-of-sight direction. It is possible to adopt a configuration for detecting the occurrence.

  In addition, any object whose position varies with time corresponds to the moving object, and may be detected by extracting a predetermined shape from the image, or the moving object may be detected by comparing images taken at different times. It may be detected, and various configurations can be adopted. The movement of the moving body only needs to be detected based on, for example, information indicating the position, moving direction, speed, acceleration, etc. of the moving body, and if these movements can be detected for the moving body detected as described above. Good. Further, it is possible to adopt a configuration that compares images taken at different times and detects the moving direction and speed based on the shooting interval of each image and the position of the moving body specified based on each image. . Of course, the coordinate system for specifying the position of the moving body only needs to be able to define the relationship with the coordinate system when defining the above-described line-of-sight direction, and may be different coordinate systems or the same coordinate system. Also good.

  The moving body warning means only needs to detect the presence of the moving body in the warning area set around the host vehicle based on the movement of the moving body and perform control for warning. Therefore, a warning area is set around the own vehicle in advance, and the presence of the moving object in the warning area is detected from the relative relationship between the moving object and the own vehicle, for example, the position of the own vehicle and the position of the moving object. do it. In the warning, it is sufficient that the driver can recognize the presence of the moving body, and the driver may be made to recognize the presence of the moving body by various means such as voice guidance, image display, and vibration. Therefore, the moving body warning means only needs to be able to control a speaker, a display unit, a vibrating body, and the like that perform these warnings. The warning area may be set in advance as an area to warn of the presence of the moving body. For example, an area at a predetermined distance from the own vehicle or an area of a specific shape set around the own vehicle is set as the warning area. Is possible.

  Furthermore, as a configuration example in which the gaze direction detection and warning according to the present invention are performed in a necessary scene, a configuration using voice generated by the driver may be employed. For example, it is possible to adopt a configuration in which voice information indicating voice generated by the driver is acquired and the line-of-sight direction is detected when voice corresponding to a moving body is detected. That is, in this configuration, when the sound corresponding to the moving body is not detected, the line-of-sight direction is not detected, and as a result, no warning is given regarding the presence of the moving body in the warning area. According to this configuration, no warning is given to any moving object, and the driver is targeted for warning when the driver emits a sound corresponding to the moving object that the driver should pay attention to. be able to.

  Furthermore, as a configuration example in which the detection and warning of the moving body in the present invention is performed in a necessary scene, the configuration may be such that the moving body existing in the line-of-sight direction included in the range excluding the predetermined range including the traveling direction of the host vehicle is detected. Good. That is, the predetermined range including the traveling direction of the host vehicle is almost always included in the driver's field of view. Therefore, if the moving object existing in the predetermined range is a warning target, it is troublesome for the driver, and attention to the moving object outside the predetermined range that should be noted can be distracted.

  Therefore, a warning is required by detecting a line-of-sight direction included in a range excluding the predetermined range, and detecting a moving object existing in the line-of-sight direction included in the range excluding the predetermined range as a warning target. A mobile object existing in the area can be reliably set as a warning target. In addition, a moving body that exists in an area where no warning is required can be excluded from the warning target. The predetermined range may be a viewing angle at which the driver can always pay attention, and may be specified in advance, for example, a range of ± 20 ° in the traveling direction.

  Furthermore, even if a certain moving body is to be noted, the driver needs to turn his gaze in multiple directions, such as an object different from the moving body and the traveling direction. Accordingly, although it is natural that the driver removes his / her line of sight from the moving body to be noted during driving, it is a heavy burden on the driver to keep an eye on the moving body that has lost his / her line of sight. Therefore, a configuration may be adopted in which an operation is detected for a moving body that has been detected to exist at least once in the driver's line of sight.

  That is, it is detected that a certain moving body exists at least once in the driver's line-of-sight direction, and after that, even if the moving body deviates from the driver's line-of-sight direction, the movement is detected for the moving body, To do. According to this configuration, even a moving body that has removed the line of sight after the driver has gazed once can be reliably warned.

  Further, the warning may be performed when the moving body is present in the warning area, but the warning may be configured by clarifying the conditions that require the warning. For example, the distance between the moving body and the own vehicle is detected based on the movement of the moving body and the own vehicle, and a warning is issued when it is detected that the moving body is approaching the own vehicle within the warning area. The configuration to be performed can be adopted.

  That is, if the distance between the moving body and the host vehicle is detected and the time change is specified, the time change of the distance between the two is found, so whether the moving body is relatively close to the host vehicle or not. Can be determined. Therefore, if the moving body approaches the host vehicle within the warning area, the moving body moving away from the host vehicle within the warning area can be excluded from the warning target. It is possible to reliably call attention to a moving body that a person should be careful of.

  Further, the method of detecting the motion of the moving body existing in the line-of-sight direction and issuing a warning when the moving body exists in the warning area set around the host vehicle as in the present invention is also applied as a program or method. Is possible. In addition, the above-described driving support device, program, and method may be realized as a single driving support device, or may be realized by using parts shared with each part provided in the vehicle. The embodiment is included. For example, it is possible to provide a navigation device, a method, and a program that include the driving support device as described above. Further, some changes may be made as appropriate, such as a part of software and a part of hardware. Furthermore, the invention is also established as a recording medium for a program for controlling the driving support device. Of course, the software recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium to be developed in the future.

Here, embodiments of the present invention will be described in the following order.
(1) Configuration of navigation device:
(2) Driving support processing:
(2-1) Warning processing:
(3) Other embodiments:

(1) Configuration of navigation device:
FIG. 1 is a block diagram showing a configuration of a navigation device 10 including a driving support device according to the present invention. The navigation device 10 includes a control unit 20 including a CPU, RAM, ROM, and the like and a storage medium 30, and the control unit 20 can execute a program stored in the storage medium 30 or the ROM. In the present embodiment, the navigation program 21 can be implemented as one of the programs, and various information is output by a display unit or a speaker, which will be described later, to implement route guidance of the host vehicle. In addition, the navigation program 21 has a function of issuing a warning regarding a moving object existing in a warning area set around the host vehicle as one of its functions.

  The own vehicle (the vehicle on which the navigation device 10 is mounted) has a GPS receiver 40 a, a vehicle speed sensor 40 b, a gyro sensor 40 c, a microphone 41, an in-vehicle camera 42, and an out-of-vehicle camera 43 in order to realize the above functions by the navigation program 21. A display unit 44 and a speaker 45 are provided, and exchange of signals between these units and the control unit 20 is realized by an interface (not shown).

  The GPS receiver 40a receives radio waves from GPS satellites and outputs information for calculating the current position of the host vehicle via an interface (not shown). The control unit 20 acquires this signal and acquires the current position of the host vehicle. The vehicle speed sensor 40b outputs a signal corresponding to the rotational speed of the wheels provided in the host vehicle. The control unit 20 acquires this signal via an interface (not shown), and acquires the speed of the host vehicle. The gyro sensor 40c outputs a signal corresponding to the angular velocity of the host vehicle. The control unit 20 acquires this signal via an interface (not shown), and acquires information indicating the direction of the host vehicle. The vehicle speed sensor 40b and the gyro sensor 40c are used for correcting the current position of the host vehicle specified from the output signal of the GPS receiver 40a and detecting the direction of the host vehicle.

  The microphone 41 generates an electrical signal corresponding to sound generated in the passenger compartment of the host vehicle, and outputs it through an interface (not shown). The control unit 20 acquires this signal and acquires audio information corresponding to the audio. The in-vehicle camera 42 is attached to the host vehicle so as to include the face of the driver who drives the host vehicle in the field of view, and outputs image information indicating the captured image. The control unit 20 acquires this image information via an interface (not shown), and identifies the orientation of the face included in the image using a three-dimensional coordinate system in which a fixed origin is defined at the reference position of the host vehicle. Detect direction. Of course, in addition to the above-described configuration, the coordinate system may be a coordinate using the reference set on the road surface as the origin, a coordinate using latitude or longitude, and 2 defined in a plane parallel to the road surface. The line-of-sight direction may be detected by a dimensional coordinate system, and various configurations can be employed.

  The outside camera 43 is attached to the own vehicle so as to include the periphery of the own vehicle in the field of view and outputs image information indicating the photographed image in order to acquire an image including a moving body that should alert the driver. To do. The control unit 20 acquires this image information via an interface (not shown). In the present embodiment, the outside camera 43 is attached to the host vehicle so as to capture a range around the host vehicle that can be included in the field of view of the driver of the host vehicle.

  FIG. 2 is an explanatory view showing an arrangement example of the in-vehicle camera 42 and the out-of-vehicle camera 43 in the own vehicle. In FIG. 2, the outer periphery of the host vehicle 100 and the interior of the passenger compartment are schematically shown together with the driver D. In this example, the in-vehicle camera 42 is attached to the periphery of the rearview mirror in the passenger compartment of the host vehicle 100 and can capture an image with a field of view including the face of the driver D. In FIG. 2, the field of view of the in-vehicle camera 42 is indicated by a one-dot chain line. In addition, the outside camera 43 is attached to the left and right side mirrors S of the host vehicle 100. In FIG. 2, the field of view of the outside camera 43 is indicated by a two-dot chain line.

  Further, the control unit 20 outputs a signal for outputting an arbitrary image and a signal for performing arbitrary audio output via an interface (not shown). As a result, the display unit 44 and the speaker 45 output an arbitrary sound and image corresponding to the above-described signal. In the present embodiment, as will be described later, the display unit 44 and the speaker 45 are used to give a warning regarding a moving object, route guidance of the host vehicle, and the like.

  The storage medium 30 stores map information 30a for performing guidance by the navigation program 21. The map information 30a includes node data indicating nodes set on a road, link data indicating connection between nodes, data indicating a target, and the like, and is used for specifying the position of the own vehicle and guiding to a destination. Is done. The data indicating the target is used for guiding the target itself, and includes information (for example, facility name, facility type, etc.) regarding the target. Furthermore, the navigation program 21 in the present embodiment has a function of adding data when there is no data indicating the target. Therefore, after the data is once added, guidance based on the added data is performed.

  In the present embodiment, the navigation program 21 causes the navigation device 10 to function as a driving support device according to the present invention by cooperating with the above-described units. Therefore, the navigation program 21 includes a navigation processing unit 21a, a sound detection unit 21b, a line-of-sight direction detection unit 21c, an image information acquisition unit 21d, a moving body motion detection unit 21e, and a moving body warning unit 21f.

  The navigation processing unit 21a is a module that performs processing for outputting a route for guiding the host vehicle to the above-mentioned target from the above-described display unit 44 and the speaker 45, route search processing, and the like. A host vehicle motion detection unit 21a1 that detects the position of the vehicle is provided. The host vehicle motion detection unit 21a1 acquires the current location of the host vehicle based on signals output from the GPS receiver 40a, the vehicle speed sensor 40b, and the gyro sensor 40c and the map information 30a, and acquires the direction of the host vehicle. . Note that the position and orientation of the host vehicle need only be specified with respect to the road surface, and a coordinate system for specifying the position and orientation on the road surface may be defined in advance, or the latitude and longitude with the specified node The position and orientation on the road surface may be specified by the relative relationship, and various configurations can be adopted.

  The voice detection unit 21b acquires voice information based on a signal output from the microphone 41, and detects a word that may be uttered by pointing at the moving body based on the voice information. Here, the words that may have been pointed to the moving body are the types, names, pronouns, etc. of objects that can move, such as words such as “pedestrian”, “bicycle”, “car”, and “that” It is possible to adopt a combination such as “that bicycle”.

  The line-of-sight direction detection unit 21c detects the line-of-sight direction of the driver based on a signal output from the in-vehicle camera 42. In the present embodiment, a configuration is adopted in which the line-of-sight direction is detected based on the direction of the driver's face, and the line-of-sight direction detection unit 21c detects the eyes and nose from the image of the in-vehicle camera 42 by pattern matching, and the relative The direction of the face is determined from the relationship, and the direction of the face is set as the line-of-sight direction.

  Since the line-of-sight direction is defined by a coordinate system based on a fixed origin set in the host vehicle, in this embodiment, in order to identify the line-of-sight direction on the road surface, the current position of the host vehicle is further determined. The direction is specified as described above. Then, the line-of-sight direction with respect to the road surface is detected based on the line-of-sight direction in the coordinate system based on the fixed origin set for the vehicle, and the current location and direction of the vehicle. Of course, the configuration for detecting the line-of-sight direction is not limited to the processing based on the face direction as described above, and the position of the eye, the direction of the eyeball, and the like are detected by a camera in the vehicle interior of the host vehicle and various sensors. A configuration in which the line-of-sight direction is detected based on the position of the eye, the direction of the eyeball, and the like may be employed.

  Furthermore, in this embodiment, in order to perform the process of determining whether or not to add data indicating the target to the map information 30a, the line-of-sight direction detection unit 21c detects the line-of-sight direction detected as described above. It is determined whether data indicating the corresponding target has already been stored in the map information 30a.

  The image information acquisition unit 21d is a module that acquires image information output from the vehicle camera 43. The image information acquisition unit 21d acquires image information at specific time intervals and stores it in a memory (not shown). The moving body motion detection unit 21e acquires the image information stored in the memory and analyzes the image indicated by the image information to detect the motion of the moving body existing in the above-described line-of-sight direction. In other words, in the present embodiment, each position of the image captured by the outside camera 43 in advance corresponds to a direction viewed from a reference position (for example, the position of the driver's head) set in the host vehicle. The direction seen from the driver is specified by specifying the position of the image.

  Further, in the present embodiment, the image information output by the outside camera 43 is acquired at specific time intervals. Therefore, by performing inter-frame comparison between consecutive images taken at different times, Can be detected. Therefore, by comparing the position in the image corresponding to the line-of-sight direction detected by the line-of-sight direction detection unit 21c and the position of the moving body in the image, the moving body motion detection unit 21e is a moving body that exists in the line-of-sight direction. Detecting the movement of In the present embodiment, the movement of the moving body is detected by detecting the position of the moving body based on the image. As the coordinate system for specifying the position of the moving body, various coordinate systems such as the above-described three-dimensional coordinate system can be used.

  Further, in the present embodiment, the moving body motion detection unit 21e detects the line-of-sight direction included in a range excluding the predetermined range including the traveling direction of the host vehicle, and exists in the line-of-sight direction included in the range excluding the predetermined range. A moving object is detected and a warning is made. In the present embodiment, in the above-described three-dimensional coordinate system, a line that is parallel to the traveling direction of the host vehicle and passes through the origin of the driver's line of sight is defined in advance, and is parallel to the road surface with respect to the line. The range of ± α ° (for example, ± 20 °) is set so as to be excluded from the detection range of the moving body. In other words, since the predetermined range including the traveling direction of the host vehicle is almost always included in the driver's field of view, it is troublesome for the driver to move the moving object existing in the predetermined range to the warning target. Attention to moving objects outside the range can be distracting.

  Therefore, in the present embodiment, by detecting a moving body that exists in the line-of-sight direction included in the range excluding the predetermined range, a configuration that does not detect the moving body that exists in the line-of-sight direction included in the predetermined range, The moving object existing in the area where the warning is required is configured to be a warning target. In addition, with this configuration, it is possible to exclude a moving body that exists in an area where no warning is required from being a warning target. The predetermined range only needs to be a viewing angle that allows the driver to always pay attention, and various ranges can be set as the predetermined range.

  Furthermore, the moving body motion detection unit 21e adds data indicating the target to the map information 30a when the moving body existing in the line-of-sight direction is not detected in the image acquired by the image information acquisition unit 21d. Process. That is, it is determined that the data indicating the target corresponding to the above-described line-of-sight direction is not stored in the map information 30a in the above-described line-of-sight direction detection unit 21c, and the movement that exists in the line-of-sight direction in the moving body motion detection unit 21e When a body is not detected, the image information acquired by the image information acquisition unit 21d is associated with the position where the image indicated by the image information is captured, and information indicating these is added to the map information 30a.

  The moving body warning unit 21f is configured so that the moving body is located around the own vehicle based on the position of the moving body specified as described above and the position of the own vehicle specified by the own vehicle operation detecting unit 21a1. It is determined whether or not the moving object is approaching the host vehicle. A warning is issued when it is determined that the moving body is approaching the host vehicle within the warning area.

  Note that the warning area is determined in advance in the above-described three-dimensional coordinate system, and in this embodiment, the warning area is at a certain distance from the reference position set on the vehicle (for example, the origin of the above-described three-dimensional coordinate system). The area is a warning area. Further, whether or not the moving body exists in the warning area can be specified by the position of the moving body detected as described above, and whether or not the moving body is approaching the host vehicle. The determination can be made by specifying the time change of the position of the moving body and the position of the host vehicle.

  In the present embodiment, a warning is provided by the display unit 44 and the speaker 45. That is, the moving body warning unit 21f outputs a control signal for outputting information indicating the warning (for example, image information indicating the warning or audio information such as “the moving body is approaching”) when performing the warning. The information is output to the display unit 44 and the speaker 45, and the information is provided to the driver.

  In the above configuration, when a moving body existing in the line-of-sight direction is detected, the subsequent movement of the moving body is detected, and the warning is given when the moving body is present in a warning area set around the host vehicle. I do. Therefore, when a driver pays attention to a moving object existing around the host vehicle, even if the database is not prepared in advance for the moving object, a warning is given to the moving object, and the driver's attention Can be aroused. Furthermore, after the driver pays attention to the moving body, a warning is given when the moving body approaches the host vehicle even if the mobile body is not always paying attention to the moving body. Very small and easy to secure safe delivery.

(2) Driving support processing:
Next, the process which the navigation apparatus 10 implements in the above structure is demonstrated. When the navigation program 21 is executed by the navigation device 10, the processing shown in FIG. 3 is executed by the voice detection unit 21b to the moving body warning unit 21f at regular intervals (for example, 100 ms). The navigation processing unit 21a performs processing in parallel with the processing shown in FIG. 3. In the navigation processing unit 21a, the own vehicle motion detection unit 21a1 detects the position and orientation of the own vehicle in the course of this processing. ing.

  In the process shown in FIG. 3, first, the voice detection unit 21b determines whether or not the driver has spoken voice corresponding to the moving body (step S100). That is, the voice detection unit 21b specifies voice information based on the output signal of the microphone 41, and indicates whether or not the voice is a voice corresponding to a predetermined moving body, that is, pointing to the moving body. Determine whether a possible word is included.

  Next, the gaze direction detection unit 21c detects the gaze direction of the driver, and the moving body motion detection unit 21e determines whether or not the gaze direction is out of a predetermined range (that is, the gaze direction includes the traveling direction of the host vehicle). It is determined whether it is included in a range excluding the predetermined range (step S105). FIG. 4 is an explanatory diagram for explaining an example of the own vehicle traveling toward the intersection, the line-of-sight direction, and the moving body. In FIG. 4, a line (broken line shown in FIG. 4) parallel to the traveling direction of the host vehicle passing through the center of the driver's seat is used as a reference for defining the angle of the line of sight, and the reference set for the driver's seat The range of ± α ° viewed from the point (hatched area shown in FIG. 4) is excluded from the detection range of the moving object.

  Therefore, the line-of-sight direction detection unit 21c acquires the output signal of the in-vehicle camera 42, detects eyes and nose by pattern matching from the captured image of the in-vehicle camera 42 based on the signal output from the in-vehicle camera 42, The orientation of the face is determined from the relative relationship, and the orientation of the face is set as the line-of-sight direction. Then, the moving body motion detection unit 21e determines whether or not the line-of-sight direction is included in the predetermined range. If it is determined in step S105 that the line-of-sight direction is included in the predetermined range including the traveling direction of the host vehicle, the driver's line-of-sight is considered to be the line-of-sight direction by normal driving, and no processing is performed on the object in the line-of-sight direction. The process is terminated without performing.

  When it is determined in step S105 that the line-of-sight direction is included in a range excluding the predetermined range including the traveling direction of the host vehicle, the moving body motion detection unit 21e further includes a target existing at a position included in the line-of-sight direction. It is determined whether or not data indicating an object has already been stored in the map information 30a (step S110). That is, since the position of each target included in the map information 30a is specified, the line-of-sight direction detection unit 21c is included in the line-of-sight direction with respect to the road surface based on the position and direction of the host vehicle and the line-of-sight direction. It is determined whether or not there is a target.

  When it is determined in step S110 that the data indicating the target existing at the position included in the line-of-sight direction is already stored in the map information 30a, in this embodiment, the driver pays attention to the target instead of the moving object. The navigation processing unit 21a guides information about the target (step S115). That is, the navigation processing unit 21a refers to the map information 30a, acquires information on the target at a position overlapping in the line-of-sight direction, and outputs a signal for guiding the target to the display unit 44, the speaker 45, and the like. Output.

  As a result, information on the target can be displayed on the display unit 44 or the display without merely registering the driver for guidance or the like, or by simply viewing the target without operating the navigation device 10 or the like. Guided from the speaker 45 or the like. Therefore, the driver can make a proposal for the target without performing complicated operations during driving.

  On the other hand, when it is not determined in step S110 that the data indicating the target existing at the position included in the line-of-sight direction has been stored in the map information 30a, the situation where the driver focuses on a moving body other than the target; It can be assumed that the driver pays attention to the target but the map information 30a does not store data related to the target and thus cannot perform guidance.

  Therefore, in the present embodiment, if it is not determined in step S110 that the data indicating the target existing at the position included in the line-of-sight direction has been stored in the map information 30a, the image information acquisition unit 21d performs the camera outside the vehicle. Image information is acquired from 43 (step S120). Then, the moving body motion detection unit 21e determines whether or not there is a moving body in the driver's line-of-sight direction based on the image indicated by the image information (step S125). That is, the moving body motion detection unit 21e performs inter-frame comparison based on the image information to detect the moving body in the image, and a position in the image corresponding to the line-of-sight direction detected by the line-of-sight direction detection unit 21c; By comparing the position of the moving body, it is determined whether or not there is a moving body that exists in the line-of-sight direction.

  If it is determined in step S125 that a moving body is present in the direction of the driver's line of sight, warning processing for performing a warning regarding the moving body is executed (step S130). If it is not determined in step S125 that a moving body exists in the driver's line of sight, the image information acquired by the image information acquisition unit 21d is associated with the position where the image indicated by the image information is captured. Then, a registration process for adding information indicating these to the map information 30a is performed (step S135).

(2-1) Warning processing:
Next, details of the warning process performed in step S130 will be described. When it is determined that a moving body exists in the driver's line-of-sight direction, first, the moving body motion detection unit 21e acquires the movement of the moving body that exists in the driver's line-of-sight direction (step S200). That is, the moving body in the image is detected by performing the inter-frame comparison between the images indicated by the image information acquired by the image information acquisition unit 21d, and the position of the moving body is detected.

  In this embodiment, in step S200, once the moving body is detected to be present in the driver's line-of-sight direction and the detection of the movement is started, the movement is traced. That is, in the present embodiment, when the navigation program 21 is being executed, the processing shown in FIG. 4 is also executed at regular intervals (for example, 100 ms) in addition to the processing shown in FIG. At this time, the detection of the movement is continued unless a certain condition (for example, when moving away from a predetermined distance beyond the rear of the host vehicle) is reached for the moving body once the detection of the movement is started. As a result, for a moving body that is detected to exist at least once in the direction of the driver's line of sight, the operation is detected until a certain condition is reached.

  Next, the host vehicle motion detection unit 21a1 acquires information indicating the current position of the host vehicle that has been continuously acquired by the navigation processing unit 21a (step S205). That is, information indicating the position of the host vehicle is acquired. The information indicating the movement of the moving body acquired in step S200 and the information indicating the movement of the host vehicle acquired in step S205 are transferred to the moving body warning unit 21f.

  The moving body warning unit 21f determines whether or not there is a moving body in the warning area set around the own vehicle based on the information indicating the operation of the moving body and the information indicating the operation of the own vehicle. Step S210). That is, the position of the moving body and the host vehicle is detected based on the information indicating the operation of the moving body and the information indicating the operation of the host vehicle, and the distance between the two is specified. Then, the distance from the reference position set to the host vehicle to the boundary of the warning area at a certain distance is compared with the distance from the host vehicle to the moving body, and the distance from the reference position set to the host vehicle is set to a fixed distance. When the distance to the boundary of a certain warning area is longer than the distance from the vehicle to the moving body, it is determined that the moving body exists in the warning area.

  If it is not determined in step S210 that there is a moving body in the warning area set around the host vehicle, the process after step S215 is skipped and the process is terminated. On the other hand, when it is determined in step S210 that a moving body exists in the warning area set around the host vehicle, the moving body warning unit 21f determines whether each moving body is approaching the host vehicle ( Step S215).

  That is, the moving body warning unit 21f detects the positions of the moving body and the host vehicle based on the information indicating the operation of the moving body and the information indicating the operation of the host vehicle, and a plurality of continuous distances between the two. Detection is based on image information over frames. And it is specified whether the relative distance of a mobile body and the own vehicle has become short by detecting the time change of distance.

  In step S215, a warning is issued for a moving object that is determined to be approaching the host vehicle (step S220), and for a moving object that is not determined to be approaching the host vehicle, step S220 is skipped and the warning is issued. Not done. That is, when there is a moving body that is determined to be approaching the host vehicle, the moving body warning unit 21f detects the moving body and displays image information and audio information that warn of the approach of the moving body. 44 and the speaker 45. As a result, guidance for warning the approach of the moving body is output from the display unit 44 or the speaker 45. Therefore, even if the driver removes his / her line of sight from the moving body once focused, the driver is alerted when a situation in which the driver should pay attention is reached.

  That is, even if a certain moving body is to be noted, the driver needs to turn his / her line of sight in multiple directions, such as an object different from the moving body and the traveling direction. Accordingly, although it is natural that the driver removes his / her line of sight from the moving body to be noted during driving, it is a heavy burden on the driver to keep an eye on the moving body that has lost his / her line of sight. Therefore, in the present embodiment, a configuration is adopted in which an operation is detected for a moving body that has been detected to exist at least once in the driver's line-of-sight direction. Then, even if it is detected that a certain moving body exists at least once in the driver's line-of-sight direction, and then deviates from the driver's line-of-sight direction, the operation is detected for the moving body and set as a warning target. With this configuration, even a moving body that has removed its line of sight after the driver has watched it can be reliably warned and the burden on the driver can be reduced.

In FIG. 4, the boundary of the warning area is indicated by a sector A indicated by a solid line, and pedestrians P _{0 to} P ₂ are illustrated as examples of the moving body. In the example shown in FIG. 4, the pedestrians P ₀ and P ₁ are included in the warning area indicated by the sector A and not included in the pedestrian P ₂ . Further, the movement direction vectors of the pedestrians P ₀ and P ₁ are illustrated as V ₀ and V ₁ , respectively. In the example shown in the figure 4, the pedestrian P ₀ is walking toward the lower side, it is approaching the host vehicle. Furthermore, the host vehicle is traveling toward the intersection on the upper side of the page.

Accordingly, the pedestrian P ₀ is a moving body approaching the host vehicle within the warning area. For this reason, in this example, for the pedestrian P ₁ , the moving body warning unit 21 f outputs information for warning to the display unit 44 and the speaker 45. As a result, for example, audio information such as “left front, moving object is approaching” is output from the speaker 45. Therefore, even if the driver pays attention to the pedestrian P ₀ and then removes his / her line of sight, the driver is warned when the pedestrian P ₀ approaches the own vehicle, and the driver is warned.

On the other hand, the pedestrian P ₁ is walking toward the right side, moving away from the vehicle. The own vehicle is traveling toward the intersection on the upper side of the page. Accordingly, the pedestrian P ₁ is not approaching the host vehicle within the warning area. For this reason, in this example, for the pedestrian P ₁ , the moving body warning unit 21 f does not output information for warning to the display unit 44 or the speaker 45, and the warning about the pedestrian P ₁ is not performed. That is, when the driver pays attention to the pedestrian P ₁ and then removes his line of sight, no warning is given if the pedestrian P ₁ does not affect the safety of the host vehicle. For this reason, it is possible to prevent frequent unnecessary warnings.

  Further, in the above-described embodiment, since the line-of-sight direction is detected when sound corresponding to the moving body is detected by the sound detection unit 21b, the sound corresponding to the moving body is assumed to be a moving body to be noticed by the driver. When the is issued, the moving body can be targeted for warning. That is, when the sound corresponding to the moving body is not detected, the line-of-sight direction is not detected, and a warning regarding the presence of the moving body in the warning area is not given. For this reason, no warning is given to any moving object. For example, it is possible to prevent a moving body such as a bird flying around the host vehicle from being detected as a warning target.

(3) Other embodiments:
The above embodiment is an example for carrying out the present invention, as long as it can detect a motion of a moving body existing in the driver's line-of-sight direction and can warn when it exists in the warning area. Various embodiments can be employed. For example, in the above-described embodiment, the line-of-sight direction is detected before the image information is acquired to determine whether or not the target is visually recognized. Of course, this determination may be performed based on the image. Good. For example, based on the image information, the object existing in the line-of-sight direction is specified as a stationary object or a moving object, and when it is a stationary object, it is determined whether or not the above-mentioned target object exists. It is set as the structure which detects operation | movement of a moving body when it is. According to this configuration, even when a moving body exists between the target and the driver, it is possible to accurately warn the moving body.

  Further, in the detection of the gaze direction, it is only necessary to be able to detect the gaze direction of the driver. In addition to the configuration for detecting the direction of the driver's face, the eye position, the direction of the eyeball, etc. A configuration may be adopted in which the line-of-sight direction is detected based on the position of the eye, the direction of the eyeball, and the like, detected by a camera or various sensors. Of course, when detecting the gaze direction of the driver, when the gaze is maintained within a certain range for a certain time, the direction may be adopted as the gaze direction.

  Furthermore, the above-mentioned pedestrian is an example of a moving body, and in the present invention, any object whose position varies with time corresponds to the moving body. The structure for detecting the moving body is a structure for detecting the moving body by extracting a predetermined shape from the image in addition to a structure for detecting the moving body by comparing images taken at different times. Various configurations may be employed.

  The operation of the moving body only needs to be information for specifying whether or not to warn the host vehicle. In addition to the configuration for detecting the position of the moving body as described above, the moving direction, speed, The movement may be specified based on information indicating acceleration or the like. If the images taken at different times are compared, the moving direction and speed can be detected based on the shooting interval of each image and the position of the moving body specified based on each image. Of course, the coordinate system for specifying the position of the moving body only needs to be able to define the relationship with the coordinate system when defining the above-described line-of-sight direction, and may be different coordinate systems or the same coordinate system. Also good.

  Further, in the warning, it is sufficient that the driver can recognize the presence of the moving body, and it is possible to adopt a configuration in which the driver recognizes the presence of the moving body by means of vibration or the like in addition to voice guidance and image display. . The warning area may be set in advance as an area to warn of the presence of the moving body. In addition to the area at a predetermined distance from the own vehicle, an area having a specific shape set around the own vehicle is set as the warning area. Is possible.

  Furthermore, in the above-described embodiment, the configuration is such that a warning is given when the moving object is present in the warning area and the vehicle is approaching, but the warning is simply given when the moving object is present in the warning area. It may be. With this configuration, it is not necessary to detect a temporal change in the relative relationship between the host vehicle and the moving body, so that it is possible to reduce the processing burden on whether or not to issue a warning.

It is a block diagram of the navigation apparatus containing a driving assistance device. It is explanatory drawing which shows the example of arrangement | positioning with a vehicle camera and a vehicle camera outside. It is a flowchart of a driving assistance process. It is explanatory drawing for demonstrating the example of the own vehicle, a gaze direction, and a moving body. It is a flowchart of a warning process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Navigation apparatus, 20 ... Control part, 21 ... Navigation program, 21a ... Navigation processing part, 21a1 ... Own-vehicle motion detection part, 21b ... Audio | voice detection part, 21c ... Gaze direction detection part, 21d ... Image information acquisition part, 21e ... moving body motion detection unit, 21f ... moving body warning unit, 30 ... storage medium, 30a ... map information, 40a ... GPS receiving unit, 40b ... vehicle speed sensor, 40c ... gyro sensor, 41 ... microphone, 42 ... in-vehicle camera, 43 ... External camera 44 ... Display unit 45 ... Speaker

Claims (8)

Gaze direction detection means for detecting the gaze direction of the driver driving the host vehicle;
Image information acquisition means for acquiring image information indicating an image around the host vehicle;
A moving body motion detecting means for detecting a moving body existing in the line-of-sight direction based on the image information and detecting the motion of the moving body;
A moving body warning means for detecting that the moving body is present in a warning area set around the own vehicle based on the operation of the moving body and performing control for warning;
A driving support apparatus comprising: Voice detection means for detecting voice corresponding to the moving body by acquiring voice information indicating voice generated by the driver;
The line-of-sight direction detecting means detects the line-of-sight direction when detecting sound corresponding to the moving body.
The driving support device according to claim 1. The moving body operation detecting means detects a moving body existing in the line-of-sight direction included in a range excluding a predetermined range including a traveling direction of the host vehicle.
The driving support device according to claim 1. The moving body motion detection means detects a motion of a moving body that has been detected to exist at least once in the driver's line-of-sight direction.
The driving assistance apparatus in any one of Claims 1-3. The moving body motion detecting means detects the motion of the moving body by comparing image information of the images acquired at different times.
The driving support device according to any one of claims 1 to 4. Comprising own vehicle operation detecting means for detecting the operation of the own vehicle;
The moving body warning means detects a distance between the moving body and the own vehicle based on an operation of the moving body and an operation of the own vehicle, and the moving body approaches the own vehicle within the warning area. Perform the warning when it is detected that
The driving support device according to any one of claims 1 to 5. A line-of-sight direction detecting step for detecting a line-of-sight direction of a driver driving the vehicle;
An image information acquisition step of acquiring image information indicating an image around the host vehicle;
A moving body operation detecting step of detecting a moving body existing in the line-of-sight direction based on the image information and detecting an operation of the moving body;
A moving body warning step for detecting that the moving body is present in a warning area set around the host vehicle based on the operation of the moving body, and performing control for warning;
Driving support method including. A gaze direction detection function for detecting the gaze direction of the driver driving the host vehicle;
An image information acquisition function for acquiring image information indicating an image around the host vehicle;
A moving body motion detection function for detecting a moving body existing in the line-of-sight direction based on the image information and detecting the motion of the moving body;
A moving body warning function for detecting that the moving body is present in a warning area set around the host vehicle based on the operation of the moving body and performing control for warning;
Driving support program that realizes the computer.

Images