JP6459856B2 - Vehicle driving support device, vehicle driving support method, and program - Google Patents

Description

  The present invention relates to a vehicle driving support device, and more particularly to a vehicle driving support device that is suitable for estimating the possibility that a vehicle driver will perform random driving.

For example, a method as disclosed in Patent Document 1 has been proposed as a method for detecting a vague driver's driving. In Patent Document 1, in advance, by defining the statistical distribution of vehicle driver's gaze information at the time of casual driving, by evaluating the similarity with the statistical distribution of vehicle driver's gaze information obtained at the time of evaluation, Estimate the possibility of casual driving.

Further, for example, a method as disclosed in Patent Document 2 has been proposed as a method for detecting a vague driver's driving. In Patent Document 2, a gaze direction of a vehicle driver is detected for a predetermined time, and gaze distribution information indicating a gaze direction distribution is generated. The predetermined position (for example, the center position) of the distribution indicated by the obtained line-of-sight distribution information is greater than or equal to a predetermined threshold value above the standard position in the line-of-sight direction in the state of non-random driving indicated by the predefined reference information. If it is away, it is determined that the driver is driving casually.

Further, as a method for preventing the vehicle driver from driving freely, for example, a method as disclosed in Patent Document 3 has been proposed. In Patent Document 3, an individual distribution that is an attention distribution based on object, image information, and specific information is generated based on an attention model prepared in advance and specific information detected at that time. Moreover, the position (gaze point) where the driver's attention is directed is stochastically estimated by generating an integrated distribution which is an attention distribution obtained by integrating these individual distributions with the same weight or individually weighted. . Further, the attention amount of the vehicle driver for each object extracted from the front image is calculated according to the integrated distribution. Finally, based on the attention amount, information related to an object that the vehicle driver should recognize is provided by display, voice, vibration, or the like.

Japanese Patent Laid-Open No. 8-178712 JP 2008-243031 A JP 2012-103850 A Japanese Patent Laid-Open No. 3-165737 JP 2010-211460 JP 2007-233487 A JP 2007-89094 JP 2010-97410 A JP 2013-37615 JP 2012-141922 A JP 2012-84003 A JP 2007-241469 A JP 2006-4173 A Japanese Patent No. 3965067 Japanese Patent No. 5042496 Japanese Patent No. 5127182 JP 2006-301464 A JP 2009-107582 JP

Kanamori, T., Hido, S., & Sugiyama, M., A least-squares approach to direct importance estimation, Journal of Machine Learning Research, vol.10 (Jul.), Pp.1391-1445, 2009. Yamada, M., Suzuki, T., Kanamori, T., Hachiya, H., & Sugiyama, M., Relative density-ratio estimation for robust distribution comparison, Neural Computation, vol.25, no.5, pp.1324 -1370, 2013. Sugiyama, M., Kanamori, T., Suzuki, T., du Plessis, MC, Liu, S., & Takeuchi, I., Density difference estimation, IEICE Technical Report IBISML2012-8, pp.49-56, Kyoto, Japan, Jun. 19-20, 2012.

  However, the techniques disclosed in the above Patent Documents 1-3 have the following problems.

Patent Document 1 defines in advance the statistical distribution of the vehicle driver's gaze information at the time of casual driving, and evaluates the similarity with the statistical distribution of the vehicle driver's gaze information obtained at the time of evaluation, We estimate the possibility of driving casually. Therefore, the technique such as Patent Document 1
There is a problem that the estimation accuracy is reduced for a statistical distribution of the gaze information of the vehicle driver at the time of casual driving.

Patent Document 2 discloses a predetermined position of the distribution indicated by the gaze distribution information of the vehicle driver,
Based on the difference from the standard position in the line-of-sight direction in the state that is not the random driving indicated by the reference information stored in advance, the possibility of the random driving is estimated. In the technique such as Patent Document 2, the difference in the gaze distribution information between the state where the driver is not driving and the state where the driver is not driving is not the standard position, but the difference in the statistical distribution (for example, kurtosis, skewness, variance, etc. ), The estimation accuracy is reduced.

Further, Patent Document 3 is based on an attention model prepared in advance and specific information detected from time to time, and distributes objects, image information, individual distributions that are attention distributions based on specific information, and weights the same or individually. An integrated attention distribution is generated. The technique such as Patent Document 3 has a problem that the estimation accuracy decreases when the optimum weight for the individual distribution cannot be set.

  In view of the above-described problems, an object of the present invention is to provide a technique capable of accurately estimating a rough driving.

A first aspect of the present invention is a vehicle driving support device for detecting a vague driving of a vehicle driver,
An image acquisition means for acquiring an image in the forward direction of the vehicle, a gaze information extraction means for extracting gaze information of the vehicle driver, and a time series feature amount based on the gaze information at the time of evaluation of the vehicle driver Series feature quantity calculation means, normal data storage means for storing time series feature quantities based on gaze information during normal operation, and difference calculation means for calculating differences between the time series feature quantities during the normal operation and during the evaluation And a rough driving detection means for detecting the rough driving of the vehicle driver based on the magnitude of the difference.

A second aspect of the present invention is a vehicle driving support method for detecting a vague driving of a vehicle driver,
An image acquisition step for acquiring an image in the forward direction of the vehicle, a gaze information extraction step for extracting gaze information of the vehicle driver, and a time series feature amount based on the gaze information at the time of evaluation of the vehicle driver A difference calculation step of calculating a difference between the time series feature quantity calculated in the series feature quantity calculation step, the time series feature quantity calculated in the evaluation data calculation step, and the time series feature quantity based on gaze information calculated in normal operation in advance; And a rough driving detection step of detecting the rough driving of the vehicle driver based on the magnitude of the difference.

  According to the present invention, it is possible to improve the estimation accuracy of the casual driving.

FIG. 1 (A) is a functional configuration diagram of the vehicle driving support apparatus according to the first embodiment, FIG. 1 (B) is a flowchart showing a flow of learning processing in the first embodiment, and FIG. 3 is a flowchart showing a flow of a random driving detection process in the first embodiment. FIG. 2 is a diagram for explaining the function of the gaze information extraction unit 11 of the vehicle driving support device according to the first embodiment. FIG. 3 is a diagram for explaining the function of the time-series feature amount calculation unit 12 of the vehicle driving support device according to the first embodiment. 4 (A) -4 (B) are diagrams for explaining functions of the difference presentation unit 16 of the vehicle driving support device according to the first embodiment. FIG. 5 (A) is a functional configuration diagram of the vehicle driving support device according to the second embodiment, and FIG. 5 (B) illustrates the function of the gaze information extracting unit 21 of the vehicle driving support device according to the second embodiment. FIG. FIG. 6 is a diagram for explaining the function of the pedestrian detection unit 213 of the vehicle driving support apparatus according to the second embodiment. FIG. 7 is a diagram for explaining a function of the vehicle detection unit 214 of the vehicle driving support device according to the second embodiment. FIG. 8 is a diagram for explaining the function of the sign detection unit 215 of the vehicle driving support apparatus according to the second embodiment. FIG. 9 is a diagram for explaining the function of the traffic light detection unit 216 of the vehicle driving support apparatus according to the second embodiment. FIGS. 10A and 10B are diagrams for explaining the function of the distance information calculation unit 217 of the vehicle driving support apparatus according to the second embodiment. FIG. 11 is a functional configuration diagram of the vehicle driving support apparatus according to the third embodiment.

<First Embodiment>
A first embodiment of the present invention will be described with reference to FIGS.

(Constitution)
FIG. 1 (A) is a block diagram showing a schematic configuration of the vehicle driving support apparatus 1 according to the first embodiment of the present invention. The vehicle driving support apparatus according to the present embodiment can be realized using a semiconductor integrated circuit (LSI). As shown in FIG. 1 (A), the vehicle driving support device 1 includes a gaze information extraction unit 11, a time-series feature amount calculation unit 12, a normal data storage unit 13, an evaluation data storage unit 14, a difference calculation unit 15, and a difference presentation. Unit 16 and stimulus generation unit 17. These components correspond to the functions performed by the vehicle driving support device 1.

The gaze information extraction unit 11 extracts the gaze position of the vehicle driver. The gaze information extraction unit 11 is a camera provided on the vehicle, which is an image obtained by photographing the front direction of the vehicle (hereinafter referred to as a front image) and an image obtained by photographing the eyes of the vehicle driver (hereinafter referred to as an eye image). Or the like from a device external to the vehicle driving support device 1. The gaze information extraction unit 11 analyzes the front image and the eye image, and calculates the gaze position (x _e (t), y _e (t)) of the vehicle driver in the front image at time t as shown in FIG. Extract. The gaze information extraction unit 11 outputs the obtained gaze information f (t) = (x _e (t), y _e (t)) at time t to the time-series feature amount calculation unit 12.

Here, the line-of-sight direction of the vehicle driver is, for example, a difference image between two eye images captured by a visible light-sensitive camera and an infrared light-sensitive camera, as in the line-of-sight detection method of Patent Document 4, with an appropriate threshold value. By comparing with, the iris portion can be extracted to detect the line-of-sight direction. Then, based on the positional relationship between the camera that captures the eye image and the camera that captures the front image, the gaze position (x _e (t), y _e (t)) of the vehicle driver at time t can be detected. it can.

  The time-series feature amount calculation unit 12 calculates the time-series feature amount F (t) using the gaze information f (t) obtained by the gaze information extraction unit 11, and the normal data storage unit 13 and the evaluation data storage unit 14 Output to. Specifically, as shown in FIG. 3, the time series feature amount F (t) is calculated by concatenating K gaze information f (t) continuous in time.

For example, when K = 3, the time-series feature amount F (0) at time t = 0 is
F (0) = (f (0), f (1), f (2)) = (x _e (0), y _e (0), x _e (1), y _e (1), x _e ( 2), y _e (2))
Of 6 (2 × K) dimensional vectors. In addition, the time-series feature amount F (1) at time t = 1 is
F (1) = (f (1), f (2), f (3)) = (x _e (1), y _e (1), x _e (2), y _e (2), x _e ( 3), y _e (3))
Of 6 (2 × K) dimensional vectors. Furthermore, the time-series feature amount F (2) at time t = 2 is
F (2) = (f (2), f (3), f (4)) = (x _e (2), y _e (2), x _e (3), y _e (3), x _e ( 4), y _e (4))
Of 6 (2 × K) dimensional vectors. Thereafter, time-series feature values F (3), F (4),... Can be calculated in the same procedure. Here, the times t = 0, 1, 2, ... are consecutive times in this order (
Time step).

Here, the connection number K of gaze information may be determined by trial and error. Or you may determine according to the discriminating performance of the casual driving required for the whole system. Or you may determine according to the processing time requested | required of the whole system.

Here, a case has been described in which the time-series feature amount F (t) at time t is a combination of continuous K pieces of gaze information starting from time t, but it can be defined by other methods. For example, F (t) = (f (t-2), f (t-1), f (t)), and so on. Can be defined as a combination of gaze information. Alternatively, it is defined as a combination of continuous K gaze information centered on time t, such as F (t) = (f (t-1), f (t), f (t + 1)) Also good.

The normal data storage unit 13 stores a time-series feature amount extracted in advance for a predetermined time in a state where it is not a random operation (hereinafter referred to as normal operation), and outputs it to the difference calculation unit 15. Specifically, time series feature values F (0), F (1),..., F (T) from time t = 0 to T obtained by using the time series feature quantity calculation unit 12 are stored. Here, the time T may be determined according to the size of the memory capacity of the normal data storage unit 12. Or you may determine according to the processing speed requested | required of the whole system.

The evaluation data storage unit 14 stores a time series feature amount extracted for a predetermined time at the time of evaluation, and outputs it to the difference calculation unit 15. Specifically, time series feature values F ′ (0), F ′ (1),..., F ′ (T from time t ′ = 0 to T ′ obtained using the time series feature quantity calculation unit 12 ') Is stored. Here, the time T ′ may be determined according to the memory capacity of the evaluation data storage unit 14. Or you may determine according to the processing time requested | required of the whole system. Alternatively, the time T in the normal data storage unit 12 may be given.

The difference calculation unit 15 includes T time series feature values F (0), F (1), F (2), stored in the normal data storage unit 13.
..., F (T) and T 'time-series feature values F' (0), F '(1), F' (2), ..., F '(T') stored in the evaluation data storage unit 14 Is calculated and output to the difference presentation unit 16 and the stimulus generation unit 17.
Here, the difference D in the statistical distribution of the two time-series feature quantities is that a sample consisting of T time-series feature quantities F (0), F (1), F (2), ..., F (T) Based on the probability density followed and the probability density followed by the sample consisting of T 'time series features F' (0), F '(1), F' (2), ..., F '(T') presume.

For example, using the density ratio estimation method of Non-Patent Document 1, the probability density for T time series feature quantities F (0), F (1), F (2),..., F (T) and T ′ Estimate the probability density ratio (density ratio) for each of the time series features F '(0), F' (1), F '(2), ..., F' (T ') The difference D may be calculated based on the ratio. The probability density ratio can be calculated by, for example, the relative density ratio estimation method described in Non-Patent Document 1, the least square density ratio fitting method described in Non-Patent Document 2, the probabilistic classification method, the product moment fitting method, and the like. . Alternatively, the difference D in the statistical distribution of two time-series feature quantities is, for example, the probability density for T time-series feature quantities F (0), F (1), F (2), ..., F (T) And estimate the probability density difference (density difference) for T 'time-series features F' (0), F '(1), F' (2), ..., F '(T'). You may calculate based on the obtained density difference. The probability density difference can be calculated by a probability density difference estimation method described in Non-Patent Document 3, for example.

The difference presentation unit 16 presents the difference D obtained by the difference calculation unit 15 to the vehicle driver in the form of visual information. For example, the difference presentation unit 16 visualizes the difference D in an intuitively easy-to-understand figure and presents it to the vehicle driver. Specifically, as shown in FIG. 4 (A), the larger the difference D, the higher the possibility of a rough driving (bad), and the smaller the difference D, the higher the possibility of normal driving (good). ) Should be presented in an intuitive manner. Alternatively, a figure as shown in FIG. 4B may be used. Alternatively, a figure as shown in FIG. 4C may be used. Alternatively, a figure as shown in FIG. 4 (D) may be used. Thus, the vehicle driver can be alerted by presenting the figure as shown in FIGS. 4 (A) -4 (D) to the vehicle driver. In addition to the graphic, the difference presentation unit 16 may present the difference D to the vehicle driver by outputting characters (text).

Here, the information presentation unit 16 is described to always present the difference D, but the difference D may be presented only when the magnitude of the difference D is larger than a predetermined threshold Th. That is, presentation with visual information may be performed only when there is a high possibility of casual driving. The threshold value Th can be calculated using, for example, an intersection confirmation method.

The stimulus generation unit 17 gives a stimulus to the vehicle driver based on the difference D obtained by the difference calculation unit 15. Specifically, the driver's seat may be vibrated (vibrated) according to the magnitude of the difference D. Or you may vibrate the handle which a vehicle driver grasps according to the size of difference D. Alternatively, the seat belt may be vibrated by controlling the tightening strength of the seat belt according to the size of the difference D. Alternatively, a warning sound may be emitted according to the magnitude of the difference D.
Or, depending on the size of the difference D, “It ’s safe driving. That ’s how it ’s going.” “It looks like the attention is distracting.” “Would you be tired? Rest in the next service area. You may call attention with a voice such as "Would you like to try it?" Moreover, you may utilize combining these some methods.

  If the magnitude of the difference D obtained by the difference calculation unit 15 is smaller than the predetermined threshold Th ′, it is determined that the driving is normal driving (not random driving), and the output by the stimulus generation unit 17 is not performed. May be. The threshold value Th ′ may be the same value as or different from the threshold value Th described above.

(Method)
A flow of processing performed by the vehicle driving support device 1 of the present embodiment will be described.

First, normal data learning processing will be described with reference to FIG. In step S11, the forward image and the eye image taken in normal driving (non-manage driving) are input to the gaze information extraction unit 11, and the gaze information extraction unit 11 extracts gaze information from these images by the method described above. To do. In step S12, the time-series feature amount calculation unit 12 calculates the time-series feature amount using the extracted gaze information. When a plurality of images related to normal driving are prepared, gaze information is extracted from each of the plurality of images and a time-series feature amount is calculated. In step S13, the time-series feature amount calculation unit 12 stores the time-series feature amount calculated in step S12 in the normal data storage unit 13 as normal data.

Next, with reference to FIG. 1 (C), a process at the time of detecting a random operation will be described. The process shown in FIG. 1 (c) is repeatedly executed periodically during operation, for example. In step S21, the forward image and the eye image taken during driving are input to the gaze information extraction unit 11, and the gaze information extraction unit 11 extracts gaze information from these images by the above-described method. And in step S22,
The time series feature quantity calculation unit 12 calculates a time series feature quantity using the extracted gaze information. In step S23, the difference calculation unit 15 calculates the difference D between the time series feature quantity calculated in step S22 and the statistical distribution of the time series feature quantity stored in the correct data storage unit 13 during normal operation. .
In step S24, the difference presentation unit 16 and the stimulus generation unit 17 alert the vehicle driver by outputting an image and applying a stimulus according to the magnitude of the difference D calculated in step S23.

(Advantageous effects of this embodiment)
According to the vehicle driving support device 1 according to the present embodiment, it is possible to accurately detect the vague driving of the vehicle driver. Moreover, by detecting the tendency of random driving and giving a stimulus to the vehicle driver based on the tendency, it is possible to prevent the rough driving.

  As in the prior art, the method of discriminating between normal driving and rough driving based on the statistical distribution of vehicle driver's line-of-sight information extracted during rough driving cannot deal with unknown abnormal patterns or has complicated factors. There is a problem that it is difficult to define an entangled pattern. In this embodiment, the statistical distribution of the vehicle driver's line-of-sight information extracted during normal driving is stored as correct data, and casual driving is determined based on the consensus with the statistical distribution of the line-of-sight information during driving. Yes. Therefore, it is possible to deal with unknown abnormal patterns. Also, it is not necessary to define an abnormal pattern involving complicated factors. As described above, according to the present embodiment, it is possible to accurately detect a rough driving.

<Second Embodiment>
A second embodiment of the present invention will be described with reference to FIGS.

FIG. 5 (A) is a block diagram showing a schematic configuration of the vehicle driving support apparatus 2 according to the second embodiment of the present invention. The vehicle driving support apparatus according to the present embodiment can be realized using a semiconductor integrated circuit (LSI). As shown in FIG. 5A, the vehicle driving support device 2 includes a gaze information extraction unit 21, a time-series feature amount calculation unit 22, a normal data storage unit 23, an evaluation data storage unit 24, a difference calculation unit 25, a difference presentation Part 26 and stimulus generator 27. These components correspond to the functions performed by the vehicle driving support device 2. Note that in the present embodiment, only differences from the first embodiment will be mentioned.

In the first embodiment, the gaze information extraction unit 11 calculates the gaze position (x _e (t), y _e (t)) of the vehicle driver in the forward image at time t, and the obtained time t While the gaze information f (t) = (x _e (t), y _e (t)) is output to the time-series feature amount calculation unit 12, in this embodiment, the vehicle driver's time t
Eye movement as well _{(x e (t), y} e (t)), a variety of other information (e.g., vehicle operator's line of sight information (x _e (t), the object in the y _e (t)) The attribute information, distance information, or biological information of the vehicle driver is used to increase the amount of gaze information.

As shown in FIG. 5 (B), the gaze information extraction unit 21 includes an image acquisition unit 211, a gaze position detection unit 212, a pedestrian detection unit 213, a vehicle detection unit 214, a sign detection unit 215, a traffic light detection unit 216, and distance information. The calculation unit 217 includes a biological information acquisition unit 218 and a gaze information calculation unit 219.

The image acquisition unit 211 acquires an image in the forward direction of the vehicle (hereinafter referred to as a forward image) input from the outside of the vehicle driving support device 2, and a line-of-sight position detection unit 212, a pedestrian detection unit 213, a vehicle detection unit 214
, Output to the sign detection unit 215, the traffic light detection unit 216, the distance information calculation unit 217, and the biological information acquisition unit 218. The image acquisition unit 211 also acquires an image of the eyes of the vehicle driver (hereinafter referred to as an eye image) input from the outside of the vehicle driving support device 1 and outputs the acquired image to the line-of-sight position detection unit 212.

The line-of-sight position detection unit 212 analyzes the front image and the eye image, and calculates the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver in the front image at time t as shown in FIG. Extract. The line-of-sight position extraction method may be performed by a method similar to the method described in the first embodiment.

As shown in FIG. 6, the pedestrian detection unit 213 performs horizontal and horizontal positions of a rectangular area including the position (x _p (t), y _p (t)) of the pedestrian p in the forward image at time t and the pedestrian p. The vertical size (h _p (t), v _p (t)) is calculated and output to the gaze information calculation unit 219. Here, the position of the pedestrian p in the forward image (x _p (t), y _p (t)) and the size of the rectangular area encompassing the pedestrian p (h _p (t), v _p (t)) may be calculated by any existing method. For example, the methods described in Patent Documents 5, 6, and 7 can be used.

At this time, the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t is the size (h in the position (x _p (t), y _p (t)) in the forward image. _When it is inside the rectangular area of _p (t), v _p (t)), it can be seen that the vehicle driver visually observed the pedestrian p at time t. On the other hand, the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t is a size (h _p at the position (x _p (t), y _p (t)) in the forward image. When it is outside the rectangular area of (t), v _p (t)), it can be seen that the vehicle driver did not visually observe the pedestrian p at time t. Also, when pedestrian p is not detected in the front image at time t, the horizontal and vertical size of the front image is (W, H), and (x _p (t), y _p (t)) = (W / 2, H / 2), (h _p (t), v _p (t)) = (W / 2, H / 2) may be output.

As shown in FIG. 7, the vehicle detection unit 214 is arranged in the horizontal and vertical directions of the position (x _c (t), y _c (t)) of the vehicle c in the forward image at the time t and the rectangular area including the vehicle c. Calculate the size (h _c (t), v _c (t))
Output to the gaze information calculation unit 219. Here, the position (x _c (t), y _c (t)) of the vehicle c in the front image and the horizontal and vertical sizes (h _c (t), v _c ( t)) may be calculated by any existing method. For example, the methods described in Patent Documents 8 and 9 can be used.

At this time, the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t is a size (h in the position (x _c (t), y _c (t)) in the forward image. _When it is inside the rectangular area of _c (t), v _c (t)), it can be seen that the vehicle driver visually observed the vehicle c at time t. On the other hand, the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t is a size (h _c ) at the position (x _c (t), y _c (t)) in the forward image. When it is outside the rectangular area (t), v _c (t)), it can be seen that the vehicle driver did not visually observe the vehicle c at time t. If the vehicle c is not detected in the front image at time t, the horizontal and vertical sizes of the front image are (W, H), and (x _c (t), y _c (t)) = ( W / 2, H / 2), (h _c (t), v _c (t)) = (W / 2, H / 2) may be output.

As shown in FIG. 8, the sign detection unit 215 is arranged in the horizontal and vertical directions of the rectangular area including the position (x _s (t), y _s (t)) of the sign s in the forward image at the time t and the sign s. Calculate the size (h _s (t), v _s (t))
Output to the gaze information calculation unit 219. Here, the position of the sign s in the forward image (x _s (t), y _s (t)) and the horizontal and vertical size (h _c (t), v _c ( t)) may be calculated by any existing method. For example, the methods described in Patent Documents 10 and 11 can be used.

At this time, the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t is a size (h in the position (x _s (t), y _s (t)) in the forward image. _When it is inside the rectangular area of _s (t), v _s (t)), it can be seen that the vehicle driver has visually observed the sign s at time t. On the other hand, the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t is the size (h _s ) at the position (x _s (t), y _s (t)) in the forward image. When it is outside the rectangular area of (t), v _s (t)), it can be seen that the vehicle driver did not visually observe the sign s at time t. In addition, when the sign s is not detected in the front image at the time t, the horizontal and vertical sizes of the front image are (W, H), and (x _s (t), y _s (t)) = ( W / 2, H / 2), (h _s (t), v _s (t)) = (W / 2, H / 2) may be output.

As shown in FIG. 9, the traffic light detection unit 216 is arranged in the horizontal and vertical directions of the rectangular area that includes the position (x _g (t), y _g (t)) of the traffic light g in the forward image at the time t and the traffic light g. The size (h _g (t), v _g (t)) is calculated and output to the gaze information calculation unit 219. Here, the position (x _g (t), y _g (t)) of the traffic light g in the front image and the horizontal and vertical sizes (h _g (t), v _g ( t)) may be calculated by any existing method. For example, the methods described in Patent Documents 12 and 13 can be used.

At this time, the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t is a size (h in the position (x _g (t), y _g (t)) in the forward image. _When it is inside the rectangular area of _g (t), v _g (t)), it can be seen that the vehicle driver visually observed the traffic light g at time t. On the other hand, the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at the time t is a size (h _g ) at the position (x _g (t), y _g (t)) in the forward image. When it is outside the rectangular area (t), v _g (t)), it can be seen that the vehicle driver did not visually observe the traffic light g at time t. In addition, when the traffic light g is not detected in the front image at the time t, the horizontal and vertical sizes of the front image are (W, H), and (x _g (t), y _g (t)) = ( W / 2, H / 2), (h _g (t), v _g (t)) = (W / 2, H / 2) may be output.

The distance information calculation unit 217, as shown in FIGS. 10 (A) and 10 (B), is distance information z (t) of the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t. Is output to the gaze information calculation unit 219. FIG. 10 (a) shows the line-of-sight position (x _e (t), y _e (t)) of the vehicle driver at time t. Similarly, FIG. 10 (b) represents the distance information z (x _e (t), y _e (t)) of the gaze position (x _e (t), y _e (t)) of the vehicle driver at time t. . Here, the distance information z (x _e (t), y _e (t)) of the line-of-sight position (x _e (t), y _e (t)) at the time t of the vehicle driver is obtained by any existing method. What is necessary is just to calculate. For example, the stereo ranging method described in Patent Documents 13, 14, and 15 may be used, or the TOF (Time Of Flight) method may be used.

The biological information acquisition unit 218 extracts the biological information v (t) of the vehicle driver at time t and outputs it to the gaze information calculation unit 219. Here, the biological information v (t) of the vehicle driver at time t may be any biological information. For example, one or more of the vehicle driver's heartbeat, pulse, electrocardiogram, respiration, body temperature, weight, body fat, blood pressure, sweating, line of sight, myoelectricity, skin impedance, etc. may be extracted as biometric information. Any existing method can be adopted as a method for extracting such biological information. For example, if it is a heartbeat, the method of patent document 17, 18 can be used. For the pulse, electrocardiogram, respiration, body temperature, weight, body fat, blood pressure, sweating, line of sight, myoelectricity, and skin impedance, the technique described in Patent Document 18 can be used.

ここで、上式の分母の各定数は次式で与えられる。

The gaze information calculation unit 219 includes the gaze position (x _e (t), y _e (t)) of the vehicle driver in the front image at the time t obtained by the gaze position detection unit 212, and the pedestrian detection unit 213. The position of the pedestrian p in the forward image at the obtained time t (x _p (t), y _p (t)) and the horizontal and vertical size (h _p (t ), v _p (t)), and a rectangular area including the position of the vehicle c (x _c (t), y _c (t)) and the vehicle c in the forward image at the time t obtained by the vehicle detection unit 214 Horizontal and vertical dimensions (h _c (t), v
_c (t)), the position (x _s (t), y _s (t)) of the sign s in the forward image at time t obtained by the sign detection unit 215, and the horizontal direction of the rectangular area including the sign s and The vertical size (h _s (t), v _s (t)) and the position of the traffic light g in the forward image at the time t obtained by the traffic light detection unit 216 (x _g (t), y _g (t )) And the size (h _g (t), v _g (t)) of the rectangular area including the traffic light g, and the line of sight of the vehicle driver at the time t obtained by the distance information calculation unit 217. Gaze information using distance information z (t) of position (x _e (t), y _e (t)) and biological information v (t) of the vehicle driver at time t obtained by biological information acquisition unit 218 f (t) = (X (t), Y (t), z (t), v (t)) is calculated and output to the time-series feature amount calculation unit 22. Specifically, (X (t), Y (t)) is defined by the following equation.

Here, each constant of the denominator of the above equation is given by the following equation.

X (t) and Y (t) included in the gaze information f (t) indicate the difference between the position of the object (pedestrian, vehicle, sign, traffic signal) to be watched and the gaze position of the vehicle driver. It can be said that it is information that is normalized according to the above, and is information that indicates how much the object that the vehicle driver is gazing at. Therefore, as long as it can represent how much the vehicle driver is gazing at the object to be watched, definitions other than the above may be adopted. For example, although the difference between the object position and the line-of-sight position is obtained separately in the X direction and the Y direction, the distance (difference) between the object position and the line-of-sight position may be evaluated with one value.

  According to the present embodiment, since various information such as the attribute information and distance information of the object at the line-of-sight position of the vehicle driver or the biological information of the vehicle driver is used as the gaze information, the information amount of the feature amount can be increased. it can.

<Third embodiment>
A third embodiment of the present invention will be described with reference to FIG.

FIG. 11 is a block diagram showing a schematic configuration of the vehicle driving support apparatus 3 according to the third embodiment of the present invention. The vehicle driving support apparatus according to the present embodiment can be realized using a semiconductor integrated circuit (LSI). As shown in FIG. 11, the vehicle driving support device 3 includes a gaze information extraction unit 31, a time-series feature amount calculation unit 32, a normal data storage unit 33, an evaluation data storage unit 34, a normal data selection unit 35, and a difference calculation unit 36. And a difference presentation unit 37 and a stimulus generation unit 38. These components correspond to the functions performed by the vehicle driving support device 3, respectively. Note that in the present embodiment, only differences from the first embodiment will be mentioned.

In the first embodiment, the statistical distribution difference D between all the time series feature values stored in the normal data storage unit 13 and the time series feature value stored in the evaluation data storage unit 14 is calculated. And
In the present embodiment, the normal data selection unit 35 is used to create a subset S of time series feature values stored in the normal data storage unit 33, and the obtained subset S and the evaluation data storage unit 34 are stored. This is characterized in that the difference D in statistical distribution with the time series feature quantity is calculated.

The normal data selection unit 35 is a subset S of time-series feature values stored in the normal data storage unit 33 based on the driving state of the vehicle and the attribute information of the vehicle driver input from the outside of the vehicle driving support device 1. And the obtained subset S is output to the difference calculation unit 36. Specifically, the subset S is a set of normal data having attribute information similar to the vehicle driver to be evaluated based on the vehicle driver attribute information (for example, personal name, age group, gender, etc.). Create it. Alternatively, the subset S may be the driving state of the vehicle (for example, when driving on an urban area, driving on a highway, driving on a suburb, daytime when driving, nighttime when driving, or clear when driving) Or may be created as a set of normal data having a running state similar to the running state to be evaluated. Alternatively, the subset S may be determined based on both the attribute information of the vehicle driver and the traveling state of the vehicle.

  In the present embodiment, the data stored in the normal data storage unit 33 is stored so that a subset based on the attribute information of the vehicle driver and the driving state of the vehicle can be selected from the data stored in the normal data storage unit 33. The data is stored in association with the attribute information of the vehicle driver and the traveling state of the vehicle. Further, the normal data selection unit 35 acquires the attribute information of the vehicle driver at the time of evaluation and the running state of the vehicle, and uses them for selecting normal data. The attribute information of the vehicle driver is stored in advance in association with the vehicle driver, and the normal data selection unit 35 obtains identification information (for example, biometric information such as a fingerprint, an iris, and a voiceprint, an identification code) acquired from the vehicle driver. And the corresponding attribute information may be acquired based on the selection input of the registrant from the vehicle driver. The traveling state of the vehicle can be acquired based on information obtained from various sensors included in the vehicle.

  According to the present embodiment, comparison with normal data with similar vehicle driver attribute information and vehicle running state is possible, and therefore, more accurate and easy driving detection is possible.

<Other embodiments>
In the above embodiment, image display and stimulus generation are performed based on the difference in statistical distribution of time-series feature values during evaluation and normal operation, but if the difference is greater than or equal to a threshold value, random operation occurs. It is also possible to display the image and generate a stimulus only in that case. In addition, the processing when the occurrence of the sloppy driving is detected is not limited to the above, and a notification notifying the occurrence of the sloppy driving may be output to another device.

  In the above-described embodiment, the vehicle driving support device has been described as calculating gaze information during normal driving and time-series feature amounts based on the gaze information. However, these pieces of information are calculated by other devices to support vehicle driving. You may store in the normal data storage part of an apparatus.

The embodiment is not limited to a semiconductor integrated circuit (LSI), and may be implemented using a computer-executable program or the like.

1 ... Vehicle driving support device
11… Gaze information extraction unit
12… Time series feature quantity calculation unit
13: Normal data storage
14 ... Evaluation data storage
15 ... Difference calculator
16 ... Difference display
17 ... Stimulus generator

Claims (16)

A vehicle driving support device that detects a vague driving of a vehicle driver,
Image acquisition means for acquiring an image in the forward direction of the vehicle;
Gaze information extracting means for extracting gaze information of the vehicle driver;
Based on the gaze information at the time of the evaluation of the vehicle driver, time-series feature quantity calculating means for calculating a time-series feature quantity in time series representing information on the position watched by the vehicle driver in the image ;
Normal data storage means for storing time-series feature values based on gaze information during normal operation;
A difference calculating means for calculating a difference in statistical distribution of the time-series feature quantity during the normal operation and during the evaluation;
Based on the magnitude of the difference, the rough driving detection means for detecting the rough driving of the vehicle driver,
A vehicle driving support device comprising: The time-series feature amount includes information representing, in time series, coordinates of a position in which the vehicle driver gazes in the image.
The vehicle driving support device according to claim 1. The time-series feature amount further includes information representing the depth of the position watched by the vehicle driver in the image in time series.
The vehicle driving support device according to claim 2 . The time series feature amount further includes information representing the biological information of the vehicle driver in time series.
The vehicle driving support device according to claim 2 or 3 . The gaze information is calculated based on the line-of-sight position of the position and the size and the vehicle driver of a predetermined object in said forward direction of the image,
The vehicle driving support device according to any one of claims 1 to 4. The time-series feature amount is given by connecting a predetermined number of the gaze information continuous in time.
The vehicle driving support device according to any one of claims 1 to 5. The difference is calculated based on a probability density ratio between the time-series feature amount during the normal operation and the time-series feature amount during the evaluation.
The vehicle driving support device according to any one of claims 1 to 6. The difference is calculated based on a difference in probability density between the time series feature amount during the normal operation and the time series feature amount during the evaluation.
The vehicle driving support device according to any one of claims 1 to 6. The normal data storage means stores the time-series feature amount based on the gaze information during the normal driving in association with the attribute information or the driving state of the vehicle driver during the normal driving,
Normal data for acquiring attribute information or driving state of the vehicle driver at the time of the evaluation and selecting from the normal data storage means attribute information similar to the attribute information or driving state or time series feature amount associated with the driving state Further comprising a selection means,
The difference calculation means calculates a difference in statistical distribution between the time series feature quantity at the time of the evaluation and the time series feature quantity selected by the normal data selection means.
The vehicle driving support device according to any one of claims 1 to 8. Visualizing the size of the difference and further comprising a difference presentation means for presenting to the vehicle driver,
The vehicle driving support device according to any one of claims 1 to 9. Further comprising stimulus generation means for giving a stimulus to the vehicle driver based on the magnitude of the difference;
The vehicle driving support device according to any one of claims 1 to 10. The stimulus generating means vibrates the driver's seat based on the magnitude of the difference;
The vehicle driving support device according to claim 11. The stimulus generating means emits a warning sound based on the magnitude of the difference;
The vehicle driving assistance device according to claim 11 or 12. The stimulus generating means warns the vehicle driver by voice based on the magnitude of the difference.
The vehicle driving support device according to any one of claims 11 to 13. A vehicle driving support method for detecting vague driving of a vehicle driver,
An image acquisition step of acquiring an image in a forward direction of the vehicle;
A gaze information extracting step for extracting gaze information of the vehicle driver;
Based on the gaze information at the time of evaluation of the vehicle driver, a time series feature quantity calculating step for calculating a time series feature quantity that represents information on the position in which the vehicle driver gazes in the image in time series,
A difference calculating step of calculating a difference in statistical distribution between the time series feature amount calculated in the time series feature amount calculation step and a time series feature amount based on gaze information calculated in normal operation in advance;
Based on the magnitude of the difference, the rough driving detection step of detecting the rough driving of the vehicle driver;
Including a vehicle driving support method.   A program for causing a computer to execute the steps of the method according to claim 15.

Images