JP4780711B2 - Driving motion analysis apparatus and driving motion analysis method - Google Patents

Description

  The present invention relates to a driving motion analysis device and a driving motion analysis method, and more particularly to a driving motion analysis device and a driving motion analysis method for analyzing a driving motion of an automobile driver, for example.

  An example of a conventional operation analysis device of this type is disclosed in Patent Document 1. According to the snoozing driving detection device disclosed in Patent Document 1, the center point of the steering wheel is set to “O”, and the period of the steering wheel passing through the central point “O” while the driver is operating the steering wheel is measured. After the driver starts depressing the accelerator pedal, the cycle is measured 2 to 5 times, and the longest cycle (longest cycle) is memorized. Thereafter, when the period during which the steering wheel passes the center point “O” during the steering operation exceeds the maximum period, it is determined that the driver is dozing and a warning is issued.

Non-Patent Document 1 reports the effect of mounting a video recording type drive recorder. According to this non-patent document 2, video storage type drive recorders are mounted on taxis, trucks and buses. For example, in a drive recorder for taxis, information from a camera unit that captures images outside the vehicle and the acceleration sensor built into the camera unit is sent to the drive recorder body, and at the same time, a brake signal, blinker signal, vehicle speed pulse signal is sent from the vehicle. And power is supplied. The data sent to the drive recorder is recorded on a recording medium (flash memory card) from which data for 15 seconds before and after the signal can be removed by a preset trigger (video recording trigger) signal. The drive recorder can also record data by pressing a manual switch when the driver feels dangerous. As a result, the taxi drive recorder stores the situation of the vehicle and the surrounding environment as a video and vehicle sensor information about the near-miss event in addition to the accident, and is used for operation management work.
JP 2004-310738 A “Survey report on the effect of the 2004 video recording drive recorder” March 2005 Ministry of Land, Infrastructure, Transport and Tourism

  However, in the technique of Patent Document 1, since a warning is issued when the cycle of the steering wheel operation exceeds the maximum cycle, for example, when driving on a gentle and large curve, whether or not the driver is asleep. Nevertheless, there is a possibility that the cycle of the steering wheel operation exceeds the maximum cycle. That is, it may be erroneously determined that the driver is a snooze driving.

  In addition, in the technology of Non-Patent Document 1, it is intended to be used for operation management work from accidents or near-miss events recorded in the drive recorder, but this only indirectly measures the result of the driving operation of the driver. Therefore, it is impossible to know how the driver operates and what the behavior of the car has become. In other words, it cannot be said that sufficient information is obtained to prevent accidents. Also, when the combined acceleration of the front-rear direction and the left-right direction fluctuates by 0.4 G (G is gravitational acceleration) in 0.5 seconds (or when the combined acceleration exceeds 0.8 G), a near-miss occurs. Although accredited, this degree of acceleration occurs directly on a sharp curve or a slight sudden start. For this reason, it is impossible to accurately determine whether the near-miss is caused by a really abnormal (dangerous) driving operation.

  Therefore, a main object of the present invention is to provide a novel driving operation analysis apparatus and driving operation analysis method.

  Another object of the present invention is to provide a driving motion analysis device and a driving motion analysis method capable of accurately detecting an abnormal driving motion.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate correspondence relationships with embodiments described later to help understanding of the present invention, and do not limit the present invention in any way.

According to the first aspect of the present invention, driving operation detection means for detecting driving operation data regarding the driving operation of the subject, frequency analysis means for frequency analysis of driving operation data detected by the driving operation detection means, and analysis results of the frequency analysis means Based on the feature quantity extraction means for extracting the feature quantity of the driving motion of the subject and the feature quantity is mapped to a predetermined space, the feature quantity extracted by the feature quantity extraction means is assigned a predetermined label by multiple instance learning. A driving action analysis device comprising driving action analysis means for detecting an abnormal driving action of a subject when it is determined that a set of reference feature values for the attached reference driving action exists at a point where the set does not exist It is.

In the first aspect of the present invention, the driving motion analysis device (10) includes a driving motion detection means (12, 14, 16, 18, 20, S51), a frequency analysis means (12, S55, S71, S73), and a feature amount extraction. Means (12, S75) and driving action analysis means (12, S33, S79, S81) are provided. The driving motion detection means detects driving motion data regarding the driving motion of the subject. The frequency analysis means performs frequency analysis (wavelet analysis) on the detected operation data. The feature amount extraction means extracts the feature amount of the driving motion of the subject based on the analysis result of the frequency analysis means. When the feature value is mapped to a predetermined space , the driving action analysis means is configured such that the feature quantity extracted by the feature quantity extraction means is a reference feature quantity for a reference driving action that is given a predetermined label by multiple instance learning. When it is determined that the set is present at a point where no set exists, the abnormal driving motion of the subject is detected. Here, multiple instances Learning: In (Multiple Instance Learning MIL), for example, a set of feature data based on the operating behavior of the subject as positive bag, a set of reference feature amount as a negative bag, detects the true positives bag . Here, the bag refers to a set of feature quantities (instances) based on a driving operation for a fixed time. Also, positive and negative are bag labels, positive labels are attached to bags that are predicted (assumed) that they have always performed abnormal driving behavior once, and all negative labels are correct driving behavior (standard It is attached to a bag that is predicted to be the driving operation). Therefore, when an instance of a bag with a positive label exists at a point where there is no instance of the bag with a negative label, the instance is detected as an abnormal driving action.

According to the present invention, the feature amount of the subject driving operation, based on a set of reference features of driving operation of a predetermined label is affixed reference by multiple instances learning, abnormal operation of the subjects' Since the motion is detected, the abnormal driving motion of the subject can be detected accurately and easily.

  The invention of claim 2 is dependent on claim 1 and further comprises warning means for warning the subject that the abnormal driving action is detected when the abnormal driving action is detected by the driving action analyzing means.

  In the invention of claim 2, the warning means (12, 22, S85) warns the subject that the abnormal driving action is detected when the abnormal driving action is detected by the driving action analyzing means. For example, a warning that the driving operation is abnormal is notified by sound (voice), flashing or lighting of light, or by vibration. In some cases, it is notified by text display. However, any two or more of these may be executed in combination.

  According to the invention of claim 2, since the subject is warned of an abnormal driving operation, a near-miss event or an accident can be prevented in advance.

  The invention of claim 3 is dependent on claim 1 or 2, and the reference feature amount is a feature amount when an exemplary driver performs a driving action.

  In the invention of claim 3, the reference feature amount is a feature amount when a driver who becomes a model such as a veteran or an instructor of a driving school performs a driving operation.

  According to the invention of claim 3, since the driver who serves as a model is used as a reference, it is possible to easily detect an abnormal driving motion of a subject such as a beginner.

  The invention of claim 4 is dependent on claim 1 or 2, and the reference feature value is a feature value when the subject performs a normal driving operation.

  In the invention of claim 4, the reference feature amount is a feature amount when a normal driving operation is performed in a state where the subject is healthy both physically and mentally. For example, the feature quantity regarding the initial driving operation when the driving is started corresponds.

  According to the fourth aspect of the invention, since the abnormal driving motion is detected based on the normal driving motion of the subject, for example, the abnormal driving due to the accumulation of fatigue of the subject can be detected. Therefore, a warning that a break should be made is possible.

The invention of claim 5 is according to any one of claims 1 to 4, further comprising a driving operation data storing means for storing running operation data when an abnormal operating behavior is detected by the running operation analysis means, OPERATION operation analysis means further includes a feature amount extracted by the feature extraction means, can a feature quantity stored in the running operation data storing means based on the driving operation data and you approximated, abnormal operating behavior of the subject Detect .

In the invention of claim 5, the driving operation data storage means (12, 12a, S41) stores in advance driving operation data when an abnormal driving operation is detected by the driving operation analysis means. Therefore, for example, a database about abnormal driving operations is created . At the subsequent times operation, the running operation analyzing means includes a feature quantity extracted by the feature extraction means, can a feature quantity stored in the running operation data storing means based on the driving operation data and you approximated, subject Detects abnormal driving behavior .

  According to the invention of claim 5, since driving operation data of abnormal driving operation is stored in advance, it is only determined whether or not it is an abnormal driving operation in consideration of this in the subsequent driving. Processing costs can be reduced, and abnormal driving operations can be quickly determined.

The invention of claim 6 is according to any one of claims 1 to 5, the operating dynamic Sakuken detecting means detects the output of the first acceleration sensor mounted on at least the subject's wrist.

  In the invention of claim 6, the first acceleration sensor (14, 16) is attached to at least the wrist of the subject. By detecting the output of the acceleration sensor in time series, a series of driving actions (driving action data) of the subject is detected. For example, the movement of the hand when the subject runs on a certain round course is detected.

  According to the invention of claim 6, since only the output of the acceleration sensor is detected, the driving operation can be easily detected.

  The invention of claim 7 is dependent on claim 6 and is attributed to the automobile based on the output of the second acceleration sensor from the second acceleration sensor attached to the automobile and the driving action data detected by the driving action detecting means. The apparatus further includes a removing unit that removes noise.

  In the invention of claim 7, the second acceleration sensor (20) is attached to, for example, a dashboard of an automobile. The removing means (12, S9, S11, S61, S63) is noise (engine vibration, road surface condition, automobile, etc.) caused by the automobile based on the output of the second acceleration sensor from the driving action data detected by the driving action detecting means. Removing its own acceleration).

  According to the seventh aspect of the present invention, noise caused by the automobile is removed from the driving operation data, so that the driving operation of the subject can be accurately detected.

The invention of claim 8 (a) detects driving operation data about the driving operation of the subject, (b) analyzes the frequency of the driving operation data detected in step (a), and (c) performs step (b). Based on the analysis result, the feature amount of the driving motion of the subject is extracted, and (d) when the feature amount is mapped to a predetermined space, the feature amount extracted in step (c) is determined by the multiple instance learning. This is a driving motion analysis method for detecting an abnormal driving motion of a subject when it is determined that a set of reference feature values for a reference driving motion with a label is present at a point where it does not exist .

  In the invention of claim 8, similarly to the invention of claim 1, the abnormal driving motion of the subject can be detected accurately and easily.

  According to the present invention, the abnormal driving operation of the subject is detected by the learning method for detecting the abnormal driving operation based on the characteristic amount of the driving operation of the subject and the characteristic amount of the reference driving operation. It is possible to detect an abnormal driving motion of the subject accurately and easily.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  With reference to FIG. 1, the driving | operation operation | movement analysis apparatus 10 of this Example contains the general purpose computer 12 like PC or PDA. A plurality of (four in this embodiment) acceleration sensors 14, 16, 18, and 20 are connected to the computer 12, and an output device 22 is connected to the computer 12. In FIG. 1, each of the acceleration sensors 14, 16, 18, and 20 is described as being connected to the computer 12 by wire, but actually, it is connected wirelessly. As will be described later, the acceleration sensors 14 and 16 are mounted on the wrist of the driver, and the acceleration sensor 18 is mounted on the toe of the right foot of the driver, so that it does not interfere with driving. . The acceleration sensor 20 is installed at a predetermined position of the vehicle body (dashboard in this embodiment).

  As the acceleration sensors 14, 16, 18, and 20, general-purpose multi-axis (three-axis in this embodiment) acceleration sensors can be used. For example, a three-axis acceleration sensor (model: H48C) manufactured by Hitachi Metals, Ltd. can be used. Further, in this embodiment, the output device 22 is constituted by a speaker, a lamp, a vibrator, or any two or more of them, and gives sound (sound), light, vibration, or those to the driver. Any two or more of these combinations are used to issue a warning to the driver that the operation is abnormal (dangerous). However, the output device 22 may include a display device such as a CRT monitor or LCD. For example, when this driving motion analysis apparatus 10 is used for automobile training or driving simulators (including simulation games), abnormal driving motion details (points) and driving skill levels or scores are displayed on the screen. It is possible to do it.

  FIG. 2 shows an example of a state in which the driver is driving a car with the acceleration sensors 14, 16, and 18 attached. However, in FIG. 2, only a part of the automobile (driver's seat part) is shown. As can be seen from FIG. 2, the acceleration sensor 14 is attached to the right wrist, the acceleration sensor 16 is attached to the left wrist, and the acceleration sensor 18 is attached to the toe of the right foot. The acceleration sensor 20 is attached to the dashboard of the automobile.

  In FIG. 2, the change lever, the brake pedal, the accelerator pedal, and the like of the automobile are omitted for easy understanding of the mounting state of the acceleration sensors 14, 16, and 18.

  Although it is difficult to understand in the drawings, the acceleration sensors 14, 16, and 18 are attached to the limbs using a fixing member such as a band with a Velcro tape (registered trademark) or a wristband. The acceleration sensor 20 is attached to the dashboard using a fixing member such as an adhesive tape.

  Although not shown, each of the acceleration sensors 14-20 detects acceleration in three axial directions, and in a state where the acceleration sensor 14-20 is placed on the horizontal plane, the acceleration sensor 14-20 generates an acceleration in a direction orthogonal to the horizontal plane (left-right direction, front-rear direction). Detection is performed on each of the two axes (X-axis and Y-axis), and acceleration in a direction perpendicular to the two axes (vertical direction) is detected on the remaining one axis (Z-axis). That is, acceleration in the left-right direction (X-axis direction), acceleration in the front-rear direction (Y-axis direction), and acceleration in the vertical direction (Z-axis direction) are detected. However, in this embodiment, with the acceleration sensor 14-20 in a horizontal plane, acceleration appearing in the right direction, forward direction, and upward direction is positive, and acceleration appearing in the left direction, backward direction, and downward direction is negative direction. .

  Further, although not shown, the acceleration sensor 14 is attached to the right wrist so that the vertical direction with respect to the back of the hand coincides with the Z-axis direction and the fingertip direction in the state where the finger is extended coincides with the Y-axis direction. Similarly, the acceleration sensor 16 is attached to the left wrist. The acceleration sensor 18 is the back of the subject's right foot, and is attached to the toe of the right foot so that the vertical direction to the back matches the Z-axis direction and the toe direction of the foot matches the Y-axis direction. Further, the acceleration sensor 20 is attached to the dashboard so that the front-rear direction of the automobile and the Y-axis direction coincide with each other, and the vertical direction of the automobile coincides with the Z-axis direction.

  For example, in the driving motion analysis apparatus 10, the output of the acceleration sensor 14-20 when the subject drives the automobile according to a certain course (route) is detected in time series. In this embodiment, the test subject is a beginner or a veteran (exemplary person) among motor vehicle drivers. FIG. 3 is a graph showing the time change of the output of the acceleration sensor (16) when a beginner drives a car and travels on a certain course. However, in FIG. 3, for the sake of simplicity, only the left-hand operation among the driving operations is shown. In FIG. 3, the vertical axis of the graph represents acceleration, and the horizontal axis of the graph represents time. In this embodiment, the time is represented by a 9-digit numerical value, and represents time (2 digits), minutes (2 digits), seconds (2 digits), and milliseconds (3 digits) in order from the beginning. Therefore, for example, when “142203201” is described, the time is “14: 22: 03.201 seconds”.

  On the other hand, FIG. 4A and FIG. 4B show the same course that the beginner drove, and two different veterans (veteran 1, veteran 2) are cars (both the same cars that the beginners drove) The graph of the time change of the output of the acceleration sensor (16) at the time of driving | running | working is shown. For the veterans 1 and 2, the detection results of only the left-hand motion among the driving motions are also shown.

  Here, when a general drive recorder is used, when the output of the acceleration sensor attached to the vehicle body exceeds a predetermined threshold value, a near-miss (meaning a near-miss event) occurred. Or determine that an accident has occurred and record such an event. Moreover, it may be determined that a near-miss has occurred due to a driver's switch operation. Conventionally, such cases have been gathered and are being used for subsequent driving, but since the cause of the occurrence of a near-miss or accident is unknown, it is insufficient to prevent near-miss or accidents. It can be said.

  Therefore, in this embodiment, the driving operation itself of a driver such as a beginner and a driver such as a veteran is compared, an abnormal driving operation of a beginner is detected in advance, and an abnormal driving operation is detected. Occasionally, warnings are given to drivers such as beginners to prevent near-misses and accidents.

  Briefly, based on the output of the acceleration sensor 14-20, data on driving operation (driving operation data) for beginners and veterans is acquired. However, as will be described later, the acceleration data (acceleration data) output from the acceleration sensor 14-18 includes noise due to the acceleration component applied to the automobile itself. Therefore, in order to remove a noise component caused by the automobile, only a specific frequency component (level 5-9 in this embodiment) is extracted when performing wavelet transform. Thereby, acceleration data, that is, driving operation data is subjected to wavelet analysis (multi-resolution analysis).

  Also, when detecting sudden driving behavior (high frequency band) that occurs during emergency avoidance, etc., and when detecting basic normal driving behavior (low frequency band) (attention) Different frequency bands should be used. In such a case, the driving operation data is reconstructed by performing wavelet transform again with a frequency component (level) according to the purpose.

  Here, the wavelet analysis is a technique of decomposing and analyzing signals (acceleration data in this embodiment) from both sides of time and frequency using a mother wavelet function. Further, the mother wavelet function Ψ (t) is a generic name of functions satisfying Equation 1.

Various mother wavelet functions have been proposed. In this embodiment, the Daubechies wavelet function is used. Further, when the mother wavelet function is ψ (t), the wavelet expansion coefficient d _{j, k} based on the discrete wavelet transform is expressed by the following equation (2).

However, the wavelet expansion coefficient d _{j, k} is a component extracted at time 2 ^j k and frequency level 2- ^j of the acceleration data x (t). At this time, the component x _j (t) of the frequency level 2− ^j of the acceleration data x (t) can be expressed by ^Equation 3. Hereinafter, the frequency level 2- ^j is referred to as a level j component.

Therefore, for example, if the acceleration data for the beginner shown in FIG. 3 is wavelet transformed at a certain level (only the frequency band considered to be important), a waveform as shown in FIG. 5 is obtained. When the acceleration data for the veteran 1 and veteran 2 shown in FIGS. 4A and 4B is wavelet transformed at a certain level, waveforms as shown in FIGS. 6A and 6B are obtained. Are obtained respectively. However, 2 ^j samples are required to extract level j components. In this embodiment, since the sampling frequency is 100 Hz, the period of the level j (j = 5-9) is 1 / (100 × 2 ^−j ). same as below.

  Although detailed explanation is omitted, the frequency band considered to be important is an acceleration component due to the driving operation of the subject, and the frequency from which noise (acceleration component) caused by the automobile is removed from the output of the acceleration sensors 14, 16 and 18. It is a belt. That is, the frequency band of the acceleration component applied to the automobile itself is detected based on the output of the acceleration sensor 20, and wavelet transform is performed in a frequency band (level) excluding the frequency band. For example, noise caused by an automobile includes engine vibration, road surface condition, acceleration of the automobile itself, and the like. Therefore, in this embodiment, acceleration due to engine vibration (high frequency band of level 1-4) and acceleration of the automobile itself (low frequency band of level 10-12) are removed. That is, only the level 5-9 component of the driving operation data is extracted.

Specifically, when it is desired to extract components including levels α and β (frequency levels 2 ^−α and 2 ^−β ), the calculation may be performed according to Equation 4. Therefore, for example, a component including only a specific frequency band component can be extracted.

Next, the time axis of the acceleration data subjected to the wavelet transform is divided at regular intervals, and the correlation between the energy of each acceleration data and the acceleration data is calculated in each section to obtain the characteristic amount of the driving operation. For example, when acceleration sensors (three axes) are attached to both wrists, six types of acceleration data are obtained. In this case, after extracting a specific frequency band (specific level) by wavelet analysis, the time axis is divided every predetermined time (for example, 1 second), and energy of acceleration data (here, six types) and acceleration data Are calculated ( ₆ C ₂ = 15 types) and set as the feature quantity x _t in the section t. Such a feature amount x _t is obtained for each subject (beginner or veteran).

  In this embodiment, MIL is used to extract a beginner's driving action (abnormal driving action) that cannot occur in an experienced driving. This MIL is described in detail in “O. Maron and T. Lozano-Perez,“ A Framework for Multiple-Instance Learning ”, Advanced in Informal Information Processing Systems, 10, MIT Pres. The description will be omitted.

  In conventional supervised learning, all data needs to be labeled, whereas in MIL, labels are provided for each unit called a bag, which is a set of several data (instances). The attachment is done. The label is “Positive” or “Negative”. Thus, if any one of the instances in the bag is “positive”, the bag is labeled “positive”. If any one of the instances in the bag is “negative”, the bag is labeled “negative”.

  In this embodiment, acceleration data at a certain moment is regarded as an instance, and a set of acceleration data (driving operation data) at a certain fixed time (for example, one round of the round course from the beginning to the end of a turn) is regarded as a bag. The beginner's bag is labeled "positive" by assuming that the beginner has made a strange movement at least once during a certain period of driving (the moment is a true positive). be able to. In other words, you don't know which instance is truly positive, but you know that at least some are positive. Conversely, the bag can be labeled “negative” by assuming that the veteran is not doing any strange behavior (no positive instances). In order to estimate which instance is positive, it is only necessary to find a point where instances of positive bags are dense and there are no instances of negative bags. For example, as shown in FIG. 7, when each instance is mapped on a multidimensional vector space, a positive bag present at a position outside the area or range where the negative bags are dense is regarded as a true positive bag. However, the negative bag is veteran driving operation data, and the positive bag is beginner driving operation data.

  For convenience of drawing, FIG. 7 shows a two-dimensional vector space, but in reality, as described above, the instance is a correlation between acceleration data energy (6 types) and acceleration data (15 types). It is expressed by a 21-dimensional vector indicating

  Therefore, when an instance of a positive bag exists at a point where no instance of a negative bag exists, it can be regarded as “a sudden operation deviating from a normal driving operation (abnormal driving operation)”. In addition, if positive bag instances are clustered at a point where there are no negative bag instances, it is regarded as “an abnormal driving operation and a person's driving habits that should be corrected (corrected)”. You can also.

Here, it is assumed that the motion feature vector x _i, t in trial i and section t is an instance, and the bag B _i = {x _{i, t} } in which the instances are grouped for each trial i. The final purpose is to extract the instance x _{i, t} at the moment when the abnormal driving operation occurs. That is, it is detected at which point an abnormal driving operation has occurred. As described above, the instance at the moment when this abnormal driving operation occurs is a truly positive instance.

  Equation 5 is used as a measure of how many negative bags exist around a positive bag instance x. In Equation 5, ‖a−b‖ represents a distance between a and b (for example, Euclidean distance).

  The closer the value of this negative non-crowdness is to 0, the more negative instances there are. That is, there is a high probability that the instance x is data at the moment when an abnormal driving operation is performed.

  On the other hand, when positive bags are densely packed around the instance x, the instance x is considered to be a habit of a person who performs an abnormal driving operation. The positive density is calculated according to Equation 6.

  The closer the value of the positive density is to 0, the more dense the positive instances are.

  As described above, the instance x having a negative non-crowd value close to 0 is driving operation data when an abnormal driving operation deviates from the driving operation theory. In addition, if the negative non-crowding value is close to 0 and the positive crowding value is close to 0, it indicates that the abnormal driving behavior is often a habit of the person.

  FIG. 8A and FIG. 8B show the change in time of the beginner's driving motion (left hand) data and the negative non-crowding degree + (and) positive density (Diversity Density) for the driving motion extracted by MIL. : Hereinafter, simply referred to as “DD”). However, the graph shown in FIG. 8A is the same as the graph of the beginner's driving operation data shown in FIG. Here, if the value of DD shown in FIG. 8B is close to 0, it can be said that the driving operation is abnormal. Therefore, as shown by surrounding with a dotted line, for example, by obtaining driving operation data (feature vector of driving operation composed of acceleration data) at a time when the value of DD is close to 0, an abnormal driving operation (here, (Left hand movement)

  FIG. 9A and FIG. 9B are graphs showing temporal changes in data of a beginner's driving operation (right hand and left hand) and DD with respect to the driving operation extracted by the MIL. Although illustration is omitted, when this driving operation data was acquired, there was a moment when a beginner completely released both hands from the handle when returning the cut handle. When the hand is instantaneously released from the handle, it can be seen from the change in the driving operation data in FIG. 9A in the same time zone that a large acceleration is generated in the radial direction from the center of the handle as shown by the dotted line. . The veteran will slip the handle when returning the cut handle, but will never let go of the handle. In other words, the movement that releases both hands when returning the cut handle does not appear at all in the veteran driving operation, so MIL detects such movement as abnormal.

  However, in this embodiment, an abnormal driving operation that occurs instantaneously (hereinafter sometimes referred to as “sudden driving operation”) and an abnormal operation that occurs in a relatively long time (hereinafter referred to as “continuous operation”). In order to detect a component of a specific level (level 5-9), the driving operation data obtained by wavelet transform so as to extract a component of a specific level (level 5-9) and a low frequency band (level 5-6) Reconfiguration is performed with the frequency band (level 7-9).

  FIG. 10A shows the wavelet transform of the beginner driving operation data shown in FIG. 3 to remove noise caused by the automobile, and then extracts only the high frequency band (levels 5 and 6). The graph at the time of reconstruction is shown. FIG. 10B is a wavelet transform of the beginner driving operation data shown in FIG. 3 to remove noise caused by the automobile, and then only the low frequency band (level 7-9) is extracted and driving operation is performed. The graph when data is reconstructed is shown.

  Similarly, FIG. 11 (A) extracts only the high frequency band (levels 5 and 6) after wavelet transforming the driving operation data of the veteran 1 shown in FIG. 4 (A) and removing noise caused by the automobile. And the graph at the time of reconstructing driving | operation operation | movement data is shown. In addition, FIG. 11B is a wavelet transform of the driving operation data of the veteran 1 shown in FIG. 4A, and after removing noise caused by the automobile, only the low frequency band (level 7-9) is extracted. Then, a graph when the operation data is reconstructed is shown.

  Further, FIG. 12 (A) is a wavelet transform of the driving operation data of the veteran 2 shown in FIG. 4 (B), and after removing noise caused by the automobile, only the high frequency band (levels 5 and 6) is extracted. Then, a graph when the operation data is reconstructed is shown. In addition, FIG. 12 (B) performs wavelet transform on the driving operation data of the veteran 2 shown in FIG. 4 (B), removes noise caused by the automobile, and then extracts only the low frequency band (level 7-9). Then, a graph when the operation data is reconstructed is shown.

  In this way, the level to be extracted is appropriately changed according to the abnormal driving behavior to be detected according to the difference in frequency, so that the abnormal driving behavior of the beginner can be accurately detected, and a near-miss event or accident A warning can be issued before the occurrence.

Specifically, the computer 12 shown in FIG. 1 executes real-time processing shown in offline processing or 1 5 and 1 6 as shown in FIGS. 13 and 14. In the offline processing shown in FIG. 13 and FIG. 14, for example, abnormal driving behaviors of beginners are extracted based on driving motion data of beginners and veterans acquired (detected) in advance, and a memory such as the hard disk 12a is extracted. To remember. That is, a database for abnormal driving operations (feature vectors) is constructed. Then, in a real-time processing shown in FIG. 1 5 and 1 6, results obtained in off-line processing using a (database), are to be determined whether novice driver operation is abnormal.

  As illustrated in FIG. 13, when the computer 12 starts the offline processing, the computer 12 acquires driving operation data of a certain number of frames (for example, 512 frames). Here, the first 512 frames are acquired from the driving operation data detected in advance for the beginner. The reason why the operation data for 512 frames is acquired is that it is the minimum amount of data necessary for the wavelet transform process described later. In the subsequent step S3, wavelet transform is performed using the driving operation data for 512 frames acquired in step S1. Here, components of level 5-9 are extracted by executing wavelet transform.

  In subsequent step S5, the operation data subjected to wavelet transform is decomposed for each frequency level. In the next step S7, the frequency level number i is initialized (i = 1). In step S9, it is determined whether the frequency band is expected to be noise. Here, it is determined whether the noise is caused by the automobile or the road surface. Here, the noise caused by the automobile or the road surface includes engine vibration, vertical vibration due to road surface conditions, acceleration of the vehicle body itself, and the like. Specifically, in the wavelet transform, the law of conservation of energy is established, so that it is possible to know how much frequency band components are contained by performing energy analysis. Therefore, by performing wavelet analysis on the acceleration data of the acceleration sensor 20 installed on the dashboard of the automobile, it is possible to know in which frequency band the noise caused by the automobile and the road surface is distributed. It is determined whether the level corresponds to this frequency band. As described above, in this embodiment, levels 1-4 and levels 10-12 are removed.

  If “NO” in the step S9, that is, if the frequency band is not expected to be noise caused by the automobile, the process proceeds to a step S13 as it is. If “YES” in the step S9, that is, if the frequency band is expected to be noise caused by the automobile, the frequency band (level) is removed from the driving operation data in a step S11, and the process proceeds to the step S13. In step S13, it is determined whether all frequency levels have been processed. That is, it is determined whether the number i is 12 or more.

  If “NO” in the step S13, that is, if the processing has not been completed for all the frequency levels, the frequency level number i is incremented by 1 in a step S15 (i = i + 1) in order to execute the processing for the next level. Return to step S9. On the other hand, if “YES” in the step S13, that is, if all the frequency levels have been processed, it is determined whether or not a sudden driving operation is extracted in a step S17 shown in FIG.

  If “NO” in the step S17, that is, if a continuous driving operation is extracted, the driving operation data is reconstructed using only the low frequency band component in the step S19, and the process proceeds to the step S23. For example, in step S19, the driving operation data is reconfigured so as to extract the level 7-9 frequency band. On the other hand, if “YES” in the step S17, that is, if a sudden driving operation is extracted, the driving operation data is reconstructed using only the high frequency band component in a step S21, and the process proceeds to the step S23. . For example, in step S21, the driving operation data is reconfigured so as to extract the level 5-6 frequency band.

In step S23, a driving action feature quantity (feature quantity x _t ) is extracted. Specifically, in the reconstructed driving operation data, as described above, the time axis is divided every second, the energy of acceleration data and the correlation of acceleration data are calculated in each section, and the feature amount x in the section t is calculated. _{Let t} .

In the next step S2 5, it is determined whether the processed for all the running operation data. That is, it is determined whether or not all driving operation feature values for trial i have been extracted. If “NO” in the step S25, that is, if all the driving operation data has not been processed, the driving operation data of the next predetermined number of frames is acquired in a step S27, and the process returns to the step S3 shown in FIG. . On the other hand, if “YES” in the step S25, that is, if all the driving operation data have been processed, the analysis target driving operation data is generated in a step S29. That is, a positive bag for a beginner is generated.

  In step S31, model driving operation data is acquired. Here, the negative bag about the veteran is acquired. However, the negative bag is obtained by executing the same processing as steps S1 to S27 based on the veteran driving operation data acquired in advance.

  Subsequently, in step S33, an abnormal driving operation is extracted by MIL. That is, a true positive instance is extracted from the positive bag. In step S35, a feature vector (driving feature vector) corresponding to the abnormal driving operation is stored in the hard disk 12a, and the offline processing is terminated. That is, a database for abnormal driving operations is created. However, the driving feature vector does not need to be stored in the hard disk 12a of the computer 12, and although not shown, it is stored in a recording medium (disk recording medium, memory stick, USB memory, etc.) attached to the computer 12. Also good. Alternatively, it may be stored in a database (not shown) connected to the computer 12 directly or via a network such as the Internet.

  As described above, FIGS. 15 and 16 show real-time processing of the computer 12. Hereinafter, real-time processing will be described, but processing similar to offline processing will be briefly described.

  For example, when driving of a car is started, as shown in FIG. 15, the computer 12 starts real-time processing, and starts acquiring driving operation data in step S51. That is, recording of acceleration data from the acceleration sensor 14-20 is started. At this time, for example, the computer 12 stores the acceleration data together with the recording time at regular intervals. The computer 12 acquires time from a timer (not shown) provided therein.

  In a succeeding step S53, it is determined whether or not driving operation data having a certain number of frames (for example, 512 frames) or more has been acquired. If “NO” in the step S53, that is, if driving operation data of a certain number of frames is not acquired, the process returns to the same step S53 as it is. On the other hand, if “YES” in the step S53, that is, if driving operation data for a certain number of frames is acquired, in step S55, wavelet transform is performed using driving operation data for a certain past frame including the current frame.

  In subsequent step S57, the operation data subjected to wavelet transform is decomposed for each level j. Next, in step S59, the frequency level number i is initialized (i = 1). In step S61, it is determined whether the frequency band is expected to be noise. If “YES” in the step S61, the frequency band (level) is removed in a step S63, and the process proceeds to a step S65. On the other hand, if “NO” in the step S61, the process proceeds to a step S65 as it is.

  In step S65, it is determined whether all frequency levels have been processed. If “NO” in the step S65, the frequency level number i is incremented by 1 (i = i + 1) in a step S67, and the process returns to the step S61. On the other hand, if “YES” in the step S65, it is determined whether or not a sudden driving operation is extracted in a step S69 shown in FIG. If “NO” in the step S69, the driving operation data is reconstructed using only the low frequency band component in a step S71, and the process proceeds to the step S75. On the other hand, if “YES” in the step S69, the driving operation data is reconstructed using only the high frequency band component in a step S73, and the process proceeds to the step S75.

In step S75, an exercise action feature quantity (feature quantity x _i ) is extracted from driving action data obtained by extracting only a predetermined level. In a succeeding step S77, it is determined whether or not it is a simple process. For example, at a driving school such as a driving school, it is determined whether or not it is an abnormal driving action with relatively simple processing, and when driving on public roads, it is determined whether or not it is an abnormal driving action with strict processing. To do. This is an item set by the administrator of the driving motion analysis apparatus 10 or the like. However, the simple processing has an advantage that the processing cost can be reduced, but there is a disadvantage that it cannot cope with an unknown driving operation. On the other hand, the strict processing has an advantage that it can cope with an unknown driving operation, but there is a disadvantage that the processing cost increases as the number of instances increases.

  Note that when only strict processing is executed, offline processing (FIGS. 13 to 14) is unnecessary.

  If “YES” in the step S77, that is, if it is a simple process, a distance (approximation) with the abnormal driving feature vector extracted in the offline process is calculated in a step S79, and the process proceeds to a step S83. That is, template matching is executed. On the other hand, if “NO” in the step S77, that is, if it is not a simple process, the negative non-congestion degree is calculated using the exemplary driving operation data in a step S81, and the process proceeds to the step S83. However, the model driving operation data may be acquired in real time or may be acquired offline in advance. In addition, when acquiring in real time, the computer 12 is connected with the computer which detects the output (driving operation data) of the acceleration sensor mounted in the motor vehicle which a veteran drives directly or via a network. However, before calculating the negative non-crowdness, the high frequency band or the low frequency band is extracted by preprocessing for the model operation data.

  In step S83, it is determined whether or not the threshold value is exceeded. Here, when the simple process is executed, it is determined whether or not the distance exceeds a predetermined distance (first threshold) and approaches (approximates). That is, it is determined whether the distance is equal to or less than the first threshold value. On the other hand, when strict processing is executed, it is determined whether or not the negative non-crowdness exceeds the second threshold and is close to 0. If “NO” in the step S83, that is, if the threshold value (first threshold value, second threshold value) is not exceeded, it is determined that the driving operation is not abnormal, and the process proceeds to the step S87. On the other hand, if “YES” in the step S83, that is, if the threshold value (first threshold value, second threshold value) is exceeded, the driver (subject) is warned in step S85 that the driving operation is abnormal. Then, the process proceeds to step S87. For example, it is warned that an abnormal driving operation is performed by sound (sound), vibration, color of light to be lit or blinking of light, or any two or more of them.

  In step S87, it is determined whether or not the real-time processing is completed. For example, it is determined whether or not the driving of the car is finished (whether the engine is turned off). If “NO” in the step S87, that is, if the real-time processing is not ended, the process returns to the step S53 shown in FIG. If “YES” in the step S87, that is, if the real-time processing is ended, the real-time processing is ended.

  According to this embodiment, since only the degree of approximation with an abnormal operation registered in advance is detected, the processing cost can be reduced. In addition, since it is possible to compare with the characteristic amount of the driving operation of the driver who is a model in real time, even an abnormal driving operation that does not exist in the database can be accurately determined.

  In the offline processing of the above-described embodiment, the driving feature vector for the abnormal driving operation is stored. However, it is also possible to store a habit of a beginner's driving operation. In such a case, in step S33, when an abnormal driving operation is extracted by the MIL, it is only necessary to acquire a driving feature vector when both the negative non-crowding degree and the positive crowding degree are close to zero. That is, it is only necessary to obtain a driving feature vector when DD is close to zero.

  In the offline processing of the above-described embodiment, the driving feature vector for the abnormal driving operation is stored. However, the abnormal driving operation may be stored as a near-miss event. In such a case, it is possible to accurately determine the occurrence of a near-miss event by linking the conventional drive recorder and the driving operation analysis device 10. Moreover, a database of near-miss events can be automatically created.

  Furthermore, in the real-time process of the above-described embodiment, the simple process (S79) or the strict process (S81) is executed. However, when the simple process is executed, an abnormal driving operation cannot be detected. In addition, a strict process may be executed.

  Furthermore, in the above-described embodiment, an abnormal driving operation is detected in advance, and this is made into a database as a near-miss event, which is useful for the subsequent driving operation. However, the present invention is not limited to this.

  For example, at a driving school, driving operation data of a trainee and an instructor are acquired, the driving operation data of the instructor is set as a positive bag, the driving operation data of the instructor is set as a negative bag, and an instructor is incorrect by MIL It is also possible to detect and indicate an abnormal driving operation, or to indicate the driving operation of a trainee when the driving operation of the instructor is a perfect score. This can be realized by offline processing or online processing.

  Regardless of whether it is a beginner or a veteran, driving operation data when the driver drove in a state where both the mind and body are healthy is stored in advance, and this is used as a negative bag. However, it is also possible to store the initial driving operation data when the driver starts driving and use this as a negative bag. On the other hand, every time a certain period of time (for example, 15 minutes to 30 minutes) during which the driver is driving the vehicle has passed, the driver's driving operation data is detected, and the detected driving operation data is set as a positive bag. And When a true positive instance is detected by the MIL, the driver's fatigue is accumulated, and it can be determined that the driving has been disturbed, and a warning that the user should take a break can be issued.

  Further, the same subject can drive an automobile having the same basic part but different driver seat layout, and a driver seat layout that is difficult for an novice to cause an abnormal operation can be selected. In such a case, driving operation data when a veteran driver drives a car with a different driver seat layout is stored as negative. And driving operation data in case a test subject like a beginner drives is made positive. For each driver seat layout, the presence of a true positive can be detected by the MIL, and for example, the driver seat layout when there is no true positive can be selected as being less likely to cause an abnormal operation.

  Furthermore, in the above-described embodiment, the acceleration sensor is attached to the toes of both wrists and the right foot to detect the subject's driving motion. For example, the feature amount of the beginner's driving motion and the veteran driving motion Since the beginner's abnormal driving motion is detected based on the feature amount, even if the acceleration sensor is only attached to one of the left wrist, right wrist, or right toe, It is also possible to detect a driving action. However, in consideration of safety, in the above-described embodiment, the movements at the three positions are detected.

  In the above-described embodiment, the acceleration sensor is used to detect the driving motion of the subject. However, the present invention should not be limited to this. A motion capture system can also be used. For example, an optical motion capture system of VICON (http://www.vicon.com/) can be applied as the motion capture system. In such a case, a marker is pasted to the toes of both wrists and the right foot, and this is photographed by a plurality of cameras having an infrared irradiation function, and the three-dimensional movement (position) is tracked in time series from the photographing result. (Detection). Basically, since the subject remains in the driver's seat, a relatively small number of cameras may be provided, and noise caused by the automobile can be ignored. However, the motion capture system is not limited to an optical system, and various known systems can be applied.

FIG. 1 is an illustrative view showing one example of an operation analysis apparatus of the present invention. FIG. 2 is an illustrative view showing one example of a state in which a subject drives an automobile with the acceleration sensor shown in FIG. 1 embodiment attached. FIG. 3 is a graph showing changes in driving operation data over time when a beginner runs a course on a car. FIG. 4 is a graph showing changes in driving operation data over time when a veteran travels on the same course as the course on which a beginner traveled. FIG. 5 is a graph obtained by wavelet transforming the driving operation data shown in FIG. 3 for levels 1-12. FIG. 6 is a graph obtained by wavelet transforming the driving operation data shown in FIG. 4 for levels 1-12. FIG. 7 is an illustrative view showing an example in which a negative bag and a positive bag are mapped on a multidimensional vector space. FIG. 8 is a graph of time change of beginner driving operation data (only the left hand) and a time change graph of DD. FIG. 9 is a graph of a time change of beginner's driving operation data (both hands) and a DD time change graph. FIG. 10 is a graph of changes over time when the beginner driving operation data is reconstructed by extracting only the high frequency band or only the low frequency band. FIG. 11 is a graph showing changes over time when the operation data of the veteran 1 is reconstructed by extracting only the high frequency band or only the low frequency band. FIG. 12 is a graph showing temporal changes when the operation data of the veteran 2 is reconstructed by extracting only the high frequency band or only the low frequency band. FIG. 13 is a flowchart showing a part of the offline processing of the computer shown in FIG. FIG. 14 is another part of the offline processing of the computer shown in FIG. 1, and is a flowchart subsequent to FIG. FIG. 15 is a flowchart showing a part of the real-time processing of the computer shown in FIG. FIG. 16 is another part of the real-time processing of the computer shown in FIG. 1, and is a flowchart subsequent to FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Driving | running operation analysis apparatus 12 ... Computer 14, 16, 18, 20 ... Acceleration sensor 22 ... Output device

Claims (8)

Driving action detection means for detecting driving action data about the driving action of the subject;
Frequency analysis means for analyzing the frequency of the driving action data detected by the driving action detection means;
Feature quantity extraction means for extracting the feature quantity of the driving motion of the subject based on the analysis result of the frequency analysis means; and
In the case of mapping the feature quantity in a predetermined space, the feature quantity extracted by the feature amount extracting means, a set of reference feature amount for driving operation of a predetermined label is affixed criteria exist by multiple instances Learning when it is determined that present in non point including the driving operation analysis means for detecting an abnormal operation operation before Symbol subject, operational behavior analysis apparatus.   The driving operation analysis device according to claim 1, further comprising a warning unit that warns the subject that the driving operation is abnormal when an abnormal driving operation is detected by the driving operation analysis unit.   The driving behavior analysis device according to claim 1, wherein the reference feature value is a feature value when a model driver performs a driving action.   The driving behavior analysis device according to claim 1, wherein the reference feature value is a feature value when the subject performs a normal driving action. A driving operation data storage unit for storing driving operation data when an abnormal driving operation is detected by the driving operation analysis unit;
Before SL running operation analyzing means further wherein the feature amount feature amount extracted by the extraction means, when said driving operation data storage means stored feature based on the operating behavior data you approximation, of the subject The driving | running operation analysis apparatus in any one of Claim 1 thru | or 4 which detects abnormal driving | operation operation | movement. The operating dynamic Sakuken detecting means detects the output of the first acceleration sensor mounted on the wrist of at least said subject, driving motion analysis device according to any one of claims 1 to 5. A second acceleration sensor attached to the vehicle, and a removal unit that removes noise caused by the vehicle based on an output of the second acceleration sensor from driving operation data detected by the driving operation detection unit. Item 6. The operation analysis device according to item 6. (a) Detect driving action data about the driving action of the subject,
(b) frequency analysis of the driving operation data detected in the step (a),
(c) based on the analysis result of the step (b) to extract the feature quantity of the subject's driving action, and
(d) In the case of mapping the feature quantity in a predetermined space, feature amount extracted by said step (c), a set of reference feature value for the reference driving operation of a predetermined label is affixed by multiple instances Learning when there it is determined that at spot does not exist, detecting abnormal operation operation before Symbol subject, operating operation analyzing method.