KR101276770B1 - Advanced driver assistance system for safety driving using driver adaptive irregular behavior detection - Google Patents

Abstract

The present invention relates to a safe driving assistance system based on user adaptive singular behavior detection, and more specifically, to track a driver's driving pattern and specific behaviors that may occur from external and external risk factors for the driver's safe driving in real time. The present invention relates to a safe driving assistance system of a user adaptive singular behavior detector panel that assists a driver in driving safely by inducing the occurrence of abnormal behavior and the inference of the risk and outputting a response to cope with it.
The present invention for achieving this object, the input unit for detecting data for determining whether the specific behavior of the inside and outside of the vehicle, and receives data according to the driver's driving pattern; A database unit for storing data received from the input unit and normal behavior reference values for determining whether the vehicle has unusual behavior inside and outside the vehicle; A topic analysis unit for analyzing whether or not a specific behavior and risk of data received in real time from the input unit using the reference value stored in the database unit; And a risk determination counter that controls to output a countermeasure according to whether the risk is increased or decreased by using the risk analyzed by the topic analysis unit. .

Description

 Safe Driving Assistance System for User-Adaptive Singular Behavior Detector Panels ADAPTIVE IRREGULAR BEHAVIOR DETECTION

The present invention relates to a safe driving assistance system of a user adaptive singular behavior detector panel, and more particularly, to track in real time a specific behavior that may occur from driver's driving pattern and risk factors inside and outside the vehicle for the driver's safe driving. The present invention relates to a safe driving assistance system of a user adaptive singular behavior detector panel that assists a driver in driving safely by inducing the occurrence of abnormal behavior and the inference of the risk and outputting a response to cope with it.

In Korea, about 210,000 accidents occur annually, which incurs 380,000 casualties and 9 trillion won in traffic accident handling costs. Many people in the world as well as in Korea are injured and even killed by traffic accidents. Causes include drunk driving, drowsy driving, dangerous driving habits, speeding and limited vision. Recently, in Korea, various high-tech electronic devices such as vehicle dynamic control (VDC) and smart cruise control (SCC) are used, and the danger that may occur when speeding or approaching other vehicles. Can be reduced. And Australia's Cohda Wireless is able to detect signals from cars moving within 500 feet by communicating between cars using communication in-car technology using Wi-Fi, which will be commercialized in 2012. I see it. These existing systems, however, focus primarily on reducing the risks of speeding or limited visibility. In other words, the development of an intelligent system that determines the danger from the driver's current state, habit or external environment is still inadequate.

In addition, with respect to Patent Publication No. 10-2010-0028253 (name of the invention: the driving assistance apparatus and the driving assistance method), the driver's state is analyzed using the driver's image photographing the driver, and the driver's state information is The present invention provides a driving assistance apparatus and a driving assistance method that can assist a driver in driving safely. In this case, however, the driver's drowsiness judgment is used to determine the risk factors, and the vehicle's surroundings and internal conditions cannot be judged comprehensively. Therefore, only the vision sensor is used, so the actual pressure, heart rate, There was a problem that it is not possible to detect the driver's state more accurately because it is unable to detect changes in the vital signs of various drivers such as pulse, body temperature, tension.

The present invention has been made in view of the above problems, and provides an intelligent system that determines the danger of the driver's current state, habit or external environment, and detects not only the driver's internal problems but also external traffic conditions. Accordingly, the present invention provides a safe driving assistance system for a user adaptive singular behavior detector panel that assists a driver to drive a safer driving.

In addition, by using various sensors or cameras to detect pressure, heart rate, pulse rate, body temperature, and tension, the data of the vehicle are not only directly detected, but also the driver's driving pattern is recognized. Accordingly, the present invention provides a safe driving assistance system for a user adaptive singular behavior detector panel that outputs a corresponding method.

The present invention for achieving the above technical problem, the input unit for detecting data for determining whether the specific behavior of the interior and exterior of the vehicle, and receives the data according to the driver's driving pattern; A database unit for storing data received from the input unit and normal behavior reference values for determining whether the vehicle has unusual behavior inside and outside the vehicle; A topic analysis unit for analyzing whether or not a specific behavior and risk of data received in real time from the input unit using the reference value stored in the database unit; And a risk increase / determination determining module 410 that controls to output a corresponding method according to whether the risk increases or decreases, using the risk analyzed by the topic analysis unit. .

According to the present invention as described above, by providing an intelligent system that determines the danger of the current state, habits or external environment of the driver, by detecting not only the driver's internal problems, but also the external traffic conditions, safer driving The driver can be assisted to do so.

In addition, not only the data inside and outside the vehicle may be directly detected using various sensors or cameras, but also the driver's driving pattern may be recognized and a corresponding method may be output according to each driver's driving pattern.

1 is a block diagram of a safe driving assistance system of a user adaptive singular behavior detector panel according to an embodiment of the present invention.
2 is a schematic diagram of a Bayesian State Space Model (SSM) for use as an embodiment of the present invention.
3 is a flowchart illustrating a safe driving assistance method of a user adaptive singular behavior detector panel according to an embodiment of the present invention.

Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. It is to be noted that the detailed description of known functions and constructions related to the present invention is omitted when it is determined that the gist of the present invention may be unnecessarily blurred.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings.

Regarding the safe driving assistance system of the user adaptive singular behavior detector panel of the present invention, it will be described with reference to FIGS.

1 is a block diagram of a safety driving assistance system for a user adaptive singular behavior detector panel according to an embodiment of the present invention. This will be described with reference to the following.

The safe driving assistance system of the user adaptive singular behavior detector panel of the present invention includes an input unit 100, a database unit 200, a topic analysis unit 300, and a risk determination counterpart 400.

The input unit 100 detects data for determining whether the vehicle has a specific behavior inside and outside the vehicle, and receives data according to a driver's driving pattern. The vehicle internal behavior detection module 110, the vehicle exterior behavior detection module 120, and the driver adaptive behavior input module 130 are included.

The in-vehicle behavior detection module 110 detects in-vehicle behavior data in real time in order to determine whether the in-vehicle specific behavior is detected, and detects the face and behavior of the driver in the vehicle. Driver health status information, driver driving pattern information, driver careless information, and the like.

Driver health state information includes drowsiness detection (blinking, yawning, heart rate, etc.), driver driving pattern information includes average speed, vehicle handling information, and driver careless information including the position of a driving hand, speech sounds, and the like.

The vehicle internal motion detection module 110 detects data according to a driver's driving pattern and data according to a biosignal change, and a video sensor (RGB) or a biosignal change for detecting data according to the driver's driving pattern Various audio, chemistry, tactile, temperature, pulse, and infrared sensors are used to detect the data according to this.

The data according to the driver's driving pattern means the facial expression change and eye state, head gesture, and motion information about the driving behavior according to the driver's condition, and the data according to the biosignal change includes pressure, heart rate, pulse, body temperature, and tension Etc. However, the above-described detection data is an example and the present invention is not limited thereto. In-vehicle behavior mainly refers to detection of unusual behavior generated from the driver, and of course, any data necessary for safe driving assistance can be detected.

The vehicle exterior behavior detection module 120 detects vehicle exterior behavior data in real time to determine whether the vehicle exterior behavior is unusual, and detects driving information and road environment information of the vehicle. Detects and tracks objects such as lanes, cars, and pedestrians by using an external situation camera, a distance detection object, and an object detection sensor to detect unusual behaviors such as reverse driving, speeding, and lane violations from the exterior of the vehicle. It monitors the traffic situation around the vehicle, and uses the lane detection sensor, front and rear side observation sensor, and external situation measurement sensor to detect driving information (speed detection, lane detection, distance between front and rear vehicles, Driving information, pedestrian detection, etc.) and road environment information (road construction information, display information on speed limit or one-way, etc.) can be detected.

In addition, the vehicle external behavior detection module 120 includes a communication module that can receive driving status information of each other by sharing the driving status information of the vehicles in the communicable area with each other. By transmitting and collecting each other's information so that the drivers of each vehicle can know each other's driving status information, it holds data that can cope with dangerous situations in advance.

The driver adaptive action input module 130 recognizes the driver's driving pattern through innate or acquired learning, builds the driver's information into a profile, and receives data that can adaptively deal with a dangerous situation for each driver. That is, the driver may modify the appropriate normal behavior criteria stored in the database unit 200 according to his driving pattern. The driving pattern may be different for each driver, and since the bio signal change value is different, the driver may continuously update the database through the driver's own recognition and update the driver's safety driving system.

As an example, if a driver normally blinks badly, and the 5 blinks are normally the normal behavior threshold, then the driver may raise it to 10, or the speed threshold is actually 100km / h. However, if one driver is a driver at 80km / h at the highest speed, the driver then lowers the normal behavior reference value to 80km / h so that each driver adapts the data to adapt to the dangerous situation.

If the driver is aware of the standard from the beginning, it can be input by changing the normal behavior standard value, or even if it is not, the driver can identify the continuous driving pattern through learning and modify the standard value. Can be. It is possible to implement a kind of user adaptive driving specific behavior detection system.

The database unit 200 stores all data received from the input unit 100 and a normal behavior reference value for determining whether the vehicle exhibits unusual behavior inside and outside the vehicle. The vehicle internal data module 210, the vehicle external data module 220, and the driver adaptive data module 230 are included.

The vehicle internal data module 210 stores normal behavior reference values corresponding to data according to a driver's driving pattern detected by the vehicle interior behavior detection module 110 and data and data according to a change in a biosignal. In-vehicle behavior data that is detected in real time by the vehicle interior behavior detection module 110 is stored, and includes driver health status information, driver driving pattern information, and carelessness information. Driver health status information includes drowsiness detection (blinking, yawning, heart rate, etc.), driver driving pattern information including average speed, vehicle handling information, and driver careless information including the location of a driving hand, speech sounds, etc. One content is only an example, but is not limited thereto.

The vehicle external data module 220 stores normal driving reference values corresponding to driving information, road environment information, and information of the vehicle detected by the vehicle external behavior detection module 120. The driving information of the vehicle includes speed detection, lane detection, distance detection between front and rear cars, driving information of adjacent vehicles, pedestrian detection, etc., and road environment information includes road construction information, speed limit or one-way display. Information, etc., but the above description is only one example, but is not limited thereto. In particular, the vehicle external data module 220 stores driving state information of vehicles in a communicable area transmitted by the communication module included therein.

The driver adaptive data module 230 stores normal behavior reference values corresponding to the driver's information and data through the driver's recognition received from the driver's adaptive action input module 130. Initially, the default is stored for the general person, but the driver's driving pattern can be grasped and the reference value can be changed through acquired learning. By default, the reference value may be modified and stored according to the driver, or when there are a plurality of drivers, a database may be built for each driver. It also stores data about the driver's status (hat, glasses, etc.).

The topic analysis unit 300 analyzes the internal and external data received in real time from the input unit 100 by using the reference value stored in the database unit 200, and analyzes whether the specific behavior is recognized and the risk level.

Each of the risk factors includes a partial topic analysis module 310 for analyzing whether the behavior is a specific topic, and an overall topic analysis module 320 for analyzing whether or not the corresponding situation contributes to a threat to safety.

In general, the topic analysis unit 300 uses a topic model. Using the method of extracting and modeling pixel-like movement patterns such as optical flow, we analyze the singular behavior that occurs in the inside and outside of the vehicle using unsupervised learning techniques such as topic modeling.

Topic model is an algorithm mainly used for natural language processing and finds a topic by substituting a small movement occurring in a scene into a word and substituting such a set of unit movements into a document. Topics analyzed in this way can be classified into meaningful regions, and can be learned using unsupervised learning techniques without prior information on vast input data.

However, the topic model used as an embodiment of the present invention is a Hierarchical Dirichlet Processes that can co-cluster words and documents without limiting the number of topic models for occurrence behavior. Singular behaviors are detected using non-parametric techniques using HDP). In addition, since each behavior can occur simultaneously in mixture form in consecutive frames, Co-Occurrence of each of these behaviors is calculated and the Spatio-temporal dependency is extracted using the Markov Model. To do this, the behavior trajectory information categorized by a specific topic corresponding to the meaning of each behavior is viewed as a sequence of observations, which is then learned using the Infinite Hidden Markov Model (IHMM) to calculate the probability of the behavior trajectory of a new moving object for the learned IHMM. If the probability value is low, it is determined as a singular behavior.

However, when only the global probabilistic model is used, it is difficult to accurately detect the location of the regional specific behavioral patterns that occur along with various normal behaviors. Therefore, when the input data is image data, the present invention divides the image data detected by the input unit into predetermined block units, constructs a local spatial model and a local temporal model, and then constructs a global probability model based on the two models. Use a hierarchical method. The local spatial model is modeled based on the location (block) of the stored motion information through quantization, and the local temporal model additionally includes the temporal information from which the movement of a specific object is extracted, and the occurrence time information is the global probability model. Is used to calculate the temporal association in. The global probability model considers the meaning of all movements occurring simultaneously in the entire image at the time of observation, not just the movements occurring in a certain area (one image block). Determine the cognition.

In the present invention, the word (word) is the characteristic information detected by the input unit 100 (characteristics such as the driver's eye blink, nodding, hand gestures, heart rate and other biosignals, vehicle speed, driving information of the adjacent vehicle, etc.) (document) is the image clip (1 second or 2 second unit) and sensor information of unit time when these feature information is collectively displayed, and the topic is a threat to safe driving, that is, whether the image is normal behavior or not. Whether the behavior is abnormal is a topic.

At this time, a topic includes a local topic and a global topic. The partial topic analysis module 310 analyzes a local topic, and the whole topic analysis module 320 includes the entire topic. Analyze global topics. A topic corresponding to a local topic may be reverse driving, speeding, lane violations, or the user's drowsiness occurring at a specific location, and a global topic may be determined by the local topic. Based on the topic, determine whether the video is a threat to safe operation.

The partial topic analysis module 310 analyzes the partial topic by determining whether the corresponding situation received in real time through the input unit 100 is an unusual behavior. It analyzes whether it is a specific behavior about each behavior such as reverse driving, speeding, lane violation, and user's drowsiness occurring at a specific location. Takes various data input from the input unit 100 and analyzes according to the corresponding.

First, in the case of detecting and using data on the vehicle external behavior from the vehicle external behavior detection module 120, in order to secure the safety of the vehicle in operation, the vehicle is being driven from the front lane detection sensor and the front and rear vehicle observation sensors. Continuously collect data on external environment and use it as observation data for unusual behavior analysis. To this end, the vehicle external situation measurement sensor analyzes the positional relationship between the direction of the currently driving lane and the driving route through edge detection technology and GPS-based moving path and speed analysis process, and then performs lane detection through lane edge detection and road template matching. The angle between the vehicle and the direction of travel. By using the angle information thus determined to determine whether the vehicle is operating normally, it analyzes the sensing data on emergency situation such as change of lane, departure from lane, and overspeed driving, and also the proximity of moving vehicle in front and rear Observe the driving information of the vehicle and the pedestrian situation, calculate the approach distance and the risk between each vehicle or the pedestrian, measure the actual distance between the running vehicle and the vehicle detected as a hazard, and from the princess distance and the braking distance By inferring the predicted stopping distance, we analyze the correlation between the risk factors generated from external factors and each detection data.

In case of detecting and using the data about the vehicle's internal behavior from the vehicle's internal behavior detection module 110, tactile and infrared sensors for collecting health status information such as heart rate, pulse, body temperature, and tension of the driver who are driving in the vehicle Information such as data, video and audio sensor data for determining attention information such as driver's eyes, and distractions of eyes, and olfactory sensor data for determining whether a driver is drunk are collected. In addition, it analyzes whether it is an unusual behavior by analyzing the relationship with the situation such as attention distraction, driving carelessness, drinking, speeding, and physical injury in the graph model for safe driving.

In particular, through the process of detecting and tracking the driver's face using the video sensor, whether the specific behavior is determined using the image data that detects the facial expression change, the eye state, the head gesture, and the like according to the driver's state. In the case of determining, feature information is extracted from the video data by using an active shape model, etc. in order to recognize a driver's eye blink and gaze dispersion.

을 이용한다. 통계적인 모델링을 위해 M개의 학습 데이터

로부터 Procrustes 알고리즘을 이용하여 정렬한 후 학습 집합을 구성한 후 평균적인 형태 모델과 PCA를 이용하여 형태의 변화를 모델링한다. 이렇게 구성된 형태모델

은 아래 수학식 1과 같이 표현할 수 있다.The active outgoing model is a modeling method that statistically models the shape and shape change of the face and then detects the face tracking and eye and mouth area, and extracts the feature information of the corresponding image. The contour features of the face, called landmarks to create, are the face contours and N points labeled for eyes, nose, mouth, etc.

. M training data for statistical modeling

We then use the Procrustes algorithm to sort and construct the learning set, then model the shape change using the average shape model and PCA. Shape model constructed in this way

May be expressed as Equation 1 below.

[Equation 1]

은 얼굴 형태이고,

은 평균 형태 벡터이고,

는 형태의 공분산에 대한 고유벡터의 열로 이루어진 행렬,

는 형태 파라미터 벡터이다. here

Is in the form of a face,

Is the average shape vector,

Is a matrix of columns of eigenvectors for the covariance of the form,

Is a type parameter vector.

에 의해 결정되고, 이때 변환은 영상에서 모델의 위치 이동

과 회전 변환

및 크기변환 s의 파라미터로부터 결정된다. 생성된 형태 모델을 이용하여 입력 영상으로부터 얼굴의 특징점을 추출하는 과정은 형태모델의 특징점과 입력 영상에서의 특징점 각각을 일대일로 매칭하는 방법으로 볼 수 있다. The shape in the actual input image is the shape parameter obtained by the conversion from the model coordinate frame to the image coordinate frame.

Determined by, where the transformation is the movement of the model's position in the image.

And rotation conversion

And a parameter of the size conversion s. Extracting the feature points of the face from the input image using the generated shape model can be seen as a method of matching the feature points of the shape model with the feature points in the input image one-to-one.

를 0으로 초기화 한 다음, 형태모델의 모양을 수학식 1의 수식을 통해 구한 다음, 형태모델의 모양

과 입력영상에서의 형태 모양

사이의 관계로부터 파라미터 [

]를 구하고, 이를 역 변환하여 모델 형태 프레임으로

을 투영시킨다. 그리고

에 정합되도록 모델의 형태 파라미터

를

의 값으로 갱신해 나가면서

의 모든 값들이 제한 조건 범위를 만족할 때까지 모델 투영과 파라미터 값 갱신을 반복한다. 이와 같은 방법을 통하여 입력 영상으로부터 얼굴 영역의 위치뿐 만 아니라 눈, 입의 위치를 정확하게 검출하고 이를 연속적인 프레임에 적용함으로써 효율적인 추적이 가능하다. 이때, 능동외향모델 방법을 이용한 얼굴 특징점 검출 및 정합에서는 형태 모델의 초기 값에 따라 그 성능의 정확도가 좌우되는데, 입력 영상의 첫 번째 프레임에 대해서 Adaboost방법을 이용하여 얼굴 영역의 초기 위치와 크기 정보를 추출하여, 이 정보를 얼굴과 몸통모델의 초기 값으로 이용하면, 성능을 향상시킬 수 있다. Type parameter

Is initialized to 0, and then the shape of the shape model is obtained using the equation (1).

Shape shape in and input image

From the relationship between the parameters [

], And inversely transform it into a model frame

Project the. And

The shape parameters of the model to match

To

While updating to the value of

The model projection and parameter value update are repeated until all of the values in X are within the constraint range. Through this method, it is possible to efficiently track not only the position of the face region but also the position of the eyes and mouth from the input image and apply it to successive frames. In this case, the accuracy of the performance depends on the initial value of the shape model in face feature point detection and matching using the active outward model method. The initial position and size information of the face region using the Adaboost method for the first frame of the input image By extracting and using this information as initial values for face and torso models, performance can be improved.

를 구하여 이용하는데,

를 구하는 방법으로는 상술한 능동외향모델을 사용한다. 이전 영상(

)으로 기준으로 형태파라미터

에 의해 현재 영상(

)을 분석하여, 정상행동인지 특이행동인지를 분석할 수 있다.
That is, when the data input from the input unit 100 is video image data, in order to determine whether the current input image is normal behavior or singular behavior, the shape parameter

To obtain and use

The active outgoing model described above is used as a method of obtaining. Previous video (

Shape parameters as standard

By current video (

), It can be analyzed whether it is normal behavior or unusual behavior.

The entire topic analysis module 320 determines the overall risk using the partial topic analyzed by the partial topic analysis module 310 and the corresponding situation that is received in real time through the input unit 100 is a threat to safe operation. Analyze the entire topic to determine if it is or not.

When the result data analyzed by the partial topic analysis module 310 is defined as 'state' and the data input from the input unit 100 is 'observation data', state and observation according to each time Define data and model their relationships as graphical models.

2 is a schematic diagram of a Bayesian State Space Model (SSM) used as an embodiment of the present invention. The Bayesian state space model uses a probability graph model to model a relationship between observation data and a risk situation over time.

는 각 시간에서의 관찰 데이터이고, 각 스테이트

는 위험 상황들을 원소로 갖는 벡터로써

와 같이 표현된다. 여기서

들은 각각 졸음, 음주, 심장마비 등의 위험 상황들을 나타내며, 0 또는 1 값을 갖는 이진 변수이다. 하지만 위험 상황이 발생했다고 해서 항상 대처를 해야 하는 것은 아닌데, 예를 들어 운전자가 차를 주차한 상태로 잠깐 졸고 있을 경우, 이는 운전 중이 아니므로 사고가 발생할 가능성은 없다. 따라서 스테이트 안에 추가적으로 현재 상황이 위험한지 그렇지 않은지를 나타내는 이진 변수

를 두어, 위험 상황에 대한 대응(action)이 필요한지 불필요한지를 나타내도록 한다. Emission 확률인

는 훈련 데이터로부터 Gaussian Mixture Model(GMM) 로 모델링 할 수 있다.

Is observation data at each time, and each state

Is a vector of elements with dangerous situations

. here

Are risky situations such as drowsiness, drinking, and heart attack, respectively, and are binary variables with zero or one values. However, a dangerous situation does not always require you to respond, for example, if the driver is asleep while parked in the car, it is not driving and there is no chance of an accident. Thus, additionally a binary variable in the state indicating whether the current situation is dangerous or not

To indicate whether an action is needed or unnecessary. Emission probability

Can be modeled as a Gaussian Mixture Model (GMM) from training data.

On the other hand, since the topic model does not consider the temporal order of words in a document, it is difficult to continuously detect and track an object with unusual behavior. In addition, the specific behavior is difficult to predict by defining a limited number of topic models due to its variety, and in general, the occurrence frequency is lower than the normal behavior, and it is difficult to detect due to the occurrence of the normal behavior at the same time. Using the Latent Dirichelt allocation (LDA) based on the existing Unsupervise learning technique, the movement trajectories can be classified into words and documents, and classified according to the subjects representing the meaning of the behavior, since the temporal order of the classified behaviors is not modeled. For example, even if a moving object is reversed in the area, it cannot be detected as an unusual behavior, and the number of behaviors used for learning must be defined in advance. In the present invention, a non-parametric technique using Hierarchical Dirichlet Processes (HDP), etc., capable of co-clustering words and documents without limiting the number of topic models for occurrence behavior. Application to the system for using has already been described above.

As an example of the present invention, a method of analyzing a topic of a corresponding image using the topic model for drowsiness detection is described below. The partial topic analysis module 310 determines whether or not the state in each frame is drowsy, but since it is difficult to judge only by the corresponding frame, the current drowsiness is calculated by using Likelihood by using several previously accumulated information or previous information. The probability value of the recognition or not is calculated, and if it is less than or equal to the threshold value θ, it is determined as a risk. Determination of whether or not the risk is the result data of the partial topic analysis module 310, the whole topic analysis module 320 is the result data from the partial topic analysis module 310, that is, the state is the number of each risk situation (assuming infinite) ), And the value from each state becomes the input value of the higher state and is judged hierarchically, ultimately analyzing whether the current situation is a threat to safe operation.

The Markov chain model and the decision rule for Likelihood-ratio (LR) are shown in Equation 2.

&Quot; (2) "

은,

와 같은 일련의 행동에 대한 스테이트를 의미한다.
here

silver,

State for a series of actions, such as

The risk determination response unit 400 determines whether the risk is increased or decreased by using the risk of each state analyzed by the topic analysis unit 300 and outputs a countermeasure for each case. do. The risk increase / decrease determination module 410 and the corresponding display module 420 are included.

를 추론한다. 더불어 transition 확률

을 이용하여 위험도증감을 판단할 수 있다. 상기 위험도 분석부의 결과데이터인 스테이트간의 transition 확률을 분석한다.
The risk increase / decrease determination module 410 determines whether the current situation detected by the input unit 100 increases or decreases the risk by using the overall risk determined by the topic analyzer 300. The GMM model used in the whole topic analysis module 320 is used. State at each time with maximum a posteriori (MAP) in GMM model

Infer With transition probability

The risk increase can be judged using. Analyze the probability of transition between states, the result data of the risk analysis unit.

The corresponding display module 420 outputs a corresponding method according to whether the risk increases or decreases determined by the risk increase / decrease determination module 410, and outputs a corresponding method according to the driver's response. When the risk is increased, a warning may be issued and a countermeasure for outputting the engine may be output depending on the situation, and when the risk is reduced, normal operation may be controlled to be possible. In addition, depending on whether the driver performs the output response, a higher intensity response may be output, or the intensity may be reduced to output the response.

The correspondence display module 420 uses a Markov Decision Process (MDP). The Markov Decision Process (MDP) is a model of how a driver handles a particular environment. It is defined by state set, action set, state transition probability, and reward function. State is defined according to the surrounding environment. The result data analyzed by the topic analysis unit 300 becomes a state. The action that is output to the driver consists of actions to prevent the driver and pedestrian from being in danger.

To complement the MDP, a Partially Observable Markov Decision Process (POMDP) is used. This can be used when it is impossible to infer the correct state of the state, by observing by adding partially observable data to the MDP, when the state is not known accurately, but only partial data detection is possible instead. The process is the same as MDP, but the parameters of POMDP use value iteration algorithm to learn in a short time.

There may be various and various types of actions, but as an example, action 1 warns with a warning notification and a light, takes direct action (vibration, etc.) on the user's chair and seat belt, Measure 2 (action 2) to control the warning of the rear warning light, forcibly control the running speed of the vehicle, and response 3 (action 3) to force the vehicle rear warning light There may be a countermeasure 4 (action 4) for controlling and warning, but the present invention is not limited thereto. If there is a required action, it can be added. If there is no action required, it can be omitted.

State transition probability is information about which state changes to the next state when the driver takes a specific action.

Rewards are expressed as a function of the current state, the action, and the next state. In a car, the reward is set to give the driver and pedestrians priority. For example, when the topic analysis unit 300 analyzes the user's specific behavior as driving carelessness (such as holding a steering wheel with one hand or talking with the person next to him) as a specific behavior, the result data is one state. Report it to the driver and output action 1. It requires driving in the correct posture with Action 1, where the system controls the more intense Action 2 if the user does not respond to this Action 1 and continues to make mistakes. . When the driver returns to the correct driver posture, the driver does not proceed to the response 3, but outputs the lower response 1, or stops outputting the response. The output of the next action according to the action is called Reward.

As another example, when the driver speeds more than usual, does not follow the lane, and the biological signal is not good, the topic analyzer 300 recognizes this as an unusual action and outputs a response 1 to the driver. If there is no reaction, take action 2 (action 2), and if there is no other action, control step by step 3,4 (action 3, 4). When the driver responds immediately by judging the driver's response that occurs after each action output, the response returns to its initial state.

3 is a flowchart illustrating a safe driving assistance method of a user adaptive singular behavior detector panel according to an embodiment of the present invention. Referring to this, the safe driving assistance method is as follows.

The database unit 200 stores the normal behavior reference value for each data for determining whether to behave unusually (S10). Each of the data for determining whether a particular behavior is the data stored in the input unit 100 includes driver health status information, driver driving pattern information, driver careless information, vehicle driving information and road environment information. Detailed data contents have been described above.

At this time, the database unit 200 modifies and stores the normal behavior reference value according to the driver's driving pattern from the driver adaptive behavior input module 130 (S20). Although the driver's driving pattern can be modified through acquired learning, in the case of data whose driving pattern is already known, it can be modified and stored when the reference value is naturally stored in step S10.

The input unit 100 detects singular behavior inside and outside the vehicle (S30). In-vehicle behavior detection module 110 detects driver health status information, driver driving pattern information, driver careless information to track the specific behavior in the vehicle (S31). Detailed data contents have been described above. The vehicle exterior behavior detection module 120 detects information about the exterior of the vehicle to track the unusual behavior of the external situation of the vehicle (S32). Further, the communication module receives the driving status information of each other by sharing the driving status information of the vehicles in the communication area (S33).

The topic analysis unit 300 analyzes partial topics about each behavior such as reverse driving, speeding, lane violation, presence or absence of a user's drowsiness, etc. using the internal and external data detected in step S30 (S40). ). As described above, the topic model is used, and in order to analyze the partial topic, it is determined whether it is an unusual behavior by using normal normal behavior reference values for each data stored in steps S10 to S20. The method is the same as described above, and the active outward model is used for the video image obtained by using the data received from the vehicle exterior behavior detection module 120 and the vehicle interior behavior detection module 110, and in particular by using a video sensor. The feature information is extracted using to determine the specific behavior.

The topic analysis unit 300 analyzes the entire topic using the partial topic analyzed in step S40 (S50). The whole topic also uses a topic model, which defines one analysis topic analyzed at step S40 as 'state', and uses this to analyze the risk of whether the current situation is dangerous or not.

The risk determination response unit 400 determines whether the current risk is increased by using the risk analyzed in step S50 (S60). When it is determined that the risk is increased, a corresponding action is output (S71), and when it is determined that the risk is not increased, the process proceeds to step S101 in which the driver's driving pattern is determined (S72). The MDP to the POMDP is used as a method of responding to the risk even after the risk is determined, which has been described above.

The risk determination response unit 400 determines whether or not the driver responds to the response action output in step S71 (S80). In this case, if the driver does not respond to the response action, another In response to an action instruction (S91), if there is a response, another action instruction is suspended (S92).

The input unit 100 recognizes the driver's driving pattern through acquired learning and reflects it to the normal normal behavior reference value for each data for determining whether the driver has a specific behavior (S100). The process returns to the step S20 through acquired learning, and corrects and stores the normal normal behavior reference value for each data for determining whether the driver behaves in accordance with the individual driver's driving pattern through the driver's recognition (S20).

While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be appreciated by those skilled in the art that numerous changes and modifications may be made without departing from the invention. And all such modifications and changes as fall within the scope of the present invention are therefore to be regarded as being within the scope of the present invention.

100; An input unit 110; In-vehicle Behavior Detection Module
120; Vehicle exterior behavior detection module 130; Driver adaptive action input module
200; A database unit 210; In-vehicle data module
220; Vehicle external data module 230; Driver Adaptive Data Module
300; Topic analysis unit 310; Partial Topic Analysis Module
320; All topic analysis module 400; Risk Assessment Response Department
410; Risk increase determination module 420; Correspondence display module

Claims (8)

An input unit 100 that detects data for determining whether the vehicle is singly behaving inside and outside the vehicle and receives data according to a driver's driving pattern;
A database unit 200 for storing data received from the input unit 100 and a normal behavior reference value for determining whether the vehicle has an unusual behavior inside and outside the vehicle;
A topic analysis unit (300) for analyzing whether or not a specific behavior and risk of the data received from the input unit (100) by using the reference value stored in the database unit (200); And
A risk determination counterpart 400 for controlling the output of a countermeasure according to whether the risk is increased or decreased by using the risk analyzed by the topic analyzer 300; Including but not limited to:
The topic analysis unit 300,
A partial topic analysis module 310 for analyzing a partial topic by determining whether the corresponding situation received in real time through the input unit 100 is an unusual behavior; And
Determining the overall risk using the partial topic analyzed by the partial topic analysis module 310 to analyze the entire topic on whether or not the corresponding situation received through the input unit 100 is a threat to safe operation. Topic analysis module 320; Safe operation assistance system of the user adaptive singular behavior detector panel comprising a.
The method of claim 1,
The input unit (100)
An internal vehicle behavior detection module 110 that detects at least one of a face, an action, and a biosignal change of a driver in the vehicle; And
A vehicle external behavior detection module 120 for detecting at least one of driving information and road environment information of the vehicle; Safe operation assistance system of the user adaptive singular behavior detector panel comprising a.
delete 3. The method of claim 2,
The vehicle external behavior detection module 120,
A communication module configured to receive driving state information of each other by sharing driving state information of vehicles in a communication area; Safe operation assistance system of the user adaptive singular behavior detector panel comprising a.
The method of claim 1,
The topic analysis unit 300,
Analyze whether the corresponding situation received through the input unit 100 is singular behavior by collecting (Co-Clustering) a combination of the internal and external behavior data received from the input unit 100 and data combining the characteristics of these data, A safe driving assistance system for a user adaptive singularity behavior detector panel that uses the analyzed data to determine whether the situation is a threat to safe operation.
delete The method of claim 1,
The partial topic analysis module 310,
When the corresponding situation received through the input unit 100 is image data including a shape, the feature points of the statistical shape model and the feature points of the shape included in the input image are matched one-to-one, and each shape is formed from the relationship between the matched feature points. Safe driving assistance of a user adaptive singularity behavior detector panel that obtains parameters, performs inverse transformation using Equation 1, and analyzes a partial topic that determines whether a behavior is abnormal by analyzing position information of a form included in the image data. system.
[Equation 1]

here

Is a form included in the image data,

Is the average form vector,

Is a matrix of columns of eigenvectors for the covariance of the form,

Is a type parameter vector.
The method of claim 1,
The risk determination counterpart 400 is,
A risk increase / determination module 410 that determines whether the risk is increased or decreased by using the overall risk determined by the topic analysis unit 300; And
A corresponding display module 420 for outputting a countermeasure according to whether the risk increases or decreases determined by the risk increase / decrease determination module 410 and controlling the countermeasure according to the driver's response; Safe operation assistance system of the user adaptive singular behavior detector panel comprising a.

Images