KR101393851B1 - Method and apparatus for sturdy classification of drive incident based on two step layer - Google Patents

Abstract

Provided are a method and an apparatus for classifying sturdy driving incidents based on a two-step layer. The sturdy driving incident classifying method based on the two-step layer, provided by the present invention, includes the following steps of extracting motion information on an inputted vehicle image, and setting grid division-based areas of the image; selecting an interest area for an entire area classification of the set areas, and building a motion feature-based entire area motion classifying unit for the entire area classification; filtering an entire area motion for a partial area classification, and selecting an interest area for the partial area classification; building a motion feature-based partial area motion classifying unit for the partial area classification; and generating the result of an automated driving incident classification which has the layer of the entire area/partial area classifications.

Description

TECHNICAL FIELD [0001] The present invention relates to a two-stage layer-based robust driving event classification method and apparatus,

The present invention relates to a vehicle image analysis based event classification algorithm, and more particularly, to a two-layer based robust driving event classification method and apparatus.

A system has been developed that recognizes pedestrians and vehicles in front of a vehicle using recent vehicle images and warns a driver when there is a risk of collision or automatically performs braking control or steering control to avoid collision. For such a system to function properly, it must be preceded by a quick and accurate determination of whether the object in front of the vehicle is a pedestrian, a vehicle, or any other object. In addition, in the case of a vehicle, it is necessary to accurately determine the movement of the vehicle. As a method of recognizing a forward object, there are a method using a camera and a method using a range sensor, which are currently being used.

In addition, a driving event classification apparatus using a vehicle image is also being developed. Such a traveling event classifier needs to analyze local motion of a specific area reflecting current statistical characteristics of motion distributed in each area through region-based motion analysis of the image. Also, it requires a traveling event classification device that can process all classification processes in real time by performing event classification that occurs in the traveling process in real time.

In order to construct a classification section for a more robust classification, the present invention sets two classification stages corresponding to a global classification and a regional classification, independently classifies global and local events of the current traveling information, Layer based robust driving event classification method and apparatus.

According to an aspect of the present invention, there is provided a robust driving event classification method based on a two-stage layer proposed in the present invention, comprising the steps of: extracting motion information of an input vehicle image and setting an area based on a grid division of the image; Constructing a global motion classification unit based on a motion feature for sorting a region of interest and for global classification, global motion filtering for region classification, selecting a region of interest for the region classification, Constructing a feature-based local motion classifier, and generating an automated driving event classification result having the global classifier and the local classifying layer.

In the step of extracting motion information of the input vehicle image and setting an area based on the grid division of the image, a grid partition area may be set by adjusting the size of the area in order to set a region of interest in the image.

The step of selecting the region of interest for the global classification and the global motion classification unit based on the motion feature for the global classification for the set region are different in the importance of the motion information for each region, A histogram-based region of interest can be selected for use.

The global motion classification unit based on the motion feature for global classification can be constructed by extracting the statistical parameters of the extracted motion vectors in each region after sorting the ROIs.

The step of constructing the local motion classifier based on the motion feature for the local classification may classify the regions using the extracted feature vectors for a predetermined number of local events.

According to another aspect of the present invention, there is provided a robust driving event classification apparatus based on a two-layer structure proposed in the present invention, including a motion information extraction unit for extracting motion information on an input vehicle image, A global motion classification unit based on a motion feature for selecting a region of interest for the global classification and performing the global classification for the set region, a filtering unit for performing global motion filtering for region classification, And a local motion classifier based on a motion feature for classifying the region of interest and performing the local classification.

According to embodiments of the present invention, region-based motion characteristics of an image can be extracted through motion vector estimation and statistical parameter extraction. Based on the region-based motion analysis of the image, it is possible to classify the current driving information based on the analysis of the local motions of a specific region, reflecting the statistical characteristics of the motions distributed in each region

FIG. 1 is a flowchart illustrating a robust driving event classification method based on a two-level layer according to an embodiment of the present invention.
2 is a diagram for explaining a process of setting a video region of a grid split according to an embodiment of the present invention.
3 is a diagram showing a pseudo color map for an accumulated value of an angular histogram in a motion tensor P corresponding to a specific global driving event according to an embodiment of the present invention.
4 is a diagram showing a pseudo color map of a local driving event after global motion filtering in the motion tensor W _j according to an embodiment of the present invention.
5 is a diagram illustrating a configuration of a two-layer layer based robust traveling event classification apparatus according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The proposed two - layer based robust driving event classification method and apparatus can extract region - based motion features of an image through motion vector estimation and statistical parameter extraction. Based on the region-based motion analysis of the image, it is possible to classify the current driving information based on the analysis of the local motions of the specific region, reflecting the statistical characteristics of the motions distributed in each region. In addition, in order to construct a classification section for more robust classification, a two-step classification step corresponding to global and local classification can be set up, and information on global and local events related to current traveling information can be independently classified and provided. Through this process, it is possible to construct a system capable of detecting the subdivision of driving information and detecting multiple events. By realizing all classification processes in real time, it is possible to construct online real-time driving information assistance service instead of offline analysis in actual driving environment. In the step of constructing the classification part, the data collection and processing assumption is designed to operate offline, but it can be designed to classify the events occurring in the traveling process in real time by using the classification part once constructed.

FIG. 1 is a flowchart illustrating a robust driving event classification method based on a two-level layer according to an embodiment of the present invention.

A robust driving event classification method based on a two-stage layer includes a step 110 of extracting motion information of an input vehicle image and setting an area based on a grid division of the image (110), selecting a region of interest for global classification A step 120 of constructing a global motion classification unit based on a motion feature for the global classification, a global motion filtering for a local classification, a step 130 selecting a region of interest for the regional classification, Based local motion classification unit 140, and generating 150 an automated driving event classification result having the global classification and the regional classification layer.

In step 110, motion information extraction for the inputted vehicle image and grid-division-based area of the image can be set. In other words, in order to set the region of interest in the image, the size of the region can be adjusted stepwise to set the grid partition region.

First, motion information on the input image can be extracted. For example, when an image of two frames is inputted by the camera, motion information extraction can be performed. In order to extract real-time motion information, optical flow measurement technique based on energy minimization technique developed by Horn et al. Or linear modeling based optical flow measurement technique developed by Lucas et al can be used . And motion estimated from each algorithm can be stored as one motion vector value (u, v) _{x, y, t} over all pixel positions. In this case, u and v are the horizontal and vertical sizes of the motion vector estimated by the optical flow measurement technique, respectively, and the subscripts x, y, and t correspond to (x, y) ≪ / RTI >

Next, the grid-division-based region of the image can be set. As an initial step for classification of driving events, it is possible to set the image area based on the grid division. For example, the size of the region may be stepwise adjusted to set the number of grid partition regions by s in order to set whether or not a region of interest is within the image for classification of a driving event. And, it can have an area scale of a total g according to the size of the divided area. Will be described in more detail with reference to FIG.

2 is a diagram for explaining a process of setting a video region of a grid split according to an embodiment of the present invention.

As shown in FIG. 2, the number of the grid partition areas can be set by gradually adjusting the size of the area in order to set whether or not the area of interest is within the image for the travel event classification. 2, the number of divided grid areas in FIG. 2 (a) is 25, the number of divided grid areas in FIG. 2 (b) is 100, and the number of divided grid areas in FIG. . For example, the total area scale g is 3 at this time.

In step 120, a global motion classification unit based on a motion feature for global classification and a region selection for a global classification may be constructed for the set region.

First, the histogram-based region of interest can be selected. Depending on the type of travel information, the importance of the region information of the motion indicated by the specific class may be distributed differently. To take advantage of these features, we can perform histogram-based region-of-interest screening. For example, it is assumed that a motion tensor W of hxwxk size having a time interval of k frames in which a global event of a specific class occurs in an event space having a total of n global driving events is extracted. At this time, a total of n motion tensors W ₁ , W ₂ , ..., W _n can be extracted according to each global event. At this time, there may be a coordinate transformation W _i ? P _i for the polar coordinate tensor P _i having an angle and a magnitude with respect to the i-th motion tensor W _i existing on the Cartesian coordinate for the vertical and horizontal motion vectors. A histogram h having a total of four bins can be generated based on the angle &thetas; value of the motion vector in s regions with respect to the motion tensor P _i on this polar coordinate. At this time, the histogram h in the i-th region can be defined as Equation (1).

수학식1

Equation 1

If the maximum histogram accumulation value for all driving event classes is above a certain threshold value k, that is, the interest area within the partition area can be extracted based on the max h> k criterion. Will be described in more detail with reference to FIG.

3 is a diagram showing a pseudo color map for an accumulated value of an angular histogram in a motion tensor P corresponding to a specific global driving event according to an embodiment of the present invention.

Since the importance of the motion information per domain is different, the histogram-based region of interest can be selected to utilize the feature of motion information per region. For example, FIG. 3 (a) shows a pseudo color map for the accumulated value of the angle histogram indicating the straight ahead, and FIG. 3 (b) shows the pseudo color map for the accumulated value of the angle histogram showing the right turn. Fig. 3 (c) shows a pseudo color map for the accumulated value of the angle histogram indicating the left turn, and Fig. 3 (d) shows the pseudo color map for the accumulated value of the angular histogram showing the deceleration.

Next, a global motion classification unit based on motion characteristics for global classification can be constructed. In other words, the global motion classification unit based on motion characteristics for global classification can be constructed by extracting the statistical parameters of the extracted motion vectors in each region after sorting the ROIs. After selecting the region of interest, statistical parameters can be extracted for the set of extracted motion vectors u and v in each region. The statistical parameters obtained from u and v have a total of 12 elements, and the final extracted feature vector x = [x ₁ , x ₂ , ..., x ₁₂ ] ^T can be defined as follows.

For example, n-class classification for each region can be performed using Equation (2) through SVM (Support Vector Machine) classification unit using the finally extracted feature vector x for n global events.

수학식2

Equation 2

으로 정의되며 최대값 c_max에 대한 정규화 값을 그 영역의 가중치 값으로 할당할 수 있다. 최종적으로 전역 주행사건에 대한 분류는 각 영역에서 분류 결과에 대한 수학식3을 만족하는 형태로 얻을 수 있다.
y is a class variable with y ∈ {0, 1, ..., n}, K is a kernel function, and λ is a dual-type optimization problem that maximizes the Lagrangian multiplier α with normalization parameters . We can extract one class vector ri = [r ₁ , r ₂ , ..., r _g ] in the _g scale regions of the i-th interested region r _i among the smallest size region. And, each r _i can have a classification variable value between 1 and n. Let R be the matrix containing the class information for the whole region of interest with each r _i as a row. Let R denote the cumulative vector c = {c ₁ , c ₂ , ..., c _g } Can be obtained. The decision classification value for the _ith region of interest r _i is

And the normalization value for the maximum value c _max can be assigned as the weight value of the area. Finally, the classification of global driving events can be obtained in a form satisfying Equation (3) for classification results in each region.

수학식3

Equation 3

는 최소 크기의 영역 내 관심영역의 총 개수이다. 모든 부류에서 얻어진 정규화된 가중치 부류의 누적값을 최대화하는 형태로 현재 분류를 위한 전역 주행사건을 선택할 수 있다.

Is the total number of regions of interest within the minimum size region. The global driving event for the current classification can be selected in such a manner as to maximize the cumulative value of the normalized weight class obtained from all classes.

In step 130, global motion filtering for region classification and region of interest for the regional classification may be selected. Global motion filtering can be performed to classify robust local driving events. To do this, a statistical motion model for each global driving event can be constructed. The average motion vector in the i-th region from the motion tensor W _j obtained from the j-th global driving event among the n total global driving events can be defined as shown in Equation (4).

수학식4

Equation 4

If the current driving event is classified from the global classification unit to the jth class, the average motion in the negative direction may be added to all the regions to perform Equation (5).

수학식5

Equation 5

에 대하여 전역 주행사건에 대한 관심영역 추출과 같은 방법으로 누적 히스토그램

을 생성할 수 있다. 전역 주행사건의 경우와 같이

기준을 바탕으로 지역 주행사건의 관심 영역을 추출할 수 있다. 도 4를 참조하여 더욱 상세히 설명한다.
Motion tensor obtained after global motion filtering

In the same way as the extraction of the region of interest for the global driving event, the cumulative histogram

Can be generated. As in the case of the global driving case

Based on the criteria, it is possible to extract the area of interest of the local driving event. This will be described in more detail with reference to FIG.

4 is a diagram showing a pseudo color map of a local driving event after global motion filtering in the motion tensor W _j according to an embodiment of the present invention.

에 대한 지역 주행사건의 의사색채 맵을 나타내었다. 도 4(a)는 우측 차선변경을 나타내는 지역 주행사건의 의사색채 맵을 나타내는 도면이고, 도 4(b)는 좌측 차선변경을 나타내는 지역 주행사건의 의사색채 맵을 나타내는 도면이다. 그리고, 도 4(c)는 우측 끼어듦을 나타내는 지역 주행사건의 의사색채 맵을 나타내는 도면이고, 도 4(d)는 좌측 끼어듦을 나타내는 지역 주행사건의 의사색채 맵을 나타내는 도면이다. 그리고, 도 4(e)는 전방차량 우측 차선변경을 나타내는 지역 주행사건의 의사색채 맵을 나타내는 도면이고, 도 4(f)는 전방차량 좌측 차선변경을 나타내는 지역 주행사건의 의사색채 맵을 나타내는 도면이다.
4, global motion filtering is performed on a motion tensor W _j corresponding to a specific global driving event, and a motion tensor

The results of this study are as follows. FIG. 4A is a diagram showing a pseudo color map of a local driving event showing a right lane change, and FIG. 4B is a diagram showing a pseudo color map of a local driving event showing a left lane change. FIG. 4 (c) is a diagram showing a pseudo color map of a local driving event indicating a right pincushion, and FIG. 4 (d) is a diagram showing a pseudo color map of a local driving event indicating a left pincushion. Fig. 4 (e) is a diagram showing a pseudo color map of a local driving event showing a right lane change of the front vehicle, and Fig. 4 (f) to be.

In step 140, a local motion classifier based on the motion feature for the regional classification may be constructed. At this time, classification for each region can be performed using the extracted feature vectors for a predetermined number of local events.

For example, n-class classification for each region based on the SVM (Support Vector Machine) classifier can be performed using Equation 6 using the extracted feature vector x for m local events that are defined in advance .

수학식6

Equation 6

으로 정의되며 최대값 c_max에 대한 정규화 값을 그 영역의 가중치 값으로 할당할 수 있다. 최종적으로 지역 주행사건에 대한 분류는 각 영역에서 분류 결과에 대한 다음 수식을 만족하는 형태로 수학식7과 같이 나타낼 수 있다.
y is a class variable with y ∈ {0, 1, ..., m}, K can be computed as a kernel function, and λ can be computed as a dual-form optimization problem that maximizes the Lagrangian multiplier σ as a normalization parameter . And to extract one of the class of the i-th region of interest of the smallest partition r _i is in the g region of the scale vector _{ri = [r 1, r 2} , ..., r g] T, each r _i is from 1 m. < / RTI > Let R be the matrix containing the class information for the whole region of interest with each r _i as a row. Let R denote the cumulative vector c = {c ₁ , c ₂ , ..., c _g } Can be obtained. The decision classification value for the _ith region of interest r _i is

And the normalization value for the maximum value c _max can be assigned as the weight value of the area. Finally, the classification of local driving events can be expressed as Equation (7) in the form of satisfying the following formula for classification results in each area.

수학식7

Equation 7

는 최소 크기의 영역 내 관심영역의 총 개수이다. 모든 부류에서 얻어진 정규화된 가중치 부류의 누적값을 최대화하는 형태로 현재 분류를 위한 지역 주행사건을 선택할 수 있다.

Is the total number of regions of interest within the minimum size region. A local driving event for the current classification can be selected in a form maximizing the cumulative value of the normalized weight class obtained from all classes.

In step 150, an automated driving event classification result having the global classification and the regional classification layer may be generated. After both the global classification and the regional classification step, an automated case classification result with two layers can be generated. Based on the predefined n global driving events and m local driving events, it is possible to perform multi-class classification for a total of nxm events.

5 is a diagram illustrating a configuration of a two-layer layer based robust traveling event classification apparatus according to an embodiment of the present invention.

The robust running event classification apparatus based on the two-stage layer may include a motion information extracting unit 510, an area setting unit 520, a motion classifying unit 530, a filtering unit 540, and a motion classifying unit 550 .

The motion information extraction unit 510 may extract motion information on the input vehicle image. For example, when an image of two frames is inputted by the camera, motion information extraction can be performed. In order to extract real-time motion information, optical flow measurement technique based on energy minimization technique developed by Horn et al. Or optical flow measurement technique based on linear modeling developed by Lucas et al can be used .

The area setting unit 520 can set a grid-based area of the image.

As an initial step for classification of driving events, it is possible to set the image area based on the grid division. For example, the size of the region may be stepwise adjusted to set the number of grid partition regions by s in order to set whether or not a region of interest is within the image for classification of a driving event. And, it can have an area scale of a total g according to the size of the divided area.

The global motion classifier 530 is a global motion classifier based on a motion feature that selects a region of interest for global classification and performs the global classification on the set region. Depending on the type of travel information, the importance of the region information of the motion indicated by the specific class may be distributed differently. To take advantage of these features, we can perform histogram-based region-of-interest screening. Then, the global motion classification unit based on the motion feature for global classification can be constructed by extracting the statistical parameters of the extracted motion vectors in each region after selecting the region of interest. It is also possible to generate automated driving event classification results after passing both the global classification and the regional classification step.

The filtering unit 540 may perform global motion filtering for local classification. In other words, global motion filtering can be performed for robust local driving event classification. To do this, a statistical motion model for each global driving event can be constructed.

The local motion classifying unit 550 is a local motion classifying unit based on a motion feature that selects a region of interest for the region classifying and performs the region classifying. In other words, the local motion classifier 550 can select a region of interest for local classification and construct a local motion classifier based on motion characteristics for local classification. At this time, classification for each region can be performed using the extracted feature vectors for a predetermined number of local events. It is also possible to generate automated driving event classification results after passing both the global classification and the regional classification step.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (6)

In a driving event classification method,
Extracting motion information of the input vehicle image and setting an area based on the grid division of the image;
Constructing a global motion classification unit based on a motion feature for global classification and a region of interest for global classification for the set region;
Global motion filtering for local classification and selection of a region of interest for the regional classification;
Constructing a local motion classifier based on a motion feature for the regional classification; And
Generating an automated driving event classification result having the global classification and the regional classification layer
Wherein the driving event classification method comprises the steps of: The method according to claim 1,
Wherein the step of extracting motion information of the input vehicle image and setting an area of the image based on the grid division,
The size of the region is stepwise adjusted to set the region of interest in the image to set the grid partition region
Driving event classification method. The method according to claim 1,
Wherein the step of constructing a global motion classification unit based on a motion feature for global classification and a region-of-interest selection for global classification for the set region comprises:
Since the importance of the motion information per area is differently distributed, the histogram-based interest region is selected to utilize the feature of the motion information per region
Driving event classification method. The method of claim 3,
After the region of interest is selected, statistical parameters of the motion vectors extracted in each region are extracted to construct a global motion classification unit based on motion characteristics for global classification
Driving event classification method. The method according to claim 1,
Wherein the step of constructing the local motion classifier based on the motion feature for local classification includes:
And performs classification on each of the regions using the extracted feature vectors for a predetermined number of local events
Driving event classification method. A driving event classification apparatus comprising:
A motion information extraction unit for extracting motion information on the input vehicle image;
An area setting unit for setting a grid division based area of the image;
A global motion classification unit based on a motion feature for selecting a region of interest for the global classification and performing the global classification for the set region;
A filtering unit for performing global motion filtering for local classification; And
A motion-feature-based local motion classifier for classifying the region of interest for the regional classification and performing the regional classification;
Wherein the traveling classifier comprises:

Images