CN107992899A - A kind of airdrome scene moving object detection recognition methods - Google Patents

Abstract

The invention provides a method for detecting and recognizing moving targets on an airport scene. The specific process is as follows: firstly, using a tendency flow method to obtain area suggestions for moving targets, and then using a convolutional neural network to identify the moving targets. This method overcomes the shortcomings of the traditional target extraction method for incomplete extraction of moving targets of different sizes and more noise, and makes up for the shortcomings of the deep learning target detection network for the detection of small targets on the airport scene. The recognition network designed by the invention has fewer convolutional layers, smaller feature dimensions, less calculation, and higher accuracy, which meets the requirements of airport scene monitoring.

Description

A method for detection and recognition of moving objects in airport scenes

technical field

The invention relates to the technical field of digital image processing, in particular to an airport moving target detection and recognition method.

Background technique

With the rapid development of my country's civil aviation transportation industry, the number of aircraft, vehicles and personnel at the airport has increased rapidly, and the operating environment of the airport scene has become increasingly complex. It is necessary to introduce airport scene surveillance system.

The traditional airport scene surveillance method is mainly based on the scene surveillance radar, and domestic large-scale airports such as Beijing Capital Airport and Shanghai Pudong Airport are equipped with scene surveillance radar. However, due to the high installation and maintenance costs of surface surveillance radars, the vast majority of small and medium airports in China are not equipped with surface surveillance radars, but rely on the controller's visual and human operations to realize the surveillance function of the scene, which will greatly Increase the risk of airport surface operations.

With the development of computer vision, airport scene surveillance technology based on video technology has emerged in recent years. This technology is low in cost, and compared with the surveillance radar method, it can cover a wider area, so it is more flexible. At present, the researches on dynamic targets on airport scenes mainly focus on the tracking of known types of targets, but there are not many researches on target detection. Object detection methods can be mainly divided into two categories: methods based on manual features and methods based on deep learning networks. Common target detection methods based on manual features include algorithms such as optical flow method, frame difference method, and ViBe. These methods have low accuracy and high time cost, and are not suitable for airport scene surveillance. Target detection algorithms based on deep learning networks include R-CNN, Faster R-CNN, SSD (Single Shot MultiBox Detector) and other methods. These methods have the characteristics of high detection accuracy and high efficiency, but due to the design of such network models characteristics, its effect on small target detection is not good, especially the detection accuracy of airport far-field small targets is low. Given that most of the targets in the airport scene are relatively fixed, for the purpose of security surveillance, we are much more interested in the dynamic targets in the airport scene than in the static targets.

Contents of the invention

In view of this, the present invention uses the tendency flow method to detect the movement information of the moving target, and designs a recognition depth network to recognize the target, so as to realize the detection and recognition of the moving target on the airport scene including small targets. The invention divides the airport moving objects into three categories of aircraft, automobiles and pedestrians.

Technical scheme of the present invention is concretely realized as follows:

(1) Obtain region proposals for moving objects using the tendency flow method, in the form of rectangular frame coordinates. The specific steps of calculating the moving target tendency flow are as follows:

(a) First obtain the optical flow vector of the moving target For a pixel point (x, y) on the image, the brightness at time t is E(x, y, t), and the brightness after a period of time Δt is E(x+Δx, y+Δy, t+Δt ). When Δt approaches infinitesimal, it is considered that the brightness of the point remains unchanged:

E _x u+E _y v+E _t ＝0

in, Respectively represent the pixel point along x, y,

t Gradients in three directions. Respectively represent the velocity components of the optical flow in the x and y directions, that is, the optical flow vector. Select a neighborhood window of n×n (n>1) size, and establish the neighborhood pixel system equation to solve the optical flow vector. From the pixels 1, 2, ..., i=n ² , a system of equations can be obtained:

The above formula can be written in matrix form:

Ad=-b

Its least squares solution, that is, the optical flow vector is:

(b) Calculate the pulse line Q _i of the moving pixel from the optical flow vector. According to the optical flow vector obtained in the first step, set the point Represents the position of a particle of a moving target at time t in the i-th frame, and its initial position is point q. The veins are defined as:

The translation of each particle on the vein line is:

The vein line is equal to the collection of these particles, obtained by the following formula:

Q _i = {x _i (t), yi (t), u _i , v _i }

in,

(c) Calculate the tendency flow Ω _i of the moving target. The inclined flow is defined as Q _s =(u _s , v _s ) ^T , where u _s and v _s respectively represent the velocity vectors of the inclined flow in two directions. Let the set U=[u _i ], consider that u _i is the linear interpolation of three adjacent pixels:

u _i ＝b ₁ u _s (k ₁ )+b ₂ u _s (k ₂ )+b ₃ u _s (k ₃ ) where k _j represents the index number of the adjacent pixel,

b _j represents the known triangular basis function of the jth adjacent pixel in this domain. For all points in U, use the above formula to form a system as:

Bu _s = U

Among them, B is composed of elements b _i , and u _s can be solved using the least square method. In the same way, v _s can be calculated, and then the tendency flow can be obtained

Ω _s = (u _s , v _s ) ^T .

(d) Take the maximum enclosing rectangular frame of each target extracted by the tendency flow, that is, take the rectangular frame surrounded by [x _min , y _min ] and [x _max , y _max ] for each target to form a moving target area suggestion.

(2) Based on the moving target captured by the tendency flow method, the moving target is identified using a convolutional neural network.

The convolutional neural network of the present invention has 14 layers in total, including 12 convolutional layers and 2 fully connected layers. The convolutional neural network of the present invention aims at the characteristics of large size differences of airport moving targets, uses five Inceptiion structures, and uses three convolution kernels of 1×1, 3×3, and 5×5 to extract multi-scale features of the target. In order to reduce network overfitting and speed up network training and testing time, the number of feature maps and convolution kernel size of each layer are carefully designed;

The identification steps of the identification network for moving targets are as follows:

(a) The recognition network extracts the moving target features. Performing a convolution operation on an image is shown in the following formula:

Here, the convolutional layer inputs K feature maps and outputs L feature maps. The number of required convolution kernels is LK, and the size of the convolution kernel is IJ. F _l represents the output lth feature map, and M _k represents the input feature map. k feature maps, X _{k, l} represent the two-dimensional convolution kernel of row k and column l, and b _kl represents the bias value;

The extracted features are weighted and summed, and processed using the ReLU (Rectified Linear Units) unsaturated activation function:

σ(x)=max{0,x}

After being processed by the activation function, the network will perform a maximum pooling operation on the feature map to form a new feature map as:

where ^Y'l represents the feature map after pooling;

According to the hierarchical arrangement of the recognition network of the present invention, repeated execution of the above-mentioned convolution and pooling work will obtain the semantic abstraction of the two-dimensional format data from low level to high level.

(b) The recognition network classifies moving objects: After the above features are connected through the fully connected layer, the feature vector is finally input into the softmax function for classification. The invention divides the airport moving objects into three categories of aircraft, automobiles and pedestrians.

Beneficial effects: the present invention first uses the tendency flow method to extract the area of the moving target in the image, and then constructs a convolutional neural network to identify the moving target in the area proposed by the tendency flow method, forming an efficient and accurate airport moving target detection Identify the frame.

Description of drawings

Fig. 1 is a flow chart of area suggestion proposed by the inclined flow method in the embodiment of the present invention.

Fig. 2 is a structural diagram of Inception in the embodiment of the present invention.

Fig. 3 is an effect diagram of moving target detection in the embodiment of the present invention.

Fig. 4 is a structural diagram of an identification network in an embodiment of the present invention.

Claims (5)

1. A detection and identification method for an airport scene moving target is characterized by comprising the following specific processes:

acquiring a region suggestion of a moving target by using a tendency flow method;

and (II) identifying the moving target by using a convolutional neural network.

2. The method for detecting and identifying the moving object on the airport surface according to claim 1, wherein the specific process of the step (one) is as follows:

step one, obtaining an optical flow vector of a moving objectThe value is determined by:

<mrow> <mfenced open = "[" close = "]"> <mtable> <mtr> <mtd> <mi>u</mi> </mtd> </mtr> <mtr> <mtd> <mi>v</mi> </mtd> </mtr> </mtable> </mfenced> <mo>=</mo> <msup> <mrow> <mo>(</mo> <msup> <mi>A</mi> <mi>T</mi> </msup> <mi>A</mi> <mo>)</mo> </mrow> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </msup> <msup> <mi>A</mi> <mi>T</mi> </msup> <mrow> <mo>(</mo> <mo>-</mo> <mi>b</mi> <mo>)</mo> </mrow> </mrow>

the above equation isA least squares solution of; wherein E represents the brightness of the pixel point; respectively representing gradients of pixel points along the three directions of x, y and t;optical flow vectors respectively representing optical flows in x and y directions;

step two, obtaining the pulse line Q of the motion pixel by the optical flow vector_iThe value is determined by:

Q_i＝{x_i(t)，y_i(t)，u_i，v_i}

wherein,dotThe position of a certain particle representing a moving object in the ith frame at time t is the initial position of a point q;

step three, calculating the trend flow omega of the moving target_sThe value is determined by:

Ωs＝(u_s，v_s)^T

wherein u is_sAnd v_sRespectively representing the velocity vectors of the inclined flow in two directions; by u_sFor example, the value is found by a least squares solution of the following equation:

u_i＝b₁u_s(k₁)+b₂u_s(k₂)+b₃u_s(k₃)

wherein k is_jIndex number representing adjacent pixel, b_jA triangular basis function representing the known j-th adjacent pixel in the domain;

step four, taking each target maximum bounding rectangle extracted by the trend stream, namely taking [ x ] for each target_min,y_min]And [ x ]_max,y_max]And forming a motion target area suggestion by the enclosed rectangular frame.

3. The method for detecting and identifying the moving object on the airport surface according to claim 1, wherein the specific process of the step (two) is as follows:

step one, identifying a network to extract moving target characteristics, and performing convolution operation on a picture as shown in the following formula:

<mrow> <msub> <mi>F</mi> <mi>l</mi> </msub> <mrow> <mo>(</mo> <mi>m</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>=</mo> <msub> <mi>M</mi> <mi>k</mi> </msub> <mrow> <mo>(</mo> <mi>m</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>*</mo> <msub> <mi>X</mi> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </msub> <mrow> <mo>(</mo> <mi>m</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>+</mo> <msub> <mi>b</mi> <mi>l</mi> </msub> <mo>=</mo> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>k</mi> <mo>=</mo> <mn>0</mn> </mrow> <mrow> <mi>K</mi> <mo>-</mo> <mn>1</mn> </mrow> </munderover> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>0</mn> </mrow> <mrow> <mi>I</mi> <mo>-</mo> <mn>1</mn> </mrow> </munderover> <munderover> <mo>&amp;Sigma;</mo> <mrow> <mi>j</mi> <mo>=</mo> <mn>0</mn> </mrow> <mrow> <mi>J</mi> <mo>=</mo> <mn>1</mn> </mrow> </munderover> <msub> <mi>M</mi> <mi>k</mi> </msub> <mrow> <mo>(</mo> <mi>m</mi> <mo>+</mo> <mi>i</mi> <mo>,</mo> <mi>n</mi> <mo>+</mo> <mi>j</mi> <mo>)</mo> </mrow> <msub> <mi>X</mi> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> </mrow> </msub> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>+</mo> <msub> <mi>b</mi> <mrow> <mi>k</mi> <mi>l</mi> </mrow> </msub> </mrow>

the convolutional layer inputs K feature maps and outputs L feature maps, the required number of convolutional kernels is LK, and the sizes of the convolutional kernels are IJ and F_lThe ith feature map, M, representing the output_kK characteristic diagram, X, representing input_k,lTwo-dimensional representation of the k-th row, l columnConvolution kernel, b_klRepresents a bias value;

the extracted features are weighted and summed and processed using a ReLU (Rectified Linear Units) unsaturated activation function:

σ(x)＝max{0，x}

after the activation function processing, the network performs maximum pooling operation on the feature map to form a new feature map as follows:

<mrow> <msup> <mi>Y</mi> <mrow> <mo>&amp;prime;</mo> <mi>l</mi> </mrow> </msup> <mrow> <mo>(</mo> <msup> <mi>m</mi> <mo>&amp;prime;</mo> </msup> <mo>,</mo> <msup> <mi>n</mi> <mo>&amp;prime;</mo> </msup> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mi>max</mi> <mrow> <mi>r</mi> <mo>&amp;Element;</mo> <mi>R</mi> </mrow> </munder> <msup> <mi>Y</mi> <mi>l</mi> </msup> <mrow> <mo>(</mo> <mi>m</mi> <mo>+</mo> <mi>r</mi> <mo>,</mo> <mi>n</mi> <mo>+</mo> <mi>r</mi> <mo>)</mo> </mrow> </mrow>

wherein Y'^lA graph representing pooled features;

according to the hierarchical arrangement of the recognition network, the convolution and pooling are repeatedly executed to obtain semantic abstraction of two-dimensional format data from low level to high level;

step two, classifying the moving objects by the recognition network: and after the features are connected through the full-connection layer, finally inputting the feature vector into a softmax function for classification.

4. The method for detecting and identifying the moving object on the airport surface according to claim 3, characterized in that the moving object on the airport is divided into three categories of airplane, automobile and pedestrian; the convolutional neural network of the invention has 14 layers in total, comprising 12 convolutional layers and 2 full-connection layers; aiming at the characteristic of large size difference of airport moving targets, the convolutional neural network uses 5 Inceposition structures, and extracts multi-scale features of the targets by respectively utilizing three convolution kernels of 1 × 1,3 × 3 and 5 × 5; in order to reduce the network overfitting and accelerate the network training and testing time, the number of feature maps and the size of convolution kernels of each layer are carefully designed.

5. The method for detecting and identifying the moving object on the airport surface according to claim 1, wherein the method comprises the steps of firstly extracting the area of the moving object in the image by using a tendency flow method, and then constructing a convolutional neural network to identify the moving object in the area proposed by the tendency flow method, so that an efficient and accurate airport moving object detection and identification framework is formed.