WO2021098163A1

WO2021098163A1 - Corner-based aerial target detection method

Info

Publication number: WO2021098163A1
Application number: PCT/CN2020/089937
Authority: WO
Inventors: 苗锋; 白俊奇; 杨沛; 朱伟; 杜瀚宇; 马浩; 刘�文; 邱文嘉; 翟尚礼
Original assignee: 南京莱斯电子设备有限公司
Priority date: 2019-11-18
Filing date: 2020-05-13
Publication date: 2021-05-27
Also published as: CN110852323B; CN110852323A

Abstract

A corner-based aerial target detection method, comprising: firstly, receiving an infrared image or a visible image, and placing image data in a buffer area; performing color space conversion on the image to obtain a corresponding gray-scale image; performing edge detection on the gray-scale image to obtain an edge closed region; obtain all corners conforming to a computation rule in the image by using a corner detection algorithm, and screening the corners according to a screening rule; grouping the corners by using the closed region, and taking the corners in the same contour as the same group; respectively calculating the outer contour of the central position of each group of corners as location information of the detection target; and outputting the location and size information of each target to complete target detection. By means of the method, the location of the target can be quickly determined, and the cloudy background interference in the air can be effectively eliminated, so that the target loss phenomenon occurring when an aerial target passes through the cloudy background is broken, and the aerial target detection problem under the cloudy background is solved.

Description

An air target detection method based on corner points

Technical field

The invention relates to image detection technology, in particular to a corner point-based aerial target detection method.

Background technique

In photoelectric early warning detection, photoelectric imaging equipment converts optical signals into electrical signals through imaging methods and transmits them to the processing terminal. The processing terminal detects the target in the video by image detection means and extracts the target position at the same time. Image target detection usually uses the characteristics of different objects in the image to show different colors, grays, and textures from the background, and distinguishes the target by detecting the edges of the target. The disadvantage of this method is that under cloudy weather conditions, the clouds show different shapes, colors and infrared radiation characteristics, forming a complex and irregular image background, and the image edges are cluttered, which affects the system’s impact on aircraft, drones, etc. The detection and tracking effect of similar early warning targets.

In the process of photoelectric early warning and detection, in cloudy weather, when aircraft and other targets pass through the cloud background, the cluttered edge characteristics will affect target detection and tracking. For important military targets or safety prevention and control targets, it is necessary to improve target detection probability and tracking stability Sex. Therefore, research on target detection methods suitable for different severe weather conditions is one of the important topics in the current image detection field. The key issues that need to be solved in air target detection technology are:

1. Select an image feature that can effectively distinguish the target from the cloudy background;

2. Design a method to accurately extract the location and size of the target.

Summary of the invention

Objective of the invention: The objective of the present invention is to provide a corner point-based aerial target detection method that can overcome the defects of the prior art.

Technical solutions:

The invention discloses a corner point-based aerial target detection method, which includes the following steps:

Step S1: Receive the video image, unpack the video and place it in the video buffer to obtain the video information. The video information includes the image resolution R×C, the frame rate N _f , and the image color space type, where R represents the number of horizontal pixels of the image , C represents the number of vertical pixels of the image;

Step S2: According to the video image color space type, extract the gray image, which is marked as Im _gray ;

Step S3: Image edge detection: for the gray image Im _gray , an edge detection algorithm is adopted to obtain the edge image Im _edge , and all the closed edges in the edge image Im _{edge are extracted, that is, the connected domain;}

Step S4: Image corner detection: For the gray image Im _gray , adopt the corner detection algorithm to obtain all the characteristic corner points in the image, record it as the characteristic corner point set P={p ₁ ,p ₂ …p _n }, record each Corner coordinates

Step S5: Perform corner grouping, and divide the feature corner set P into m subsets;

Step S6: Target extraction: extract the outer contours of the corner points in the m subsets respectively, find the smallest circumscribed rectangular frame of each group of corner points, and at the same time find the target centroid of the group as the target position, calculate the centroid to each side of the circumscribed rectangle The maximum width and height of, take the centroid as the center and the maximum width and height as the sides to make a rectangle, as the final target frame;

Step S7: Target information output: output the target position and target frame.

In the present invention, in step S1, the video image is a visible light or infrared video image.

In the present invention, step S2 includes:

For RGB images, RGB images are separated into R channel (R means red), G channel (G means green), and B channel image (B means blue). The method of obtaining Gray image is as follows:

Gray=R*0.299+G*0.587+B*0.114,

For YUV images, the Y channel is a grayscale image; YUV is a type of true-color color space (color space) compiled. Proper nouns such as Y'UV, YUV, YCbCr, YPbPr, etc. can all be called YUV, which overlap with each other. "Y" represents brightness (Luminance or Luma), which is the grayscale value, and "U" and "V" represent chrominance (Chrominance or Chroma), which are used to describe the color and saturation of the image, and are used to specify pixels s color.

For HSV images, the V channel is a grayscale image. HSV (Hue, Saturation, Value) is a color space created by A.R. Smith in 1978 based on the intuitive characteristics of colors, also called Hexcone Model. The color parameters in this model are: hue (H), saturation (S), and lightness (V).

In the present invention, step S3 includes the following steps:

S3.1: Perform Gaussian convolution on each pixel of the grayscale image Im _gray _{, the Gaussian template is a rectangular structure with a size of l 1} ×l ₂ , the standard deviation in the column direction is σ _x , and the standard deviation in the row direction is σ _y , to obtain smoothness The rear image Im _gauss , l ₁ and l ₂ respectively represent the length and width of the rectangular structure;

S3.2: Use the canny edge detection algorithm to _{extract the edge of the image Im gauss} to obtain the binarized edge image Im _edge . The dual threshold parameters are T _h and T _{l respectively} . Common edge detection algorithms such as sobel edge detection and canny edge Detection, etc. (Reference: Canny JA Computational Approach to Edge Detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,PAMI-8(6):679-698.);

S3.3: Perform the closed operation in geometric morphology on the image Im _edge to form a new binary edge image Im _morp . The structure element used is a rectangular structure with _{a size of l 3} ×l ₄ _{, l 3} , l ₄ Respectively indicate the length and width of the rectangular structure;

S3.4: image Im _morp connectivity detection region, adjacent the same edge point is marked as the communication area, communication domain obtained, the j-th communication domain referred to as _{C j, j = 1,2, ...} , m, using The adjacent structural element of is _{a rectangular structure with a size of l 5} ×l ₆ _{, and l 5} and l ₆ represent the length and width of the rectangular structure, respectively.

In the present invention, in step S4, the corner detection algorithm uses an orb feature point detection method (reference: Ruble E, Rabaud V, Konolige K, et al. ORB: an efficient alternative to SIFT or SURF[C]/ /IEEE International Conference on Computer Vision,ICCV 2011,Barcelona,Spain,November 6-13,2011.IEEE,2011.), using standard direction FAST feature point detection, and then the feature points are described by BRIEF feature, and the best one is selected. More than N characteristic corner points, obtain the characteristic corner point set P={p ₁ ,p ₂ …p _n }, where n≤N, p _n represents the nth characteristic corner point, and the kth characteristic corner point coordinate is

In the present invention, step S5 includes: the corner points in the same closed edge are grouped into the same group, and two or more closed edges divide the characteristic corner point set P into m subsets G ₁ , G ₂ … G _m , G _m represents The mth subset, and the subset satisfies:

Among them, the characteristic corner point subsets G ₁ , G ₂ ... G _{m are} non-empty and disjoint subsets.

In the corner grouping process, because the characteristic corner points are usually near the edge of the object, the edge detection in step S2 can ensure that all the characteristic corner points are distributed within the connected domain of each edge, satisfying the characteristic corner point subsets G ₁ , G ₂ …G _{m is a} non-empty and disjoint subset, and

In the present invention, step S6 includes the following steps:

S6.1: Calculate the centroid (X _i ,Y _i ) of the i-th feature corner subset G _i:

S6.2: Obtain the circumscribed rectangular boundary of all the feature corner points in the i-th feature corner point subset G _i;

S6.3: Calculate the x-direction distance X _r from the centroid to the side of the circumscribed rectangle, and the y-direction distance Y _r from the centroid to the side of the circumscribed rectangle;

S6.4: Calculate the boundary of the target frame.

In S6.1, calculate the i-th feature corner subset G _i centroid (X _i ,Y _i ) according to the following formula:

Among them, i = 1, 2, ..., m, n _i is the number of feature corners in the i-th feature corner subset G _i , and p _k is the kth in the i-th feature corner subset G _i Feature corner points.

In S6.2, the upper, lower, left, and right boundary coordinates x _left , x _right , y _up , y _down of the circumscribed rectangle of all the characteristic corner points in the i-th characteristic corner point subset G _i are obtained according to the following formula:

among them,

Is the x coordinate of the n _i- th characteristic corner point,

Is the y coordinate of the n _i- th characteristic corner point.

In the present invention, in S6.3, the following formula is used to calculate the radius from the centroid to the circumscribed rectangle:

X _r =max(abs(X _i -x _left ), abs(X _i -x _right )),

Y _r =max(abs(Y _i -y _up ), abs(Y _i -y _down )).

In the present invention, in S6.4, the following formulas are used to calculate the upper, lower, left, and right boundary coordinates x′ _left , x′ _right , y′ _up , and y′ _{down of the} final target frame:

x′ _left =X _i -X _r ,

x′ _right =X _i +X _r ,

y′ _up =Y _i -Y _r ,

_y'down =Y _i +Y _r .

Beneficial effects:

Compared with the prior art, the present invention has the following beneficial effects:

(1) Use corner detection to extract unique features of man-made targets, such as airplanes, helicopters, drones, etc., and effectively distinguish them from common interferences such as background clouds to increase the detection probability while reducing the false alarm rate;

(2) By combining corner points and edges, the corner points can be grouped and multiple targets can be distinguished;

(3) Using the distribution of feature points, accurately determine the target center and boundary, and improve the target detection display effect and tracking accuracy.

Description of the drawings

In the following, the present invention will be further described in detail with reference to the accompanying drawings and specific embodiments, and the above-mentioned or other advantages of the present invention will become clearer.

Figure 1 is a flow chart of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the drawings and embodiments.

As shown in Figure 1, the present invention discloses an air target detection method based on corner detection, which includes the following steps:

S1: Receive visible light or infrared video image: unpack the video and place it in the video buffer to obtain video information: image resolution R×C, frame rate N _f , and image color space type;

S2: Extract the corresponding gray image according to the image color space type: extract the gray image Im _{gray for} different color spaces such as RGB image, YUV image, HSV image, single-channel image, etc.;

S3: Image edge detection: For the gray image Im _gray , the edge detection algorithm is adopted to obtain the edge image Im _edge , and extract all the closed edges C _j _{in the edge image Im edge} , which is mainly divided into the following four steps:

S3.1: Perform Gaussian convolution on each point of the gray image Im _gray _{, the Gaussian template is a rectangular structure with a size of l 1} ×l ₂ , the standard deviation in the column direction is σ _x , and the standard deviation in the row direction is σ _y , after obtaining the smoothed Image Im _gauss , where the typical parameter values are l ₁ =7, l ₂ =7, σ _x =1.5, and σ _y =1.5;

S3.2: Use the canny edge detection algorithm to _{perform edge extraction on Im gauss} to obtain a binarized edge image Im _edge . The dual threshold parameters are T _h and T _l , and the typical parameter values are T _h =40 and T _l =30.

S3.3: _{Perform closed operations in geometric morphology on Im edge} to form a new binary edge graph Im _morp . The structure element used is _{a rectangular structure with a size of l 3} ×l ₄ , and the typical parameter value is l ₃ =5, l ₄ =5;

S3.4: Im _morp connectivity for detection region, adjacent the same edge point is marked as the communication area to obtain communication domain _{C j, j = 1,2, ...} , m, the adjacent structural elements used are of size l _{For a 5} ×l ₆ rectangular structure, typical parameter values are l ₅ =3 and l ₆ =3.

S4: Image corner detection: For the gray image Im _gray , use the corner detection algorithm to obtain all the characteristic corners in the image, record it as the point set P = {p ₁ ,p ₂ …p _n }, record the coordinates of each corner

Where k = 1, 2, ..., n; in this step, the corner point detection uses the orb feature point detection method, using the standard direction FAST feature point detection, and then the feature point is described by the BRIEF feature, and the optimal one is not more than N Obtain the characteristic corner point set P={p ₁ , p ₂ …p _n }, where n≤N, and the typical parameter value is N=50.

S5: Corner point grouping: the corner points within the same closed edge C _j are grouped into the same group. Multiple closed edges divide the point set P in S4 into subsets G ₁ , G ₂ … G _m . Since the characteristic corner points are usually near the edge of the object, the edge detection in S2 can ensure that all the characteristic corner points are distributed in Within the connected domain of each edge, the feature corner subsets G ₁ , G ₂ … G _{m are} non-empty and disjoint subsets, namely:

S6: Target extraction: extract the outer contours of the corner points of each subset respectively, find the smallest bounding rectangle of each group of corner points, and at the same time find the target centroid of the group as the target position, and calculate the maximum from the centroid to each side of the rectangle Width and height, take the centroid as the center and the maximum width and height as the sides to make a rectangle, as the final target frame; this step is specifically divided into the following four steps:

S6.1: Find the center of the characteristic corner point set G _i (X _i ,Y _i ), the formula is as follows

Among them, i=1,2,...,m, n _i is the number of characteristic corner points in the point set G _i;

S6.2: Calculate the circumscribed rectangle boundary of all the characteristic corner points in the point set G _{i, the formula is as follows:}

S6.3: Calculate the x-direction distance X _r from the centroid to the side of the circumscribed rectangle, and the y-direction distance Y _r from the centroid to the side of the circumscribed rectangle. The formula is as follows:

X _r =max(abs(X _i -x _left ), abs(X _i -x _right ))

Y _r ＝max(abs(Y _i -y _up ), abs(Y _i -y _down ))

S6.4: Calculate the boundary of the target frame, the formula is as follows:

x′ _left =X _i -X _r

x′ _right =X _i +X _r

y′ _up ＝Y _i -Y _r

y′ _down =Y _i +Y _r

S7: Target information output: output target position and target frame.

The present invention provides a method for detecting aerial targets based on corner points. There are many specific methods and ways to implement this technical solution. The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, In other words, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention. All the components that are not clear in this embodiment can be implemented using existing technology.

Claims

An air target detection method based on corner points, characterized in that it comprises the following steps:

Step S1: Receive the video image, unpack the video and place it in the video buffer to obtain the video information. The video information includes the image resolution R×C, the frame rate N f , and the image color space type, where R represents the number of horizontal pixels of the image , C represents the number of vertical pixels of the image;

Step S2: According to the video image color space type, extract the gray image, which is marked as Im gray ;

Step S3: Image edge detection: for the gray image Im gray , an edge detection algorithm is adopted to obtain the edge image Im edge , and all the closed edges in the edge image Im edge are extracted, that is, the connected domain;

Step S4: Image corner detection: For the gray image Im gray , adopt the corner detection algorithm to obtain all the characteristic corner points in the image, record it as the characteristic corner point set P={p 1 ,p 2 …p n }, record each Corner coordinates

Step S5: Perform corner grouping, and divide the feature corner set P into m subsets;

Step S6: Target extraction: extract the outer contours of the corner points in the m subsets respectively, find the smallest circumscribed rectangular frame of each group of corner points, and at the same time find the target centroid of the group as the target position, calculate the centroid to each side of the circumscribed rectangle The maximum width and height of, take the centroid as the center and the maximum width and height as the sides to make a rectangle, as the final target frame;

Step S7: Target information output: output the target position and target frame.
The method according to claim 1, wherein in step S1, the video image is a visible light or infrared video image.
The method according to claim 2, wherein step S2 comprises:

For RGB images, RGB images are separated into R channel, G channel, and B channel images. The method of obtaining Gray image is as follows:

Gray=R*0.299+G*0.587+B*0.114,

For YUV images, the Y channel is the grayscale image;

For HSV images, the V channel is a grayscale image.
The method according to claim 3, wherein step S3 comprises the following steps:

S3.1: Perform Gaussian convolution on each pixel of the grayscale image Im gray , the Gaussian template is a rectangular structure with a size of l 1 ×l 2 , the standard deviation in the column direction is σ x , and the standard deviation in the row direction is σ y , to obtain smoothness The rear image Im gauss , l 1 and l 2 respectively represent the length and width of the rectangular structure;

S3.2: Use the edge detection algorithm to perform edge extraction on the image Im gauss to obtain the binarized edge image Im edge ;

S3.3: Perform the closed operation in geometric morphology on the image Im edge to form a new binary edge image Im morp . The structure element used is a rectangular structure with a size of l 3 ×l 4 , l 3 , l 4 Respectively indicate the length and width of the rectangular structure;

S3.4: image Im morp connectivity detection region, adjacent the same edge point is marked as the communication area, communication domain obtained, the j-th communication domain referred to as C j, j = 1,2, ... , m, using The adjacent structural element of is a rectangular structure with a size of l 5 ×l 6 , and l 5 and l 6 represent the length and width of the rectangular structure, respectively.
The method according to claim 4, characterized in that, in step S4, the corner detection algorithm selects the method of orb feature point detection, uses the standard direction FAST feature point detection, and then performs the BRIEF feature description on the feature points, and selects the most There are no more than N characteristic corner points, and the characteristic corner point set P={p 1 ,p 2 …p n } is obtained, where n≤N, p n represents the nth characteristic corner point and the kth characteristic corner point The coordinates are
k=1, 2,..., n.
The method according to claim 5, characterized in that step S5 comprises: the corner points within the same closed edge are grouped into the same group, and two or more closed edges divide the characteristic corner point set P into m subsets G 1 , G 2 …G m , G m represents the m-th subset, and the subset satisfies:

Among them, the characteristic corner point subsets G 1 , G 2 ... G m are non-empty and disjoint subsets.
The method according to claim 6, wherein step S6 comprises the following steps:

S6.1: Calculate the centroid (X i ,Y i ) of the i-th feature corner subset G i:

S6.2: Obtain the circumscribed rectangular boundary of all the feature corner points in the i-th feature corner point subset G i;

S6.3: Calculate the x-direction distance X r from the centroid to the side of the circumscribed rectangle, and the y-direction distance Y r from the centroid to the side of the circumscribed rectangle;

S6.4: Calculate the boundary of the target frame.
The method according to claim 7, characterized in that, in S6.1, the i-th feature corner subset G i centroid (X i ,Y i ) is calculated according to the following formula:

Among them, i = 1, 2, ..., m, n i is the number of feature corners in the i-th feature corner subset G i , and p k is the k-th feature angle in the i-th feature corner subset G i point.
The method according to claim 8, characterized in that, in S6.2, the upper, lower, left, and right boundaries of the circumscribed rectangle of all the characteristic corner points in the i-th characteristic corner point subset G i are obtained according to the following formula Coordinates x left , x right , y up , y down :

among them,
Is the x coordinate of the n i- th characteristic corner point,
Is the y coordinate of the n i- th characteristic corner point.
The method according to claim 9, wherein in S6.3, the following formula is used to calculate the radius from the centroid to the circumscribed rectangle:

X r =max(abs(X i -x left ),abs(X i -x right )),

Y r =max(abs(Y i -y up ),abs(Y i -y down )),

Among them, abs() is the absolute value operation;

In S6.4, the following formulas are used to calculate the coordinates x′ left , x′ right , y′ up , and y′d own of the new target frame's upper, lower, left, and right boundary:

x′ left =X i -X r ,

x′ right =X i +X r ,

y′ up =Y i -Y r ,

y'down =Y i +Y r .