CN103942786B - The self adaptation block objects detection method of unmanned plane visible ray and infrared image - Google Patents

Abstract

The invention discloses a kind of overall block objects detection method being applicable to unmanned plane visible ray and infrared image, belong to image processing field.Described method is from image object essence, agglomeration according to target, target in the range of image overall is detected automatically, adapt to the demand of unmanned plane multiple image load, this method does not relies on the movable information of target, and sound target is respectively provided with the suitability, utilizes the parameter of unmanned aerial vehicle onboard LDMS and imaging device simultaneously, set up interesting target template size table, further speed up processing speed.Use the inventive method, it is possible to make full use of unmanned plane multi-load resource, improve efficiency and the accuracy rate of target detection.

Description

The self adaptation block objects detection method of unmanned plane visible ray and infrared image

Technical field

The invention belongs to unmanned plane technical field of image processing, be specifically related to a kind of unmanned plane visible ray and infrared image of being applicable to Self adaptation block objects detection method.

Background technology

Along with the development of science and technology, unmanned plane can carry multiple image load (such as visible ray, infrared sensor etc.), right The view data of different images sensor passback carries out Automatic Targets, is to ensure that unmanned plane has the base of round-the-clock fight capability Plinth.Unmanned plane image is to shoot to obtain under high-altitude, movement environment, has the following characteristics that target relative image is less, target Variation and easily occur profile variation and background complicated and changeable etc..In conjunction with above unmanned plane feature of image and multi-load should By demand, need to study a kind of simple effectively, be not only suitable for unmanned plane visible images be also applied for unmanned plane infrared image Object detection method.

Automatic Targets always machine vision one is important and has the research direction of challenge.Currently, for this problem Main research method can be divided into following a few class: 1, Target Modeling method, extracts target characteristic, characterizes target, sets up To Template, scans for coupling in global image scope；2, background modeling method, background is modeled, deducts in the picture Background i.e. can get target；3, image segmentation, image same area is split, thus background and target are separated.

Unmanned plane feature of image limits traditional object detection method.When Small object, target characteristic is unstable, no It is sufficient for matching detection accurately, and same target shows different features, no under infrared image and visible images Adapt to the demand of multiple image load.Background modeling method usually requires that photographic head is fixed, to obtain a relatively stable background Model, unmanned plane image then shoots during rapid movement and obtains, if application background modeling, is necessary for front and back Image registrates, and on the one hand registration process is the most more, and the detection in real time to target has considerable influence, the most infrared figure As the most ripe method for registering.Image, at certain feature space, is carried out clustering iteration, finds identical by image segmentation Region, thus be partitioned into target and background, it is adaptable to target compared with big and target and background in the widely different situation of feature space, In unmanned plane image, target is less, and infrared image is not easily found consistent feature space with visible images.

From upper analysis, currently carry out the main problems faced of target detection for unmanned plane multi-load image as follows: be both suitable for Be applicable to again visible images, Small object in infrared image, target is changeable and unknown, background is complicated and changeable, existing target Detection method all can not fully meet actual demand.

Summary of the invention

The present invention is to solve that unmanned plane multi-load Image Automatic Target detects problem encountered, it is proposed that a kind of from target essence Set out, be independent of the target automatic testing method of the particular type of image, the most comprehensive unmanned plane during flying height and imaging parameters, Set up target size table, the target of the multiple size of self adaptation.

No matter in visible images or in infrared image, unmanned plane target generally occurs, with it with block overall form Ambient background presents more significantly difference, referred to as agglomeration.The present invention is the intrinsic propesties of target from unmanned plane image, Utilize the agglomeration that it is presented, and according to unmanned plane imaging characteristic, propose one and be not only suitable for unmanned plane visible images but also fit Object detection method for unmanned plane infrared image.

The unmanned plane visible ray of the present invention and the self adaptation block objects detection method of infrared image, including following step:

The first step, sets up interesting target size template table

Second step, determines sliding window traversing graph picture

3rd step, carries out agglomeration detection to target

4th step, detects target under comprehensive many sizes

The invention have the advantages that

1) unmanned plane visible images and infrared image are adapted to；

2) make full use of other load of unmanned plane and parameter information, set up interesting target size table, adapt to the target of multiple size；

3) utilizing target agglomeration and the difference with ambient background thereof to detect, method is the most effective.

Accompanying drawing explanation

Fig. 1 is the population structure schematic diagram of the present invention；

Fig. 2 is that schematic diagram chosen by image slide window；

Fig. 3 is image-region pixel summation schematic diagram；

Fig. 4 is target detection schematic diagram；

Fig. 5 is target detection flow chart under same size；

Fig. 6 is the comprehensive schematic diagram of target under many sizes.

Detailed description of the invention

Below in conjunction with the accompanying drawings, the present invention is described in detail.

The present invention is a kind of object detection method being applicable to unmanned plane visible ray and infrared image based on agglomeration, population structure Schematic diagram is as it is shown in figure 1, specific implementation method comprises the following steps:

The first step, sets up the size table of interesting target according to the flight parameter of unmanned plane.

The interesting target being likely to occur is scouted in video for unmanned plane, such as moving vehicle, military architecture, landmark etc., Statistics target actual size size, in conjunction with the unmanned plane during flying height measured by unmanned aerial vehicle onboard laser ranging unit and unmanned Machine imaging load imaging parameters, can estimate roughly target Pixel Dimensions in the picture, then form the pixel of interesting target Size table.The Pixel Dimensions of target not exact value, but owing to follow-up detection method is not required to the most accurate target picture Element size, design object Pixel Dimensions table, primarily to quickly determine the size of detection window, improves the efficiency of detection.

Concrete:

Unmanned plane is during performing reconnaissance mission, and generally to specific objective, such as moving vehicle, military architecture, terrestrial reference is built Building, can add up its general actual size, this step can complete before flight.Target actual size (long and Wide) as shown in table 1,

Table 1 interesting target actual size table

Target	Long (unit: rice/m)	Wide (unit: rice/m)
			Target 1	L1	W1
Target 2	L2	W2
			…	…	…
Target n	Ln	Wn

According to the flying height that unmanned plane is different, and the imaging parameters of airborne imaging load, unmanned plane image can be calculated each Length Δ x (m) in reality representated by individual pixel, it is known that pixel dimension is Δ u (m), focal length is f (m), and flying height is H (m), M representation unit, for rice, the formula of the formula (1) physical length representated by each pixel in calculating image.

$\mathrm{\Δx}=\frac{H}{f}\mathrm{\Δu}---\left(1\right)$

Target length and width shared number of pixels in the picture can be obtained, respectively by l according to Δ x (m) and target actual size_iAnd w_i Representing, specific formula for calculation is as follows:

$\left\{\begin{array}{c}{l}_i=\frac{L_i}{\mathrm{\Δx}}\\ w_i=\frac{W_i}{\mathrm{\Δx}}\end{array}\right.---\left(2\right)$

Wherein: L_iRepresent the physical length of i-th target, W_iThe developed width of i-th target, l_iRepresent i-th target length At shared number of pixels, w in the picture_iExpression i-th target width is in shared number of pixels in the picture, and unit is rice, 1≤i≤n, n are the quantity obtaining interesting target；

Table 2 is unmanned plane interesting image object pixel size table,

Table 2 interesting target Pixel Dimensions table

Target	Long (unit: pixel)	Wide (unit: pixel)
			Target 1	l1	w1
Target 2	l2	w2
			…	…	…
Target n	ln	wn

Second step, according to the sliding window of the suitable size of object pixel size Selection.

According to the object pixel size table formed inside the first step, determine corresponding sliding window size, during detection, Will be according to the agglomeration of target, and the difference of target and ambient background, sliding window size is greater than the Pixel Dimensions of target. Then choose traversal rule, travel through global image with sliding window.Choosing of traversal rule can be flexible and changeable, mainly to inspection Surveying precision and real-time does a balance, in ergodic process, slide anteroposterior windows overlay rate is the highest, and Detection results is the best, but Need accordingly to consume the more time.

Concrete:

Sliding window size is chosen as the square window of p × p, owing to, in unmanned plane image, target is less, presents agglomeration, L under normal circumstances_i,w_i(1≤i≤n) is more or less the same, and different target is likely to have similar size, and the computing formula of p is as follows:

$p_i=\left\{\begin{array}{cc}\max (l_i,w_i)& \left(\right|l_i-w_i|>15)\\ 1.25*\frac{l_i+w_i}{2}& \left(\right|l_i-w_i|\≤15)\end{array}\right.---\left(3\right)$

For 1≤i≤n, by p_iSort from small to large, p₁≤...≤p_i≤...≤p_n, then with 15 for step-length, template is divided into m Individual grade, whereinThe template of each grade is averaging, obtains new window size q_k, as follows

Table 3 window size correspondence table

Size sequence number	Sliding window width (unit: pixel)
		1	q1
2	q2
		…	…
m	qm

For size k (1≤k≤m), use q_k×q_kImage is traveled through by the sliding window of size, schematic diagram as in figure 2 it is shown, The specific strategy of window traversal is as follows:

(1) sliding window is using the image upper left corner as original position；

(2) scanning to the right by row, step-length is

(3), after scanning through a line, sliding window moves down at up scanning starting positionContinuing to scan to the right by row, step-length is (4) repeat the above steps is until sliding window travels through entire image；

3rd step, in sliding window, carries out agglomeration detection to target.

Calculating the integrogram of image to be detected, the size of integrogram is as the size of original image, at sliding window traversing graph During Xiang, calculate the agglomeration in each sliding window, in then judging sliding window, whether there is target, for possible There is the window of target, record the information such as corresponding sliding window size, position, agglomeration, and determine target with the most all Contrast, get rid of the target of duplicate detection under same sliding window size, it can be ensured that the target detected under each size Do not repeat.When the sliding window under a size has traveled through image, the sliding window of other sizes is taked identical traversal And detection method, until the sliding window of all sizes has traveled through image.

Concrete:

The agglomeration of unmanned plane target refers to, target concentrates due to energy and ambient background exists significant difference and shows Character, the agglomeration of target is target essential attribute, is independent of the feature that concrete image type is formed, is therefore not only suitable for Unmanned plane infrared image, is applicable to again unmanned plane visible images.Image to be detected be designated as i (x, y), integrogram be designated as ii (x, y), Integrogram ii position (x, value y) be in image i (x, y) the gray value sum of all pixels of upper left, whereinI (x ', y ') represents the image i gray value at (x ', y ').As it is shown on figure 3, tetra-angle points of region R Position in the picture is respectively (x_A,y_A),(x_B,y_B),(x_C,y_C),(x_D,y_D), the pixel in the R of region and S_RCan be by formula (4) Calculate,

S_R=ii (x_D,y_D)+ii(x_A,y_A)-ii(x_B,y_B)-ii(x_C,y_C) (4)

Fig. 4 is target detection schematic diagram, and the sliding window length of side is q_k, target window and sliding window Gong Yige center, target window The mouth length of side is denoted as t_k, wherein t_k=0.75q_k, the agglomeration of target is denoted asExamine in conjunction with Fig. 5 target agglomeration Flow gauge figure, specifically comprises the following steps that

(1) if image visible images, (x y), if image unmanned plane infrared image, is directly entered first to be converted into gray level image i Next step；

(2) calculate image integrogram ii (x, y), wherein s (x ,-1)=0, ii (-1, y)=0；

S (x, y)=s (x, y-1)+i (x, y) (5)

Ii (x, y)=ii (x-1, y)+s (x, y) (6)

(3) calculate in sliding window all pixel grey scales and, be designated as

$S_{\mathrm{si}}^k=\mathrm{ii}(x_{\mathrm{Dk}},y_{\mathrm{Dk}})+\mathrm{ii}(x_{\mathrm{Ak}},y_{\mathrm{Ak}})-\mathrm{ii}(x_{\mathrm{Bk}},y_{\mathrm{Bk}})-\mathrm{ii}(x_{\mathrm{Ck}},y_{\mathrm{Ck}})$

Wherein: tetra-angle points of sliding window k position in the picture is respectively (x_Ak,y_Ak),(x_Bk,y_Bk),(x_Ck,y_Ck),(x_Dk,y_Dk)；

(4) according to formula (4) calculate in target window all pixel grey scales and, be designated as

$S_{\mathrm{ti}}^k=\mathrm{ii}(x_{\mathrm{Dkt}},y_{\mathrm{Dkt}})+\mathrm{ii}(x_{\mathrm{Akt}},y_{\mathrm{Akt}})-\mathrm{ii}(x_{\mathrm{Bkt}},y_{\mathrm{Bkt}})-\mathrm{ii}(x_{\mathrm{Ckt}},y_{\mathrm{Ckt}})$

Wherein, (x_Akt,y_Akt),(x_Bkt,y_Bkt),(x_Ckt,y_Ckt),(x_Dkt,y_Dkt) represent that in sliding window k, four angle points of target are at image Position, and

$\left\{\begin{array}{c}(x_{\mathrm{Akt}},y_{\mathrm{Akt}})=\left(\right[x_{\mathrm{Ak}}+\frac{1}{8}{q}_k],[y_{\mathrm{Ak}}+\frac{1}{8}{q}_k\left]\right)\\ (x_{\mathrm{Bkt}},y_{\mathrm{Bkt}})=\left(\right[x_{\mathrm{Bk}}-\frac{1}{8}{q}_k],[y_{\mathrm{Bk}}+\frac{1}{8}{q}_k\left]\right)\\ (x_{\mathrm{Ckt}},y_{\mathrm{Ckt}})=\left(\right[x_{\mathrm{Ck}}+\frac{1}{8}{q}_k],[y_{\mathrm{Ck}}-\frac{1}{8}{q}_k\left]\right)\\ (x_{\mathrm{Dkt}},y_{\mathrm{Dkt}})=\left(\right[x_{\mathrm{Dk}}-\frac{1}{8}{q}_k],[y_{\mathrm{Dk}}-\frac{1}{8}{q}_k\left]\right)\end{array}\right.$

Wherein: [A] expression carries out rounding operation to A；

Then in sliding window all pixel grey scales of background and

(5) agglomeration is calculated according to formula (7)；

${\mathrm{Agglomeration}}_i^k=|\frac{S_{\mathrm{ti}}^k}{t_k\×t_k}-\frac{S_{\mathrm{bi}}^k}{q_k\×q_k-t_k\×t_k}|---\left(7\right)$

(6) ifThen it is judged to target, size k of record sliding window, and position

(7) willWithCompare, if $\left(\right|x_i^k-x_j^k|+|y_i^k-y_j^k\left|\right)<10,$ Wherein 1≤j≤i-1,WithIt is then the same target under size k, removes the target that agglomeration is less；

4th step, is carried out comprehensively the target detected under different size sliding window

During the 3rd step target agglomeration detection, under each sliding window size, all can obtain an object chain, Detection between different size is separate, can only ensure the target not having to repeat in the object chain under each size, it is impossible to row Occuring simultaneously except existing between the object chain under different size sliding window, i.e. one target is repeated inspection under different size sliding window Situation about surveying, as shown in Figure 6.The all targets detected under different size sliding window are verified, get rid of and repeat target, And export final target.For for two sliding windows that size is k, s (1≤k, s≤m), ifThen it is judged to same target, removes the target that agglomeration is less.Should in reality During with, the destination number in a width unmanned plane image is few, and therefore many size objectives being carried out checking can't be to real-time Bring significantly impact.

Claims (2)

1. the object detection method being applicable to unmanned plane visible ray and infrared image based on agglomeration, comprises the following steps:

The first step, obtains the size of interesting target according to the flight parameter of unmanned plane；

According to the task of unmanned plane, obtaining the actual size of interesting target, the physical length obtaining i-th target is L_i, width For W_i, unit is rice, and 1≤i≤n, n are the quantity obtaining interesting target；

Physical length representated by acquisition each pixel of unmanned plane image:

$\mathrm{\&Delta;}x=\frac{H}{f}\mathrm{\&Delta;}u---\left(1\right)$

Wherein: Δ u represents that pixel dimension, H represent that flying height, f represent focal length, and unit is rice；

According to Δ x, L_i、W_iObtain target length and width shared number of pixels in the picture:

$\left\{\begin{array}{c}{l}_i=\frac{L_i}{\mathrm{\&Delta;}x}\\ w_i=\frac{W_i}{\mathrm{\&Delta;}x}\end{array}\right.---\left(2\right)$

Wherein: l_iRepresent i-th target length shared number of pixels, w in the picture_iRepresent that i-th target width is in the picture Shared number of pixels；

Second step, obtains the size of sliding window according to object pixel size；

If sliding window is p_i×p_iSquare window, p_iComputing formula as follows:

$p_i=\left\{\begin{array}{cc}max(l_i,w_i)& \left(\right|l_i-w_i|>15)\\ 1.25*\frac{l_i+w_i}{2}& \left(\right|l_i-w_i|\&le;15)\end{array}\right.---\left(3\right)$

By p_iSort from small to large, p₁≤...≤p_i≤...≤p_n, with 15 for step-length, template is divided into m grade, whereinBeing averaging the template of each grade, obtain k new window, size is q_k, 1≤k≤m；

For kth new window, use q_k×q_kImage is traveled through by the sliding window of size；

3rd step, in sliding window, carries out agglomeration detection to target；

In the sliding window k of second step, target is carried out agglomeration detection, particularly as follows:

(1) if image visible images, it is converted into gray level image, if image unmanned plane infrared image, is directly entered next Individual step, if (x y) represents that image i is at (x, y) gray value of position to i；

(2) (x, y), in position, (x, value y) is (x, y) upper left institute in image i to integrogram ii to the integrogram ii of calculating image i There is the gray value sum of pixel, i.e.I (x ', y ') represents the image i gray value at (x ', y ')；

IfS (x, y) be in image before xth row the gray scale of y pixel and, obtain ii (x, y):

Ii (x, y)=ii (x-1, y)+s (x, y) (4)

S (x, y)=s (x, y-1)+i (x, y) (5)

S (x ,-1)=0, ii (-1, y)=0 (6)

(3) obtain in sliding window k all pixel grey scales and:

$S_{si}^k=ii(x_{Dk},y_{Dk})+ii(x_{Ak},y_{Ak})-ii(x_{Bk},y_{Bk})-ii(x_{Ck},y_{Ck})---\left(7\right)$

Wherein: tetra-angle points of sliding window k position in the picture is respectively (x_Ak,y_Ak),(x_Bk,y_Bk),(x_Ck,y_Ck),(x_Dk,y_Dk)；

(4) obtain sliding window k internal object pixel grey scale and:

$S_{ti}^k=ii(x_{Dkt},y_{Dkt})+ii(x_{Akt},y_{Akt})-ii(x_{Bkt},y_{Bkt})-ii(x_{Ckt},y_{Ckt})---\left(8\right)$

Wherein, (x_Akt,y_Akt),(x_Bkt,y_Bkt),(x_Ckt,y_Ckt),(x_Dkt,y_Dkt) represent that in sliding window k, four angle points of target are at image Position, and

$\left\{\begin{array}{c}\left(x_{Akt},y_{Akt}\right)=\left(\&lsqb;x_{Ak}+\frac{1}{8}{q}_k\&rsqb;,\&lsqb;y_{Ak}+\frac{1}{8}{q}_k\&rsqb;\right)\\ \left(x_{Bkt},y_{Bkt}\right)=\left(\&lsqb;x_{Bk}-\frac{1}{8}{q}_k\&rsqb;,\&lsqb;y_{Bk}+\frac{1}{8}{q}_k\&rsqb;\right)\\ \left(x_{Ckt},y_{Ckt}\right)=\left(\&lsqb;x_{Ck}+\frac{1}{8}{q}_k\&rsqb;,\&lsqb;y_{Ck}-\frac{1}{8}{q}_k\&rsqb;\right)\\ \left(x_{Dkt},y_{Dkt}\right)=\left(\&lsqb;x_{Dk}-\frac{1}{8}{q}_k\&rsqb;,\&lsqb;y_{Dk}-\frac{1}{8}{q}_k\&rsqb;\right)\end{array}\right.---\left(9\right)$

Wherein: [A] expression carries out rounding operation to A；

Then all pixel grey scales of background and

(5) agglomeration is calculated according to following formula；

${\mathrm{Agglomeration}}_i^k=|\frac{S_{ti}^k}{t_k\&times;t_k}-\frac{S_{bi}^k}{q_k\&times;q_k-t_k\&times;t_k}|---\left(10\right)$

Wherein,

(6) ifThen it is judged to target, the position of record sliding window k

(7) willWithCompare, ifWherein 1≤j≤i-1,WithIt is then the same target under sliding window k, removes the target that agglomeration is less；

4th step, is carried out comprehensively the target detected under different size sliding window

For two sliding windows that size is k, s, 1≤k, s≤m, if Then it is judged to same target, removes the target that agglomeration is less.

A kind of target inspection being applicable to unmanned plane visible ray and infrared image based on agglomeration the most according to claim 1 Survey method, in second step, uses q_k×q_kImage is traveled through by the sliding window of size method particularly includes:

(1) sliding window is using the image upper left corner as original position；

(2) scanning to the right by row, step-length is

(3), after scanning through a line, sliding window moves down at up scanning starting positionContinue to scan to the right by row, step-length For

(4) repeat the above steps is until sliding window travels through entire image.