CN102359788A

CN102359788A - Series image target recursive identification method based on platform inertia attitude parameter

Info

Publication number: CN102359788A
Application number: CN 201110266278
Authority: CN
Inventors: 杨卫东; 朱鹏; 朱虎; 黎云; 杨洋; 张桥
Original assignee: Huazhong University of Science and Technology
Current assignee: Huazhong University of Science and Technology
Priority date: 2011-09-09
Filing date: 2011-09-09
Publication date: 2012-02-22
Anticipated expiration: 2031-09-09
Also published as: CN102359788B

Abstract

The invention provides a series image target recursive identification method based on platform inertia attitude parameters. Based on an initial whole image capture result, the method utilizes imaging attitude parameters to carry out recursive prediction on target point pixel coordinates in a series image and combines with a local detection identification method to carry out accurate positioning, so as to reduce search scope in a series image target identification process, inhibit interferences of a background image on the target detection identification, reduce positioning calculating time of the target identification, effectively increase reliability and positioning precision of the target identification, and provide a reliable and efficient target identification strategy for moving platform imaging.

Description

Sequential image target recursive identification method based on platform inertial attitude parameters

Technical Field

The invention belongs to the technical field of autonomous navigation and imaging target identification of aircrafts, and particularly relates to a sequential image target recursive identification method based on platform inertia attitude parameters, which is used for realizing rapid and accurate positioning of targets in sequential images and providing a new target identification tracking method and strategy for imaging accurate navigation.

Background

The problem of how to improve the recognition rate and the real-time performance of the automatic target recognition system under the condition of limited computing resources is a difficult problem in the technical field of navigation recognition and positioning, and the method has wide application value for the research of the technology.

Under the conditions of different time phases, climates, viewpoints, heights and the like of the aircraft navigation, the change of the characteristics of the target background in the optical image acquired by the imaging sensor is large. Especially, under the condition of remote imaging, the ground target under a complex background is identified, because the target is small, the number of repeated modes similar to the target in a view field is large, a large amount of interference exists, and the difficulty of identification and accurate positioning of the target is increased.

In the field of digital image processing, a traditional target identification method is to search in a full image range of a real-time image by using a prepared target feature template image and perform relevant matching identification so as to position the pixel coordinate position of a target point. However, when the scene where the target is located is complex, a plurality of repetitive patterns of target feature recognition exist in the background, or the contrast of the formed real-time image is low under the condition that the imaging condition is not ideal, and a plurality of areas close to the target features are formed in the background, the target recognition by using the traditional matching recognition method often generates a false capture phenomenon, so that the requirement of target recognition and positioning cannot be met. Meanwhile, the calculation amount of the full-field search and target identification is large, the real-time identification difficulty of the sequence image target is large under the condition that the calculation resources of the aircraft platform are limited, and a rapid identification method needs to be developed.

In the existing published documents, no report exists on a technology for quickly and automatically positioning and identifying a sequence image target specially aiming at a complex imaging environment, and the problem of automatic positioning and identifying of the target is a technical problem which has to be solved in the navigation positioning field and needs a new technical scheme to solve.

How to fully utilize the imaging attitude parameters acquired by the platform imaging system to predict the coordinates of the target in the image so as to carry out local identification and accurate positioning on the target has very important significance for the accurate navigation of the aircraft.

Disclosure of Invention

The invention provides a sequential image target recursive identification method based on platform inertia attitude parameters, and aims to solve the problems that due to the change of optical imaging conditions or the existence of a pattern which is matched with target features in a scene, the contrast of an imaged image is low, the features of a target are not obvious compared with a background, or the target is difficult to accurately identify and position under the condition that a plurality of repeated patterns which are similar to the target exist in an image background.

A sequential image target recursion identification method based on platform inertial attitude parameters specifically comprises the following steps:

(1) calculating the position of a light axis point in the current frame image in a target coordinate system according to the target initial pixel coordinate;

(2) estimating the position of the optical axis point in the target coordinate system in the next frame of image according to the position of the optical axis point in the target coordinate system in the current frame;

(3) predicting the final pixel coordinate of the target in the next frame image according to the position of the optical axis point in the next frame image in the target coordinate system;

(4) and constructing a local search window in the next frame image by taking the target final pixel coordinate as a center, updating the target final pixel coordinate by capturing the target in the local search window, and taking the updated target final pixel coordinate as the next identified target initial pixel coordinate.

The initial pixel coordinate of the target in the first recognition is obtained by full-image capture in the history sequence image.

The step (1) is specifically as follows: position (x) of optical axis point in target coordinate system in current frame_o0，y_o0，z_o0) The calculation method comprises the following steps:

y_o0＝0

wherein

dA_v＝ROW/φ

α₀To imager azimuth angle, θ₀For imager pitch angle, h₀Is imager height, gamma₀Is the rolling angle of the imager, phi is the longitudinal imaging field angle of the imager,

for the imager transverse imaging angle, ROW is the number of real-time imaging lines, COL is the number of real-time imaging columns, (u)_c0，v_c0) Is the target initial position.

The step (2) is specifically as follows: position (x) of the optical axis point in the target coordinate system in the next frame_o(i)，y_o(i)，z_o(i)) The calculation method comprises the following steps:

x_o(i)＝x_o0+ΔP_x(i)

z_o(i)＝z_o0+ΔP_z(i)

y_o(i)＝0

wherein,

ΔP_x(i)＝h_ictgθ_icosα_i-h₀ctgθ₀cosα₀+Δx_i

ΔP_z(i)＝h_ictgθ_isinα_i-h₀ctgθ₀sinα₀+Δz_i

and,

Δx_i＝Δt_i×w_x×dA_u

Δz_i＝Δt_i×w_z×dA_v

dA_v＝ROW/φ

(x_o0，0，z_o0) Is the position of the optical axis point of the current frame in the target coordinate system, Δ t_iRepresenting the imaging time difference from the current frame to the next frame; omega_xFor the angular velocity, w, of the imaging platform along the x-axis of the target coordinate system_zThe angular velocity of the imaging platform along the direction of the z axis of the target coordinate system; alpha is alpha₀、θ₀、h₀Respectively representing the azimuth angle, the pitch angle and the height of the current frame image; alpha is alpha_i、θ_i、h_iRespectively representing the imaging azimuth, the pitch angle and the height corresponding to the next frame of image; phi is the longitudinal imaging field angle of the imager,for the lateral imaging angle of the imager, ROW is the number of real-time imaging lines, and COL is the number of real-time imaging columns.

The step (3) is specifically as follows: target final pixel coordinate (u)_c(i)，v_c(i)) The calculation method comprises the following steps:

u_c(i)＝dA_u×[(A_v(i)-θ_i)×sinγ_i+A_u(i)×cosγ_i]

v_c(i)＝dA_v×[(A_v(i)-θ_i)×cosγ_i-A_u(i)×sinγ_i]

wherein,

dA_v＝ROW/φ

α_i、θ_i、h_iand gamma_iRespectively representing the azimuth angle, the pitch angle, the height and the roll angle corresponding to the next frame of image imaging, (x)_o(i)，0，z_o(i)) Is the position of the optical axis point in the next frame image in the target coordinate system, phi is the longitudinal imaging field angle of the imager,

for the lateral imaging angle of the imager, ROW is the number of real-time imaging lines, and COL is the number of real-time imaging columns.

The length T _ Row and the width T _ Col of the local search window in the step (4) are respectively:

T_Row＝L_Row+2×biasX

T_Col＝L_Col+2×biasZ

wherein,

<math> <mrow> <mi>L</mi> <mo>_</mo> <mi>Row</mi> <mo>=</mo> <mn>2</mn> <mo>×</mo> <mfrac> <mi>ROW</mi> <mi>φ</mi> </mfrac> <mo>×</mo> <mrow> <mo>(</mo> <mi>arctan</mi> <mfrac> <mi>h</mi> <mrow> <mfrac> <mi>h</mi> <mrow> <mi>tan</mi> <mi>θ</mi> </mrow> </mfrac> <mo>-</mo> <mfrac> <mi>Length</mi> <mn>2</mn> </mfrac> </mrow> </mfrac> <mo>-</mo> <mi>θ</mi> <mo>)</mo> </mrow> </mrow> </math>

biasX＝maxω_x×Δt×dA_u

biasZ＝maxω_z×Δt×dA_v

length, Width represents the actual Length and Width of the target under the geodetic coordinates, phi is the longitudinal imaging field angle of the imager,

the imaging angle is the transverse imaging angle of the imager, ROW is the real-time imaging line number, COL is the real-time imaging column number, and theta and h are the imaging pitch angle and height respectively; max omega_x，maxω_zRespectively representing the maximum angular speed of the aircraft in the direction of an x axis and the direction of a z axis; at represents the time difference between two adjacent frames of images of the aircraft, dA_uAnd dA_vRepresenting the lateral and longitudinal angular resolutions, respectively.

The technical effects of the invention are as follows:

due to various reasons, the optical image has more repeated patterns similar to the target features, so that the target cannot be accurately identified and positioned. According to the method, the coordinate position of the target pixel in the sequence image is predicted by utilizing the imaging attitude parameter acquired by the platform imaging system, and then accurate positioning is realized in a local matching positioning mode, so that the robustness and the adaptability of the target automatic identification positioning system are ensured, and the defect that accurate identification positioning cannot be realized due to more repeated modes in a real-time image is overcome.

In the initial capturing and optical axis point estimating stage of the target, the position of the optical axis point under the target coordinate system is estimated according to the target pixel coordinate obtained by initial capturing by taking the target point coordinate as a reference, and the position of the ground optical axis point of the next frame of image in the sequence is predicted according to the imaging attitude parameter. And in the stage of recursive prediction of the pixel coordinates of the target point, the pixel coordinate position of the target in the image is reversely deduced according to the ground optical axis point position and the imaging attitude parameters, and then the local positioning is carried out by taking the position as the basis, so that the accurate positioning of the target is realized. The method has the advantages that the target is accurately identified and positioned under the conditions that the contrast of the imaged image is low, the characteristic of the target is not obvious compared with the background due to the change of the optical imaging condition or the existence of the mode which is similar to the target characteristic in the scene, or more repeated modes which are similar to the target exist in the image background, a new thought is provided for increasing the identification rate of the automatic identification of the target, and the method has great significance for the research and development of the automatic identification and navigation positioning technology of the target.

The invention predicts the pixel coordinates of the target point in the process of identifying and positioning the target, and then replaces the whole-image search by using a local positioning mode, thereby greatly reducing the target search range in the calculation process, saving the identification calculation time, improving the performance of the whole automatic target identification system, and providing a feasible method and a new thought for the application occasion with higher real-time requirement on target identification.

Drawings

FIG. 1 is a general flow chart of a sequential image target recursive identification method based on platform inertial attitude parameters according to the present invention;

FIG. 2 is a diagram of an imaging geometry mathematical model defined in the present invention, including a target coordinate system, an imaging platform measurement coordinate system, and a line-of-sight coordinate system;

FIG. 3 is a schematic view of the imaging attitude parameter determination and imaging field range in the present invention;

FIG. 4 is a schematic diagram of the principle of the present invention for performing mobile pipeline filtering on an initial target point pixel coordinate trajectory;

FIG. 5 is a schematic diagram of the transformation of space and geometric coordinates according to the present invention;

FIG. 6 is a schematic diagram of the location estimation of the ground optical axis point in the present invention;

FIG. 7 is a schematic diagram of predicting pixel coordinates of a target point according to a ground optical axis point position in the present invention;

FIG. 8 is a graph comparing the height with the abscissa deviation curve of the target recognition and location results in the top and bottom frame images, with and without the algorithm of the present invention;

FIG. 9 is a graph comparing the height with the vertical coordinate deviation curves of the target recognition and location results in the top and bottom frame images, with and without the algorithm of the present invention.

Detailed Description

A general flow chart 1 of a sequential image target recursive prediction identification method based on platform inertial attitude parameters is shown in the figure, and the specific process of the method is as follows:

1. target point initial pixel coordinate capture

(1.1) coordinate System and inertial attitude parameter definition

In order to establish an imaging geometric mathematical model, a target coordinate system, an imaging platform measurement coordinate system and a sight line coordinate system are defined, and the following steps are respectively carried out:

(1) target coordinate system o_gAx_gAy_gAz_gA

Target coordinate system o_gAx_gAy_gAz_gAFor the world coordinate system of the target observed by the imager, i.e. the geometric center o of the target point detected at the initial imaging moment_gAIs the origin, o_gAy_gAThe axis is consistent with the normal line of the local reference ellipsoid and points upwards, and the sky is positive; o_gAx_gAThe local horizontal plane is oriented to the north, i.e. to the north_gAz_gAShaft and o_gAx_gA、o_gAy_gAThe axes are in a right-handed orthogonal coordinate system, east as shown in fig. 2.

(2) Aircraft coordinate system o_Rx_Ry_Rz_R

With the aircraft centroid o_RIs the origin, o_Rx_RAlong the axial direction of the platform and pointing to the head, when the installation error is not counted, the platform measurement coordinate system is overlapped with the aircraft coordinate system. The imaging platform outputs angle information of a pitch angle theta, an azimuth angle alpha and a roll angle gamma of the aircraft.

(3) Line of sight coordinate system o_Rx_Sy_Sz_S

The observation angle measurement coordinate system is defined by taking a target as a reference. With the aircraft centroid o_RIs the origin, o_Rx_sThe axis being coincident with the aircraft's line of sight to the target, o_Rz_sThe axis is parallel to the horizontal plane of the target point and is o_Rx_sThe axis pointing vertically to the right, o_Ry_sShaft and o_Rx_sShaft o_Rz_sThe axes form a right-handed orthogonal coordinate system.

(1.2) initial Capture of target Point

(1.2.1) obtaining the track coordinates of the target in the previous n frames of images by full-image matching

In the process of matching the whole image, methods such as a gray level cross-correlation matching algorithm, a phase cross-correlation matching algorithm, a mutual information matching algorithm and a feature-based matching algorithm can be adopted. The invention adopts the gray level cross-correlation matching algorithm of mean value removal normalization and correspondingly simplifies the algorithm according to the real-time requirement.

Assume a template is M and its size is R_r×C_rF is the image to be matched and R is the size_s×C_sAnd R is_s＞R_r，R_s＞R_rThen the calculated traversal range is (R)_s-R_r)×(C_s-C_r). According to a calculation formula of the mean value removing normalized cross-correlation algorithm, after the template and the image to be matched are convolved, the variance of the template and the variance of the current position of the image to be matched are divided.

Under the condition of high real-time requirement, besides ensuring the accuracy of the calculation result, the method of replacing the calculation time by the storage space is also a very effective way. Each point traversed by the point to be matched is divided by the variance of the template, so that the variance of the template can be eliminated in actual calculation, and the variance of the template is eliminated after the maximum value of the calculated correlation coefficient is obtained, thereby omitting one division operation. The normalized de-mean cross-correlation match metric given above is converted into:

<math> <mrow> <mi>NC</mi> <mrow> <mo>(</mo> <mi>m</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <munder> <mi>Σ</mi> <mi>x</mi> </munder> <munder> <mi>Σ</mi> <mi>y</mi> </munder> <mo>[</mo> <mi>M</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <mover> <mi>M</mi> <mo>&OverBar;</mo> </mover> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>×</mo> <mo>[</mo> <mi>F</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>+</mo> <mi>m</mi> <mo>,</mo> <mi>y</mi> <mo>+</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>-</mo> <mover> <msub> <mi>F</mi> <mi>mn</mi> </msub> <mo>&OverBar;</mo> </mover> <mo>]</mo> </mrow> <msqrt> <munder> <mi>Σ</mi> <mi>x</mi> </munder> <munder> <mi>Σ</mi> <mi>y</mi> </munder> <msup> <mrow> <mo>[</mo> <mi>F</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>+</mo> <mi>m</mi> <mo>,</mo> <mi>y</mi> <mo>+</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>-</mo> <mover> <msub> <mi>F</mi> <mi>mn</mi> </msub> <mo>&OverBar;</mo> </mover> <mo>]</mo> </mrow> <mn>2</mn> </msup> </msqrt> </mfrac> </mrow> </math>

simplifying the formula, and removing the root number as follows:

<math> <mrow> <msup> <mi>NC</mi> <mn>2</mn> </msup> <mrow> <mo>(</mo> <mi>m</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <msup> <mrow> <mo>{</mo> <munder> <mi>Σ</mi> <mi>x</mi> </munder> <munder> <mi>Σ</mi> <mi>y</mi> </munder> <mo>[</mo> <mi>M</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>-</mo> <mover> <mi>M</mi> <mo>&OverBar;</mo> </mover> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>×</mo> <mo>[</mo> <mi>F</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>+</mo> <mi>m</mi> <mo>,</mo> <mi>y</mi> <mo>+</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>-</mo> <mover> <msub> <mi>F</mi> <mi>mn</mi> </msub> <mo>&OverBar;</mo> </mover> <mo>]</mo> <mo>}</mo> </mrow> <mn>2</mn> </msup> <mrow> <munder> <mi>Σ</mi> <mi>x</mi> </munder> <munder> <mi>Σ</mi> <mi>y</mi> </munder> <msup> <mrow> <mo>[</mo> <mi>F</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>+</mo> <mi>m</mi> <mo>,</mo> <mi>y</mi> <mo>+</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>-</mo> <mover> <mrow> <msub> <mi>F</mi> <mi>mn</mi> </msub> <mo>]</mo> </mrow> <mo>&OverBar;</mo> </mover> </mrow> <mn>2</mn> </msup> </mrow> </mfrac> </mrow> </math>

the molecules of the above formula are simplified to give:

<math> <mrow> <msup> <mi>NC</mi> <mn>2</mn> </msup> <mrow> <mo>(</mo> <mi>m</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <msup> <mrow> <mo>[</mo> <munder> <mi>Σ</mi> <mi>x</mi> </munder> <munder> <mi>Σ</mi> <mi>y</mi> </munder> <mi>M</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>×</mo> <mi>F</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>+</mo> <mi>m</mi> <mo>,</mo> <mi>y</mi> <mo>+</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>-</mo> <mi>Δ</mi> <mo>×</mo> <mover> <mi>M</mi> <mo>&OverBar;</mo> </mover> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>×</mo> <mover> <msub> <mi>F</mi> <mi>mn</mi> </msub> <mo>&OverBar;</mo> </mover> <mo>]</mo> </mrow> <mn>2</mn> </msup> <msub> <mi>D</mi> <mi>mn</mi> </msub> </mfrac> </mrow> </math>

wherein:

<math> <mrow> <msub> <mi>D</mi> <mi>mn</mi> </msub> <mo>=</mo> <munder> <mi>Σ</mi> <mi>x</mi> </munder> <munder> <mi>Σ</mi> <mi>y</mi> </munder> <msup> <mrow> <mo>[</mo> <mi>F</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>+</mo> <mi>m</mi> <mo>,</mo> <mi>y</mi> <mo>+</mo> <mi>n</mi> <mo>)</mo> </mrow> <mo>-</mo> <mover> <msub> <mi>F</mi> <mi>mn</mi> </msub> <mo>&OverBar;</mo> </mover> <mo>]</mo> </mrow> <mn>2</mn> </msup> </mrow> </math>

Δ＝R_r×C_r

wherein M (x, y) is the gray value of the pixel point (x, y) in the template,

is the mean value of the gray levels of the template, D_mnF (x + m, y + n) is the gray value of the point (x + m, y + n) of the image to be matched, which is the square of the variance of the template size range at the point (m, n) of the image to be matched,

is the gray average value of the image to be matched, wherein m is more than or equal to 0 and less than or equal to (R)_s-R_r)，0≤n≤(C_s-C_r)。

(1.2.2) determining initial target pixel coordinates

(1) Determining a target point fluctuation range

Because the angular velocity of the aircraft changes at a constant speed according to a relatively stable value in an imaging period, the fluctuation range of the pixel coordinate positions of the target points of two adjacent frames of images in the sequence can be estimated according to the condition. The specific calculation process is as follows:

Δx＝maxω_x×Δt×dA_u

Δy＝maxω_y×Δt×dA_v

wherein, Deltax, Deltay respectively represent the maximum fluctuation range of the horizontal and vertical coordinates of the target point of two adjacent frames of images, max omega_x，maxω_yRespectively representing the maximum angular speed of the aircraft in the direction of an x axis and the direction of a z axis; at represents the time difference between two adjacent frames of images of the aircraft, dA_u，da_vRespectively representing the lateral and longitudinal angular resolutions,

(2) mobile pipeline filtering

According to the change rule of the angular velocity of the aircraft in the flying process, the coordinate tracks of the target points in the formed sequence images fluctuate within a certain range, so that a time domain pipeline filter based on the tracks of the target points in the formed images can be established according to the fluctuation range of the pixel coordinate positions of the target points of two adjacent frames of images in the estimated sequence, n track points obtained through global matching are filtered, the error capturing result is filtered, and the accurate initial coordinate position of the target point is obtained, as shown in fig. 4.

And performing full map matching on the prepared target reference map and the first n (n > -3) frame image of the image sequence to obtain a track of a target point, wherein the track is expressed as: a. the₀，A₁...A_nThe corresponding pixel coordinate position points are respectively as follows: (x)₀，y₀)，(x₁，y₁)...(x_n，y_n) And the size of n is selected according to the real-time requirement of the algorithm.

Assuming that the filter pipeline length is N (N ≦ N, N is typically an odd number), the gate size is denoted by M (l, d), where l and d denote the length and width of the filter gate, respectively, and the filter pipeline is implemented in a first-in-first-out manner.

The candidate target point A_i(iN) as the center of the current pipeline, N-1 frames of images (before and after each) adjacent to the current pipeline are calculated

Frame) and all other track points in the pipeline meet the following conditions, and the judgment formula is as follows:

x_j-Δn×Δx＜＝x_i＜＝x_j+Δn×Δx

y_j-Δn×Δy＜＝y_i＜＝y_j+Δn×Δy

where Δ N denotes a frame difference between the previous and subsequent sequence images, Δ x, Δ y denote fluctuation ranges of pixel coordinate positions of target points of two adjacent frame images in the sequence estimated from 1.2.2, j < i < N, and there are:

Δn＝i-j+1

Δx＝maxω_x×Δt×dA_u

Δy＝maxω_y×Δt×dA_v

the dots satisfying the condition are counted by a counter, k_tIndicating that if the candidate target point in the pipeline meets the judgment condition, k is set_tIs added by 1 when

When the number of the effective points in the pipeline is less than the preset value, the point in the pipeline is considered to be effective

Then, the center point of the pipeline is considered as the accurate coordinate (x) of the target point in the current frame image_i，y_i)^·。

Continuously updating the image in the pipeline, filtering the track coordinates of the target points in the selected image sequence through the pipeline, and filtering the error capturing result to obtain the accurate coordinates (x) of the target points₀，y₀)^·，(x₁，y₁)·...(x_s，y_s)^·. Selecting the coordinate (x) of the target point with the frame number closest to n_s，y_s)^·As initial coordinate position (u) of target point_c0，v_c0) Subsequent calculations are performed.

2. Optical axis point position estimation

(2.1) estimation of initial position of optical axis point

(2.1.1) target coordinate System Down-conversion model

As shown in fig. 5, the position coordinates of the target point in the target coordinate system are transformed to the pixel coordinates of the target point in the image coordinate system, and the calculation method is as follows:

in a target coordinate system, assuming that the position of a platform corresponding to the imaging time is P, defining imaging attitude parameters as (h, alpha, theta and gamma), and defining the origin of a platform measurement coordinate system as O_R(x_R，x_R，z_R) The coordinate of the ground point (hereinafter referred to as ground optical axis point) to which the optical axis of the camera is directed is O_o(x_o，y_o，z_o) Then, there are:

x_o＝x_R+h·ctgθ·cosα

z_o＝z_R+h·ctgθ·sinα

y_R＝h

y_o＝0

in the case where the roll angle γ is not considered, the coordinates of the target point in the image are (u, v), and there are:

u＝COL/2+A_u·dA_u

v＝ROW/2+(A_v-θ)·dA_v

wherein the transverse angular resolution

And longitudinal angular resolution dA_v＝ROW/φ，

The field angle is transverse, phi is longitudinal, ROW is the number of real-time imaging lines, and COL is the number of real-time imaging columns.

<math> <mrow> <msub> <mi>A</mi> <mi>v</mi> </msub> <mo>=</mo> <msup> <mi>tg</mi> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </msup> <mo>[</mo> <mfrac> <mi>h</mi> <mrow> <mi>hctgθ</mi> <mo>-</mo> <msub> <mi>x</mi> <mi>o</mi> </msub> <mi>cos</mi> <mi>α</mi> <mo>-</mo> <msub> <mi>z</mi> <mi>o</mi> </msub> <mi>sin</mi> <mi>α</mi> </mrow> </mfrac> <mo>]</mo> </mrow> </math>

The pose roll angle results in a rotational transformation of the target point in the image plane with the coordinate (u)_c，v_c)：

As can be seen from the above formula, in the target coordinate systemNext, the position of the ground optical axis point (x) is determined_o，y_o，z_o) Then, given the imaging attitude parameters, the coordinates (u) of the target point in the real-time image can be calculated_c，v_c). The above model can be expressed in the form:

(u_c，v_c)＝F[(x_o，z_o)|h，α，θ，γ]

the corresponding inverse transformation model is:

(x_o，z_o)＝F^-1[(u_c，v_c)|h，α，θ，γ]

(2.1.2) estimation of ground optical axis Point position

Since the pixel coordinates of the target point in the image have been obtained at the initial coordinate capturing stage of the target point, the initial coordinate position (u) of the target in the resulting image is determined according to the inverse transformation model described above, as shown in fig. 5_c0，v_c0) As a known quantity, the corresponding ground optical axis point position (x) in the target coordinate system is solved_o0，y_o0，z_o0) The specific calculation process is as follows:

firstly, calculating the field angle (A) of the target point and the optical axis point in the transverse and longitudinal directions_u0，A_v0)：

The change model comprises:

by inverse transformation of the matrix one can obtain:

then:

according to the obtained opening angle (A) in the transverse and longitudinal directions_u0，A_v0) Solving the corresponding ground optical axis point position (x) under the target coordinate system_o0，y_o0，z_o0)：

y_o0＝0

Wherein alpha is₀To an imaging azimuth angle, theta₀For imaging pitch angle, gamma₀For the imager roll angle, h₀Is the imaging height, phi is the longitudinal imaging field angle of the imager,

for the imager transverse imaging angle, ROW is the number of real-time imaging lines, COL is the number of real-time imaging columns, dA_u，dA_vRepresenting the lateral and longitudinal angular resolutions, respectively.

(2.2) recursive estimation of optical axis point position

Under the imaging condition, the position of the ground optical axis point changes due to the motion of the platform, and the coordinates of the target point in the formed image correspondingly change. The position offset (delta P) of the ground optical axis point in the target coordinate system can be calculated according to the position of the initial ground optical axis point in the target coordinate system and by combining the imaging attitude parameters of the current frame and the next frame_x(i)，ΔP_z(i)) Thereby estimating the position (x) of the ground optical axis point of the next frame image_o(i)，y_o(i)，z_o(i)) The method comprises the following steps:

as shown in fig. 6, the target coordinate system is chosen as the reference frame, assuming in the initial case the position of the imaging platform is: p₀(x_R0，h₀，z_R0) The corresponding ground optical axis positions are: q₀(x_o0，0，z_o0) And the position of the current frame image imaging platform is as follows: p_i(x_R(i)，h_i，z_R(i)) The corresponding ground optical axis positions are: q_i(x_o(i)，0，z_o(i)) Then, the following relationship is given:

P₀and Q₀The following components are arranged in between:

x_R0＝x_o0-h₀·ctgθ₀·cosα₀

z_R0＝z_R0-h₀·ctgθ₀·sinα₀

P_iand Q_iThe following components are arranged in between:

x_R(i)＝x_o(i)-h_i·ctgθ_i·cosα_i

z_R(i)＝z_o(i)-h_i·ctgθ_i·sinα_i

due to P₀And P_iIs the coordinate position difference caused by the movement of the platform, and the following relationship exists between the two types of the platform:

x_R(i)＝x_R0+Δx_i

z_R(i)＝z_R0+Δz_i

Δx_i、Δz_ithe movement amounts (which may be negative values) of the imaging platform along the x-axis and the z-axis under the target coordinate system are respectively expressed by the following calculation formula:

Δx_i＝Δt_i×w_x×dA_u

Δz_i＝Δt_i×w_z×dA_v

dA_v＝ROW/φ

where Δ t_iRepresenting the platform motion time, ω, from the initial frame to the current frame_xFor the angular velocity, w, of the imaging platform in the direction of the x-axis of the target coordinate system_zThe angular velocity of the imaging platform in the direction of the z-axis of the target coordinate system.

This can be calculated as:

x_o(i)＝x_o0+ΔP_x(i)

z_o(i)＝z_o0+ΔP_z(i)

y_o(i)＝0

and has the following components:

ΔP_x(i)＝h_ictgθ_icosα_i-h₀ctgθ₀cosα₀+Δx_i

ΔP_z(i)＝h_ictgθ_isinα_i-h₀ctgθ₀sinα₀+Δz_i

in many cases, when the difference between the frame numbers of the two images is small and the difference between the imaging times is short, as shown in FIG. 8, the near vision can be considered as P₀And P_iThe projections on the XOZ plane in the target coordinate system are overlapped, and the movement amount of the imaging platform in the directions of the x axis and the z axis in the target coordinate system is ignored, and then:

Δx_i＝Δz_i＝0

in the derivation process described above, α₀、θ₀、h₀Respectively representing the azimuth angle, the pitch angle and the height of the current frame; alpha is alpha_i、θ_i、h_iRespectively representing the azimuth angle, the pitch angle and the height of the next frame; phi is the longitudinal imaging field angle of the imager,

3. Target point coordinate prediction and accurate positioning

(3.1) target Point coordinate prediction

As shown in FIG. 7, the position (x) of the ground optical axis of the next frame image is obtained by estimation_o(i)，y_o(i)，z_o(i)) Thereafter, the pixel coordinate position (u) of the target point in the next frame image can be predicted according to the imaging attitude parameters thereof_c(i)，v_c(i))。

Firstly, the field angle (A) between the target point and the ground optical axis point is calculated_u(i)，A_v(i)) If the target coordinate system is selected as the reference system, the following steps are performed:

according to the opening angle and the imaging roll angle gamma_iPredicting the coordinate position (u) of the target point pixel in the image_c(i)，v_c(i)) The calculation is as follows:

then there are:

wherein alpha is_i、θ_i、h_iRespectively representing the azimuth angle, the pitch angle and the height of the next frame; ROW is the number of real-time imaging lines, COL is the number of real-time imaging columns, dA_u，dA_vRepresenting the lateral and longitudinal angular resolutions, respectively.

(3.2) accurate positioning of coordinates of target points

In order to obtain accurate target point coordinates, a local search window is constructed by taking the predicted target point coordinates as a center, and identification and positioning are carried out in the local search window.

And when the local search window range is determined, estimating the size of the target in the sequence image according to the flight attitude parameters, and then giving out pixels in the column direction and the row direction as the size of a target search area in the image according to the imaging characteristics of the imager and various errors after comprehensive consideration.

The region size of the target region in the sequence image is:

length and Width represent the actual Length and Width of the target under the geodetic coordinate, ROW is the number of real-time imaging lines, COL is the number of real-time imaging columns, and theta and h are the imaging pitch angle height respectively.

Because the photoelectric platform has certain error when measuring the imaging attitude parameter in the actual imaging process. After various errors are comprehensively considered, performing pixel external expansion biasX and biasZ on the search area in the column direction and the row direction respectively, wherein the size of the search area is as follows:

T_Row＝L_Row+2×biasX

T_Col＝L_Col+2×biasZ

wherein the sizes of biasX and biasZ are estimated according to the movement limit speed of the imaging platform in the actual imaging process, and the calculation formula is expressed as:

biasX＝maxω_x×Δt×dA_u

biasZ＝maxω_z×Δt×dA_v

maxω_x，maxω_zrespectively representing the maximum angular speed of the aircraft in the direction of an x axis and the direction of a z axis; at represents the time difference between two adjacent frames of images of the aircraft, dA_u，dA_vRepresenting the lateral and longitudinal angular resolutions, respectively.

(3.3) optical axis Point position update

And obtaining accurate target point coordinates in the current frame image in a local identification and positioning mode. Because the position of the ground optical axis is deduced according to the imaging attitude parameters, and the imaging platform has certain errors in the measurement process of the imaging attitude parameters, if the errors are always calculated, the errors are inevitably accumulated, thereby causing the capture failure.

And updating the position of the ground optical axis by using the accurate coordinates of the target point in the current frame image obtained by local identification and positioning, and eliminating the accumulated error of the target point coordinate estimation.

Selecting a target coordinate system as a reference system, and according to an opening angle calculation formula, the relation between the pixel coordinate position of a target point in an image and the opening angles of the target point and the ground optical axis point in the transverse and longitudinal directions is as follows:

according to the obtained opening angle (A) in the transverse and longitudinal directions_u(i) ^·，A_v(i) ^·) And updating the position (x) of the corresponding ground optical axis under the target coordinate system_o(i) ^·，0，z_o(i) ^·) The specific method comprises the following steps:

wherein alpha is_i、θ_i、h_iRespectively representing the azimuth angle, the pitch angle and the height of the next frame; ROW is the number of real-time imaging lines, and COL is the number of real-time imaging columns；dA_u，dA_vRepresenting the lateral and longitudinal angular resolutions, respectively.

After the ground optical axis position is obtained, the next frame of image ground optical axis position is predicted, and the target point coordinate prediction positioning in the sequence image is realized through recursion.

Claims

1. A sequential image target recursion identification method based on platform inertial attitude parameters specifically comprises the following steps:

2. The method for recursive identification of sequence images targets based on platform inertial gesture parameters according to claim 1, wherein the initial pixel coordinates of the target in the first identification are obtained by full image capture in the historical sequence images.

3. The method for recursive identification of sequence image targets based on platform inertial attitude parameters according to claim 1, wherein the step (1) is specifically as follows: position (x) of optical axis point in target coordinate system in current frame_o0，y_o0，z_o0) The calculation method comprises the following steps:

y_o0＝0

wherein

dA_v＝ROW/φ

4. The method for recursive identification of sequence image targets based on platform inertial attitude parameters according to claim 2, wherein the step (2) is specifically as follows: position (x) of the optical axis point in the target coordinate system in the next frame_o(i)，y_o(i)，z_o(i)) The calculation method comprises the following steps:

x_o(i)＝x_o0+ΔP_x(i)

z_o(i)＝z_o0+ΔP_z(i)

y_o(i)＝0

wherein,

ΔP_x(i)＝h_ictgθ_icosα_i-h₀ctgθ₀cosα₀+Δx_i

ΔP_z(i)＝h_ictgθ_isinα_i-h₀ctgθ₀sinα₀+Δz_i

and,

Δx_i＝Δt_i×w_x×dA_u

Δz_i＝Δt_i×w_z×dA_v

dA_v＝ROW/φ

(x_o0，0，z_o0) Is the position of the optical axis point of the current frame in the target coordinate system, Δ t_iRepresenting the imaging time difference from the current frame to the next frame; omega_xFor the angular velocity, w, of the imaging platform along the x-axis of the target coordinate system_zThe angular velocity of the imaging platform along the direction of the z axis of the target coordinate system; alpha is alpha₀、θ₀、h₀Respectively representing the azimuth angle, the pitch angle and the height of the current frame image; alpha is alpha_i、θ_i、h_iRespectively representing the imaging azimuth, the pitch angle and the height corresponding to the next frame of image; phi is the longitudinal imaging field angle of the imager,

5. The method for recursive identification of sequence image targets based on platform inertial attitude parameters according to claim 2, wherein the step (3) is specifically as follows: target final pixel coordinate (u)_c(i)，v_c(i)) The calculation method comprises the following steps:

u_c(i)＝dA_u×[(A_v(i)-θ_i)×sinγ_i+A_u(i)×cosγ_i]

v_c(i)＝dA_v×[(A_v(i)-θ_i)×cosγ_i-A_u(i)×sinγ_i]

wherein,

dA_v＝ROW/φ

6. The method for recursive identification of sequence image targets based on platform inertial gesture parameters according to claim 1, 2, 3, 4, or 5, wherein the length T _ Row and the width T _ Col of the local search window in step (4) are respectively:

T_Row＝L_Row+2×biasX

T_Col＝L_Col+2×biasZ

wherein,

biasX＝maxω_x×Δt×dA_u

biasZ＝maxω_z×Δt×dA_v