CN103090868A - Method for preparing typical ground object reference picture by using multispectral image classification - Google Patents

Abstract

The invention discloses a method for preparing a typical feature reference map by using multispectral image classification, extracting the class of interest from the multispectral image as a landmark, and preparing a reference map containing the landmark for indirect target detection Positioning recognition, the method specifically includes: selecting the class of interest from the multi-spectral image, extracting its spectral-spatial texture feature, and classifying the multi-spectral image according to the extracted spectral-spatial texture feature; on the basis of the classification, extracting a complete sense interest class, and select the interest class used as a landmark according to the landmark selection criteria; according to the landmark material type, use the infrared radiation formula to calculate the infrared radiation amount of the landmark under the atmospheric parameter conditions; map the infrared radiation amount to the gray value , so as to prepare a reference image including the shape of the landmark, its grayscale, and the background grayscale. The method of the invention has high accuracy and automaticity, and can improve the matching recognition algorithm performance based on the reference image.

Description

A Method for Preparing Typical Object Reference Map Using Multispectral Image Classification

technical field

The invention belongs to the technical field of image recognition, and in particular relates to a method for preparing a reference map using multispectral image classification, which is used for preparing reference maps of typical ground objects and is suitable for target recognition in aircraft navigation and guidance.

Background technique

With the development of infrared imaging and guidance technology, the research on the characteristics of target background radiation and its infrared imaging features has been paid more and more attention by researchers at home and abroad, which is of great significance in the field of aircraft navigation and guidance research. Under different time phases, different climates, different lighting conditions, and long-distance imaging conditions, the optical images acquired by the imaging sensor present different characteristics and are complex and changeable, especially when the target features are not obvious enough or the target is blocked, using direct navigation Location identification methods are often unreliable. However, in a large-scale high-resolution image at a certain distance, there are often certain ground objects with significant characteristics in the surrounding area of the target, which are called typical ground objects, such as rivers, bridges, roads, and building groups. The size, geometric shape, visibility and radiation characteristics of such ground objects are used as ground markers (referred to as landmarks) to indirectly locate targets, which helps to solve the problem of target recognition in complex environments. Therefore, it is particularly important to prepare feature reference maps containing landmarks.

Guo Wenpu et al. proposed a method for preparing a visible light forward reference map for scene matching in the paper "Generation Method of Forward View Reference Map Based on 3D Visual GIS". For the image pair, the digital elevation model data is generated by a 3D reconstruction algorithm, the orthophoto images are superimposed, and finally the front-view reference image sequence is generated to match the front-view real-time map. However, this method prepares a target reference map, which cannot meet the needs of navigation and guidance when the target characteristics are not significant, and the target reference map generated does not consider the constraints on the gray scale of the target under actual imaging conditions. Chinese patent document 200910273308.2 discloses a scheme, which uses a digital surface model to simulate the flight scene on the preset route, determines the plane landmarks used for navigation on the route, and then makes a multi-viewpoint and multi-scale optical feature map of the landmarks to calculate the correlation of each landmark. Parameters, to prepare a reference graph with parameters. The reference image prepared by this method does not consider the gray value assignment of the landmarks during actual imaging, and is mainly prepared manually, lacking in automation. In addition, Liu Jing and others proposed a multi-feature forward-looking reference map generation method in the paper "Research on Generation of Forward-Looking Infrared Scene Matching Guidance Reference Map" (Microcomputer Information, Vol. 23, No. 30, 2007). The image is the data source. The geometric view of the target and the background is extracted and the temperature and radiation characteristics are calculated respectively. After synthesis and adding the effect of atmospheric effect, the forward-looking infrared scene matching multi-viewpoint and multi-scale reference map is obtained according to different viewpoints. Based on image segmentation and multi-image extraction, this method obtains the background and target geometric views respectively, and performs modeling on the target and Vega infrared simulation to generate a benchmark image. It has many input parameters, a large amount of calculation, and the accuracy of the calculation results is comparable to that of image segmentation, Methods such as object modeling are closely related, and the automaticity and real-time performance are insufficient.

The above method basically prepares feature reference maps containing landmarks/targets by offline manual participation. This kind of method has a certain degree of subjectivity and low efficiency. Empirical gray values are assigned to landmarks/targets without considering the actual The infrared radiation characteristics of landmarks/targets under imaging conditions, so the prepared reference image does not predict the infrared characteristics of landmarks/targets, which has a direct impact on the target matching and recognition algorithm. For the reference image containing the target, the navigation positioning problem when the target is apparently weak in long-distance imaging is not considered, and the prepared reference image cannot meet the precise navigation requirements when the target is far away.

Contents of the invention

The purpose of the present invention is to provide a reference map preparation method using multispectral image classification, which uses a variety of feature information to quickly classify multispectral images, extract relatively complete classes of interest, select some of them as landmarks, and Predict the infrared characteristics of landmarks to solve the problem of indirect positioning and recognition of targets in complex environments by preparing feature reference maps containing landmarks when the target features are not obvious in the navigation and guidance process of the aircraft. support.

A method for preparing a reference map using multispectral image classification to achieve the purpose of the present invention, comprising the following steps:

(1) Select the class of interest from the multispectral image, extract its spectral-spatial texture features, and classify the multispectral image according to the extracted spectral-spatial texture features;

(2) On the basis of classification, use the object shape, mathematical morphology method and object spatial relationship to optimize the interest class, extract a relatively complete interest class from it, and use the landmark selection criterion to select the interest class as a landmark kind;

(3) Calculate the radiation amount of the above-mentioned landmarks according to the infrared radiation formula according to the material characteristics and radiation characteristic parameters of the ground objects;

(4) Perform grayscale mapping on the radiation value, and prepare a reference map accordingly.

As an improvement of the present invention, the spectral-spatial texture feature is formed by splicing the spectral feature and the spatial texture feature of the class of interest after being extracted and normalized respectively.

As an improvement of the present invention, in extracting the spatial texture features of the class of interest, first calculate the rotation-invariant balanced local binary mode value of the central pixel on each image data cube on the multispectral image, according to the mode value and mode The mapping relationship of the number of occurrences, the value of the number of occurrences of the pattern is obtained, and used as the spatial texture feature.

As an improvement of the present invention, the infrared radiation formula is as follows:

L _d ＝(L _direct +L _thermal )*τ _path +L _b

In the formula, L _d is the radiation amount when imaging on the surface of the detector, L _direct is the solar radiation reflected by the object, L _thermal is the radiation of the object itself, L _b is the path radiation transmitted by the atmosphere, and τ _path is the average transmittance of the atmosphere.

As an improvement of the present invention, the mapping of the infrared radiation amount to the gray value specifically refers to mapping the maximum radiation amount and 0 to the gray value 255 and 0 respectively, and the intermediate value is linearly mapped, thereby normalizing to [0,255 ].

As an improvement of the present invention, the prepared reference image is a bottom-view reference image, which specifically includes: extracting the grayscale-mapped landmarks and superimposing them on the original-sized background image to prepare the bottom-view reference image.

As an improvement of the present invention, the front reference image can be obtained after the perspective transformation of the bottom view reference image.

As an improvement of the present invention, after the landmark is determined, the spatial position constraint relationship between the target and the landmark in the multispectral image can be calculated for target matching.

As an improvement of the present invention, before the class of interest is selected from the multispectral image and classified, the multispectral image to be processed may also be preprocessed to obtain an input image suitable for feature extraction.

As an improvement of the present invention, as an optional operation, the multispectral image is also preprocessed;

As an improvement of the present invention, in the present invention, the class of interest used as a plane landmark is determined within the field of view along the flight path, and the spatial position constraint relationship between the target and the landmark in the multispectral image is calculated.

The invention prepares a reference map including landmark infrared characteristic prediction through fast classification and infrared radiation calculation, and provides important navigation reference data for a navigation system based on matching recognition, so that the matching recognition performance based on the reference map is better. The method of the invention has high accuracy, high efficiency and automaticity, can solve the problem that the target cannot be directly recognized under the long-distance imaging condition, and provides support for the forward-looking target recognition based on the matching reference image.

Description of drawings

Fig. 1 is the overall flowchart of the embodiment method of the present invention;

Fig. 2 is a multispectral image including rivers, roads, and building groups according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of the spectral-spatial combination feature rich in discriminant extraction of the class of interest in the embodiment of the present invention;

Fig. 4 is the result figure that classifies rivers, roads, and building groups according to an embodiment of the present invention;

Fig. 5 is the result map that extracts rivers, roads, building groups as landmarks on the basis of Fig. 4;

FIG. 6 is a diagram of the relative relationship between landmarks and targets according to an embodiment of the present invention;

Fig. 7 is a schematic diagram of the sun angle during infrared radiation calculation according to an embodiment of the present invention;

Fig. 8 is a comparison diagram of temperature change and self-radiation amount of typical materials at different observation times in the embodiment of the present invention;

Fig. 9 is a bottom view reference diagram after infrared radiation calculation and grayscale mapping according to an embodiment of the present invention;

Fig. 10 is a spatial constraint relationship diagram between a landmark-shaped special point set and a target centroid according to an embodiment of the present invention;

FIG. 11 is a multi-scale and multi-viewpoint front view reference view after perspective transformation of FIG. 9 .

Detailed ways

In order to make the object, technical solution and effect of the present invention more clear, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. The following examples are only for explanatory purposes and do not constitute a limitation to the present invention. All modifications that do not depart from the spirit of the present invention belong to the protection scope of the present invention.

Under a certain imaging distance, flying height and suitable flying angle, the visibility of ground features such as rivers, roads, and building groups can be guaranteed, so the present invention is considered under the condition that these ground features are visible. The present invention will be further described in detail below by taking the multispectral image shown in FIG. 2 as an example.

As shown in Figure 1, the specific process for preparing the reference diagram in this embodiment is as follows:

(1) For the multispectral image, select the class of interest, extract the spectral-spatial texture features of the interested class, and use these spectral-spatial texture features for classification.

(1.1) Spectral-spatial texture feature extraction. As shown in Figure 2(a)-(c), in the high-resolution multispectral image, through the analysis of the surrounding objects in the scene, select the local area of interest on the route with certain significance, visibility and contrast Objects are the class of interest. For the selected class of interest, or a part of the class of interest as a training sample, its spectral-spatial texture features are extracted. specifically is,

First, on the input image, calculate the rotation-invariant balanced local binary pattern (LBP ^riu2 ) value of the central pixel on each image data cube by means of a sliding window of the central pixel, according to the mapping relationship between the pattern value and the number of pattern occurrences , to calculate the rotation-invariant balanced histogram feature (that is, the spatial texture feature) of the pixel, and repeat this process to calculate the rotation-invariant balanced histogram feature for all predefined training samples.

TS∈R^d。光谱特征的计算在本领域具有多种成熟的方法，本实施例中采用各种方法均可实现。Then, calculate the spectral features of the class of interest, and normalize them with their spatial texture features, and then concatenate them into a comprehensive feature vector, denoted as

TS ∈ R ^d . There are many mature methods in this field for the calculation of spectral features, and various methods can be used in this embodiment.

TS∈R^d。其中l,d,u分别表示训练样本个数，特征维数与测试样本个数。通常需要对高维的LBP^riu2特征进行约简，使得不同类别的特征之间更加可分，记新的特征维数为d'。Repeat this calculation process for the test image to obtain the test sample set, denoted as

TS ∈ R ^d . Among them, l, d, u represent the number of training samples, feature dimension and number of test samples respectively. It is usually necessary to reduce the high-dimensional LBP ^riu2 features to make the features of different categories more separable, and record the new feature dimension as d'.

Figure 3 shows a schematic diagram of the spectral-spatial features extracted in some typical information categories, including 3 spectral features and 10 compact spatial histogram features, from which it can be seen that the classifier introducing spatial features will improve the classification performance .

(1.2) Classification.

In this embodiment, kernel-based support vector data domain (SVDD) is used for classification, and of course other methods may also be used for classification, such as one-class support vector machine (OCSVM) classification.

First, the training samples of the class of interest obtained in step (1.1) are randomly divided into training subsets, cross-validation subsets and test subsets in a ratio of 5:1:4, which are used to train the kernel-based support vector data domain ( SVDD) model for fast classification of test images. Secondly, since one class of classification only needs to provide samples of the class of interest, the optimal kernel parameters (C, gamma) will be obtained by cross-validation based on the training subset and the test subset, and a final kernel SVDD model will be trained using these parameters. Finally, the kernel SVDD model is used to quickly classify the regions of interest in the image. The kernel SVDD model can accurately identify the interesting class and reject the abnormal class, and can perform precision analysis on the basis of real class data.

In order to verify the effectiveness of the above spatial structure feature extraction method, this embodiment simultaneously extracts five texture features (contrast, entropy, angular second-order moment) of data samples (training samples and test samples) based on gray level co-occurrence matrix (GLCM , inverse moment and correlation) for comparison. Figure 4(a)-(c) are the classification results of the feature extraction method in this embodiment on three experimental images respectively. It can be seen from FIG. 4 that the feature extraction method in this embodiment is effective in spatial classification, especially when the classes of interest are randomly distributed.

Before feature extraction and classification, as an optional step, the multispectral image to be processed can be preprocessed to obtain an input image suitable for feature extraction. For example, perform data width conversion (16-bit to 8-bit), data normalization, and dimensionality reduction.

(2) Feature extraction. On the basis of classification, based on the knowledge of object shape information, mathematical morphology processing and spatial relationship, the optimization of interest classes is carried out, and a relatively complete interest class is extracted.

Through the analysis of the interested classes in the scene such as rivers, roads, and building groups, it can be seen that they have different shape information, such as rivers are curved, roads are straight, and building groups are approximately rectangular, and they have different aspect ratios. , eccentricity; In terms of spatial geometry, two kinds of ground features will appear at the same time, such as buildings often adjacent to rivers, and roads often appear around airports. Therefore, the initial class of interest (target class) is extracted by comprehensively utilizing the shape information of ground objects and the knowledge of spatial relationship, and enhanced by mathematical morphology methods to obtain a more complete final class of interest. Figure 5(a)-(c) are the results of the rivers, roads, and building groups of interest extracted from the multispectral image with a field of view of 1156 m × 725 m.

(3) Landmark selection. According to the geometric size, shape, visibility, radiation characteristics and other criteria of the class of interest, one or more objects of interest are selected as landmarks.

Analyze the geometric size, shape, visibility, and reflected radiation characteristics of the extracted classes of interest, and screen multiple extracted objects of interest that meet the user's set values, using relatively uniform materials and significant optical imaging And one or more well-known features of interest are selected as landmarks. In this example, the rivers, roads, and building groups extracted above are used as landmarks. As shown in Figure 6(a)-(c), they are the relative relationship diagrams of river landmarks, roads, building group landmarks and targets, and the straight lines between the centroids of rivers, roads, building group landmarks and their respective target centroids The distances are 236 meters and 154 meters respectively, and the straight-line distances between the landmark centroids of the three building groups and the target centroids are 225 meters, 220 meters and 190 meters respectively, and the route directions are also given in the figure.

In order to use the reference image for subsequent matching and recognition, the spatial position relationship between the landmark and the target can be calculated first. The spatial position relationship between the landmark and the target includes the shape point of the landmark, the coordinates of the centroid of the target, and the relative position between the landmark and the target. The position of the target minus the position of the landmark. And write the spatial position relationship information of the landmark and the target into the parameter file bound to the reference map, so as to be called directly when matching. In this embodiment, the parameter file is a text file format, including:

Reference image resolution: R meters

Flight altitude: h meters

Flight entry angle: alpha degrees

Landmark visible pitch angle: (beta1, beta2), unit: degree

The distance range where the landmark appears: (D _i1 , D _i2 ), unit: meter

Landmark and target position deviation: (X _LT , Y _LT )

(4) Calculation of landmark infrared radiation. That is, through material mapping, the infrared radiation amount of the landmark under atmospheric parameter conditions is calculated by using the infrared radiation formula.

The radiation characteristics of typical materials such as water, soil, asphalt, etc. are mainly affected by the reflected solar radiation of the object, the radiation of the object itself, and the radiation of the atmospheric transmission path. The surface radiation intensity of an object is closely related to the surface temperature of the object, and its temperature varies with the sun angle, season and altitude. During the day, the radiation on the surface of an object mainly comes from radiation such as the sun and the atmosphere; at night, when the solar radiation is weakened, the radiation part of the object itself is more prominent, and the surface temperature of the object will also decrease accordingly. Figure 7 shows the comparison of temperature change and self-radiation of typical materials at different observation times.

In order to obtain the radiation amount of objects under actual imaging conditions, according to the infrared radiation mechanism, heat conduction and radiation mechanism, the infrared radiation formula is used to calculate the radiation amount of ground objects.

The infrared radiation calculation formula determined in the present embodiment is as follows:

L _d ＝(L _direct +L _thermal )*τ _path +L _b

In the formula, L _d is the radiation amount when imaging on the surface of the detector, L _direct is the solar radiation reflected by the object, which is equal to ρ E ₀ (CCF τ _s +τ _d ), where CCF is the cloud cover coefficient, and the weather is clear 1 when , otherwise 0, E ₀ is the solar constant, τ _s , τ _d are the corresponding atmospheric transmittance and scattering rate when the sun is directly irradiated and diffused radiation, and the calculation formula is:

τ _s =(1+0.034cos(2πn/365))·P ^m cosθ

${\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; sinh</span>}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; P</span>}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1.4</span>}\mathrm{<span\; class="notranslate">\; ln</span>}\mathrm{<span\; class="notranslate">\; P</span>}}{\mathrm{<span\; class="notranslate">\; cos</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\frac{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}$

in,

cosθ=cosβsinh+sinβcoshcosγ, $\cos \mathrm{\&gamma;}=\frac{\sinh \sin \mathrm{\&phi;}-\sin \mathrm{\&delta;}}{\cosh \cos \mathrm{\&phi;}},$ $\mathrm{\&delta;}=23.45\sin (360\&times;\frac{284+\mathrm{no}}{365})$

In the formula, h, θ, γ, δ, β, φ are respectively the solar altitude angle, solar incident angle, solar azimuth angle, solar declination angle, object inclination angle and local latitude, and the relationship between these angles is shown in Figure 8 Show. m and P are air quality and air transparency respectively. The air quality can be calculated by the following formula (the meaning of h in the formula is the same as above):

In the formula, P and m are atmospheric transparency and air quality, respectively, and the two are related. In this embodiment, m is preferably set to 2, and the atmospheric transparency P corresponding to the value of m can be obtained according to the solar observation data.

L _thermal is the radiation of the object itself, which is equal to εσT ⁴ , where ε, σ, and T are the emissivity of the object, the Stefan-Boltzmann constant and the absolute temperature of the object surface (in K).

L _b is the path radiation transmitted by the atmosphere, and the calculation complexity of this factor is large, but the influence is small.

通过查询预定义数据表、插值计算和迭代积分可以求出。τ _path is the average transmittance of the atmosphere, equal to

It can be obtained by querying a predefined data table, interpolation calculation and iterative integration.

Because the material of the object selected as the landmark is relatively uniform, and the landmark is much smaller than the background image. Therefore, when calculating the radiation amount of the object itself, the object is regarded as an isothermal body. It is also possible to model the object and divide it into multiple grid nodes and predict the temperature of each node before calculating its own radiation.

The amount of radiation calculated by the above formula is the result of the combined effect of all radiation after atmospheric attenuation and the spectral response of the detector. At the same time, it is calculated based on the material characteristics and radiation characteristics of ground objects, so the radiation amount can reflect the actual imaging effect.

(5) Grayscale mapping, that is, mapping the infrared radiation amount calculated in step (4) to a grayscale value.

In order to obtain the final infrared reference map, each infrared radiation amount should be gray-scale mapped, specifically: the maximum radiation amount and the minimum radiation amount are mapped to gray-scale values 255 and 0, respectively, and the intermediate values are linearly mapped, and the results are attributed to One to [0,255]. In addition, according to the obtained image gray value and radiation, the mapping relationship between the two can also be solved.

(6) Calculation of the spatial relationship between landmarks and targets. In the multi-spectral image, the feature library of the spatial constraint relationship between the landmark and the target to be recognized is established.

In this embodiment, as shown in Figure 9(a)-(c), the centroids of rivers, roads, and building groups, and the points with the largest curvature are respectively selected to construct shape-specific point sets of these landmarks. The three river landmarks from left to right in Figure 9(a) are the points of maximum curvature LM _1,1 , LM _1,2 and centroid LM _1,3 , and their coordinates (Y,X) are ( 365,435), (295,480), (363,640), and the centroid coordinate T ₁ of the target is (422,412). The three road landmarks from left to right in Figure 9(b) are the maximum curvature point LC _{1, 1} , the centroid LC _{1, 2} and the maximum curvature point LC _{1, 3} , and their coordinates (Y, X) They are (312,182), (254,354), (316,546), and the centroid coordinates T ₂ of the target are (103,386). Taking the average position of three river landmark points as LM, then LM=(LM _1,1 +LM _1,2 +LM _1,3 )/3, and its value is (341,518). In the same way, the average position LC of road landmarks can be obtained, and its value is (294,361). The centroids of the three building group landmarks from left to right in Fig. 9(c) are LB _1,1 , LB _1,2 and LB _1,3 respectively, and their coordinates (Y, X) are (455,442), (125,521) and (508,650).

为第i个地标形殊点的坐标，

为目标形心坐标；⑵如图10（b1）-（b3）所示，道路地标形殊点与目标形心之间的约束关系

$\mathrm{\&Delta;}{\mathrm{LC}}_1^j=\left[(Y_{T_2}-Y_{{\mathrm{LC}}_{1,i}})(X_{T_2}-X_{{\mathrm{LC}}_{1,j}})\right],$ j＝1,2,3，

为第j个地标形殊点的坐标，

为目标形心坐标。通过之前建立的地标与目标之间的相对位置关系以及在实时图中获得的地标位置值进行解算，将获得目标在前视图中的空间位置，从而实现对目标的间接定位。⑶如图10（c1）-（c3）所示，建筑物群地标形殊点与目标形心之间的约束关系

$\mathrm{\&Delta;}{\mathrm{LB}}_1^k=\left[(Y_{T_3}-Y_{{\mathrm{LB}}_{1,k}})(X_{T_3}-X_{{\mathrm{LB}}_{1,k}})\right],$ k＝1,2,3为三个建筑物群的顺序编号，

为第k个地标形心的坐标。同理，可获得该目标在前视图中的位置坐标。In this embodiment, the spatial geometric constraint relationship between each landmark and the target is shown in Figure 10, which includes: (1) As shown in Figure 10 (a1)-(a3), the distance between the special point of the landmark shape of the river and the centroid of the target constraint relationship $\mathrm{\&Delta;}{\mathrm{LM}}_1^i=\left[(Y_{T_1}-{\mathrm{<figure-callout\; id="1"\; label="Y\; LM"\; filenames="HDA00002695752600101.png"\; state="\{\{state\}\}">Y</figure-callout>}}_{{\mathrm{LM}}_{}})(x_{T_1}-{\mathrm{<figure-callout\; id="1"\; label="x\; LM"\; filenames="HDA00002695752600101.png"\; state="\{\{state\}\}">x</figure-callout>}}_{{\mathrm{LM}}_{}})\right],$ i=1,2,3,

is the coordinates of the i-th landmark point,

is the coordinates of the target centroid; (2) As shown in Figure 10 (b1)-(b3), the constraint relationship between the special point of the road landmark and the target centroid

$\mathrm{\&Delta;}{\mathrm{LC}}_1^j=\left[(Y_{T_2}-{\mathrm{<figure-callout\; id="1"\; label="Y\; LC"\; filenames="HDA00002695752600101.png"\; state="\{\{state\}\}">Y</figure-callout>}}_{{\mathrm{LC}}_{}})(x_{T_2}-{\mathrm{<figure-callout\; id="1"\; label="x\; LC"\; filenames="HDA00002695752600101.png"\; state="\{\{state\}\}">x</figure-callout>}}_{{\mathrm{LC}}_{}})\right],$ j=1,2,3,

is the coordinates of the jth landmark point,

is the target centroid coordinates. Through the calculation of the relative position relationship between the previously established landmarks and the target and the landmark position values obtained in the real-time image, the spatial position of the target in the front view will be obtained, thereby realizing the indirect positioning of the target. (3) As shown in Figure 10 (c1)-(c3), the constraint relationship between the landmark shape point of the building group and the target centroid

$\mathrm{\&Delta;}{\mathrm{LB}}_1^k=\left[(Y_{T_3}-{\mathrm{<figure-callout\; id="1"\; label="Y\; LB"\; filenames="HDA00002695752600101.png"\; state="\{\{state\}\}">Y</figure-callout>}}_{{\mathrm{LB}}_{}})(x_{T_3}-{\mathrm{<figure-callout\; id="1"\; label="x\; LB"\; filenames="HDA00002695752600101.png"\; state="\{\{state\}\}">x</figure-callout>}}_{{\mathrm{LB}}_{}})\right],$ k=1,2,3 are the sequential numbers of the three building groups,

is the coordinates of the centroid of the kth landmark. Similarly, the position coordinates of the target in the front view can be obtained.

In this embodiment, the parameters in the parameter file bound to the reference image corresponding to Figure 9(a) are actually:

Reference map resolution (m): 3

Flight altitude (meters): 4000

Flight entry angle (degrees): 180

Landmark visible pitch angle (degrees): 0~40

The distance range (meters) where the landmark appears: 0~10000

Landmark and target position deviation (meters): 59-228

(7) Preparation of landmark reference maps. According to the gray value obtained in step (5), an infrared reference image including the landmark shape, gray scale, and background gray scale can be prepared.

The landmarks after gray scale mapping are extracted and superimposed on the background image of the original size to prepare a reference image of the landmark bottom view, which mainly includes the shape and gray scale of the landmarks, as well as the gray scale of the background. The value assigned to the background gray level can be obtained similarly to the above calculation method, or the background can be extracted based on the real-time image. According to the knowledge of the radiation characteristics of the landmarks used as landmarks during the day and night, the daytime and nighttime down-view reference maps of the landmarks can be prepared. Figure 9 shows the reference image of the landmark downside after infrared radiation calculation and grayscale mapping.

Here the background uses simplification and is regarded as a simple object with 0 radiation. The background class can also be divided into regions, and the infrared radiation amount of each region of the background can be obtained through the above steps (1)-(2), (4)-(5).

Using the bottom reference image, a front reference image can also be prepared. The forward-looking reference image can be considered as a transformed reference image under the condition of simulating the imaging parameters of the aircraft by using the lower-view reference image. The aircraft track parameters include: flight altitude, imaging distance, azimuth and pitch angle, wherein the pitch angle can be obtained by dividing the flight altitude by the ratio of the imaging distance. Using the down-view reference map and the above-mentioned track parameters, the forward-view reference map can be obtained. According to the perspective transformation of the track parameters, the characteristic views of the landmarks at different scales and different viewpoints can be obtained. After the infrared radiation calculation, the forward-looking reference image and the infrared real-time image show the greatest similarity in certain characteristics, and have better Match performance.

In this embodiment, the specific method for performing perspective transformation on the down-looking infrared reference image is as follows:

Let p be the viewpoint of the sensor, h be the imaging height, α be the azimuth, θ be the pitch angle, φ be the longitudinal field of view of the sensor, is the lateral field of view of the sensor, and the coverage of the field of view is ROW×COL. T ₀ (x ₀ , y ₀ ) is the aiming point of the optical axis of the sensor, T ₁ (x ₁ , y ₁ ) is a certain imaging point, then the position of T ₀ in the front view (real-time image) is (COL/2,ROW /2). Assuming that the position of T ₁ in the front view is (T ₁ _ROW, T ₁ _COL), the process of calculating T ₁ _ROW and T ₁ _COL is as follows:

OT ₀ =h/tanθ

OM＝OT ₀ +(y ₁ -y ₀ )×cosα+(x ₁ -x ₀ )×sinα

tan(∠OMP)＝h/OM

T ₁ _ROW＝ROW/2+(∠OMP-θ)*ROW/φ

Among them, OT ₀ is the distance between the intersection point T ₀ of the optical axis pointing to the earth’s horizontal plane and the imager’s projection to point O on the earth, and point M is the intersection point of the projection of T ₁ to the longitudinal direction of the optical axis and the straight line OT ₀ . According to the position deviation between a point in the earth coordinate system and the aiming point of the optical axis, the position of the point in the front view can be determined. Therefore, the bottom-view reference image is transformed into the front-view reference image, which provides important reference data for matching recognition based on the front-view reference image.

Figure 11 shows the front view reference images obtained at different flight heights, different imaging distances and different viewpoints, where the perspective transformation parameters corresponding to Figure 11(a)-(b) are the flight height is 4km, the imaging distance They are 6km and 10km respectively, and the azimuth angle is 180°; the perspective transformation parameters corresponding to Fig. 11(c)-(d), (e)-(f) are flight altitude of 5km, imaging distances of 6km, 10km, azimuth The angle is 0°. The coordinate positions between each landmark and the target in Figure 11(a)-(f) can be obtained from the coordinate positions in the bottom view reference image and the parameters of the perspective transformation. The marks of the corresponding target and landmark-shaped special point coordinates are as follows:

T ₁ ′, LM _1,1 ′-LM _1,3 ′; T ₁ ″, LM _1,1 ″-LM _1,3 ″;

T ₂ ′, LC _1,1 ′-LC _1,3 ′; T ₂ ″, LC _1,1 ″-LC _1,3 ″;

T ₃ ', LB _1,1' -LB _1,3 '; T ₃ ", LB _1,1 "-LB _1,3 ".

For the prepared reference image, an error analysis based on matching identification can also be performed to verify the performance of the prepared reference image. For example, analyze the preparation of landmark-shaped special points and the size of the selected template, and make improvements to prepare a more accurate reference map.

Claims (9)

1. A method for preparing a reference map utilizing multispectral image classification, which extracts the class of interest from the multispectral image as a landmark, and prepares a reference map containing the landmark for matching to realize target positioning recognition, the The methods specifically include: selecting the object of interest from the multispectral image, extracting its spectral-spatial texture feature, and classifying the multispectral image according to the extracted spectral-spatial texture feature; Optimizing the features of interest according to the classification results, and selecting one or more of the optimized features of interest as landmarks; According to the material type of the landmark, determine the amount of infrared radiation of the landmark under atmospheric parameter conditions; The infrared radiation amount is mapped to a gray value, and a reference image including the background gray value, the landmark shape and its gray value can be prepared according to the gray value. 2. A kind of reference map preparation method utilizing multispectral image classification according to claim 1, is characterized in that, described spectral-spatial texture feature is that the spectral feature and the spatial texture feature of described interested class are respectively extracted And normalized splicing composition. 3. A kind of reference map preparation method utilizing multispectral image classification according to claim 2, is characterized in that, in the spatial texture feature of described extraction interest class, at first calculate center pixel on multispectral image Rotation-invariant balanced local binary mode value on an image data cube, according to the mapping relationship between the mode value and the mode occurrence times, the value of the mode occurrence times is obtained, and the rotation-invariant value of the pixel can be obtained according to this value The balanced histogram feature is the spatial texture feature. 4. A kind of reference map preparation method utilizing multispectral image classification according to any one of claims 1-3, characterized in that, the amount of infrared radiation of the landmark is calculated by the following infrared radiation formula: L _d ＝(L _direct +L _thermal )*τ _path +L _b In the formula, L _d is the radiation amount when imaging on the surface of the detector, L _direct is the solar radiation reflected by the object, L _thermal is the radiation of the object itself, L _b is the path radiation transmitted by the atmosphere, and τ _path is the average transmittance of the atmosphere. 5. A method for preparing a reference map utilizing multispectral image classification according to any one of claims 1-4, wherein mapping the infrared radiation amount to a gray value specifically refers to converting the maximum radiation Quantity and 0 are mapped to gray values 255 and 0, respectively, and the intermediate value is linearly mapped to normalize to [0,255]. 6. A method for preparing a reference map utilizing multispectral image classification according to any one of claims 1-5, wherein the prepared reference map is a bottom-view reference map, specifically: gray The landmarks after degree mapping are extracted and superimposed on the background image of the original size to prepare a bottom-view reference image. 7 . The method for preparing a reference map utilizing multispectral image classification according to claim 6 , wherein the front view reference map can be obtained after perspective transformation of the bottom view reference map. 8 . 8. A method for preparing a reference map using multispectral image classification according to any one of claims 1-7, characterized in that, after determining the landmarks, the distance between the target and the landmarks in the multispectral image can be calculated. The spatial position constraint relationship in is used for target matching. 9. A kind of reference map preparation method utilizing multispectral image classification according to any one of claims 1-8, characterized in that, before the class of interest is selected from the multispectral image and classified, further The multispectral image to be processed may be preprocessed to obtain an input image suitable for feature extraction.

Images