RU2641630C2 - Method for eliminating geometric distortions of remote sensing images obtained by slot-type scanning sensor - Google Patents

Abstract

FIELD: physics.SUBSTANCE: method comprises obtaining a sequence of frames of the slot-type scanning sensor and obtaining a reference image of the underlying surface by the frame sensor. The data received by the slot-type scanning sensor is recorded on the memory device. The image formed from these data by their spatial scanning in the survey direction is compared with the georeferenced reference image and pairs of reference points corresponding to the same objects of the underlying surface are found.EFFECT: eliminating geometric distortions of remote sensing images received by a slot-type sensor, while simultaneously bringing them to a universal coordinate grid.6 cl, 3 dwg

Description

The invention relates to methods for acquiring images with slit scanning devices for remote sensing of the earth, for example, during aerial photography. An example of a slit scanning sensor is a pushbroom hyperspectrometer.

Due to the unpredictable movement of the carrier platform under the influence of external factors, such as wind, a geometric correction of the data obtained from the sensor is necessary to obtain an undistorted image of the underlying surface.

Correction of the movement of the carrier platform can be achieved both by physical methods using the so-called gyro-stabilized platforms (gyroplatforms) that compensate for the change in the orientation of the shooting devices during shooting, and by mathematical transformation of the pixel coordinates of the resulting image. The disadvantage of the first approach is the increase in the cost and weight and size parameters of shooting equipment, in addition, in some cases it does not provide full compensation for the non-linearity of the movement of the carrier platform.

The prior art method for eliminating distortions of remote sensing images associated with a complex path of the image sensor carrier (RF Patent for the invention No. 2411449, published 02.10.2011), including obtaining a sequence of images of a frame sensor and a scanning sensor and recording them on a storage device , the comparison of successive frames received by the frame sensor with each other and the construction of the trajectory of the aircraft. A device operating by this method includes a scanning sensor and a frame sensor, and shooting by these two devices is synchronized in time. The images obtained by the frame sensor can be compared with each other using the correlation method for the subsequent restoration of the frame displacement relative to each other and the restoration of the motion path, according to which the coordinates of the images of the scanning sensor are transformed and an undistorted image is formed.

The disadvantages of this method are the lack of projection of the image on a geographic grid, the need for accurate synchronization of two shooting devices in time and spatial orientation, as well as low resistance to high-frequency distortion.

The objective of the present invention is to provide a system for eliminating geometric distortions of remote sensing images obtained by a slit sensor, while bringing them to a universal coordinate grid that allows them to be used in GIS applications.

The technical result of the invention is to increase the accuracy of the geometric correction of the sounding data of the earth's surface by scanning survey systems, the formation of a geographically linked image based on the data of the scanning survey system.

The problem is solved, and this result is achieved by the fact that in the method of obtaining a remote sensing image, which includes obtaining a picture of a frame sensor and a sequence of pictures of a scanning sensor, a picture of the frame sensor is obtained independently and geographically tied, search for points with pronounced features for which it is possible to establish their exact correspondence to the same objects of the underlying surface (reference points) in both images, for frames of scanning sen the ora containing the reference points determine the shift transverse relative to the direction of flight, then for each of these frames the rotation angle is found relative to the corresponding reference point by minimizing the difference between the calculated motion of the carrier platform from uniform, and finally, for frames that do not contain reference points, offset and rotation interpolate. As an interpolation method, linear interpolation can be used. A high-resolution camera can be used as a frame sensor, and a pushbroom hyperspectrometer can be used as a scanning sensor.

The proposed method is illustrated by diagrams and photographs presented in the drawings.

FIG. 2a is a diagram of the construction of a coordinate transformation in the proposed method, where

1 - reference point of the frame n ₁ containing the reference point, before and after the coordinate transformation;

2 - reference point of the frame n ₂ containing the reference point, before and after the coordinate transformation;

3 - reference point of the frame n ₃ containing the reference point, before and after the coordinate transformation;

4 - the middle of the segment corresponding to the frame n ₁ containing the reference point, before and after the coordinate transformation;

5 - the middle of the segment corresponding to the frame m ₁₂ that does not contain a reference point. Its position is interpolated after the end of finding the positions of frames containing reference points;

6 - the middle of the segment corresponding to frame n ₂ containing the reference point, before and after the coordinate transformation;

7 - the middle of the segment corresponding to the frame m ₂₃ that does not contain a reference point. Its position is interpolated after the end of finding the positions of frames containing reference points;

8 - the middle of the segment corresponding to frame n ₃ containing the reference point, before and after the coordinate transformation;

FIG. 2b is a diagram illustrating the transformation of the position of frames relative to reference points during geometric correction of the image of the proposed method, where

A, B - the ends of the segment corresponding to the projection of the first frame containing the reference point, on a line in the scan of the image obtained by the scanning sensor;

C is the reference point on the first of the frames containing the reference point in the scan of the image obtained by the scanning sensor;

C ₁ - reference point on the second of the frames containing the reference point, in the scan of the image obtained by the scanning sensor;

D, E - the ends of the segment corresponding to the projection of the first frame containing the reference point on the reference image;

F is the reference point in the reference image corresponding to the reference point in the first of the frames containing the reference point;

F ₁ is the reference point in the reference image corresponding to the reference point in the second of the frames containing the reference point;

FIG. 2c is a diagram illustrating the determination of the rotation angles of the projections of frames in the coordinate system of the reference image for two frames containing reference points, where

A ₁ , B ₁ - the ends of the segment corresponding to the projection of the first frame containing the reference point;

M _1, N ₁ - the ends of the segment corresponding to the projection of the second frame containing the reference point;

A is the reference point belonging to the first frame;

N is the reference point belonging to the second frame;

In - the base of the perpendicular dropped from point N to the segment A ₁ B ₁ ;

M is the base of the perpendicular dropped from point A to the segment M ₁ N ₁ ;

α, β are auxiliary angles calculated from the width of the segments and the distance between the reference points and necessary to construct the first approximation of the angle of rotation of the projections of the frames of the scanning sensor in the coordinate system of the reference image.

FIG. 2d is a diagram illustrating the optimization of the relative position of the projections of the frames of the scanning sensor on the coordinate system of the reference image, where

α ₁ , α ₂ , α ₃ , α ₄ , α ₅ , α ₆ , α ₇ , α ₈ are the internal angles of the quadrangles formed by the found projections of the frames on the reference image and the lines pairwise connecting the ends of the segments corresponding to these frames.

FIG. 3 - on the left - the image obtained by the scanning hyperspectral sensor, after correction by the proposed method and compared with the reference image of the frame sensor. Images were obtained in May 2014 in the vicinity of the city of Plavsk, Tula region.

To perform geometric correction, a geographically linked image of a high spatial resolution frame sensor is used, which can be a space-based panchromatic camera, for which the task of geo-referencing the image is much simpler than that for the scanning sensor under consideration with a complex path of the carrier platform. A pushbroom hyperspectrometer mounted on an aircraft carrier can act as a scanning sensor. To obtain a geometrically corrected and geographically referenced image, it is necessary to calculate the coordinates of each pixel of the attached image in the coordinate system of the reference image, which is considered undistorted. In the proposed method, this coordinate transformation is constructed based on information about the shooting process and reference points that can be set by the data processor operator.

To simplify the algorithm, the following assumptions are used regarding the shooting process and the source data:

1. The relief does not introduce significant distortion into the picture.

2. The territory does not contain sharp elevation changes, and the flight altitude of the platform changes smoothly.

3. Roll and pitch angles do not introduce significant distortion.

4. Each frame is shot at a time for all pixels.

5. Anchor points are set with the greatest accuracy that allows you to perform image quality.

Based on the assumptions made, each frame of the scanning sensor is projected into a straight line segment in the reference image. Each segment is defined by 4 numerical parameters (x _r , y _r , l, a): two coordinates of the center of the segment in the coordinate system of the reference image (x _r , y _r ), the length of the segment l and the rotation angle relative to the zero position a (as the zero position a horizontal segment is taken, the platform velocity vector is directed upwards).

;

для 17 и 42 кадра из подпоследовательности номеров, лежащих между вторым и третьим кадрами, содержащими опорные точки соответственно.Each pair of control points is set by their coordinates on the reference image (x _r , y _r ) and the attached image (x _t , y _t ). The y _t coordinate is the serial number of the frame received by the scanning sensor and is interpreted as an integer. The ratio of the distance from the left edge of the line corresponding to the frame to the reference point on the attached image to the length of this line is denoted by x _a (the value 0 corresponds to the point on the left edge of the image, 1 - on the right edge). Each datum provides additional information about the location of the segment corresponding to line y _t: point of this segment, which cuts off its x _a stake, is fixed at a position (x _r, y _r). For such a segment, only two parameters (l, a) remain unknown, and the parameters (x _r , y _r ) can be calculated on their basis. If there are two pairs of control points with the same y _t, based on them, you can immediately calculate the exact position of the corresponding segment. Specifying more than two pairs of control points with the same y _{t is} not allowed. We denote the set of numbers of all frames containing control points by N = {n _i }, where n _i is the sequence number of the frame containing the ith control point during shooting. We also take the set of frame numbers that do not contain reference points for {A / N} = {m _{i, i + i} }, where A is the set of all frame numbers, ai denotes an earlier frame during the shooting compared to the frame containing the reference point (if there is no such frame, i = 0). The numbering of frames that do not contain reference points in the subsequence {m _{i, i + 1} } for fixed i will be carried out using superscripts. So, the number of the first of the frames containing the reference point in the accepted notation will be written as n ₁ ; a frame that does not contain a reference point and is located between the second and third frames containing reference points will be written as m ₂₃ ; when considering several such frames, the notation may look like

;

for frames 17 and 42 from a subsequence of numbers lying between the second and third frames containing reference points, respectively.

The algorithm works as follows. The data obtained by the scanning sensor is deployed into a rectangular image that would correspond to a surface image with uniform and rectilinear motion of the carrier platform, as shown in FIG. one.

Next, on the reference image and the resulting scan find pairs of reference points corresponding to the same objects of the underlying surface. This search can be performed by the data processor operator. After finding the pairs of reference points, the identification of the frames in which the reference points are located. The list of frames is reordered according to the sequence of shooting.

The next step is the detection of the width of the segments l. For this, pairs of found frames are sequentially examined and the data of the corresponding pairs of reference points (C, F) and (C _1, F ₁ ) (Fig. 2b) with coordinates (x _t1 , y _t1 , x _r1 , y _r1 ) and (x _t2 , y _t2 , x _r2 , y _r2 ). Index 1 corresponds to the frame shot earlier, and 2 - to the frame shot later; the index t denotes the attached snapshot, r is the reference snapshot. When defining reference points by the data processor, the pair is excluded from consideration if the operator has set a fixed width for any of the points. In other cases, the pair is used to calculate the width of the segments if the condition y _t2 -y _t1 <p _dw is fulfilled, where p _dw is the algorithm parameter, which is expressed in the number of pixels of the image and is selected experimentally (for example, the value p _dw = 20 can be used). In other words, the segments should be close enough to each other in shooting time. Denote the width of the attached image in pixels as w _t , and the height (number of frames) as h _t . In the vicinity of the pair of frames under consideration, it is assumed that there is no distortion, therefore, it is possible to calculate the width of the image of the attached image on the reference image based on the width of the attached image and the known distances between the reference points:

The calculated width is assigned to both segments involved in the calculation.

Further, the widths of the segments are calculated for the remaining segments in which there are reference points. If a segment is between two segments with a calculated width, then its width is calculated by linear interpolation between these segments (the closest segment is used on each side), i.e. a linear dependence of the width of the segment on the line (frame) number is assumed. If all the segments to which the width was assigned at the previous stage are located only on one side of the considered segment (for example, the segment is at the beginning or end of the image), then the width of the nearest suitable segment is used for it. Thus, each segment containing the reference point receives a value of width w _r .

Далее,

Поворот

относительно нулевого положения (вектора (1, 0)) вычисляется как

а финальный угол поворота вектора

равен γ=α+β. Для случая A₁A>M₁N можно провести аналогичные вычисления. В этом случае α и β можно вычислить по тем же формулам, а итоговый угол поворота равен γ=α+π-β.In the next step, preferred rotation angles for each pair of segments are calculated. The calculations are made in the coordinate system of the reference image. To simplify the calculations, the difference in the width of the segments at this step can be neglected, the width of the first segment from the pair is used (in the next step, the angles will be optimized taking into account the differences in width). Each segment can rotate around its reference point. It is necessary to find the angle at which two segments form a rectangle. We denote the segments in this position as A ₁ B ₁ and M ₁ N ₁ , and their reference points A ∈ A ₁ B ₁ and N ∈ M ₁ N ₁ (Fig. 2c). A ₁ B ₁ N ₁ M ₁ is a rectangle. Without loss of generality, we put A ₁ A≤M ₁ N. Drop the perpendiculars AM on M ₁ N ₁ and NB on A ₁ B ₁ , thus constructing a rectangle ABNM.

Further,

Turn

relative to the zero position (vector (1, 0)) is calculated as

and the final angle of rotation of the vector

is equal to γ = α + β. For the case A ₁ A> M ₁ N, similar calculations can be performed. In this case, α and β can be calculated using the same formulas, and the resulting rotation angle is γ = α + π-β.

For each segment, except for the segments with the first and last reference point, two angles are calculated - one for each of the adjacent segments. The initial rotation of such a segment is calculated as the weighted average between these two angles, and the weight of the term is inversely proportional to the number of frames between the segments of the corresponding pair:

Thus, the closer the adjacent segment to the segment under consideration, the more it affects its initial angle of rotation.

The obtained angle of rotation of the segments is calculated without taking into account the difference in the length of the segments and the position of other segments. Next, iterative optimization of the rotation angles is performed taking into account the lengths and relative positions of the segments. At each iteration, for each segment, two alternative positions are considered, rotated relative to the current one by Δγ clockwise and counterclockwise (the value Δγ = 0.3 ° can be used). The quadrangles formed by this segment, adjacent segments and the sides of the image are considered (Fig. 2d, the considered segment is indicated by a dotted line). Of the two positions, one is selected that provides the smallest value of the optimization metric, for which the following value can be used:

those. the one at which the quadrangles approach the shape of the rectangle, which corresponds to the ideal rectilinear movement between the anchor points. At the end of the iteration, all segments rotate in accordance with the selected position. To determine the stabilization of the system, the total (with a sign) angle of rotation of all segments for the last k iterations is analyzed (k can be taken equal to 10). If it is less than 2Δγ, then the system has stabilized and iteration can be stopped. The lack of stabilization of the system for a reasonable number of iterations indicates an error in the placement of control points or an erroneous determination of the width of the segments, which lead to a disruption in the orientation of the quadrangles by which the metric is considered.

The resulting rotations of segments containing reference points are considered final. Based on these rotations and the positions of the control points, the coordinates of the centers of the segments are calculated. Then, using the linear interpolation, the coordinates of the centers, the lengths and angles of rotation of the segments on which there are no reference points are calculated. In the absence of segments with anchor points on both sides of the segment in question (i.e., if the segment is located before the first anchor point or after the last), the angle and width of the nearest segment with the anchor point are used, and the center position is calculated by linear interpolation over the two nearest segments with anchor points, despite the fact that they are located on one side of the segment. Such a calculation agrees with the assumption that the movement of the platform is straightforward in the areas of the image that do not contain reference points.

This is illustrated in FIG. 2a.

Finally, a geometrically corrected image is constructed from the obtained coordinates, which is also geographically attached due to the fact that the new coordinates of the image pixels are specified in the coordinate system of the reference image, which is geographically attached.

The proposed method was used to analyze the condition of crops near the city of Plavsk, Tula region based on hyperspectral remote sensing data (the correction result is shown in Fig. 3), which indicates its industrial applicability.

Claims (6)

1. A method for eliminating geometric distortions of remote sensing images obtained by a slit scanning sensor, which includes obtaining a sequence of frames of a scanning sensor and recording it on a storage device, a scan in the order of obtaining frames with the formation of a geometrically corrected image, characterized in that in the image thus obtained and reference image, independently obtained by the frame sensor, find pairs of reference points corresponding to the same objects in the images, then for each frame containing the reference point, determine its transverse position relative to this reference point, for all such frames find their rotation relative to their corresponding reference points by minimizing the unevenness of the calculated motion of the carrier platform between pairs of frames containing reference points, for frames that do not contain reference points, their position is interpolated and a true image of the probed underlying surface is formed. 2. The method according to p. 1, characterized in that a pushbroom hyperspectrometer is used as a slit scanning sensor. 3. The method according to p. 1, characterized in that the reference image is a panchromatic image of high spatial resolution. 4. The method according to p. 1, characterized in that the pairs of reference points are manually. 5. The method according to p. 1, characterized in that the sum of the deviation modules of each of the inner corners of the quadrangles formed by pairs of consecutive frames containing reference points and the segments connecting their ends from 90 ° serves as a criterion for the unevenness of the calculated motion of the carrier platform . 6. The method according to p. 1, characterized in that the interpolation of the position of the frames that do not contain reference points is linear.

Images