JPS62284214A - Attitude decision system of moving image sensor platform, etc. - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] 3. Detailed Description of the Invention [Field of Industrial Application] In a broad sense, the present invention relates to a platform that is transported by air or transported in outer space, such as an image sensor attached to a spacecraft or an aircraft. The present invention relates to a system for detecting changes in posture, and particularly relates to a system for detecting changes in posture using terrain imaging technology.

BACKGROUND OF THE INVENTION Multi-vector sensitive image scanning devices are commonly mounted on aircraft and satellites, and their purpose is to record images of the Earth's topography as they pass over the Earth's surface. The image is recorded as data, and the data consists of a series of scan lines. As the aircraft or spacecraft moves in its orbit, the relative direction of the scan line to the bar scan line changes.

The attitude of an aircraft or spacecraft is subject to perturbations such as roll, pitch, yaw, glide, and speed changes that cause irregular geometric distortions in images of the terrain. In aircraft, perturbations caused by wind perturbations cause air density changes that can lead to inadvertent and inadvertent changes in course.

Although perturbations related to artificial satellites are minimal,
This is also a buffet in the atmosphere (in low-altitude orbit), orbit I,
II control and as a result of changes in the center of inertia of the artificial satellite (i.e. changes in fuel consumption, antenna or sun position direction).

[Problems to be Solved by the Invention] In order to make so-called raw image data geometrically correct, ii! By searching on the map for the exact location of the missing picture element (in the middle of the image) of the i-image data, it is possible to confirm the ground control point and picture element on the map for the 2.3 scan line of the image data. can. The entire image can then be mathematically warped and renumbered to better fit all ground control points. However, this method is too expensive, especially in terms of time and the effort required to manually correlate the recorded image data to the location of ground control points, and corrections depend solely on the accuracy and density of the ground controls. are doing.

One solution to this problem involves horizon finding devices, star indicators, and inertial navigation systems aboard spacecraft or aircraft. These pose sensors sense and record pose change data necessary for partial geometric correction of image data.

The data cannot be said to be optimal due to at least three missing points. Firstly, the above-mentioned devices use a non-data reference to the original recorded terrain; secondly, the same position as the base or first sensor is used; and thirdly, its lff
This is because there are limitations in terms of application due to power consumption.

As mentioned above, there is a need for a lightweight, more efficient, and less expensive system for sensing and recording attitude changes of platforms, such as image sensor platforms that can be used in aircraft and satellite topographic image sensors. There is.

[Means for Solving the Problems] According to the present invention, a system for determining attitude changes of a terrain imaging Hincer platform transported by air or in space comprises a second sensor...preferably a two-dimensional golden color. It has an image sensor...and a digital image correlation device used to compare continuous, overlapping, instantaneously obtained two-dimensional border topography data.

The second sensor produces a series of main and slave images, each image becoming the main image when the next slave is created. Digital image correlation processing is performed to avoid matching multiple (preferably five) picture elements of the main image with points at the same geographic location on the minor image. This correlation! i8]Ill choose multiple (preferably 5) spatially distributed subdivisions of the slave pixel array and then compare them to the main image to find the achromatic scale correlation coefficient for each subdivision. This is done by moving the array. The position with the highest correlation coefficient is chosen as the true position of the slave segment corresponding to the main image.

The vertical and horizontal coordinates of the correlated picture elements of the slave are recorded, and the relative orientation of the slave corresponding to the main image can be found given the condition of using the same plane in the photolateral method. The relative orientation of the master and slave only determines the new attitude of the sensor platform - i.e. the change in attitude of the platform from point to point in time at which the main image and slave are recorded. Become. Is the data obtained by the two-dimensional image sensor stored for later use?
Alternatively, it is processed in real time to obtain information that defines and determines changes in the sensor platform and to modify the geometry of the terrain image data recorded by the first sensor.

As described above, the first object of the present invention is to provide a particularly simple, accurate and
It is an object of the present invention to provide a cost effective and reliable multispectral sensitive sensor for terrain imaging and a system for sensing changes in orientation of airborne or spaceborne platforms.

It is also an object of the present invention, as mentioned above, to provide such a system in which the attitude change of the sensor platform is determined using a mathematical method that can be implemented by a computer.

It is a further object of the present invention, as described above, to provide a system that can be used to correct perturbations in terrain data, including remotely sensed imagery, on a nearly real-time or off-line basis.

One of the objects of the invention, as mentioned above, is to provide a system capable of providing information relating to altitude and speed changes of any type of airlift, or of any orbiting platform.

One of the objects of the invention, as mentioned above, is to provide a system which is particularly lightweight and compact, and which can also be used for conventional airborne or spaceborne sensors.

The above objects, as well as other objects and uses of the invention, will be set forth in the following description.

C Embodiment J As can be seen from FIG. 1, the present invention broadly relates to changes in the attitude of a sensor platform mounted on a moving vehicle such as an artificial satellite 10 or an aircraft 12 flying along a selected route 13. This relates to a system that senses and measures g1. Changes in platform position and attitude manifest as changes in the orientation of the sensor along the terrain it is imaging, causing irregular geometric distortions in the imaging terrain data.

For example, the platform is equipped with a full color or multispectral sensitive scanning device (t-sensor) that operates to view the terrain beneath the platform and generate electrical signals. The signals can be processed by a computer to create an image of the terrain beneath the route 13 as the platform passes over the surface of the Earth. The image sensor is
l12i111 passing over the Earth's surface along sensing zone (swath bus) 16! Box 14 b. For this purpose, the image sensor can be considered as a first sensor of the optical-mechanical or more modern type with a one-dimensional linear array. This is generally known as “butsu plume” (
This is also known as a push broom with a long handle or a multilinear arrangement.

The latter type of full color representation is used where illustration is required. In either case, the image sensor has a photosensitive detector fitted to a common focal plane, along with other optical equipment and various types of electrical components. The direction and attitude of the focal plane, the optical aiming, and the sensor altitude above the ground determine the range of observations on the Earth's surface.

As shown in Figure 2, the simplified terrain sensor is
It has a one-dimensional or linear array sensor (first sensor) 20 in which a linear strip 18 on the terrain side is transformed by a lens 24. The repeated scanning of a continuous swath over the Earth's terrain by the linear array sensor (first sensor) 20 and the forward motion of the sensor platform essentially create a composite map of the Earth's surface within the observation range of the sensor. This results in a collection of image data scan lines that can be processed by a computer.

Ideally, the sensor platform attached to the artificial satellite 10 or aircraft 12 is stable at a constant speed, attitude, or high altitude while the earth's surface is being scanned by the topographic image sensor (first sensor) 20. Or continue moving along the straight route 13. This allows all successive image scan lines to be recorded in perfect geometric relationship to previously recorded scan lines. However, in reality, aircraft 1
2. Irregular changes in attitude, altitude and speed are buffets,
Caused by changes in air density or inadvertent route changes.

In the case of the artificial satellite 10, there are effects of buffets in the atmosphere, orbit control maneuvers, fuel consumption, antennas, changes in the center of gravity of the artificial satellite due to changes in the direction of the sun's position, etc. Such a change in the platform causes an accompanying change in the attitude of the terrain image sensor (first sensor) 20, causing a geometric error in the image data. It is very difficult to remove geometric errors in the image data by continuous image processing without sufficient and in-depth knowledge of the frequency and serrations of the perturbations imposed on the first sensor 20. The present invention seeks to provide a system for sensing attitude, altitude, and velocity changes of a terrain image sensor platform. There are two images of the platform's posture change...
・Recognition of the fact that detection and measurement can be done by comparing two two-dimensional overlapping topographical images obtained instantaneously from two points along the platform's route using the second image sensor. This is a matter included in the present invention.

In accordance with the present invention, as shown in FIG. 2, the second terrain imager or second sensor 22 is mounted on the same platform as the first sensor 20 and located at a common focal plane to the extent cost permits. The photosensitive member has a two-dimensional array attached thereto. The second sensor 22 is mounted such that its image plane is parallel to, or preferably coplanar with, the image plane of the first sensor 20 in the linear array.

Suitable electrical equipment (not shown) of a conventional nature is installed to electrically scan the second sensor 22 and provide a digital representation of all or a portion of the terrain scanned by the first sensor 20. An image raster (i.e. all image data within one frame is obtained from the same observation point along the route, i.e. instantaneously) is obtained.

The latter image data is used as a reference for detecting and measuring changes in attitude, altitude, and speed. Referring to the conceptually depicted FIG.
This overlaps the scanning area of the sensor 20. Although this may be advantageous from a systemistic sensing perspective, it is not a necessary condition as long as the second sensor 22 records overlapping two-dimensional images of the reference terrain.

Turning now to FIG. 3, it can be seen that the second sensor 22 is for recording a series of overlapping images relative to the scan area 26 on the ground. Each image, or each scan area 26, is a "main" image, and subsequent images have "slave" images to the main image. The time interval for recording successive images of the scan area 26 depends on the speed of the sensor platform, the total observation range of the sensor, the attitude and speed of the sensor platform, and the nominal degree of altitude change;
It depends on the spatial resolution of the components of the second sensor 22, which has a two-dimensional array, and the spatial resolution of the components of the first sensor 20, which has a linear array. The scan area 26 is depicted as an aircraft, and its location and size will change with changes in attitude, speed, and altitude of the aircraft 12 to which the sensor platform is attached. For example, the scan area 26a corresponds to conditions where the aircraft 12 to which the sensor platform is attached is assumed to be on a normal course and at a constant altitude and speed.

Scan area 26b represents the area of the Earth that is imaged when the aircraft 12 to which the sensor platform is attached rolls.

Scan area 26d represents the image obtained once the aircraft 12 on which the sensor platform is mounted has ascended above the altitude at which the previous image was received.

Changes in the image recorded by the topographic image sensor (first sensor) 20 are illustrated in FIG.

In any case, the changes in the image recorded by the first sensor 20 of the linear array are directly related to the changes in the image recorded by the second sensor 22.

This is because dual use 1. This is because the second sensors 20, 22 have fixed focal planes in relation to each other, so that they are similarly perturbed by external forces.

Next, the overall system will be shown and a more detailed description will be provided based on the accompanying drawings. The cause of the spatial deviation between the main image and the subsequent original image can be predicted from the speed and altitude of the aircraft 12 on which the sensor platform is mounted, and the image raster, i.e. the digital data representing the main and original images, respectively. It can be applied as a correction amount to the set of The vengeance, Lord.
The original image is registered and at least three and preferably five picture elements from the main image are correlated with picture elements corresponding to those of the main image within three to five sections from the original image.

There is no change in speed or attitude of the sensor platform (compared to expected nominal values) or attitude change (swing, roll, or pitch) in that part of the sensor platform's trajectory. A perfect correlation process is performed. If it is not perfect, a series of geometric corrections are applied to the segment of the original image and the main ii! The correlation process with iWI is repeated. The set of correction amounts applied to the sections of the original image is appropriately correlated with the main image,
Expected position (i.e. speed and altitude) and 1. It characterizes the geometrical changes from the attitude of the second sensor 20.22. In other words, three or more picture elements of the original image with the maximum correlation coefficient between the main and subordinate images are detected by the second sensor 22.
The direction of the two-dimensional array is determined, and therefore the change in position and orientation of the first sensor 20 during the time that the main/original image is recorded is determined. It is thus possible to determine relative temporal changes in the position and attitude of the sensor platform with respect to the main image data previously collected by the attitude data acquisition image sensor over a reference area on the ground.

1st. The absolute altitude of the second sensor 20.22 can be determined most appropriately if the reference area is a well-known location on the ground. However, absolute altitude determination is somewhat more practical for satellite sensor platforms that pass over the same Earth in hours or days than for aircraft sensor platforms. If the terrestrial reference area is an unfamiliar location, changes in sensor platform attitude or change from the expected position will be compared to the previously recorded attitude or position of the sensor platform over the reference area. (Second sensor) 22
Detected by This is because the first sensor 20 acquires at least two scan lines of the images recorded by changes in altitude, velocity, and attitude of the terrain image sensor (first sensor) 20 that occur within the time period. is sufficient.

Turn now to FIG. 4, which shows overlapping master and slave images represented by the symbols rAJ and rBJ, respectively. A plurality of ground control points of the driven image are indicated by circles 28. The same reference point is point (reference point) 30 on the main image.
, with arrows correlating the circle 28 and its corresponding reference point 30. As shown in FIG. 4, the master and slave images, designated A and B, respectively, are drawn on geometrically derived overlapping portions of the slave image. The overlap l is predetermined mathematically based on the time elapsed while recording the master and slave images and the expected speed and altitude of the aircraft 12 to which the sensor platform is mounted.

What is then required is to determine the geometrical orientation of the slave image relative to the main image, so that the coincident reference points 28,30 of the terrain are registered with each other. In other words, the reference points 28.30 on the ground are ``correlated and geometrically coincide. An inference can be made from the topographic points. These points show a high correlation with the same points recorded on the main image. This means that at least three, preferably five, spatially defeated criteria on the secondary image This is accomplished by choosing points that contain the points to be correlated, although it is known from simple rules of geometry that only three points are needed to determine a surface. The terrain cannot be predicted to be a simple plane, i.e. it is not possible to know the height of every point on it. Photogrammetry requires a minimum of 5 points that can be seen in both the driven image and the (correlated) main image in order to determine all the variables that solve the problem of mathematical direction with the corresponding driven image... It is necessary to rely on the relative direction principle of the method.For each of the five selected points, the coplanar condition of the photogrammetry method requires a ray that first connects the point on the ground to the camera lens, and then requires a ray that connects the camera lens to the same ground point along with the corresponding point on the film image and the corresponding point on the image on the second film, provided that both lie in one plane.

Working on the relative direction principle requires that at least five pairs of rays 'III (after using the coplanar condition) intersect in order for every pair of two rays to intersect. For details on the coplanar condition and the formula that determines it, please refer to the American Photometric Association's ``Manual of Photography''.
grammetry). (4th edition, see pages 55 onwards, edition 411944.1952, 1
96 (1, 1980, the contents of which are used here as a reference document.) Turning now to FIG. 6, the five sections 34 to 34 of the driven image
42 is selected in correlation with the reference point 37 on the main image. Each of sections 34-42 is depicted as a 9x9 array of picture elements. Correlation processing is performed to determine the correlation based on achromatic color school (brightness) between the reference point, that is, the picture element 37 of the main image, and each picture element of the sections *34 to 42 of the subordinate image facing thereto. The achromatic color scale intensity or brightness value is
It is a function of the integral average of the radiant flux of the ground point represented by the picture element. In fact, the driven image sections 34 to 42 are enlarged in the X and Y directions (FIG. 6), so that the X, Y (coordinates) of the driven image sections 34 to 42 and the corresponding reference point, that is, the picture element 37, are Correlation processing is performed between the two. This image data processing technique is sometimes used [neighborhood processing J (neighborhood processing J)
This can be seen in a technology called processinc+). The picture element of the driven image section 34-42 with the highest correlation coefficient is selected next and the pixel arrangement, that is, the X and Y coordinates within the section are recorded. The recorded coordinates of the pixel with the highest correlation coefficient in the driven image section 1 picture 34-42 determine the true coordinates of the slave section 34-42 and therefore the correct coordinates of all the driven images corresponding to the main image. Also determine the coordinates. Figure 5 shows reference points 28 and 30.
The driven image is geometrically oriented with respect to the main image by 2
A spatially registered slave image rBJ is created with respect to the dimensionally distorted main image &rAJ. - The above-described system has a first image data sensor mounted in a common plane with the first image data M'R sensor, that is, in a focal plane coplanar with it.
This can be accomplished by using one relatively large or five or more relatively small two-dimensional fixed array of photosensitive members. The array is connected to a special high-speed image sensor that performs image correlation processing in ``real-time'' (immediate response), or the image data obtained by the attitude data M'R image sensor can be sent to the image sensor for ``off-line'' processing. A synchronization signal generated by one sensor and, if necessary, a first image sensor.
The resulting image data can be recorded together with geometric modifications.

These processing devices are conventional in technology. Therefore, there is no need to describe the details here. However, such processing equipment is cost-effective and available at GeoGeo, Ann Arbor, Michigan.
A T OM (A
utoa+aticTopography Map
per ) software. The output of the processing unit may consist of coordinate positions of five or more slave image pixels correlated with the main image, or equations of the planes of the master and slave images. As sensor platform 12 moves along its path, the old slave image becomes the new primary image.

However, the Zenure Ryōkai plane can be traced back to the original main image.

As shown in Figure 7, the change in altitude of the aircraft If112, and therefore that of the sensor platform, may occur simply as a change in image size, or as the main and slave images remain parallel but not coplanar. It is detected as the distance between correlated picture elements on the main and sub-images in terms of state.

As shown in FIG. 8, if the master and slave images remain in the same plane, a change in only the velocity of the platform 12 (and therefore the image sensor as well) will be represented as a different linear transformation of the image plane. The sensor platform is shaking.

When pitching or rolling, the main and slave images are no longer coplanar or parallel. The non-coplanar state of the master and slave image planes due to simple changes in platform and sensor pitch is depicted in FIG.

Logic flowcharts for a typical computer program implementing the system described above are shown in FIGS. 10-13. Looking at Figure 10, the first step of the program, 4
4 has continuous coordinates along the route in order to obtain main and slave image data. Image correlation processing is 2
This is done within the overlapping region sections of the main and slave images to identify correlated and coincident points in a region of the imaged terrain. In a second step 46, the relative orientation of the minor image plane corresponding to the main image plane is determined using the principles of relative orientation of photogrammetry described above.

The principle requires a minimum of five correlation points located within the master and slave images. The nub-chin for step 44 is the first
1 and 12. The first step of the subroutine, step 48, is to apply the same correction G obtained from the previous master-follower image correlation process, an approximation based on the nominal platform speed as varied by the image sensor sampling interval. . The overlapping region may occur if a relatively large two-dimensional image array is used as an attitude sensor, or if five or more terrain image segments are used as five or more relatively small two-dimensional image arrays as described above. If obtained by array of regions, step 5
At 0, it is divided into at least five or more sections. Then, in step 52, a process of searching for two-dimensional correlations is performed on the head and slave image data sections. The step 52 of performing correlation processing is based on the computer-selected matching candidate region in the main image having the most contrasting part and/or points in one two-dimensional image in step 54 of FIG. It is executed by having . A similar and equivalent procedure for visual inspection would be to select groups of distinct picture elements. In step 56, if the matched candidate region in the main image does not have a correlation peak of suitable strength in the slave image, another candidate region is processed. Otherwise, control passes to step 52.

The flow routine for carrying out step 46 (FIG. 10) is shown in FIG. Step 60 consists of minimizing the square of the deviation from the ideal condition of coplanar master and slave images for five or more pairs of correlation points. A determination is made in step 62 whether the deviation is less than a limit value (determined by the geometric precision required for the image data obtained by the first image sensor); if the deviation is less than the limit value; If so, step 60 is repeated. Otherwise, the subroutine ends. What emerges from this process is to describe the position and orientation of the continuous focal plane or first image sensor and its platform. A set of mathematical formulas that describe the image data obtained by the first sensor, either in a relative sense to the images previously recorded on the "flying" part, or in an absolute sense to the topographic map. Allows precise geometric relationships. In the latter case, a region that can be specially identified as an image database on the ground is required, and it contains the map coordinates and altitude of the region, and the above-mentioned correlation 2. It can be used to orient the primary or secondary image to the terrain using the coplanar test method.

Having thus described the invention, it is understood by those skilled in the art that preferred embodiments have been chosen to describe the invention in a manner that does not depart from the spirit and scope of the invention for that art. Various modifications (drafts) and additions (drafts) to examples
may be created. Therefore, consideration should be given to possible protections contemplated herein which fairly bring the central claimed subject matter and all equivalents thereof within the scope of the invention.

[Brief explanation of the drawing]

The accompanying drawings constitute an essential part of this specification and should be read in conjunction with Figure III, and the numbers in the figures are used to indicate parts within the various figures. . Figure 1 is a schematic perspective view of the sensing band (swath bus) of a terrain imaging sensor carried by a moving platform (typically a satellite or aircraft), which introduces large geometric distortions into the terrain data. It shows a small platform perturbation that causes . FIG. 2 is a schematic perspective view showing the geometrical relationship between the image terrain sensor and the attitude sensor with respect to the scanned terrain. FIG. 3 is a schematic perspective view of successive overlapping master and slave images recorded by the altitude sensor. FIG. 4 is a diagrammatic illustration of a series of master and slave images, with each geographical point of the image indicated by a circle or dot. FIG. 5 is a schematic diagram of the master and slave images that have been correlated, with geographically coincident points between the two images indicated by matching circles and dots. FIG. 6 is a schematic diagram showing the outline of the main image, showing the divisions and arrangement of picture elements extracted from the subordinate image which are mathematically superimposed and moved. FIG. 7 is a front view of the main and slave images when the sensor platform changes only in altitude. FIG. 8 is a front view of the main and driven images when the sensor platform changes only in speed. FIG. 9 is a front view of the main and driven images when the change in the sensor home is a slope. Figures 10-13 are flowcharts for computer programs for implementing the system. □ In the figure, 10 is an artificial satellite, 12 is an aircraft, 13 is a flight route, 14 is an observation range, 16 is a sensing zone (swarth nox), 20 is a first sensor, 22 is a second sensor, 2
4 is a lens, 26 is a scanning area, 28.30.374. 34 to 42 are sections. Patent applicant: Geospectra Corporation Patent attorney: Nobuo Kinutani (1 other person)Figure
5 Figure 4 Flugura 10 Figure, 11

Claims (1)

[Claims] 1. Determine the attitude change of an airborne sensor that moves along a route at a certain altitude above the ground and records continuous two-dimensional images of the earth's surface along the route. A method having the following steps: A. creating a first data group representing pixels of a first two-dimensional image of the Earth's surface at a first point along the route; B. a second point on the Earth's surface at a second point along the route slightly spatially distant from the first point;
creating a second data group representing pixels of the second two-dimensional image, at least a portion of the two images overlapping each other; Here, the above-mentioned pixels have different strengths depending on their respective locations on the image, and the matching pixels of the first and second images in the overlapping part are mutually corrected by an amount according to the attitude change of the above-mentioned sensor. ing. C. Correlating a plurality of matching pixels of the first and second images in the overlapping portion. The relative positions of the pixels subjected to the correlation processing here represent changes in the attitude of the sensor. 2. The method according to claim 1, wherein step C comprises the following steps. Selecting at least three spatially separated N×M array of pixels within one of the two images. selecting at least three spatially separated reference picture elements (pixels) in another said image, each associated with said N×M array; Comparing the three reference pixels to each of the pixels in the associated N×M array. 3. The method according to claim 2, wherein step C comprises the following steps. recording the vertical and horizontal coordinates of each pixel in the N×M array that has the highest correlation with the reference pixel; Determining a plane passing through the pixels in the N×M array that has the highest correlation with the connected reference pixels. 4. The method of claim 3, further comprising the step of determining the pose of a plane corresponding to the plane of the first image. 5. The method of claim 1, wherein step C comprises the following steps: Selecting a plurality of spatially separated pixels within the first image. Foreshadowing the location of said selected pixel in said second image. 6. The step of foreshadowing the position of the selected pixel,
6. The method of claim 5, wherein the method is carried out by determining the location of selected pixels within a previously generated image of the Earth's surface. 7. The method of claim 1, including the step of installing a second sensor moving along the route and fixed relative to the airborne sensor such that the image planes of the sensors are parallel. 8. The method of claim 1, wherein steps A and B are performed by focusing the image of the Earth's surface onto a photosensitive object having a two-dimensional array. 9. The method according to claim 1, further comprising the step of registering the first and second images. 10. The method of claim 1, comprising the step of performing a geometric correction on said second data group.