EP0095660A2 - Stereo-photogrammetrical recording and interpretation method - Google Patents

Abstract

The invention relates to a stereophotogrammetric recording and evaluation method and a corresponding evaluation device. The intention is to obtain the orientation data of a camera flying over a terrain as well as a digital terrain elevation model. Three sensor lines arranged transversely or obliquely to the flight direction and an associated optics are used. Continuous line-by-line scanning creates three image strips of the terrain, each taken from a different perspective. For this purpose, it is proposed to preferably prescribe image points distributed in the form of a mesh in the middle image strip, to determine the corresponding image points and the assigned line numbers by area correlation in the two other image strips, on the basis of the approximately known flight movements for each point, the approximate orientation parameters of the camera and by spatial forward section to approximately determine the terrain coordinates of the point, to establish ray intersection conditions for the three rays belonging to a point and to determine the most probable and final values of the orientation parameters and the point terrain coordinates using error equations and a compensation process.

Description

The invention relates to a stereophotogrammetric recording and evaluation method for obtaining the orientation data of a camera flying over a terrain and a digital terrain elevation model (DHM) using three sensor lines A, B, C arranged transversely or obliquely to the direction of flight and associated optics for continuous line by line Scanning of the area overflown and generation of three overlapping image strips A _S , B _S , taken from different perspectives. C _S.

It is known that with optoelectronic cameras in which three sensor lines A, B, C are assigned to an optical system (see patent application P 29 40 871.7-52) and their lines transverse to the direction of flight or (according to additional application P 30 43 577.9-52) are arranged at a certain angle to each other, three image strips A _S , B _S , C _S can be generated simultaneously. It can be extended as desired by orienting a new line image onto the line group that is already known in terms of its orientation. The generation or procurement of this line image association, which is assumed to be known, is still associated with difficulties, and the stepwise addition of a new line image in each case is not advantageous from the point of view of error theory.

It is therefore the object of the invention to provide a method of the type mentioned at the outset, with which the entire line image association and the recorded terrain surface can be reconstructed in a uniform, closed method without precondition with information that comes exclusively from this image recording itself.

• a) Unter der äußeren Orientierung einer Kamera versteht man ihre Positions-Koordinaten X_o, Y, Z₀ und ihre drei Neigungskomponenten ω,ϕ,X zum Zeitpunkt der Momentaufnahme. Grundsätzlich gelten diese Definitionen auch für die elektro-optische Zeilenkamera, wobei für jede Zeilen- Perioden-Nummer N ein aus den 6 Orientierungsparametern bestehender Orientierungssatz vorhanden ist. Die Zeilenperiodendauer ist elektronisch exakt eingestellt und liegt in der Größenordnung von ca. 1/100 sec. Jede Zeilenperiode N erzeugt synchron drei Bildzeilen der Bildstreifen A_S, B_S, C_s und infolge der exakt bekannten Zeilenperiodendauer ergibt sich für jede laufende Zeilen-Nummer N ein exakter Zeitpunkt. Den Flügbewegungen entsprechend verändern sich mit der laufenden Zeilen-Nummer N die Orientierungsparameter der Kamera, und es lassen sich aus der Zeilen-Nummer N bei annähernd bekannten Flugbewegungen die Orientierungsparameter der Kamera genähert bestimmen.

If we are talking about the reconstruction of the camera orientation along the flight path and the terrain, then the following terms, some of which are already known from stereo photogrammetry, are used and definitions are meant:

• a) The external orientation of a camera means its position coordinates X _o , Y, Z ₀ and its three inclination components ω, ϕ, X at the time of the snapshot. Basically, these definitions also apply to the electro-optical line scan camera, whereby for each line period number N there is an orientation set consisting of the 6 orientation parameters. The line period is set electronically exactly and is of the order of approx. 1/100 sec. Each line period N generates three image lines of the image strips A _S , B _S , C _s synchronously and as a result of the exactly known line period duration results for each running line number N an exact time. The orientation parameters of the camera change in accordance with the flight movements with the current line number N, and the orientation parameters of the camera can be determined approximately from the line number N for approximately known flight movements.

It is not necessary for the present task that the orientation parameters of each line period N are determined, since the change of the orientation parameters takes place more or less continuously and it is therefore sufficient to determine the 6 orientation parameters at certain intervals, ie orientation intervals along the flight path. These points are referred to below as orientation interval points. Intermediate orientation parameters could be determined as a function of the orientation parameters of the neighboring orientation interval points, for example by linear interpolation. The size of the orientation intervals depends on the "ripple" of the flight movements and the desired reconstruction accuracy.

b) The same applies to the terrain to be reconstructed, which is represented by a so-called "digital terrain elevation model" (DHM). It consists of regularly or irregularly arranged terrain points, whose plan coordinates X, Y and their height coordinates Z are to be determined. The point density to be selected depends on the "ripple" of the terrain and the level of accuracy with which the terrain is to be represented. In this case too, intervening terrain points can be determined by interpolation as required.

Each terrain point and thus also every DHM point to be selected in the course of the evaluation is flown over with the camera at three different times or current line periods N _A , N _B , N at three different locations (each with an orientation set consisting of 6 parameters ) on the three picture lines A, B, C and creates one. Image point to which the image coordinates x, y are assigned in the image plane (FIG. 1).

The object of the invention is to determine the orientation parameters X _o , Y _{o '} Z _o , ω, ϕ, X of the orientation interval points and the terrain coordinates X, only from these image coordinate pairs x, y and their assigned current line period numbers N. Determine Y, Z of the DHM points and generate the rectified floor plan of each individual pixel (orthophotos) and rectified stereo images (stereo partner).

This object is achieved by the measures mentioned in the characterizing part of patent claim 1.

Accordingly, in a first process a) three image strips are recorded synchronously with the optoelectronic line camera, a clock counter registering the current line periods N of the synchronized sensors A, B and C and the discrete pixel signals assigned to the lines preferably being stored digitally.

In a second process b), image points arranged in meshes in the image strip B _S generated by the middle sensor line B and corresponding to the DHM points are preferably selected with the computer according to certain criteria and by area correlation in the other two image strips A _S and C _s the corresponding pixels of the same DHM point were found (Fig. 2). In addition, the following should be noted: Each individual pixel of an image strip, for example A _S , is defined in each case by the line number N and a discrete sensor element (sensor pixel) lying on the sensor line, the brightness signal of which is registered. Since the position of the sensor line in the image plane of the camera and the position of each pixel in the sensor line is known, the x and y coordinates in the image plane can be calculated for each sensor pixel. The image strips are therefore composed of the current lines N in the direction of flight and the current pixels of a line or a sensor in the line direction and result in a grid-like pixel matrix with which the area correlation takes place. The pixel number of the sensor line in question clearly defines the image coordinates x, y, and the line number N defines the recording position of the camera for this time and the assigned orientation parameters. As a result of this process, a list is created, which for each DHM point is usually three (only two at the beginning and end of the stripe) points on sensor lines A, B or C and their coordinate pairs x, y and a respectively assigned line period number N _A , N _B , N _C includes.

In a third process c), based on the approximately known flight movements (for example, a uniform speed, constant flight direction and flight altitude and normal flight position ω = ϕ = X = ₀ ) are assumed for each DHM point with the line numbers or times N _A , N _B , N _C preliminary orientation parameter sets (each X, Y, Z ω, ϕ, X) are calculated and thus and with the assigned image coordinates x _{A '} y _A and x _B , y _B and x _C , y _C using the spatial Forward cut preliminary approximate DHM coordinates X, Y, Z calculated.

In a fourth process d), ray intersection conditions are established for each DHM point, using either the so-called coplanarity conditions or the collinearity equations. They contain the observed image coordinates x and y, the approximated orientation parameters for the assigned time N, which are represented as functions of the orientation parameters in the orientation interval points, and the approximate DHM coordinates. Error equations are established in a known manner based on mediating observations and the most likely and final values of the orientation parameters and the DHM points are determined in any local coordinate system and scale in a least squares adjustment process according to known methods. By introducing a few so-called "control points", this model can be inserted in a higher-level geodetic or geographic coordinate system and oriented "absolutely" in a known manner.

As an explanation, it should be noted that, as already described, three DHM points are generally assigned to each of the DHM points. Through this DHM point, the projection centers X _o , Y _o , Z _{o of} the camera in the three positions or times N _A , N _B and N _C and the corresponding pixels x, y on the sensor lines A, B, C are determined. The condition that the rays intersect at the DHM point can be mathematically the so-called coplanarity condition or with the help of the so-called collinearity equations.

Using the thus found DHM coordinates and the corresponding image coordinates in the image strip B _S can now mesh as all the pixels of the image strip B _S by known transformation methods to the correct distortion free Grundrißlage transform (Figure 3).

In the same way, all the pixels of the image strips A _S and B _{S can} be transformed so that distortion-free stereo partners are created. This is done in such a way that the DHM points P are projected onto the floor plan according to P _A or P _C by an oblique parallel projection (FIG. 3). By mesh-wise transforming all the pixels of the strips A _S or C _s into this floor plan, stereo images are generated without distortion. For the calculation of the transformation parameters, the mesh points in the image strips A _S , B _S , C _S and the points P _A , P _B , P _C assigned to them in the floor plan are used.

1 shows the recording process with a three-line camera. The terrain point (DHM point) P is mapped onto the sensor line A in the line period point N _A (image coordinates X _A , Y _A ), and onto the sensor lines B (x _B , y _B ) in the line period points N _B or N _C b ^{between C} (x _C , y _C ).

2 shows the three image strips A _S , B _S , C _S with the individual image lines and the DHM pixels selected in strip B _S and correlating in strips A _S and C _s .

3 shows a DHM point with its points P _A , P _{B '} P _C projected into the floor plan.

To carry out the evaluation method according to the invention, a stereo screen is used with a computer performing the area correlation couples on which runs parallel to the automatic correlation process the point selection in the image strip B _S and the point correlation in the image strip A _s or C _s and is visibly displayed to the operator. The operator thus has the possibility of following the correlation process and, in the event that there are difficulties, making corrections by interactive intervention and / or, if necessary, restarting the interrupted correlation process.

• a) den Korrelationsbeginn zu initiieren,
• b) Paßpunkte zu identifizieren und zu markieren,
• c) interpretierende Auswertung der Objekte vornehmen zu können.

This visually interactive intervention is also required to

• a) initiate the start of correlation,
• b) identify and mark control points,
• c) to be able to perform interpretive evaluation of the objects.

• a) durch Überlagerung der beiden Stereo-Teilbilder in komplementären Farben (z.B. rot und grün) auf einem Farbbildschirm und Betrachtung mit entsprechenden Filtergläsern und Generierung einer Meßmarke, die relativ zum Bild in beiden Dimensionen (x, y) bewegt werden kann und wobei die Teilbilder gegeneinander in beiden Richtungen (x, y) verschiebbar sind.
• b) durch Darstellung der beiden Stereo-Teilbilder auf zwei Hälften eines Bildschirmes oder zwei separaten Bildschirmen und Generierung von zwei den Teilbildern zugeordneten Meßmarken, die gemeinsam und zumindest eine Meßmarke allein relativ zu dem Bild in beiden Richtungen (x und y) bewegt werden können. Die Betrachtung geschieht in diesem Fall mit Hilfe einer Stereo-Optik.

The stereo screen display happens either

• a) by superimposing the two stereo fields in complementary colors (e.g. red and green) on a color screen and viewing with appropriate filter glasses and generating a measuring mark that can be moved relative to the image in both dimensions (x, y) and where the fields are mutually displaceable in both directions (x, y).
• b) by displaying the two stereo partial images on two halves of a screen or two separate screens and generating two measuring marks assigned to the partial images, which can be moved together and at least one measuring mark alone in both directions (x and y) relative to the image. In this case, the observation is done with the help of stereo optics.

The measuring marks are generated by a special measuring mark generator from the generating electron beam and mark the respectively set pixel in the image memory. The measuring marks are either positioned manually by the computer or by the observer. The measuring marks in both pictures can be moved together and relative to each other by means of appropriate setting elements (eg trackball, handwheels, etc.). Basically, it does not matter whether the marks in the pictures are shifted or whether the marks on the screen remain in the center and so-called "panning" or "scrolling" the pictures are shifted arbitrarily relative to the fixed marks. The latter solution is generally the better because the observer's head can always stay in the same position.

Such a screen is generally important for the interpretative and measuring evaluation, whereby either the stereo measuring marks can be guided and adjusted by the observer or can be positioned by the computer.

With appropriate hardware interfaces, this screen can also be used for real-time observation of the recorded objects.

Well-known hard copy printers and image writing devices allow the graphic results to be output. The digital data can be output via alphanumeric printers, magnetic tapes and plates.

In the airplane, missile or satellite, the camera system (three-line stereo camera) is optionally installed with what is known as a "high density digital tape recorder" (HDDT).

For real-time transmission, the image data can also be sent to the ground station by telemetry and stored there on HDDT. Then the conversion of the HDDT data into computer-compatible data and the automatic and interactive evaluation takes place, either in real time or off line, Next Processing, storage and, if necessary, output of the image data using the stereo screen and the graphic and digital storage and output devices.

Claims (5)

1. Stereophotogrammetric recording and evaluation method for obtaining the orientation data of a camera flying over a terrain and a digital terrain elevation model (DHM) using three sensor lines A, B, C arranged transversely or obliquely to the flight direction and an associated optical system for continuous line-by-line scanning of the overflown terrain and production of three overlapping image strips A _S , B _S , C _S , taken from different perspectives, characterized in that a) the line production of the sensors A, B, C is synchronized and the serial numbers N of the line periods are registered by a line counter, b) in one of the image strips, preferably the central image strip B _S, according to certain criteria, meshed pixels that correspond to the points to be determined in the digital terrain elevation model (DHM) are sought, and the corresponding (homologous) pixels by area correlation in the other two image strips or their image coordinates x, y and the assigned line period numbers N _A , N _B , N _{C are} determined, c) due to the approximately known flight movements for each DHM point with line numbers N _A, N _B, N _C, the approximate orientation parameters of the camera and therefrom and with the image coordinates x _A, y _A and x _B, y _B and x _C , y _C approximate determination of the DHM coordinates X, Y, Z by spatial forward section, d) the establishment of beam intersection conditions for all three (or two) beams belonging to each DHM point is carried out by the DHM point, the projection centers of the camera in the defined by the line period numbers N _A , N _B , N _C Points and the image points lying on the sensor lines A, B, C are determined with the respective x and y coordinates, the orientation parameters being represented as functions of orientation interval points which are arranged at certain intervals along the flight path, that error equations are established based on mediating observations and that the most probable and final values of the orientation parameters in the orientation interval points and the DHM coordinates are found by a least squares adjustment process in a manner known per se. 2. The method according to claim 1, characterized in that all the pixels within a DHM mesh by means of transformation ^methods known ^{per se} from the ^{image strips} n A _S ; B _S , C _S are transformed into the floor plan, the respective DHM points in the image strips A _S , B _S , C _S and their point positions P _A , P _B , P projected into the floor plan for the transformation of a DHM mesh _{C is} used and orthophotos and stereo partners are produced in this way. 3. Stereophotogrammetric evaluation device, characterized by a stereo screen coupled to a correlating computer with corresponding image memories and electronic measurement mark generators, the measurement marks either displaying the points set and correlated by the computer or communicating the visually set points to the computer by interactive setting. 4. Evaluation method using a device according to claim 3, wherein in a known manner the stereo partial images in complementary colors on a color screen are shown superimposed and viewed with appropriate filter glasses, characterized in that a measuring mark, preferably generated in the center of the screen, and the images can be moved together and relative to one another in the x- and y-directions with known means (roller balls, handwheels) and methods (panning, scrolling) with respect to the measurement mark. 5. The method according to claim 4, characterized in that the stereo partial images are each shown separately on one half of the screen or on two separate screens, each partial image being assigned a measurement mark and the partial images can be viewed stereoscopically via optical means known per se, and both Images can be moved together, relative to one another and relative to the measuring marks by means known per se.

Images