DE3144050A1 - METHOD FOR TRANSMITTING AND PLAYING BACK VIDEO SCENES, IN PARTICULAR AERIAL VIEW SCENES, WITH REDUCED FRAME RATE - Google Patents

Description

.: '. . =. * ^: .. ^: .:. Patent attorney

Dipl.-Ing. - * - Harro Gralfs

Gialfs patent attorney Am Büigerpafk 8 D 3300 Braunschweig Geimany

Am Bürgerpark 8 D 3300 Braunschweig, Germany Telephone 0531-74798 Cable patmarks braunschweig

German research and G / WS - D 707

Research institute for air ' and space travel e.V. Linder height

5000 Cologne 90

Method for the transmission and reproduction of video scenes, in particular Aerial scenes, with reduced frame rate

The invention relates to a method for the transmission and reproduction of video scenes according to the preamble of claim 1 and provides an improvement and further development of the process according to patent application P 30 18 329.0-31.

The quality of reproduction of the process described in the main patent Intermediate images inserted in the sequence of the transmitted images depend in particular on the accuracy of the discrete image representation, the accuracy the mapping model for the generation of the intermediate images and the accuracy of the arithmetic operations. Depending on the implementation effort, there are different limits that make a flattening steeper Gray value transitions (restriction of the detail contrast) and stochastic Grayscale value fluctuations at the edges of the image (unsteady edges) and can impair the recognizability and thus the evaluability of image details.

The object of the invention is the method according to the main patent with regard to improvements in evaluability to further develop image details without the Effort must be increased significantly.

This object is achieved according to the invention in that in each case the first of two transmitted images for a period of time is displayed as a fixed image and that this fixed image display a reproduction of intermediate images follows, those from the respective still image and a subsequently transmitted one Image can be generated by the map-adapted interpolation and / or extrapolation.

In an expedient embodiment, the duration is the Still image display relative to the duration of the moving image display selectable.

The following is the procedure according to de.m main patent described with reference to the drawing.

Fiq. 1 shows the geometric arrangement for recording an aerial scene from height H above ground with the (video) camera oriented obliquely forwards. (The x and x axes are perpendicular to the plane of the drawing).

FIG. 2 shows the image recording plane of the camera with the path for two points of the object plane when the camera is moved in the direction of flight according to FIG. 1.

3 shows the temporal arrangement of two transmitted images (image 0 and image N) and an interpolated image interpolated on the reproduction side (image n) with the object-identical pixel positions Ag ₈ A _n or Bq, B _n as well as the determined by measurement _{pixel assignment C n} , C _n or C _n , C _n determined therefrom according to a linear model.

In the geometrical arrangement according to FIG. 1 , straight lines that run parallel to the y-axis in the scene (object plane) are mapped in the image recording plane (FIG. 2 ) as straight lines running obliquely to the y ^{1 -axis.} At the same time, they each represent the geometric location of the image point positions for an object point (here: the path traversed in the image) when the camera moves in the direction of flight (see FIG. 1).

The dependence of the image point position of an object point on time, ie its image-to-image offset, is determined by the regularities of the imaging geometry. In a simplified manner, as in FIG. 3 , it can be approximated by a linear curve. If the position assignment for a pair of object points in two reference images is known, for example the positions (image points) Aq, B- in image 0 and A «, B« in image N according to FIG. 3, the assignments for the rest of the image field, Determine eg for the image points Cq, C _n , from the ratio of the image point spacings according to the ray theorem. The assignment between the object-like image points C _Q and C _n with the coordinates x £ q y ' _n or xi _N , yL, and the corresponding image point C with the coordinates xi, y' in an image η is given by a specific one for this point characterized assignment straight line between the images 0 and N and may be made in the linear Xp or y 'coordinate relationships _c

^y C ^{= h} 1 ^y C ^{+ h} 2 * M

^L 0 ^L ' ^L n ^L

x '^ g ₁ XA + g ₂ (2)

and

yA ■ h ₃ y '+ h ₄ , (3)

χ 'β g, x' + g ₄ ·. (4) '

^L N ^{ό C} n ⁴

to be determined.

The coefficients f ^ to h- and g ^^ to g ₄ result from the measured pixel assignments Aq, A _n and B _Q , B _n , where h. (1 = 1,2,3,4) for each η (i.e. within each picture) and g .. for each yi (i.e. within each picture line) is constant (see below). The ^{algorithm represented by n} the equations enables line and image sequential processing in the sense of the video signal to be output and is therefore particularly suitable for the technical implementation of the method.

The one to be regenerated results from the gray values S _CQ and S _CN for the image points C _Q and C _n with the coordinates x q, y Q and xL ·, y _N determined according to equations (1) to (4) Gray value S- for the image point C by interpolation according to the relationship

or by extrapolation according to

S _r = S _r . (6a) or S _r ■ S _r (6b) ^c n _Q Si ^L N

The extrapolation is used in parts of the image in which one or more the coordinate values determined from equations (1) to (4) are outside the value range. . , -

-? - ^y C ^ C "7 '(7)

ί t ₀ C _n ί

a < _v ι _w ι <a

"" 7 " ^x r *» ^x r 7 ^{z 1} O ^C N ^Z

lie, i.e. outside the image recording plane with the width a and the height b according to Fig. 2.

In other cases, extrapolation instead of interpolation can be useful e.g. if the assignment between the reference images is temporarily only to be determined imprecisely or the overlap of the image information is relatively small.

Object movements in the scene do not appear in an image (or partial image) sequence regenerated by extrapolation, while they do not appear in the Interpolation in general by a gradual decrease and decrease of the '· Brightness intensity are marked-. This case, in which the interpolation is not based on image points of the same object, corresponds in its Effect of fading between the positions of the moving object in successive reference images.

-c-

The situation is similar in the case of sewer faults. Through disturbances in one Gray value changes caused in the reference image take place in the interpolated The image sequence gradually closes, i.e. within N image periods, and then fades away again accordingly.

_{Determine the reference point assignment A Q} , A _n and β _Q , B _n 180t ε-eh required for the method by means of an image correlation method or from on-board measurement data known at the receiving end. In order to increase the reliability of the decision, it is possible to measure the assignment for further pixels and then select them according to predetermined criteria (such as, for example, reliability or majority criteria). If reference point assignments are measured specifically for the area of an object moving within the scene, image points that are identical to the object can also be determined for the interpolation in such areas and the sequence of movements can thus be reproduced directly instead of by the abovedescribed decrease and decrease in brightness. '

In the following, the coefficients h. |. Used in equations (1) to (4) will be described and G. (i - 1,2,3,4) is defined. The selected division of the equations offers advantages for the execution of the arithmetic operations in a technical device.

• N (II ₁ ) * " ^k l ^y C _f

n ^ ^S Ii _ 1 I ") 9i ⁼ IL, il

(Nn) l ₂ ^k 2 ^(y C - ^y C

^{l c} n

The sizes h. and G. each have the same denominator. The values for h _i must be determined once per vertical period (duration of an image) and the values for g .. once per horizontal period (duration of one line) of the television signal. Via equations (1) and (3), g ^ depends on h ^.

The following auxiliary variables are used:

^d A ^ A " ^d B ^ B ^tX B" ^X A AA _n ö B _n B _n A

^k 2 ^ad A ⁽¹ " ^k l ^y V ^{+ k} l ^X A,

I ₂ - y _B -yi (1-I ₁ ) · ( ^llb ) '

^d A " ^d B

<'^ W ^(12C)

• 'x ¹ - y'

D D '

_B.

. The values for ^ y '' - ^ ^{1 k} i ^{and k} 2 ^{must be determined e1n times ye n} vertical periods.

The required coordinate values are obtained by measuring the position assignment of the same object image points A ₀ (xj ^, y ^ _Q ) and A _n (x ^ ,, y _A ") or. ⁸ O ( ^χ Β0 » ^y B0 ^ ^{and 8} N ^ BN ·» ^^ ^known ·

The method described is in principle also valid if the image point assignments within the time sequence are described by the exact relationships of the imaging geometry instead of by the more cost-effective, linear model. .

In addition or as an alternative to the described interpolation or Extrapolation of the image-to-image offset for the entire image a corresponding interpolation or extrapolation can take place for predetermined image areas, for example individually objects moving in front of the background, for which the corresponding Coordinate points are to be specified separately, or in general to increase the reproduction accuracy and / or highlight individual parts of the image.

In principle, the method can also be used for other applications with relative movement between the recording sensor and the scene.

Depending on the number of inserted intermediate images, playback can be performed at original speed, time lapse or time expansion take place.

The reproduction quality of the intermediate images inserted in the sequence of the transmitted images in the method described above is particularly of the accuracy of the discrete image display, the accuracy of the mapping model for generation the intermediate images and the accuracy of the arithmetic operations. Depending on the implementation effort, this results in different limits that flatten steep gray value transitions (Limitation of the detail contrast) and stochastic Grayscale value fluctuations at the edges of the image (unsteady edges) have consequences and the recognizability and thus the evaluability of image details can affect.

The evaluability of image details can be improved without a significant increase in effort by the fact that in each case the first of two transmitted images is displayed as a fixed image for a period of time and that this fixed image display This is followed by a display of intermediate images, those from the respective still image and a subsequently transmitted one Image through the map-adapted interpolation and / or extrapolation be generated.

The term "fast image" is intended to mean a reproduction of an image without changing the image elements. The fixed image display typically takes place by repeating the reproduction of the relevant transmitted image for the reproduction device usual picture playback frequency.

During the fixed image display that is effective for your specified time the image is free from the restrictions imposed by the reproduction accuracy caused by interpolation and / or Regenerated intermediate images are subject to extrapolation. An image observer can thus take advantage of the fixed image phase » to focus on image details (e.g. details of a target). The subsequent video display in this case serves to guide the observer largely without getting tired of the next transmitted image and It also enables him to detect any object movements in relation to the background of the image. "The observer can keep an eye on image areas of interest and immediately concentrate on the details of interest in the next fixed image. In contrast to becomes in a sequence of seamlessly consecutive fixed images generally a minimum search time is required to find the corresponding area of interest in the next image to be found again, especially with a relatively large picture-to-picture offset. With the option of special detailed information to gain in the Festbil.dphase are the limitations of the reproduction accuracy resulting from the image regeneration less significant in the moving image phase than when displaying moving images only.

When using the method in real-time operation, the Moving image phase reproduced in a time-lapse manner, as the for the display the picture-to-picture transition Time is shorter than the period of the transmitted image sequence

- 40-

and thus shorter than the corresponding time interval of the Images as they are taken. For example, in the case of a transferred Image sequence with a period of 4 seconds (corresponding to a repetition rate of 0.25 frames per second) and ' Choosing a fixed picture phase of 3 seconds is the moving picture phase instead of 4 seconds (time interval between the images when recording) only 1 second.

In the case of strong time lapse during the moving image phase, it can be useful instead of a curve with a constant one Speed a course with initially gradually increasing and then gradually decreasing speed v / orzu- ' see. Compared to the sudden onset and breakdown of image movement in the case of constant speed in the case of a gradual increase in speed, the transition from the previous or the following fixed image occurs or decrease in image accelerations or delays, which are subjectively advantageous when viewing the image impact. The speed as a function of time can, for example, be based on a triangular, angular (e.g. Sinusj Cosinus) or an exponential function or an approximation Polynomials run.

The duration and / or time of the fixed image display can be designed variably. So it is possible to use the playback device to be trained in such a way that still image sequences are more suitable from the observer Length can be optionally inserted into the moving image display, for example, when the observer of a regenerated moving image sequence encounters areas of the image in which he has details want to recognize more precisely. The switchover can be carried out manually. But it is also an automatic switchover, for example, depending on predetermined picture elements or time-dependent, possible. With sufficient over ^ The content of successive images can be lapped the possibility of - optionally extending the fixed image phase beyond the period of the transmitted image sequence, by not taking each of the images into account, but by suppressing individual images (e.g. every second image) larger picture-to-picture distance is used.

The ratio of the duration of the still image phase to the duration of the moving image phase can be different for individual image areas Select. Specifically, for example, the insertion of a still image display be restricted to an area in the center of the image, while the other parts of the image are constantly in the ·. Moving images remain.

Known additional measures for image reproduction improvement, in particular digital filter operations »in which in the moving image phase the limits of implementation are similar Restrictions can result, as in the case of image regeneration according to the main patent, can be used advantageously in the fixed image phase. .

As required, there is the option of viewing the entire scene compared to real-time operation in a shorter or longer time. Specifically, for example, one of the inserted fixed image displays can be extended in proportion to the total display time can be achieved that in the moving image display the scene movement at the speed of real-time operation, i.e. is played back without time lapse.

- • Λ? -.

Blank page

Claims (3)

Expectations A method for the transmission and reproduction of video scenes, in particular aerial scenes, with a reduced image sequence frequency with relative movement of the recorded scene with respect to the recording sensor, in which intermediate images generated on the reproduction side are inserted into the transmitted image sequence through image-adapted interpolation and / or extrapolation based on two transmitted images in each case for the inter- or extrapolation to be assigned object-like pixel positions of the two transmitted images are determined from the known assignment of the pixel positions for at least two object points lying on vertically and horizontally unequal coordinates in each of the two transmitted images, according to patent application P 30 18 329.0-31, characterized in that. that in each case the first of two transmitted images is displayed as a fixed image for a period of time and that this fixed image display is followed by a display of intermediate images that are generated from the respective fixed image and a subsequently transmitted image by the map-adapted interpolation and / or extrapolation » 2. The method according to claim .1, characterized in that the duration of the freeze frame playback is variable. 3. The method according to claim 1 or 2, characterized in that each part of the image is reproduced as a fixed image.