DE19635544A1 - Picture error compensation method during transfer of cine into video material - Google Patents

Abstract

The fault compensation method is based on a direct optical copy such that a projected film is picked-up by a video camera. Thus various faults are dealt with, such as flicker, caused by faulty synchronisation of the projector and a scanner, such as the video camera. The compensation deals also with the so called hot spots, or unequal luminance distribution, when the video image has greater luminance in the middle than at the edges, caused by quality reduction of the optical systems. Problems can be caused also by dust on the film material. All such defects are eliminated by specified steps, using a computed correction function.

Description

1. Environment

A simple method for the transfer of moving films (cinema film, narrow film) to Video material is based on a direct optical copy: the one projected by the film projector Film is recorded by a video camera.

This method only requires relatively inexpensive equipment. However, they are the results obtained were adversely affected by various effects. The The most striking of these effects are:

• 1. The flickering caused by the lack of synchronization of projection device and scanner (video camera).
• 2. An uneven brightness distribution known as a "hot spot". Even at the video image appears in even distribution of brightness of the subject the middle lighter than the edge. This error is caused by quality defects of the most commonly used inexpensive optics.
• 3. Impairment caused by dust on the film material.

2. Flicker compensation

Figure 1: Course of the brightness or intensity of a color component in the original film material.

Figure 2: Basic course of the brightness or intensity of a color component in the video generated by transfer.

A film whose original mean brightness is as shown in Figure 1 has a brightness curve as shown in Figure 2 after optical video transmission before correction.

This course is the result of the non-synchronism of projection and video scanning. in the In the amateur area, the projection takes place e.g. B. at 18 frames / s, each projected image usually interrupted three times by the sector aperture. The projection frequency then appears to be 54 fps. The video signal, however, has 50 (PAL Europe) or 60 fields / s. This difference leads to flickering.

Similar courses as in Figure 2 also result when the intensity of the individual color components red, green and blue is viewed separately.

The aim of the method is to reconstruct the brightness curve according to Figure 1 from the brightness curve according to Figure 2. The reconstruction can only be done on a luminance basis or separately for the three color components.

The basic procedure is the same for the luminance correction and the correction in the individual color components. To simplify the presentation, we will only speak of a correction of the luminance (brightness) below, even if the three color components are corrected!
The brightness distribution of an image can be described by a sum frequency diagram. The starting point for the correction are the cumulative frequency diagrams for the brightness values of the individual images.

The transferred images in the local maxima M _k form the basis of the correction. To correct the intermediate images, their brightness distribution is interpolated to match the brightness distributions of the neighboring local maxima. This interpolation can be linear in a simple form, but also in any other technically meaningful way (parabola, splines, etc.).

Alternatively, the method can also be based on the local minimums instead of on the local maximums be based. The following illustration is for using local maxima as Reference values executed. It applies analogously to local minima in the same way.  

The sequence of the flicker correction is the following (for reasons of better representability assuming a continuous course of the sum frequency function for now):

• 1. The flicker frequency is determined for the film (raw material). Building on that the local brightness maxima are determined. There is an accompanying one Plausibility assessment based on the slowly changing at most Flicker frequency to be provided: the time interval between those used for the correction local maxima must correspond approximately to the period of the flickering.
• 2. The sum frequency distribution H _{Mi of} the brightness (possibly the 3 color components) is determined for each local maximum M _i .
• 3. For an image Z₀ between the local maxima M _j and M _{j + 1} , the sum frequency distribution of the brightness (possibly the 3 color components) is determined.
• 4. The difference between the sum frequency distribution of the brightness (possibly the 3 color components) of the intermediate image Zbild and its neighboring local maxima M _j and M _{j + 1} is determined. Z₀ is then corrected.

Example of correction with linear interpolation (output data see Figure 3) at level s _{k of} the cumulative frequency diagram:

_Let H _Mj (I) be the sum frequency function of the local maximum image M _j . The description of the sum frequency functions of other images is done in the same way. H _Mj (I) = s means that s is the proportion of all points of M _j whose intensity is less than the value 1. The inverse function of H _Mj (I) is N _Mj (s).

Figure 3: Sum frequency diagrams for two neighboring local maxima and an intermediate image enclosed by these local maxima (freely accepted values, basis for the calculation example shown in the text).

The following values can be seen in Figure 3:

Intensity at level s _k

The distance between M _j and M _{j + 1} is m = 7 pictures, the distance between M _j and Z₀ is n = 5 pictures.

In the intermediate image of the example, points of 20% intensity are corrected as follows:

N _Zk (s _k ) = ((m - n) · N _Mj (s _k ) + n · N _{Mj + 1} (s _k )) / m (1)

Intensity values of 20% (or 0.2) originally present in Figure Z₀ are therefore replaced by the newly determined value N _Zk (s _k ) (in the special case 0.193).

In order to determine the corrected intermediate image Z _k from the original intermediate image Z₀, the correction for all intensity values in all color components must be carried out in this way.

• 5. Steps 2 to 4 are carried out for each intermediate image.
• 6. At the beginning and end of the scene, the flicker compensation is carried out in the simplest case only with regard to a local maximum. This is at the beginning of the scene after the image to be corrected, at the end of the scene it is in front of it. The calculation is then carried out as follows: N _Zk (s _k ) = N _Mj (s _k ) (2) Furthermore, the statements under 4 apply accordingly.

Alternatively, the correction at the beginning of the scene with respect to the first two maxima Scene (at the end of the scene with regard to the last two). The calculation is done analogously as in 4. It is only necessary to use a negative value for n.

With most technical realizations of this procedure the sum frequency as a discrete function (discrete number of pixels, discrete Number of intensity levels).

This discrete function is converted into a continuous function by linear interpolation, both for the reference and for the intermediate images. Correction _values N _Zk (s _i ) can then no longer be determined for any number of frequency levels, but only for each intensity level of the intermediate _image . The calculation is based on (1). The result obtained in this way is rounded to the nearest value permissible on the basis of the discretization.

To avoid the addition of rounding errors, one can be used instead of point-oriented integrating calculation. (3) then replaces (1).

N _Zk (s _k ) = ((m - n) * N * _Mj (s _k ) + n * N * _{Mj + 1} (s _k )) / m (3)

The N * _Mj (s _k ) occurring therein can be determined, for example, according to (4).

N * _{Mj + 1} (s _k ) is determined analogously to N * _Mj (s _k ).

3. Compensation of the hotspot

The hotspot manifests itself in a decrease in brightness at the edges, especially in the Corners. The procedure described below resembles it using one location-dependent correction function D (x, y). According to the rotational symmetry of the involved optical systems for D (x, y) it is assumed that this function only from Distance r to the center of symmetry of the hotspot depends. Usually this is Center in the middle of the picture. Deviations indicate incorrect adjustment and will discussed below.

The hotspot compensation method is based on an approach for Correction function D. Different models for the dependence on r are possible, e.g. B. a power behavior or a behavior after a cosine power. Good results can already be obtained with the approach D = a · r² = a · (x² + y²). The zero point of the The x, y coordinate system is assumed in the middle of the image.

The correction function can be calculated from the following types of image material:

• 1. Longer section of a film. In this case, a any uneven brightness distribution of the film camera also corrected.
• 2. Short shot of the blank image (projection without inserted film), in extreme cases just a single picture.

Hotspot compensation works in both cases as follows:

• 1. First, an image B _V is calculated from the video as the time average of the individual images of the video. If real film material is used, it can be assumed that the original sequence of scenes, averaged in the same way as an image B _O , has not shown a horizontal structure - provided that the averaging takes place only over a sufficiently long period of time. However, B _O will generally still have a vertical structure (e.g. due to a bright sky when shooting outdoors or light walls when shooting indoors). The brightness distribution of the averaged original image I _O (x, y) is therefore independent of x: I _O (x, y) = I _O (y) (5) If there is a blank image with only one single image, this single image replaces the averaged image and I _O is also independent of y.
• 2. As a result of the hotspot effect, the averaged video image B _V has a decrease in brightness in each line towards the edges (Figure 4) and an uneven brightness distribution I _V (x, y). Using a compensation method, the correction function D (r) can be calculated from I _V , which best approximates the brightness curve of all lines of B _V. When using the Gaussian least squares method, one obtains e.g. B. The following extreme value problem for the unknown line brightness I _O (y) and the function D (r): Since points close to the edge react most sensitively to the hotspot effect, they can optionally be weighted higher in the compensation calculation. If there is a blank, I _{O is} a constant independent of y.
• 3. The correction function D determined in the previous step is applied to each video frame applied additively. Is the overall characteristic of the video system (camera and Recording system) known, the correction can also take into account that in the Usually darker pixels require a smaller correction than light ones.  
• 4. So far it has been assumed that the maximum of the hotspot lies exactly in the middle of the screen. An offset of the hotspot due to adjustment errors during the video transfer leads to a horizontal asymmetry of the averaged video image B _V. He can optionally as above by balancing with a suitable function such. B. D (r) = a · ((x - x₀) ² + (y - y₀) ²) can be determined and corrected.

Figure 4: Single line for hotspot correction.

4. Elimination of image disturbances due to dust particles on the film material

Dust particles are present in large numbers on film material, especially in Amateur area. It is characteristic of contamination by dust particles that in an almost black pixel appears in a picture in a position that is neither in the Predecessor is still dark in the successor picture. They can be enjoyed on the video tape largely eliminate as described below.

The method examines all areas of a single image that a certain one below the adjustable luminance threshold. For each such area checked whether it is in both the predecessor and the successor image above the Threshold. If so, it is calculated using the arithmetic mean of the RGB values filled from the two neighboring pictures. Otherwise it remains unchanged.

Example of the red component of a pixel at a threshold value of 0.05:

In picture 1 + 2 the luminance threshold is undercut at this point and therefore the red component is corrected to 0.24.

Averaging on the basis of two adjacent images is the simplest Computed dirt removal solution. Other options are:

• 1. Acquisition of the RGB value directly from the previous or successor image.
• 2. Linear interpolation based on two adjacent images (e.g. Predecessor and predecessor image).  
• 3. The nonlinear interpolation based on more than two one-sided or two-sided neighboring pictures.

Claims (17)

1. A method for compensating for image errors which have been transferred to video material Narrow films in the form of hot spots, flickering images and soiling occur as described (sections 2 to 4). 2. Process for the elimination of image flicker in optical video transfer according to Description (section 2), characterized in that images using the Sum frequency curves of the brightness or the color components lightened or be darkened. 3. Method according to 2., in which a lightening or darkening by linear interpolation between the sum brightness curves of neighboring brightness maxima or minima he follows. 4. Method according to 2., in which instead of the linear interpolation a non-linear interpolation is used.   5. Procedure for hot spot elimination as described (section 3) in that the hot spot is based on a horizontal structure-free auxiliary image by means of a rotationally symmetrical compensation function is eliminated. 6. The method as in 5., characterized in that the horizontal structure-free auxiliary image is obtained by averaging the raw film material over time. 7. The method as in 5., characterized in that the horizontal structure-free auxiliary image is obtained by empty projection or by projection of neutral prestressing material. 8. The method according to 5th, 6th or 7th, characterized in that asymmetries of the averaged Video image recognized and in the compensation calculation to correct a center offset between the projection and recording axis. 9. The method according to 5th, 6th, 7th or 8th, characterized in that as a compensating function a paraboloid of revolution is used. 10. Procedure according to 5th, 6th, 7th or 8th, where the boundary points in the compensation calculation be weighted higher.   11. The method as 9, but characterized in that a compensation function any spatial function that is point-symmetrical to the image axis is used. 12. Procedure for the mathematical removal of dirt as described (Section 4), characterized in that an image due to its neighboring images is corrected if individual image areas only in the viewed, but not in the below and below a brightness threshold. 13. The method according to 12, characterized in that the correction due to only one Neighbor picture is done. 14. The method according to 12, characterized in that the interpolation of the correction value is non-linear and more than 2 images are used for correction. 15. Procedure according to 1. to 14. individually or in their entirety on a digital computer implemented. 16. Procedure according to 1. to 14. individually or in their entirety on a personal computer (PC) implemented using a video digitizing card.   17. The method according to 1. to 14. individually or in its entirety using a special analog, digital or hybrid electronic circuit implemented.