GB2139039A - Electronically detecting the presence of film dirt - Google Patents

Abstract

Dirt on a film frame is detected from a video signal obtained from the film by a telecine by comparison of the video signal derived from successive film frames. Signals A, B and C from three frames are applied to subtractors (12,14) to generate the differences A - B and C - B which are rectified (16,18) and applied to threshold circuits (20,22). A sign comparator (26) detects whether B lies between A and C. If B is not between A and C and differs from both of them by more than an amount d, then the presence of a dirt blemish is indicated. The value of d may be adaptively dependent upon signal content. Two motion detectors 52,54 each detect in a different manner the presence of movement in the scene content, and their outputs are combined in a mixer 70. The mix itself can be dependent upon the outputs of the motion detectors. Also, an unsteadiness detector 110 detects unsteadiness between film frames and modulates a signal representing the high frequency detail content of the signal to vary the value of d. <IMAGE>

Description

SPECIFICATION Electronically detecting the presence of film dirt This invention relates to a method of and apparatus for electronically detecting the presence of film dirt, after the filmed image has been converted into an electronic video signal in a telecine machine.

It is highly desirable to be able to detect dirt on a frame of a cinematographic film, or the printed image of dirt which had been adhering to a previous negative, this being termed printed dirt. The dirt can then be concealed by known methods by replacing the affected area by another similar part of the video signal or by a simulated interpolated signal.

The present invention is defined in the appended claims to which reference should now be made.

The invention will be described in more detail, by way of example, with reference to the drawings, in which Figure 1 is a waveform diagram illustrating the effect of the presence of dirt on the signal from a telecine; Figure 2 is a block circuit diagram of a first film dirt detector embodying the invention; Figure 3 illustrates the relative positions of lines of successive fields generated at a 50 Hz field rate from film running at 25 frames per second; Figure 4 is a block circuit diagram of a modified form of film dirt detector; Figure 5 is a circuit diagram of a circuit which can be used with the detector of Figure 4 to provide an adaptively controlled reference signal in response to sensed movement; Figure 6 is a block circuit diagram of a circuit suitable for sensing the degree of unsteadiness in the film;; Figure 7 is a block circuit diagram of circuitry for storing dirt flags and using them to improve the accuracy of the apparatus; and Figure 8 is a block diagram of an alternative to the circuit of Figure 5.

A telecine machine converts cinematographic film into a video signal. The following description will be given with reference to film which is run at a frame rate of 25 frames per second to produce a video signal having 50 fields per second. In this case each film frame is converted into a single television picture formed of two interlaced fields. Thus a piece of dirt adhering to or printed into one film frame will cause a corresponding signal component only in the two video fields corresponding to that frame.

A piece of dirt does not in general totally obliterate the detail in the image which it obscures. It merely reduces the transmission of light through the film, causing a drop in signal and detail amplitude, or, in the case of printed dirt, an increase in signal amplitude with a drop in detail in the obscured areas.

This is illustrated in Figure 1, which shows a portion of a hypothetical luminance waveform produced on a telecine on three successive fields, the middle one of which is marred by dirt. The three luminance signals for the previous, current and future film frames are designated A(s), B(s) and C(s) respectively. Signal B(s) shows the presence of a piece of dirt obscuring an area of uniform mean luminance but containing high frequency detail. The expected value of the signal in the absence of dirt is shown hatched and, in a stationary scene, is approximately equal to the signal values at the same point in previous and future frames.

The circuit 10 of Figure 2 serves to identify when the luminance signal lies outside the expected range. The circuit has three inputs for receiving video signals A, B and C representing three succes size film frames. Subtractors 12 and 14form the differences A - B and C - B respectively, and these difference signals are rectified in rectifiers 16,18. The rectified difference signals are applied respectively to transfer function or threshold circuits 20,22, which each receive a reference value d. If the rectified difference signal exceeds the reference value, then the threshold circuit provides an output to a threeinput logical AND-gate 24. The two circuits 20 and 22 thus provide outputs of: IA- Bl > d and IC-B > d.

A circuit 26 is connected to the outputs of the subtractors 12,14to determine whether the signs (polarities) of the subtractor outputs are the same or different. If they are different, then B lies between A and C, and no output is provided to the logic gate, but if the signs of the subtractor outputs are the same then B lies outside A and C, as shown in Figure 1. If all three inputs to the AND-gate 24 are activated, then the presence of a piece of dirt is indicated, and an output signal is provided on an output 28.

The probability distribution of dirt sizes when taken over a large sample is heavily weighted toward smaller values. In the case of 16mm film reproduced by a 625 line 50 field 5.5 MHz television system, approximately 78% by number of dirt is less than 1 pixel in size (a pixel being the smaliest resolveable dimension), and only 1% by number is greater than 2 pixels in size. The larger dirt is more noticeable than its probability of occurrence might suggest because its area increases as the square of its linear dimensions, and since most of its spectralenergy falls within the luminance bandwidth it is reproduced at its full amplitude.

The circuit of Figure 2 is found to detect dirt in stationary scenes very effectively. The value of the reference signal d is first set empirically to provide a subjectively optimum result.

As described the transfer function circuits 20,22 are threshold circuits indicating the presence or absence of a significant difference. However, the transfer functions can be changed to give a continuously variable signal, derived by the logic circuit 24 as a suitable combination of the outputs of the transfer function circuits, indicating the degree of confidence in a given dirt detection.

If the detector is used with a normal video signal, the three frames A, B and C correspond to three television pictures, and the required parts of the video signal can be made available from an incoming video signal by the use of picture delays 30 and 32. Figure 3 shows a vertical slice of a scanned image. The crosses indicate the vertical position of selected lines on six successive television fields corresponding to three successive film frames. The use of the picture delay 30,32 means that separate detections are made on odd and even fields.

An alternative would be to make the dirt detection only on odd fields and retain the output for use on the next subsequent even field. Figure 4 illustrates a number of modifications to the circuit of Figure 2 of which this is one; a one-field delay 40 is connected to the output of the AND-gate 24, and a selector switch 42 selects the undelayed and delayed outputs on "even" and "odd" fields respectively. Conventionally in this scanning, the first and second fields from a film frame are defined as "even" and "odd" fields respectively. If dirt has been detected on, say, line B1 in Figure 3, then an additional one-line delay 41 and OR-gate 43 ensure the production of a matching flag on both lines B2 and B3, i.e. symmetrically about line B1.

The amount of delay required to store the luminance signal can be reduced by detecting dirt on even fields only but by using an interpolated average from the previous odd field rather than the signal from the previous even field. Referring to Figure 3, this means that line B1 is compared with line C1 and with an interpolated line 1/2 (A2 + A3). To this end the picture delay 32 is replaced by a 312 line delay 44 and a one-line delay 46, together with a halving adder 48 connected across the input and output of the delay 46.

If several pictures of the luminance signal are held in a large random access store having several output ports these can be used to address the required signal portions A, B and C respectively, the circuitry within the block 50 being omitted.

As previously mentioned, the circuit of Figure 2 detects dirt in stationary scenes very effectively, but it is not so effective in the presence of motion in the scene, unsteadiness in the film transport system, or substantial film grain effects. Under these circumstances the 'expected' value of the normal luminance signal varies more widely. With a view to overcoming these problems, we propose to vary the reference value d applied to the transfer function circuits 20 and 22 in dependence upon these influences.

The circuit of Figure 4 thus incorporates a number of additions which serve to vary the reference value d. There are two motion detector circuits 52 and 54.

The first circuit 52 takes signal contributions from all three film frames under consideration, thus making use of the centre of the three frames, this being the one which is being monitored. We refer to this circuit 52 as the centre frame motion detector. The second circuit 54 takes contributions only from the previous and succeeding frame, and not from the current or central frame; this we refer to as the reference frame detector.

The centre frame motion detector 52 comprises a circuit 56 which is shown as an adder but which can alternatively be a circuit which selects the greater of its two inputs. The adder 56 has its inputs connected to receive the rectified difference signal IA - BI from rectifier 16 and the rectified difference signal IC - B from rectifier 18. The output of adder 56 is applied to a spatial (two-dimension) low-pass filter 58, which provides a relatively slowly varying output signal.

This is applied to a transfer function circuit 60 which effects a linear transformation by altering the gain and zero-offset. The output m1 of the circuit 60 is then representative of the amount of movement in the scene and also of the effects of gain in the film stock.

The output m1 of the transfer function circuit 60 could be used directly as the reference value dto control the circuits 20 and 22.

The modulus operation in the rectifiers 16,18 forces all picture difference components at the spatial filter input to be positive. The spatial filter output m, therefore, consists of two components; the first is a positive offset component proportional to the mean level of grain within the filter aperture, and the second is a set of large positive components surrounding the three displaced positions of the moving object in each of the three input frames.

The choice of spatial filter aperture is important to the effectiveness of this motion detector in the film dirt detection system. The filter aperture, (the picture area from which it takes its input), must be large enough for the majority of dirt signal components to be lost by virtue of their energy being spread over the whole aperture. Large pieces of dirt with dimensions comparable to the filter aperture will appear at its output unattenuated and thereby prevent their own detection. The spatial filter aperture therefore governs the maximum size of dirt that can be detected. The upper limit of the filter aperture is determined by the appearance of a 'halo' of partially concealed dirt surrounding a moving object. This is caused by the 'spreading' effect of a spatial low-pass filter.

A good compromise for broadcast quality 16mm film has been found to be an aperture of approximately 1/20 of a picture width both horizontally and vertically. Difference apertures might give improved results for other types of film.

The offset caused byfilm grain increases with increasing grain amplitude, preventing misinterpretation of grain for dirt, regardless of the amount of grain present.

The use of the centre frame motion detector 52 alone is found to provide a performance which is adequate for the majority of typical 16 and 35mm film pictures, since it can detect well over 95%, by number, of the pieces of dirt.

However, it can fail to detect some large pieces of dirt and occasionally misinterprets slow motion of 'dirt like' objects (e.g. rivets in metalwork etc).

The latter problem can be overcome by storing dirt flags from previous frames in the additional circuitry of Figure 7. Slow motion of small objects causes a succession of erroneous, closely spaced, flag signals; some flags may be missing if the motion is at all unsteady. If these erroneous flags are not prevented, they cause moving objects in the concealed picture to blink into and out of existence.

In the circuit of Figure 7, two delays 140 and 142 connected to the output of AND-gate 24 provide access to flags formed one and two frames ago respectively. The delay lengths are adjusted to provide flags which are M and N lines early with respect to the current flag signals at the output of AND-gate 24. The two flag signals are spread horizontally and vertically by spatial spreaders 146 and 148 to form two block signals B1 and B2, each of which have their stimulus flags at their exact centre.

The size of the spread function in circuit 146 is 2M x 2W where M and W are the expected displacements of the small objects in question, during a single frame period. The dimensions of spread function in circuit 148 are approximately twice those for spread function 146, since the expected motion during two frame periods is twice that in a single frame.

The two block signals B1 and B2 are logically combined in an OR-gate 150 and inverted by an invertor 152 such that their existence will prevent propagation of current flags arriving at an AND-gate 143, when either of the previous frames contained flags nearby, thus preventing small moving detail from blinking out of existence.

Flags from more than one previous frame must be used in this technique to cope with the occasional absence of flags caused by unsteady motion. In theory, any number of previous frame flags could be combined, and the corresponding motion protection would be increased. A complication arises because genuine dirt flags also produce protection blocks and the likelihood of overriding valid current flags, and therefore failing to conceal, increases both with the number of previous frames from which protection blocks are formed, and the area of spread. Two frames of previous flag protection is a good compromise offering protection for single missing erroneous flags, whilst not overriding valid flags to any significant extent, except for extremely dirty film.

The likelihood of overriding valid flags can be reduced significantly by adding two control signals P2 and P3 which are functions of scene motion, and are connected to gates 144. These signals are a logic high when motion exists, thereby allowing protection by previous flags to operate normally. When there is no motion in the picture, protection from previous flags is not needed; P2 and P3 are logic low and gates 144 prevent the formation of protection block signals B1 and B2, thereby allowing all valid flags to pass unhindered and maximising concealment.

P2 and P3 are derived by a 'global motion' detector described below with reference to Figure 8. P2 becomes logic high for a lower level of global motion than for P3, SO that protection from previous flags is introduced progressively.

Returning to Figure 4, the second motion detector 54 comprises a subtractor 62 which receives directly the previous and succeeding film frames A and C. A rectifier 64 is connected to the substractor, and a spatial low-pass filter 66 is connected to the output of the rectifier 64. Finally, a further transfer function circuit 68 of generally the same construction as the circuit 60 receives the rectifier output.

Since this detector takes no contributions from the centre frame in which the dirt detection is being attempted, no dirt signal contributions from that frame appear in its output. Dirt detection is inhibited only in moving areas and dirt of any size can be detected.

The motion detector 54 will only accurately detect motion of an amount not greater than the aperture of the spatial filter 66 in the period between the two frames A and C considered, i.e. it is only accurate for motion of less than about half of the filter aperture per film frame. Thus, in order to give adequate motion protection for reasonable speeds of motion, the spatial filter aperture in the reference motion detector needs to be larger than that used in the centree frame detector. The halo of unconcealed dirt surrounding a moving object is correspondingly bigger. Very rapidly moving detail in the scene will thus risk being misinterpreted as dirt.

The motion detector 54 could have its output m2 directly applied to the transfer function circuits 20,22 in substitution for the output m1 of the first motion detector 52, which could be omitted. However, it is seen that both motion detectors have a tendency to fail in certain circumstances; the centre frame motion detector 52 can fail to detect large dirt particles, while the reference frame motion detector can operate erroneously in the presence of rapid movement.

As shown in Figure 4, therefore, it is preferred to include both motion detectors and to select one or other output (m1 or m2) or alternatively to mix the outputs to provide the reference value. Furthermore, and as shown, it is preferred to mix the two outputs adaptively in a mixer 70 in dependence upon an adaptive control signal, which is dependent upon picture content. This enables detection of essentially all objectionable dirt in plain or stationary areas and detection of all but the largest pieces in 'busy' areas ofthe picture with no motion impairments.

One possible form for the adaptive control circuit 72 is shown in Figure 5, and comprises three integrators 74 connected respectively to integrate over a whole film frame the outputs m1, and m2 of the detectors 52,54 and also the difference m1 - m2 between them as provided by a subtractor 76. The integrated signals M1, M2 and M1,2 are applied to respective sample and hold circuits 78 which sample the integrators at the end of each frame scan. The outputs of the circuits 78 are passed into a set of cascaded delays 80 of length equal to one film frame period. These signals are analysed by a processor circuit 82 to determine whether the motion detector outputs are consistent with the motion of real objects, i.e. their first and second differentials are consistent with Newton's law of motion. The processor then develops an appropriate set of transfer functions relating the motion detector outputs to the adaptive control signal as a function of the past scene history. The transfer functions can be applied in real time during the following film frame.

More specifically, in order to detect the presence of objects moving by greater than their own dimensions in two film frames the circuit of Figure 5 records a history over past frames of the amount of motion detected by each of the two motion detectors. If the past history shows a slow variation with no large discontinuities from frame to frame then the motion is 'well behaved'. A large or erratic value of the difference signal M1,2 on the other hand indicates high movement velocities or erratic motion.

However, a preferred adaptive control circuit 72 takes the form of a global motion detector as described in our U.K. patent 2,055,495B (Inventors Storey and Roberts), modified to provide three progressive indications of the amount of motion in a scene and also a crude measure of its grain content.

As shown in Figure 8, a picture difference is formed by a picture delay 202 and a subtractor 201.

This is rectified at 203 and integrated in an integrator 204 during each line period (equivalent to a horizontal low-pass filter). The integral at the end of each line represents the mean value of the rectified picture difference and is sampled in sample-andhold circuit 205 at the line end, before the integrator is cleared in preparation for the following line. The line integral contains two components, one proportional to the film grain level and the other proportional to the motion of objects crossed by that line.

Each sampled line integral is passed to both a maximum and a minimum value detector 206,207.

At the end of the film frame the minimum detector 206 contains a signal approximately proportional to film grain and the maximum detector 207 contains a second signal proportional to grain and motion.

These two signals are subtracted in subtractor 208 to give a signal proportional to motion only, which is fed to three comparators 209,210 and 211.

Comparators 209 and 210 produce two signals P2 and P3 which control the 'Protection from Previous Flags' system of Figure 7. Normally, the signal T3 applied to comparator 209 will be greater than the signal T2 applied to comparator 210. Comparator 211 produces a signal P1 which controls the mix between centre and reference frame motion signal m1 and m2 respectively.

The minimum detector output is stored in a store 213 and gives a measure of film grain amplitude G which is used in circuit 214 to make minor adjustments to the combining transfer functions, preserving the dirt detector sensitivity for film with a high grain level.

The dirt detector is subject to errors caused by misinterpretation of unsteady fine detail. Unsteadiness in film pictures is caused by misregistration of successive images, either in the camera, during printing, in subsequent conversion to optical images or electrical signals, or a combination of all three. It manifests itself as bodily displacement of the image, in a random direction and distance, at a rate equal to the film frame rate.

This failing can be overcome in practice by deriving a signal proportional to the amount of unsteadiness between a pair of consecutive film frames, whilst the first frame of the pair is being examined for dirt. The unsteadiness value is sampled at the end of the first frame and subsequently used to modulate a signal describing the high frequency detail in the second frame of the pair while it is, in turn, being inspected for the presence of dirt.

To this end the dirt detector is connected to the output of an unsteadiness detector 110, such as illustrated in Figure 6. The unsteadiness detector forms the subject of other U.K. Patent Application No. 8306842 to which reference should be made for further details.

The unsteadiness detector 110 has an input per minal 112 for receiving the 625/50 PAL colour video signal. A delay device 114 is connected to the input terminal 112, and a subtractor 116 has one input 118 connected to the input terminal 112 and another input 20 (as shown, the inverting input) connected to the output of the delay device 114. Thus the subtractor receives the delayed and undelayed signals. The subtractor 116 forms part of a differencing circuit 122 which also includes a rectifier 124 connected to the output of the subtractor 116.

The output of the rectifier 124 is applied to an integrator 126 which integrates over a film frame period. A sampler/register 128 samples the integrator content at the end of each frame in response to an end-of-frame signal received at an input 130. The end-of-frame signal also resets the integrator 126.

The sampler-register circuit 128 provides an output 132 indicative of the measured degree of unsteadiness.

The delay device 114 provides a delay of one cine film frame period, i.e. two video fields or one interlaced picture period. The subtractor 116 thus receives signals A and B corresponding to two consecutive cine film frames and determines the difference between these two signals. The modulus of this difference is then integrated over a complete frame scan to provide a measure of the amount of unsteadiness (or motion) between the two video signals. Any differences caused by relative motion between the two frames will cause a finite output from the subtractor 116. All such outputs are made positive by the rectifier 124 and integrated through a full frame period by the integrator 126.

Provided that the motion was less than the smallest resolvable picture element, a pixel, as is normally the case, the value of the integral will increase with increasing displacement. If the displacement were greater than a pixel the reiationship between the value of the integral and the displacement may not be monotonic, but it will generally be non-zero. However, the displacemenst are in practice usually very small.

The output of the unsteadiness detector 110 is applied to a multiplier 90. The luminance signal B for the current film frame is applied to a spatial highpass filter 92. The output of filter 92 is rectified in a rectifier 94 and applied to a spatial low-pass filter 96, the output of which is applied to the other input of the multiplier 90. In this way a signal is obtained representative of the amount of high frequency detail in the current frame, which signal is modulated by the output of the unsteadiness detector 110.

The output of the multiplier 90 is applied to a function circuit 100 which can simply be an adder, or can provide a combination of its inputs, or can select the larger of them for application to the transfer function circuits 20,22. In this way the effect of unsteadiness on the accuracy of dirt detection can be materially reduced.

The principles described in relation to a 50 Hz field scanning standard can be adapted to a 60 Hz field scanning standard in which the film is scanned at 24 frames per second and alternate frames are used to generate two and three television fields. It is necessary to ensure that the three signals applied to the detector come from successive film frames by the use of appropriate delays and switching.

The output of the circuit of Figure 2 or Figure 4 can be used to control in known manner the replacement of the defective part of the video signal by, for example, the corresponding part of an adjacent film frame.

Claims (14)

1. A method of electronically detecting the presence of film dirt from a video signal representative of the film, comprising the steps of providing a compared signal representing the rectified difference between the current portion of the video signal from a current film frame and a previous video signal portion representative of the corresponding part of a preceding film frame, comparing the said compared signal with a reference value, and providing an output indicative of the presence of sensed dirt in dependence upon the resultant of the comparison.

2. Apparatus for use in the method of claim 1, comprising means for providing a compared signal representing the rectified difference between the current portion of the video signal from a current film frame and a previous video signal portion representative of the corresponding part of a preceding film frame, comparing means for comparing the said compared signal with a reference value, and means for providing an output indicative of the presence of sensed dirt in dependence upon the resultant of the comparison.

3. A method of electronically detecting the presence of film dirt from a video signal representative of the film, comprising the steps of comparing with a reference value a first compared signal representing the rectified difference between the current portion of the video signal from a current film frame and a previous video signal portion representative of the corresponding part of a preceding film frame, comparing with a reference value a second compared signal representing the rectified difference between the current portion of the video signal from the current film frame and a succeeding video signal portion representative of the corresponding part of a succeeding film frame, and providing an output indicative of the presence of sensed dirt when both the first and second compared signals exceed the reference value and the value of the current video signal portion does not lie between the values of the said previous and succeeding video signal portions.

4. Apparatus for use in the method of claim 3, comprising first comparing means for comparing with a reference value a first compared signal representing the rectified difference between the current portion of the video signal from a current film frame and a previous video signal portion representative of the corresponding part of a preceding film frame, second comparing means for comparing with a reference value a second compared signal representing the rectified difference between the current portion of the video signal from the current film frame and a succeeding video signal portion representative of the corresponding part of a succeeding film frame, and means connected to the first and second comparing means for providing an output indicative of the presence of sensed dirt when both the first and second compared signals exceed the reference value and the value of the current video signal portion does not lie between the values of the said previous and succeeding video signal portions.

5. Apparatus according to claim 2 or4, in which the reference value is variable.

6. Apparatus according to claim 5, in which the reference value is adaptively variable in dependence upon the video signal content.

7. Apparatus according to claim 6, including a movement detector which detects movement by comparing video signal portions from the previous, current and succeeding frames, and spatially lowpass filtering the resultant.

8. Apparatus according to claim 7, in which the low-pass filter has a cut-off frequency such as to give it an effective aperture of about one-twentieth of the picture width and height.

9. Apparatus according to claim 6, including a movement detector which detects movement by comparing video signal portions from the previous and succeeding frames but not the current frames, and spatially low-pass filtering the resultant.

10. Apparatus according to claim 6 and having a first movement detector as defined in claim 7 and a second movement detector as defined in claim 9, and means for selecting from or combining the outputs thereof to provide the reference value.

11. Apparatus according to claim 10, including means for adaptively selecting from or combining the two movement detector outputs in dependence upon picture content.

12. Apparatus according to any of claim 6 to 11 including an unsteadiness detector to detect film unsteadiness and vary the reference value in response thereto.

13. A method of electronically detecting the presence of film dirt from a video signal representative of the film, substantially as herein described with reference to the drawings.

14. Apparatus for electronically detecting the presence of film dirt from a video signal representative of the film, substantially as herein described with reference to and as shown in Figure 2 or Figure 4 of the drawings.