EP0261137B1 - Fernsehbild-bewegungsmessung - Google Patents

Fernsehbild-bewegungsmessung Download PDF

Info

Publication number
EP0261137B1
EP0261137B1 EP87900235A EP87900235A EP0261137B1 EP 0261137 B1 EP0261137 B1 EP 0261137B1 EP 87900235 A EP87900235 A EP 87900235A EP 87900235 A EP87900235 A EP 87900235A EP 0261137 B1 EP0261137 B1 EP 0261137B1
Authority
EP
European Patent Office
Prior art keywords
pictures
picture
motion vectors
motion
correlation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP87900235A
Other languages
English (en)
French (fr)
Other versions
EP0261137A1 (de
Inventor
Graham A. Thomas
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
British Broadcasting Corp
Original Assignee
British Broadcasting Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from GB868606809A external-priority patent/GB8606809D0/en
Application filed by British Broadcasting Corp filed Critical British Broadcasting Corp
Publication of EP0261137A1 publication Critical patent/EP0261137A1/de
Application granted granted Critical
Publication of EP0261137B1 publication Critical patent/EP0261137B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N11/00—Colour television systems
    • H04N11/06—Transmission systems characterised by the manner in which the individual colour picture signal components are combined
    • H04N11/12—Transmission systems characterised by the manner in which the individual colour picture signal components are combined using simultaneous signals only
    • H04N11/14—Transmission systems characterised by the manner in which the individual colour picture signal components are combined using simultaneous signals only in which one signal, modulated in phase and amplitude, conveys colour information and a second signal conveys brightness information, e.g. NTSC-system
    • H04N11/16—Transmission systems characterised by the manner in which the individual colour picture signal components are combined using simultaneous signals only in which one signal, modulated in phase and amplitude, conveys colour information and a second signal conveys brightness information, e.g. NTSC-system the chrominance signal alternating in phase, e.g. PAL-system
    • H04N11/167—Transmission systems characterised by the manner in which the individual colour picture signal components are combined using simultaneous signals only in which one signal, modulated in phase and amplitude, conveys colour information and a second signal conveys brightness information, e.g. NTSC-system the chrominance signal alternating in phase, e.g. PAL-system a resolution-increasing signal being multiplexed to the PAL-system signal, e.g. PAL-PLUS-system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/262—Analysis of motion using transform domain methods, e.g. Fourier domain methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N11/00—Colour television systems
    • H04N11/02—Colour television systems with bandwidth reduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N11/00—Colour television systems
    • H04N11/24—High-definition television systems
    • H04N11/28—High-definition television systems involving bandwidth reduction, e.g. subsampling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43—Hardware specially adapted for motion estimation or compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/527—Global motion vector estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/547—Motion estimation performed in a transform domain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/015—High-definition television systems
    • H04N7/0152—High-definition television systems using spatial or temporal subsampling
    • H04N7/0155—High-definition television systems using spatial or temporal subsampling using pixel blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/015—High-definition television systems
    • H04N7/0152—High-definition television systems using spatial or temporal subsampling
    • H04N7/0155—High-definition television systems using spatial or temporal subsampling using pixel blocks
    • H04N7/0157—High-definition television systems using spatial or temporal subsampling using pixel blocks with motion estimation, e.g. involving the use of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/12—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal
    • H04N7/122—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal involving expansion and subsequent compression of a signal segment, e.g. a frame, a line
    • H04N7/125—Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal involving expansion and subsequent compression of a signal segment, e.g. a frame, a line the signal segment being a picture element
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Definitions

  • the object of the present invention is to extend the known technique to measure the motion vectors in a scene containing many objects moving in different directions and speeds.
  • the process of calculating motion vectors for every pixel in the picture is broken down into two stages.
  • the first stage involves correlating two successive pictures (or fields, depending on the exact application) to determine the principal motion vectors present in the scene.
  • the second stage attempts to assign one of these vectors to every pixel. For some pixels it may be impossible to assign a motion vector, for example if the pixel corresponds to a very small object or to uncovered background.
  • FFTs 2-dimensional Fast Fourier Transforms
  • a unit length vector is calculated whose phase angle is equal to the difference in the phases of this frequency in the two pictures.
  • a reverse FFT is performed on the resulting complex array, which produces an array of real numbers giving the correlation between the two pictures.
  • the resulting phase correlation array can be thought of as a surface whose height at a particular point (x,y) is proportional to how well the two images correlate when the relative displacement between them is (x,y).
  • the correlation surface would be a delta function centered on the shift vector. The idea is that there will be a peak in this surfaced for each dominant motion vector in the scene. Measuring these motion vectors involves hunting for large peaks in the surface. The relative heights of the peaks will reflect the relative sizes of the moving objects.
  • the main novel feature of this method is to look for several peaks rather than just one, thereby allowing the detection of many velocities in one operation.
  • the first stage of the process gives us a set of motion vectors that are present in the scene, but tells us nothing about which parts of the scene are moving with which vector.
  • the second stage involves 'trying out' each possible vector on every pixel, and assigning the vector that gives the best 'fit'. In situations where motion information is required on a block-by-block basis (for example a block based bandwidth compression system), there would be no need to assign a vector to each pixel, and the assignment would be done on a block basis.
  • an 'error surface' is formed by calculating the modulus difference between the two input pictures when shifted by the motion vector under consideration. Areas of the two pictures that match well will produce low errors, and it is reasonable to assume that these areas correspond to a moving object with this particular motion vector. It is preferable to perform some sort of spatial filtering on the error surface in order to reduce the effects of noise.
  • each pixel is assigned the motion vector which gave the smallest error value.
  • a threshold error level could be set, defining the maximum acceptable error level. Pixels whose errors are above this level with every trial vector could be flagged as 'motion unknown'. Such pixels would probably correspond to areas of erratic motion, covered or uncovered background.
  • the motion vectors were measured by transforming the input pictures in blocks rather than as a whole, then it is only worth trying vectors measured in the neighbourhood of the pixel under consideration. The chances are that most pixels could have the correct vector assigned to them merely by considering the vectors measured in the block containing the pixel. Ideally the vectors measured in the immediately surrounding blocks should be tried as well, particularly for pixels near the edge of blocks. This is important in situations where a small part of a moving object intrudes into an adjacent block.
  • Phase correlation overcomes the problems with these more 'traditional' techniques. For small movements, all three techniques can be shown to be roughly equivalent but, for large movements, phase correlation comes into its own.
  • This technique has the intrinsic advantage that it detects the dominant motion vectors in a scene. These would usually correspond to large areas of background and large moving objects. These are the sort of things that the human eye can easily follow, and hence they are the most important parts of a scene to process correctly.
  • Figs. 1 to 4 and 6 to 9 show various correlation surfaces, as detailed more fully below.
  • the Z axis shows correlation peaks
  • the X and Y axes show X velocity and Y velocity respectively, denoted X-V and Y-V and measured in pixels/field period.
  • Fig. 1 shows the correlation surface for a stationary picture using a SIN(x)/x interpolator.
  • Fig. 2 shows the surface for a pair of 10 pixels per field period again using a SIN(x)/x interpolator.
  • Fig. 3 shows the surface for a stationary picture using a smoother interpolator.
  • Fig. 4 corresponds to Fig. 2 but using the smoother interpolator.
  • Fig. 5 illustrates windowing of a phase array prior to reverse transformation.
  • Fig. 6 shows the correlation surface for a pair of 10 pixels per field but with a "guard band" on the input picture.
  • Fig. 7 corresponds to Fig. 6 but further with a raised cosine aperture on the input picture.
  • Fig. 8 shows the correlation surface for an object moving over a stationary background.
  • Fig. 9 shows the correlation surface for a moving gate in a "Voit" sequence referred to below.
  • Fig. 10 illustrates how a velocity vector can be assigned in performing a motion compensated interpolation.
  • Fig. 11 shows how both a large and a small vector can fit one point.
  • the purpose of the computer simulations that have so far been performed is to find out how well the phase correlation technique really works. We have not limited the simulations to algorithms that would be easy to implement in hardware; the intention was to find out how good the best algorithm is. The subject of further simulations will be to find out how much the 'ideal' algorithm can be simplified in order to make it suitable for a real time hardware implementation for a given application.
  • the first phase of the investigations examined the 'vector measurement' stage of the algorithm.
  • the aim of this phase was to investigate the accuracy of the phase correlation technique, and see how it depended on the relative shift size, number of moving objects and object size, amount of noise, and so on.
  • Figure 1 shows the correlation surface obtained when two identical picture portions were correlated.
  • the phase array Z was padded out with zeroes to form a 128 by 128 array prior to performing the inverse FFT; this enabled values at half-integral shifts to be interpolated with a sin(x)/x impulse response. This is the cause of the 'ringing' visible around the peak.
  • Figure 2 was produced in the same way but with a horizontal shift of ten pixels between the two picture portions.
  • the location of the peak has moved, and the use of quadratic interpolation (applied independently in the x and y directions) gives the peak location as (9.99,0.02).
  • the shift has been measured to an accuracy of a few hundredths of a pixel in ten pixels.
  • the height of the peak has diminished and noise has been introduced; both effects can be attributed to the revealed and obscured material at the edges of the picture.
  • the ringing introduced by using the sin(x)/x interpolator produces spurious peaks which can sometimes mask peaks produced by small moving objects.
  • the ringing can be reduced by using a different interpolator.
  • the interpolator can be changed by windowing the phase array Z(m,n) prior to performing the reverse transform. The effects of various windows were investigated. The windows were all of the form
  • the solid line shows a simple window with frequencies 0 to f o supported in the array Z and zeroes inserted from f o to 2f o to enable interpolation of intermediate points. This gives the sin(x)/x impulse response with sharp peaks but "ringing".
  • the broken line shows a Hamming-like window leading to poorly resolved main peaks but smaller subsidiary peaks. This one dimensional windowing function was applied both horizontally and vertically. A value of zero for 'a' gives the rectangular window that produced figs. 1 and 2, and a value of 0.54 would give a Hamming window. Experiments showed that a value of about 0.8 gave a reasonable compromise between heights of spurious peaks and sharpness of each peak.
  • phase correlation technique the modulus of all the numbers in the phase array Z is set to unity. This means that the information in frequencies of low amplitude carries more weight than perhaps it should, given that the signal-to-noise ratio of these components will be low. In comparison, no such normalization takes place when performing a cross-correlation, with the result that the correlation peaks are broader and the peak heights become scene-dependent. A compromise between phase correlation and cross-correlation may give better results than using either extreme. This entails normalizing higher amplitude components and reducing the amplitude of low ones. These ideas will be investigated in the next stage of the simulation work.
  • noisy pictures were generated by adding random values (with a given peak value) to the luminance level at each pixel.
  • a correlation was performed between two pictures shifted by ten pixels (as used to generate fig.2), with different levels of noise. It was found that adding noise with a peak value 20dB below peak white level reduced the height of the correlation peak by a factor of 2.4, without significantly changing the general level of noise on the surface.
  • the measurement error increased to about 0.08 pixel.
  • Increasing the amount of noise to 10 dB below peak white level caused a further reduction of a factor 2.2, but the peak was still detectable.
  • the measurement error rose to about 0.3 pixel. This suggests that noisy pictures do not present a significant problem to this part of the motion measurement process. It is possible that the noise immunity could be increased by some 'adaptive windowing' technique on the phase array, as discussed in section 3.1.2.
  • a possible way of reducing the noise on the correlation surface would be to perform some kind of temporal filtering operation.
  • a first order recursive temporal filter applied to the correlation surface of each block would enhance stationary or slowly changing peaks, while reducing the level of any noise peaks.
  • the disadvantage of this method is that velocity peaks caused by rapidly accelerating objects would be attenuated. It should be possible to choose a filter which gives good attenuation of noise without unduly attenuating genuine moving peaks. It would be advantageous to detect shot changes and inhibit the temporal filtering action immediately after a change.
  • a portion of a picture ('Dick and Jean') 64 pixels square was extracted, and a 32 by 32 pixel portion from another picture (the blackboard cross from 'Testcard F') was inserted into the middle of this picture.
  • the edges of the inserted pictures were 'blended' with the background picture over a distance of 3 pixels so as not to produce any artificially high frequencies in the resulting picture.
  • a second similar picture was formed, but with the inserted picture portion shifted three pixels to the left and three down.
  • Figure 8 shows the resulting correlation surface.
  • the interpolated peak locations (calculated by fitting a quadratic to the sin(x)/x interpolated data as before) were accurate to 0.01 pixel.
  • the relative heights of the peaks reflect the relative areas of moving objects and background, although the ratio of the heights is less than the ratio of the object areas (1.6 compared to 4).
  • the noise around the peaks is due to the uncovered and revealed background around the inserted picture portion.
  • phase correlation technique had no problems accurately measuring this motion, some more severe experiments were tried.
  • the aim of these was to see how small a moving object can be, and still be detected.
  • a picture portion of various sizes was 'moved' with a shift vector of (5,5) over a background picture, which was 64 pixels square.
  • the object's motion vector was accurately detected for objects as small as two pixels square, which was quite surprising, especially considering that the object had moved more than twice its own length.
  • phase correlation technique does indeed work as well as claimed in the reference cited above.
  • typical vector measurement accuracies of 0.02 pixel can be obtained by first interpolating the correlation surface by a factor of two in both directions (using an interpolator with an impulse response roughly of the form sin(x)/x), then fitting a quadratic to the interpolated points either side of the maximum (for x and y separately).
  • An accuracy of about 0.2 pixel can be obtained by simply fitting a quadratic to the uninterpolated correlation surface. The accuracy of the technique was largely unaffected by noise, and it was possible to detect very small objects.
  • the aim of this part of the investigation was to simulate the second stage of the motion measurement process, namely assigning the principal motion vectors to particular pixels.
  • the first problem that had to be dealt with was to find a way of showing how well the vector measurement and assignment had been carried out. It was decided to do this by using the motion vectors to interpolate temporally a picture between the two input pictures. This is not only a stringent test of the method, but also shows what sort of results could be obtained if the technique is used to improve the motion portrayal of film, by 'making up' intermediate pictures.
  • a computer program was developed that could generate the odd fields of a sequence by temporal interpolation between the even fields (or vice versa).
  • the program allowed the user to change various parameters associated with the method, such as the size of the input picture, the size of the blocks on which correlations were performed, the number of vectors extracted per block, and so on.
  • a motion vector was assigned to every pixel; there was no upper limit set for the acceptable error. In an ideal implementation, pixels with large errors would be investigated further, and some account of uncovered background would be made.
  • the luminance value of each pixel in the output picture was calculated by averaging the values in the adjacent two fields, at locations displaced by the motion vector for the pixel.
  • Figure 10 illustrates this idea.
  • Fig.10 shows a previous field P, a next field N and the field G being generated.
  • the luminance value at the point in field G is the average of the values at points x and z in fields P and N where the points x and z are displaced from y by ⁇ V/2 and V is the velocity vector assigned at point y. This vector will have been chosen to minimise the luminance difference between points x and z (after spatial filtering).
  • the even fields from the 'VOIT' sequence were generated from the odd fields by performing a motion compensated interpolation.
  • V1 and V2 are possible motion vectors for the point P in the field G if the background is uniform
  • the object 0 might be a silver surround at the bottom of a car radiator against a background provided by the radiator itself and the black area below it in the "Voit" sequence.
  • the picture here referred to is shown in Fig.3 of BBC Research Department Report 1986/5 published July 1986.
  • the reasoning behind the gate's disappearance is slightly more subtle.
  • the gate is a periodic structure which moves about 4.7 pixels horizontally in a picture period, and repeats itself spacially about every 14 pixels. This means that there are two valid motion vectors for the gate, namely +4.7 and -9.4 pixels per picture period (disregarding the effect of the edge of the gate). If the incorrect motion vector is chosen, the gate 'breaks up' in the interpolated picture.
  • the vector measurement part of the hardware would be based around a circuit that could perform one dimensional FFTs on an array of data. Such a circuit could be built using some multipliers (or PROMs) and adders to implement one FFT 'butterfly' operation.
  • the data being transformed would be held in RAMs, and a PROM would generate the appropriate sequence of addresses to operate the butterfly between appropriate RAM addresses.
  • Another PROM would hold the 'twiddle factors'.
  • By using a pipelined design, such a circuit would take n.(log2 n) clock pulses to do an n point complex FFT.
  • Such a circuit would consist of 8 multipliers, 6 adders, a selection of PROMs holding twiddle factors, and 4 RAMs, each holding a copy of the array being transformed.
  • the physical size of such a circuit would depend on the number of bits required in the calculation (which has yet to be determined), but it would probably fit on 1 or 2 4U boards.
  • a controlling board could hold the picture portion (or the whole picture) being transformed, and pass it (one line or column at a time) to the FFT board. If the picture being transformed was n pixel square, 2n FFTs are necessary, giving a total of 2n2 log2 n clock periods per 2-D FFT.
  • the first stage of the motion vector measurement technique as described involves performing a two-dimensional phase correlation between two successive images in a sequence. For a picture portion that contains m x n elements, this involves carrying out m Fourier Transforms of length n followed by n Fourier Transforms of length m, calculating the phase difference for each of the (m x n) spatial frequencies, and performing the same number of reverse transforms. The resulting phase correlation surface is then interrogated to find the location of the dominant peaks.
  • the picture could first be vertically filtered by summing all elements in each column of pixels. A phase correlation would then be performed between the resulting row of values and the row of values that resulted from filtering the previous picture. The dominant peaks in the correlation function would give the horizontal components of the dominant motion vectors in the picture. A similar process would be performed to measure the vertical components of the dominant shifts.
  • this method cannot uniquely identify the locations of the dominant peaks, since there is no indication as to which horizontal and vertical components should be paired up. This is only a problem when more than one dominant vector is of interest. It can be shown that the location of n peaks can be uniquely identified by calculating at least n + 1 correlations on mutually non-parallel axes. As n rises, however, the probability of there being more than n possible peak locations diminishes even if there are n or less axes. In a practical case where we want to find, say, 4 peaks, 4 correlations would probably be adequate. The probability of assigning incorrect peak locations also decreases with the accuracy of the coordinate measurement. Even if some uncertainty in peak location remains, this can always be overcome by increasing the numbers of shift vectors that are tried in the vector assignment stage.
  • the amount of calculation required to find the dominant peaks in the correlation surface for two pictures can be significantly reduced by performing a number of one dimensional correlations on the filter picture.
  • the filtering operation amounts to summing the picture elements along lines perpendicular to the axis of the correlation.
  • the correlations must be performed on mutually non-parallel axes. To uniquely identify the locations of n peaks using this technique, it is necessary to carry out at least n + 1
  • correlations although for n>3, fewer correlations will usually suffice. If 4 correlations were performed on a picture portion d pixels square, the number of Fourier Transforms required per input picture would be 8, compared co 4d for the full two dimensional method. For a typical block size, this amounts to savings in hardware of the order of a factor of 30. The penalty paid is that the peaks will probably be broader and noisier, and some false peak locations may be produced.
  • the invention will be seen to lie in the combination of correlation of two pictures to determine correlation as a function of selective displacement (in the general case in two dimensions), thereby to determine a plurality of peak correlation values corresponding to respective motion vectors, followed by testing on a pixel-by-pixel or block-by-block basis which of these motion vectors gives the best fit in deriving one of the pictures from the other.
  • An advantageous application for this technique is to improve the quality of a motion adaptive bandwidth compression system. This involves measuring the principal motion vectors in a scene, and assigning them on a block-by-block basis (a block being about 6 pixels square).
  • the motion vector measurement technique can be usefully applied in any situation where it is necessary to perform temporal interpolation on a series of images or follow the movements of objects in a scene.
  • the following paragraphs describe some applications in more detail.
  • each frame is displayed twice (once scanned as an odd television field and once as an even field in an interlaced system).
  • Such an arrangement enables a film shot at 25 (or 24) frames a second to be seen at (nearly) the correct speed.
  • the process of repeating pictures on successive fields significantly reduces the level of flicker that would be experienced if each picture was only shown once, the process also introduces motion impairments.
  • the observer's eye tracks moving objects, a double image appears on the retina because each moving object is not shown at the correct place at the right time.
  • Each pixel in the intermediate picture could be generated by averaging the luminance levels at the corresponding pixels in the following and preceding pictures, displaced by plus and minus half the appropriate motion vector respectively.
  • the values of the R, G and B components would each be derived in this way.
  • motion compensated temporal interpolation is used to generate pictures at the times required by the output field rate. This process is identical to that described above for improving the motion portrayal of film, except that it is only necessary to generate every other line of the output pictures, assuming conversion to an interlaced standard.
  • Smooth slow motion could be achieved using conventional cameras if extra fields could be generated using temporal interpolation. This can be done using the process described earlier for performing standards conversion. The number of intermediate fields that need to be generated would depend on the amount by which the motion was to be slowed down.
  • the electronics required to generate the intermediate fields would be placed on the output of a conventional video tape recorder with simple slow motion playback facilities.
  • the equipment would measure the number of times each field was repeated and generate the appropriate number of intervening fields to replace the repeated fields.
  • Motion vector measurement techniques can be incorporated in thetemporal filtering process so that it may be extended into moving areas. In the case of first order recursive temporal filtering, this would entail shifting each pixel (or group of pixels) in the frame store by the corresponding motion vector prior to performing each recursion. If the filtering was performed using a transversal filter, the readout address of each store would be displaced according to the sum of the local motion vectors over the number of fields between the output field and the stored field.
  • Motion vector measurement can be used to great advantage in almost all video bandwidth reduction techniques since knowledge of localized motion vectors is the key to being able to exploit the temporal redundancy in the signal.
  • An adaptive subsampling system could use knowledge of motion vectors to enable missing samples to be reconstructed from preceding fields.
  • DPCM systems can use motion vectors as the basis of the predictive element. In both these systems, information on the motion vectors would be derived at the transmitter by performing the measurement between two successive fields, and could be transmitted to the receiver along with the rest of the signal.
  • Interpolation may be used to increase the field rate for display purposes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Television Systems (AREA)

Claims (29)

1. Maschinenverfahren zur Fernsehbild-Bewegunsmessung, umfassend einen ersten Schritt der Korrelation zweier Bilder zur Korrelationsbestimmung als Funktion einer Verschiebung, um so mehrere Peak-Korrelationswerte entsprechend jeweiliger Bewegunsvektoren zu ermitteln; und einen zweiten Schritt des Prüfens für jeden von mehreren elementaren Bereichen der Bilder, welcher dieser Bewegungsvektoren die beste Anpassung für das Ableiten eines der Bilder aus dem anderen ergibt.
2. Verfahren nach Anspruch 1, in welchem die Verschiebung zweidimensional ist und eine zweidimensionale Korrelation ausgeführt wird.
3. Verfahren nach Anspruch 1, in welchem mehrere eindimensionale Korrelationen unter Verwendung eindimensionaler Verschiebungen in verschiedenen Richtungen angewandt werden.
4. Verfahren nach Anspruch 3, in welchem die Anzahl eindimensionaler Korrelationen größer als 2 ist und mehrere Bewegunsvektoren als solche Bewegunsvektoren ermittelt werden, die in den verschiedenen Richtungen Komponenten aufweisen, die zu Peak-Korrelationswerten passe, welche durch die eindimensionalen Korrelationen geliefert werden.
5. Verfahren nach einem der Ansprüche 1 bis 4, in welchem die Bewegunsvektoren durch Interpolation der Korrelationsfunktion auf Sub-Pixelgenauigkeit gemessen werden.
6. Verfahren nach einem der Ansprüche 1 bis 5, in welchem der Korrelationsschritt bezüglich mehrerer Pixelblöcke separat ausgeführt wird und die gesamte Anordnung von hierdurch bestimmten Bewegungsvektoren im zweiten Schritt verwendet wird.
7. Verfahren nach einem der Ansprüche 1 bis 5, in welchem der Korrelationsschritt bezüglich mehrerer Pixelblöcke separat ausgeführt wird und im zweiten Schritt nur die in Bezug auf einem vorgegebenen Block ermittelten Bewegungsvektoren bei der Prüfung bezüglich dieses Blocks verwendet werden.
8. Verfahren nach einem der Ansprüche 1 bis 5, in welchem der Korrelationsschritt in Bezug auf mehrere Pixelblöcke separat ausgeführt wird und im zweiten Schritt beim Prüfen in Bezug auf einen gegebenen Block die verwendeten Bewegungsvektoren diejenigen sind, die in Bezug auf diesen Block und mehrere angrenzende Blöcke ermittelt wurden.
9. Verfahren nach Anspruc 6, 7 oder 8, in welchem jeder Block unter Verwendung einer angehobenen Kosinus- oder anderen Fensterlücke einer Fenstertechnik unterzogen wird.
10. Verfahren nach Anspruch 9, in welchem jeder der Fenstertechnik unterzogene Block mit Nullen umgeben ist.
11. Verfahren nach einem der Ansprüche 6 bis 10, in welchem die Blöcke überlappende Blöcke sind.
12. Verfahren nach einem der Ansprüche 1 bis 11, in welchem jeder elementare Bereich aus einem Pixel besteht.
13. Verfahren nach einem der Ansprüche 1 bis 11, in welchem jeder elementare Bereich ein Block aus Pixeln ist.
14. Verfahren nach einem der Ansprüche 1 bid 13, in welchem die Korrelation eine Phasenkorrelation ist.
15. Verfahren nach Anspruch 14, in welchem eine Phasenkorrelationsfunktion aus einer inversen Fouriertransformierung des Produkts von Fouriertransformierungen der beiden Bilder abgeleitet wird.
16. Verfahren nach einem der Ansprüche 1 bis 15, in welchem die Korrelationsfunktion vor der Bestimmung der Peak-Korrelationswerte einer zeitlichen Filterung unterzogen wird, um so Rauschpeaks zu vermindern.
17. Verfahren nach Anspruch 16, in welchem die zeitliche Filterung, immer wenn eine Shot-Änderung auftritt, zeitweise verhindert wird.
18. Verfahren nach einem der Ansprüche 1 bis 17, in welchem im zweiten Schritt für jeden Bewegungsvektor eine Fehlerfläche abgeleitet wird und für jedes Pixel der Bewegungsvektor, dessen Fehlerfläche den geringsten Wert zeigt, diesem Pixel zugeordnet wird.
19. Verfahren nach Anspruch 18, in welchem kein Bewegungsvektor zugeordnet wird, wenn der geringste Wert einem vorbestimmten Schwellwert übersteigt.
20. Verfahren nach Anspruch 18 oder 19, in welchem jede Fehlerfläche zur Herabsetzung des Rauscheffekts einer Ortsfilterung unterzogen wird.
21. Verfahren nach einem der Ansprüche 18 bis 20, in welchem die Fehlerflächen mit Faktoren multipliziert werden, die mit den zu den Flächen gehörenden Vektorlängen zunehmen.
22. Verfahren nach einem der Ansprüche 1 bis 21, umfassend den weiteren Schritt der Bewerkstelligung zeitlicher Interpolation der Bilder unter Verwendung der Bewegungsvektoren zur Interpolation der Positionen von sich bewegenden Objekten entsprechen mit dem Zeitpunkt des interpolierten Bildes zwischen dem vorausgehenden und darauffolgenden Bild, von dem es empfangen wird.
23. Verfahren nach Anspruch 22, in welchem die Bildinformation für nicht abgedeckten Hintergrund mit hiervon wegzeigenden Bewegungsvektoren in das interpolierte Bild vom darauffolgenden Bild hineingenommen wird und die Bildinformation des unsichtbaren Hintergrundes mit hiereinzeigenden Bewegungsvektoren in das interpolierte Bild vom vorausgehenden Bild hineingenommen wird.
24. Verfahren nach Anspruch 22 oder 23, in welchem mehrere Bilder zwischen aufeinanderfolgenden Eingangsbildern zur Erzeugung einer Zeitdehnungsbewegungssequenz oder Erhöhung der Teilbildrate für Darstellungszwecke interpoliert werden.
25. Verfahren nach einem der Ansprüche 1 bis 21, in welchem die Bilder zur Rauschverminderung einer zeitlichen Filterung unterzogen werden und die Filterung ausgedehnt wird, um die sich bewegenden Objekte, die von Teilbild zu Teilbild verschoben sind, entsprechend ihren zugeordneten Bewegungsvektoren einzuschließen.
26. Verfahren nach einem der Ansprüche 1 bis 21, in welchem die Bilder mit Bandbreitereduktion unter Einschluß von Bewegungsvektorinformation zur Rekonstruktion fehlender Abtastwerte sich bewegender Objekte übertragen oder aufgezeichnet werden.
27. Verfahren nach einem der Ansprüche 1 bis 22, in welchem schwarze und weiße Bilder durch automatisches Wiederholen zugeordneter Farben von Vollbild zu Vollbild an durch Bewegungsvektoren bestimmten Stellen koloriert werden.
28. Verfahren nach Anspruch 1, in welchem der zweite Schritt die folgenden Operationen umfaßt:
a) Ableiten eines elementaren Bereichs eines Bildes aus einem ersten von den beiden Bildern unter Verwendung jedes der im ersten Schritt bestimmten Bewegungsvektoren, um so einem Satz abgeleiteter elementarer Bereiche, jeweils entsprechend den Bewegungsvektoren, zu bilden;
b) Vergleichen jedes abgeleiteten elementaren Bereichs mit dem entsprechenden elementaren Fehler als eine Summe der Größen der Differenzen zwischen entsprecheden Punkten der elementaren Bereiche über die elementaren Bereiche;
c) die Selektion desjenigen der Bewegungsvektoren, der im kleinsten Vergleichsfehler resultiert, als die beste Anpassung für das Abeiten des elementaren Bereichs des zweiten Bildes aus dem elementaren Bereich des ersten Bildes ergebend;
d) Wiederholen der Operationen a), b), und c) für jeden der mehreren elementaren Bereiche.
29. Vorrichtung zur Verwendung im Verfahren nach Anspruch 1, aufweisend eine erste Einrichtung zum Korrelieren zweier Bilder zum Bestimmen der Korrelation als eine Funktion von Verschiebung, um so mehrere Peak-Korrelationswerte entsprechen den jeweiligen Bewegungsvektoren zu bestimmen; und eine zweite Einrichtung, die für jeden mehrerer elementarer Bereiche der Bilder prüft, welcher dieser Bewegungsvektoren die beste Anpassung für das Ableiten eines der Bilder aus dem anderen ergibt.
EP87900235A 1986-03-19 1986-12-23 Fernsehbild-bewegungsmessung Expired - Lifetime EP0261137B1 (de)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
GB8606809 1986-03-19
GB868606809A GB8606809D0 (en) 1986-03-19 1986-03-19 Tv picture motion measurement
GB8617320 1986-07-16
GB868617320A GB8617320D0 (en) 1986-03-19 1986-07-16 Tv picture motion measurement

Publications (2)

Publication Number Publication Date
EP0261137A1 EP0261137A1 (de) 1988-03-30
EP0261137B1 true EP0261137B1 (de) 1991-10-16

Family

ID=26290510

Family Applications (1)

Application Number Title Priority Date Filing Date
EP87900235A Expired - Lifetime EP0261137B1 (de) 1986-03-19 1986-12-23 Fernsehbild-bewegungsmessung

Country Status (6)

Country Link
US (1) US4890160A (de)
EP (1) EP0261137B1 (de)
JP (1) JP2609263B2 (de)
DE (1) DE3682068D1 (de)
GB (1) GB2188510B (de)
WO (1) WO1987005769A1 (de)

Families Citing this family (121)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0263112B1 (de) 1986-03-19 1992-06-03 British Broadcasting Corporation Videosignal-verarbeitung für bandbreitenverringerung
GB8626527D0 (en) * 1986-11-06 1986-12-10 British Broadcasting Corp 3d video transmission
GB8713454D0 (en) * 1987-06-09 1987-07-15 Sony Corp Television standards converters
DE3854171T2 (de) * 1987-06-09 1995-12-21 Sony Corp Bewertung von Bewegungsvektoren in Fernsehbildern.
GB8713455D0 (en) * 1987-06-09 1987-07-15 Sony Corp Television standards converters
GB2208056A (en) * 1987-08-10 1989-02-15 Philips Electronic Associated Television transmission system
DE3740066C1 (de) * 1987-11-26 1989-03-23 Messerschmitt Boelkow Blohm Einrichtung zur Auffindung und Lokalisierung eines Bildausschnittes
DE3888169T2 (de) * 1987-12-07 1994-06-09 British Broadcasting Corp Datenübertragung in aktiver bildperiode.
GB2250889B (en) * 1987-12-07 1992-09-09 British Broadcasting Corp Data transmission in active picture period
GB8728530D0 (en) * 1987-12-07 1988-01-13 British Broadcasting Corp Data transmission in active picture period
GB8814822D0 (en) * 1988-06-22 1988-07-27 British Broadcasting Corp Bandwidth reduction system for television
US5111511A (en) * 1988-06-24 1992-05-05 Matsushita Electric Industrial Co., Ltd. Image motion vector detecting apparatus
GB2222500A (en) * 1988-09-05 1990-03-07 Philips Electronic Associated Picture motion measurement
GB2222498A (en) * 1988-09-05 1990-03-07 Philips Electronic Associated Picture motion measurement
GB2222499A (en) * 1988-09-05 1990-03-07 Philips Electronic Associated Picture motion measurement
SG36672G (en) * 1988-09-13 1995-09-18 Thomson Brandt Gmbh System for transmitting a picture signal with a high temporal and spatial resolution
FR2638874B1 (fr) * 1988-11-10 1994-07-01 Thomson Csf Procede d'estimation du mouvement d'au moins une cible dans une suite d'images, et dispositif pour la mise en oeuvre de ce procede
FR2638873B1 (fr) * 1988-11-10 1990-12-14 Thomson Csf Procede pour recaler une image en rotation et dispositif pour la mise en oeuvre de ce procede
US5029000A (en) * 1988-12-23 1991-07-02 U.S. Philips Corp. High definition television system
JPH0771296B2 (ja) * 1989-03-20 1995-07-31 松下電器産業株式会社 動きベクトル検出装置
JPH07105949B2 (ja) * 1989-03-20 1995-11-13 松下電器産業株式会社 画像の動きベクトル検出装置および揺れ補正装置
GB2231750B (en) * 1989-04-27 1993-09-29 Sony Corp Motion dependent video signal processing
GB2231227B (en) * 1989-04-27 1993-09-29 Sony Corp Motion dependent video signal processing
GB2231225B (en) * 1989-04-27 1993-10-20 Sony Corp Motion dependent video signal processing
GB2231751B (en) * 1989-04-27 1993-09-22 Sony Corp Motion dependent video signal processing
GB2231748B (en) * 1989-04-27 1993-08-18 Sony Corp Motion dependent video signal processing
GB2231743B (en) * 1989-04-27 1993-10-20 Sony Corp Motion dependent video signal processing
GB2231749B (en) * 1989-04-27 1993-09-29 Sony Corp Motion dependent video signal processing
GB2231228B (en) * 1989-04-27 1993-09-22 Sony Corp Video signal to photographic film conversion
GB2231747B (en) * 1989-04-27 1993-07-07 Sony Corp Motion dependent video signal processing
GB2231745B (en) * 1989-04-27 1993-07-07 Sony Corp Motion dependent video signal processing
GB2231226B (en) * 1989-04-27 1993-09-22 Sony Corp Motion dependent video signal processing
GB2231752B (en) * 1989-04-27 1993-08-04 Sony Corp Motion dependent video signal processing
GB2231746B (en) * 1989-04-27 1993-07-07 Sony Corp Motion dependent video signal processing
GB2231744B (en) * 1989-04-27 1993-07-07 Sony Corp Motion dependent video signal processing
JP2861249B2 (ja) * 1989-05-11 1999-02-24 日本電気株式会社 画像変化検出方法および画像変化検出装置
US5640210A (en) * 1990-01-19 1997-06-17 British Broadcasting Corporation High definition television coder/decoder which divides an HDTV signal into stripes for individual processing
US5107348A (en) * 1990-07-11 1992-04-21 Zenith Electronics Corporation Temporal decorrelation of block artifacts
EP0466981B1 (de) * 1990-07-20 1997-02-26 Koninklijke Philips Electronics N.V. Vorrichtung zur Verarbeitung von Bewegungsvektoren
GB2246488B (en) * 1990-07-24 1994-04-20 Sony Corp Motion dependant video signal processing
JPH06501358A (ja) * 1990-09-20 1994-02-10 ブリティッシュ・ブロードキャスティング・コーポレーション ビデオ画像処理
GB2249907B (en) * 1990-11-15 1994-10-05 Sony Broadcast & Communication Format conversion of digital video signals
US5600731A (en) * 1991-05-09 1997-02-04 Eastman Kodak Company Method for temporally adaptive filtering of frames of a noisy image sequence using motion estimation
DE69229705T2 (de) * 1991-05-23 1999-12-23 Japan Broadcasting Corp Verfahren zur erkennung eines bewegungsvektors und vorrichtung dafür und system zur verarbeitung eines bildsignals mit dieser vorrichtung
DE69223925T2 (de) * 1991-05-24 1998-06-18 British Broadcasting Corp Videobildverarbeitung
GB9118782D0 (en) * 1991-09-03 1991-10-16 British Broadcasting Corp Video image processing
KR940008808B1 (ko) * 1991-10-05 1994-09-26 대우전자 주식회사 Hd-mac 방식 영상 신호 엔코더의 모드 결정 시스템
GB2262853B (en) * 1991-12-20 1995-07-19 Sony Broadcast & Communication Digital video signal processing
GB9204117D0 (en) * 1992-02-26 1992-04-08 British Broadcasting Corp Video image processing
GB2292040B (en) * 1992-03-24 1996-04-10 Sony Uk Ltd Motion analysis of moving images
GB2265783B (en) * 1992-04-01 1996-05-29 Kenneth Stanley Jones Bandwidth reduction employing a classification channel
FI91341C (fi) * 1992-05-20 1994-06-10 Salon Televisiotehdas Oy Menetelmä ja laite kohinan vaimentamiseksi HD-MAC-signaalista
US5335017A (en) * 1993-01-08 1994-08-02 Scott C. Harris Method for encoding transmitting and receiving high definition television signals using single variable independent equations representing each pixel over time
US5588067A (en) * 1993-02-19 1996-12-24 Peterson; Fred M. Motion detection and image acquisition apparatus and method of detecting the motion of and acquiring an image of an object
US5537529A (en) * 1993-04-22 1996-07-16 Apple Computer, Inc. Apparatus and method for creating versions of computer models and creating communications incorporating created versions therefrom
GB2278972B (en) * 1993-06-09 1997-06-04 Sony Uk Ltd Motion vector generation
GB9316153D0 (en) * 1993-08-04 1993-09-22 Avt Communications Ltd Image processing
GB9321372D0 (en) * 1993-10-15 1993-12-08 Avt Communications Ltd Video signal processing
GB9325914D0 (en) * 1993-12-18 1994-02-23 Kodak Ltd Detection of global translations between images
US5502489A (en) * 1994-01-03 1996-03-26 Daewoo Electronics Co., Ltd. Method for the motion adaptive spatial filtering of video signals in an image coding apparatus
GB9404190D0 (en) 1994-03-04 1994-04-20 Snell & Wilcox Limited Video Video signal processing
EP0697788A3 (de) 1994-08-19 1997-03-26 Eastman Kodak Co Adaptive und für globale Bewegung kompensierte Aufhebung des Zeilensprungverfahrens von aufeinenanderfolgenden Videobildern mit Nachbearbeitung
GB9419777D0 (en) * 1994-09-30 1994-11-16 Snell & Wilcox Ltd Moving image reproduction system
JP3268953B2 (ja) * 1995-02-27 2002-03-25 三洋電機株式会社 追尾領域設定装置，動きベクトル検出回路およびそれを用いた被写体追尾装置
DE19517357C1 (de) * 1995-05-11 1996-11-14 Ldt Gmbh & Co Verfahren und Vorrichtung zur Aufbereitung eines Videobildes
US5703970A (en) * 1995-06-07 1997-12-30 Martin Marietta Corporation Method of and apparatus for improved image correlation
FR2742901B1 (fr) * 1995-12-22 1998-02-13 Thomson Multimedia Sa Procede de correction d'estimation de mouvement dans des images a structures periodiques
JPH09212650A (ja) * 1996-02-05 1997-08-15 Sony Corp 動きベクトル検出装置および検出方法
TW303555B (en) * 1996-08-08 1997-04-21 Ind Tech Res Inst Digital data detecting method
US5878151A (en) * 1996-10-31 1999-03-02 Combustion Engineering, Inc. Moving object tracking
US6067367A (en) * 1996-10-31 2000-05-23 Yamatake-Honeywell Co., Ltd. Moving direction measuring device and tracking apparatus
DE19648963C1 (de) * 1996-11-26 1998-04-30 Siemens Ag Verfahren zur Bildcodierung eines digitalisierten Bildes, Verfahren zur Bilddecodierung eines digitalisierten Bildes und Anordnung zur Durchführung der Verfahren
WO1999003284A1 (en) * 1997-07-09 1999-01-21 Nippon Telegraph And Telephone Corporation Method for encoding and decoding moving vector, encoder and decoder for moving vector, and recording medium stored with encoding and decoding programs for moving vector
DE19730305A1 (de) * 1997-07-15 1999-01-21 Bosch Gmbh Robert Verfahren zur Erzeugung eines verbesserten Bildsignals bei der Bewegungsschätzung von Bildsequenzen, insbesondere eines Prädiktionssignals für Bewegtbilder mit bewegungskompensierender Prädiktion
DE69839761D1 (de) * 1997-10-15 2008-09-04 Nxp Bv Bewegungsschätzung
US6295377B1 (en) * 1998-07-13 2001-09-25 Compaq Computer Corporation Combined spline and block based motion estimation for coding a sequence of video images
US6870564B1 (en) * 1998-10-02 2005-03-22 Eastman Kodak Company Image processing for improvement of color registration in digital images
US6532264B1 (en) 2000-03-27 2003-03-11 Teranex, Inc. Processing sequential video images to detect image motion among interlaced video fields or progressive video images
KR100840133B1 (ko) * 2000-05-18 2008-06-23 코닌클리케 필립스 일렉트로닉스 엔.브이. Ｍｃ 업 컨버젼에서 헤일로를 감소시키는 움직임 추정기
KR100857731B1 (ko) * 2001-02-21 2008-09-10 코닌클리케 필립스 일렉트로닉스 엔.브이. 움직임 추정을 용이하게 하는 방법 및 장치
US7242717B2 (en) * 2001-06-08 2007-07-10 Sharp Laboratories Of America, Inc. Wavelet domain motion compensation system
US7167602B2 (en) * 2001-07-09 2007-01-23 Sanyo Electric Co., Ltd. Interpolation pixel value determining method
KR100396558B1 (ko) * 2001-10-25 2003-09-02 삼성전자주식회사 적응 움직임 보상형 프레임 및/또는 레이트 변환 장치 및그 방법
WO2003039161A2 (en) * 2001-10-29 2003-05-08 Parthusceva Ltd. Method and apparatus for motion estimation in a sequence of digital images
US7092584B2 (en) * 2002-01-04 2006-08-15 Time Warner Entertainment Company Lp Registration of separations
US7127125B2 (en) * 2002-01-04 2006-10-24 Warner Bros. Entertainment Inc. Registration of separations
US6947607B2 (en) * 2002-01-04 2005-09-20 Warner Bros. Entertainment Inc. Reduction of differential resolution of separations
US6885706B2 (en) * 2002-01-05 2005-04-26 Aiptek International Inc. Method for promoting temporal resolution of sequential images
US7623719B2 (en) * 2003-09-26 2009-11-24 The Regents Of The University Of California Video encoding methods and devices
KR20050081730A (ko) * 2004-02-16 2005-08-19 엘지전자 주식회사 움직임 보상 기반의 영상 신호 프레임율 변환 방법
WO2005093654A2 (en) 2004-03-25 2005-10-06 Fatih Ozluturk Method and apparatus to correct digital image blur due to motion of subject or imaging device
US10721405B2 (en) 2004-03-25 2020-07-21 Clear Imaging Research, Llc Method and apparatus for implementing a digital graduated filter for an imaging apparatus
US9826159B2 (en) 2004-03-25 2017-11-21 Clear Imaging Research, Llc Method and apparatus for implementing a digital graduated filter for an imaging apparatus
US20060034531A1 (en) * 2004-05-10 2006-02-16 Seiko Epson Corporation Block noise level evaluation method for compressed images and control method of imaging device utilizing the evaluation method
US8126085B2 (en) * 2004-11-22 2012-02-28 Intel Corporation Method and apparatus to estimate channel tap
EP1732329A1 (de) * 2005-06-06 2006-12-13 STMicroelectronics S.r.l. Verfahren und Vorrichtung zur Bewegtbildercodierung
TW200811204A (en) * 2006-05-19 2008-03-01 Nissan Chemical Ind Ltd Hyper-branched polymer and process for producing the same
US8553758B2 (en) 2007-03-02 2013-10-08 Sony Corporation Motion parameter engine for true motion
US8565309B2 (en) 2007-08-28 2013-10-22 Surapong Lertrattanapanich System and method for motion vector collection for motion compensated interpolation of digital video
US8861603B2 (en) 2007-08-28 2014-10-14 Samsung Electronics Co., Ltd. System and method for motion vector collection based on K-means clustering for motion compensated interpolation of digital video
US20090175539A1 (en) * 2008-01-09 2009-07-09 Authorizer Technologies, Inc. Method and system for swipe sensor image alignment using fourier phase analysis
US8571114B2 (en) 2008-01-11 2013-10-29 Zoran (France) S.A. Sparse geometry for super resolution video processing
US8363728B2 (en) 2008-04-18 2013-01-29 Sony Corporation Block based codec friendly edge detection and transform selection
US8139883B2 (en) 2008-07-29 2012-03-20 Sony Corporation System and method for image and video encoding artifacts reduction and quality improvement
US20100103323A1 (en) * 2008-10-24 2010-04-29 Ati Technologies Ulc Method, apparatus and software for determining motion vectors
US9699475B2 (en) 2009-02-12 2017-07-04 Qualcomm Incorporated Video sequence analysis for robust motion estimation
US9042680B2 (en) 2009-02-12 2015-05-26 Zoran (France) S.A. Temporal video interpolation method with 2-frame occlusion handling
GB2476298A (en) * 2009-12-18 2011-06-22 Snell Ltd Changing the temporal sample rate of a motion image sequence
US8488007B2 (en) 2010-01-19 2013-07-16 Sony Corporation Method to estimate segmented motion
US8285079B2 (en) 2010-03-19 2012-10-09 Sony Corporation Method for highly accurate estimation of motion using phase correlation
US8792559B2 (en) 2010-10-26 2014-07-29 Sony Corporation Method to improve accuracy and reliability of motion estimated with phase correlation
GB201117191D0 (en) 2011-10-04 2011-11-16 Imagination Tech Ltd +Detecting image impairments in an interpolated image
US9177390B2 (en) * 2012-01-09 2015-11-03 Intel Corporation Optical flow measurement divide
GB2502047B (en) 2012-04-04 2019-06-05 Snell Advanced Media Ltd Video sequence processing
US9241128B2 (en) 2013-02-14 2016-01-19 Warner Bros. Entertainment Inc. Video conversion technology
GB2513112B (en) * 2013-04-08 2020-01-08 Snell Advanced Media Ltd Video sequence processing
US10609328B2 (en) 2013-05-13 2020-03-31 Grass Valley Limited Video processing using a one-dimensional control function to control processing time and error
GB2514334B (en) * 2013-05-13 2020-04-22 Snell Advanced Media Ltd Video processing
GB2525137A (en) * 2013-08-30 2015-10-21 Amberfin Ltd Conversion of video frame rate
JP6412819B2 (ja) * 2015-03-30 2018-10-24 株式会社東芝 画像処理装置、画像処理方法、および画像処理プログラム
US10416275B2 (en) * 2016-05-12 2019-09-17 Isolynx, Llc Advanced tools for an object tracking system

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5215949B1 (de) * 1970-06-15 1977-05-06
GB2050752B (en) * 1979-06-07 1984-05-31 Japan Broadcasting Corp Motion compensated interframe coding system
CA1175557A (en) * 1981-06-01 1984-10-02 Akira Hirano Predictive coding system for television signals
JPS61118085A (ja) * 1984-11-14 1986-06-05 Nec Corp 画像信号の符号化方式およびその装置
JPS61200789A (ja) * 1985-03-04 1986-09-05 Kokusai Denshin Denwa Co Ltd <Kdd> 画面上の物体の動きベクトル検出方式
US4727422A (en) * 1985-06-03 1988-02-23 Picturetel Corporation Method and apparatus for efficiently communicating image sequence having improved motion compensation
AU579550B2 (en) * 1985-06-10 1988-11-24 Nec Corporation Movement compensation predictive encoder for a moving picture signal with a reduced amount of information
CA1252569A (en) * 1985-09-13 1989-04-11 Akihiro Furukawa Method and device of coding a digital video signal for reproduction of pictures with a high quality even upon occurrence of a drastical change between the pictures

Also Published As

Publication number Publication date
GB2188510A (en) 1987-09-30
DE3682068D1 (de) 1991-11-21
JP2609263B2 (ja) 1997-05-14
JPH01500238A (ja) 1989-01-26
EP0261137A1 (de) 1988-03-30
US4890160A (en) 1989-12-26
WO1987005769A1 (en) 1987-09-24
GB2188510B (en) 1989-11-15
GB8702582D0 (en) 1987-03-11

Similar Documents

Publication Publication Date Title
EP0261137B1 (de) Fernsehbild-bewegungsmessung
US4864393A (en) Motion vector estimation in television images
US4862259A (en) Motion vector reduction in television images
Castagno et al. A method for motion adaptive frame rate up-conversion
US5134480A (en) Time-recursive deinterlace processing for television-type signals
EP0294958B1 (de) Bewegungskompensierte Interpolation von digitalen Fernsehbildern
EP0294961B1 (de) Auswahl eines Bewegungsvektors in Fernsehbildern
US5012337A (en) Motion dependent video signal processing
US4862260A (en) Motion vector processing in television images
US5012336A (en) Motion dependent video signal processing
EP0395263A2 (de) Bewegungsabhängige Videosignalverarbeitung
KR0141705B1 (ko) 움직임 벡터의 계산 방법 및 장치
US4974084A (en) Method and apparatus for picture motion measurement
US5485224A (en) Motion compensated video signal processing by interpolation of correlation surfaces and apparatus for doing the same
EP0367310B1 (de) Bild-Bewegungsmessung
EP0659021B1 (de) Detektion von globaler Translation zwischen Bildern
Biswas et al. A novel motion estimation algorithm using phase plane correlation for frame rate conversion
US5226114A (en) Television pictures
GB2205713A (en) Motion compensated interpolation of digital television images
Sabri Movement compensated interframe prediction for NTSC color TV signals
GB2222498A (en) Picture motion measurement
Clarke Basic principles of motion estimation and compensation
Nojiri et al. Motion compensated standards converter for HDTV
Oistamo et al. Reconstruction of quincunx-coded image sequences using vector median
Csillag et al. Motion-compensated frame rate conversion using an accelerated motion model

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR NL

17P Request for examination filed

Effective date: 19880315

17Q First examination report despatched

Effective date: 19900711

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR NL

REF Corresponds to:

Ref document number: 3682068

Country of ref document: DE

Date of ref document: 19911121

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed
PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20051216

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20051230

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20060213

Year of fee payment: 20

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20061223

NLV7 Nl: ceased due to reaching the maximum lifetime of a patent

Effective date: 20061223