JPH0530411A - Picture motion vector detector and picture jiggling correction device - Google Patents

Abstract

PURPOSE:To reduce the influence of an error in an error vector caused in terms of a probability and to reduce a noise component superimposed on a motion vector by finding an entire pattern of an inputted picture signal or a partial motion vector. CONSTITUTION:The correction device is provided with a means detecting a motive vector from motion vector detection areas 9a-9e and a vector selection means obtaining a motion vector to obtain the entire pattern of an inputted picture signal or a partial motion vector. That is, the motion vector detected at a current field is added to the motion vector obtained precedingly to obtain the set of motion vectors, and a representative set is outputted as the motion vector of the entire pattern or the partial picture. Thus, up to the case that number of motion vectors less than a half of number of elements of the set are an error vectors is not effected of the error vectors and an accurate motion vector is detected. Moreover, the noise component superimposed on the motion vector is sufficiently reduced since the number of elements of the set is large.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image processing for image motion vector detection and image shake correction.

[0002]

2. Description of the Related Art Recently, as an image motion vector detecting and smoothing signal processing apparatus, Japanese Patent Laid-Open No. 61-107886 has been proposed.
It is shown in the publication.

A conventional image motion vector detecting and smoothing signal processing apparatus will be described below. FIG. 5 is a block diagram of a motion vector detecting device using a conventional image motion vector smoothing circuit. Further, FIG. 6 is a schematic diagram for explaining the operation. In FIG. 5, reference numeral 1 is an image signal input terminal. A representative point memory 2 stores signals of pixels corresponding to representative points in the input image signal. A difference / absolute value converter 3 outputs the absolute value of the difference between the inputs.
Reference numeral 4 is a cumulative adder, and 5 is a minimum point detection circuit. 6 is a selection means. Reference numeral 7 is a motion vector smoothing means. 4a and 4b, 8 is a screen, 9a to 9d are motion vector detection areas, and 10a to 10d are motion vectors detected from the respective areas.

The operation of the conventional image motion vector detecting apparatus configured as described above will be described below.
First, a temporally continuous image signal is input to the input terminal 1. In the representative point memory 2, first, a plurality of representative points are determined in advance in the respective detection areas 9a to 9d of the screen 8, and the signal of the pixel corresponding to the position of the representative point in the input image signal is stored. The difference / absolute value converter 3 uses the absolute value | ΔL | of the difference between the signal at the position of the representative point of the previous field and the signal at the position deviated (horizontal direction i, vertical direction j) from the representative point of the current field. Find (i, j). The cumulative adder 4 has a table corresponding to the deviation (i, j) for each of the detection areas 9a to 9d, and cumulatively adds the signal from the difference / absolute value converter 3 for each deviation (i, j). And shift this (i, j)
The correlation value Σ | ΔL | (i, j) in In the minimum point detection circuit 5, the deviation (i ′, which gives the minimum value of the correlation value,
j ′) is detected and output as the motion vectors 10a to 10d of the detection areas 9a to 9d, respectively. The selection means 6 obtains the motion vector of the entire screen from the motion vectors 10a to 10d of the respective detection areas. The motion vector smoothing means 7 outputs a temporally smoothed value of the motion vector obtained for each field as shown in the following equation.

Vo (k) = W (k) × Vi (k) + (1-W (k))
× Vo (k-1) (where Vi (k) is the input motion vector of the kth field, Vo (k) is the output motion vector of the kth field, and W (k) is the kth field. This is a weighting coefficient for the motion vector. By smoothing the motion vector in the time axis direction, if the motion vector of the entire screen detected by the selection means 6 is an erroneous vector, the motion vector due to the erroneous vector The temporal discontinuity of is reduced to improve the visual characteristics when motion compensation is performed.

It should be noted that W (k) is set to 1 in the case of a scene change or the like to follow a rapid change, and is set to about ⅛ in a steady state to enhance the noise elimination effect.

However, the following problems occur when the motion vector detecting device using the conventional motion vector smoothing circuit for an image is used, for example, to correct image shake.

First, in the correction of image shake, a direction in which an image signal is stored in a memory for one field, a motion vector of the entire screen from one field before that field is detected during that field, and the motion is corrected. Is done by shifting the read position from the memory for each field.

Therefore, the read position from the memory is basically determined by the integrated value of the motion vector detected. Therefore, a shake correction operation by the motion vector detecting device using the conventional image motion vector smoothing circuit will be described below with reference to FIGS. FIG. 7a shows the error when the motion vector of the entire screen detected by the selection means 6 in a certain field is an erroneous vector, and FIG. 7b shows the output of the motion vector smoothing means 7 at that time. 7c shows the error that occurs in FIG.
It shows the integration error when integrating it.

In a certain field, the motion vector of the entire screen detected by the selection means 6 is an erroneous vector,
When an error 20 occurs as shown in FIG. 7a, the error at the output of the motion vector smoothing means 7 is as shown in FIG. 7b.
Although the temporal discontinuity of the motion vector due to the false vector is reduced, the error of the generated false vector is only smoothed over several fields, and thus the error accumulated in the integrated value of the detected motion vector. Becomes as shown in FIG. 7c and its final integration error value 21 is not reduced. Therefore, in shake correction, the error of the generated error vector is directly affected, and a malfunction occurs such that the corrected image shifts in the direction opposite to this error.

Further, since the smoothing causes a delay time in the motion vector, in the shake correction screen, only the low frequency component of the shake is corrected and the high frequency component of the shake remains uncorrected, which is visually unnatural. Will be felt.

Furthermore, when there is a moving object on the screen that is about half the size of the screen, the motion vector outputs the motion of the moving object. When used for image shake correction, the moving object is stationary and the background is opposite to that of the moving object. It was moving in the direction of, and the image was very unnatural.

[0013]

As described above, the conventional motion vector detecting device using the motion vector smoothing circuit for an image has the drawbacks of malfunction and visual unnaturalness when used for the correction of image shake. Had.

The present invention solves the above-mentioned conventional problems by effectively reducing the influence of the error of an error vector that is stochastically generated, reducing the noise component superimposed on the motion vector, and delaying the delay time. It is a motion vector detection device for an image that suppresses the occurrence of noise.

Further, the motion vector detecting apparatus of the present invention can detect the background motion vector without using the motion vector detected from the moving object even when a moving object of about half the size of the image is present. It is a possible image motion vector detection device.

Further, the image shake correction apparatus of the present invention uses the above-described image motion vector detection apparatus to effectively reduce the effects of erroneous vectors and noise components, and to use a motion vector due to a moving object for image shake correction. An object of the present invention is to provide an image shake correction device that can obtain a good corrected image that does not cause unnaturalness to be visually sensed without using it.

[0017]

An image motion vector detection device of the present invention has a plurality of motion vector detection areas for the entire screen or a part of an input image signal, and detects a motion vector from each area. In the motion vector of each area newly detected in the current field, the motion vector of each area detected in the past less than the number of motion vectors of the newly detected area, or determined in the past. A motion vector of the whole screen or a part is added as a candidate vector, and a vector selecting means for obtaining a motion vector of the whole screen or a part by selecting a representative value from a set of candidate vectors is inputted. It is configured to obtain a motion vector of the entire screen or a part of the image signal.

[0018]

With this configuration, the motion vectors detected in the current field are added to the motion vectors obtained earlier than the current number, which is smaller than the number, to form a set of motion vectors, and a representative value of the set. Is output as a motion vector of the whole screen or a part thereof. Therefore, an accurate motion vector can be detected without being affected by the error vector until less than half of the motion vectors of the set are error vectors. In particular, the number of elements in the set can be increased, and even if the motion vector has a high probability of being an erroneous vector, an accurate motion vector can be detected without being affected by the erroneous vector. Also, the noise component superimposed on the motion vector can be sufficiently reduced because the number of elements in the set is large.

In addition, in the case of the present invention, even when the majority of the motion vectors obtained in the current field indicate the motion of the moving object, the motion vector is determined in consideration of the background motion vector obtained in the past. Therefore, it is possible to effectively eliminate the case where a majority of the detected motion vectors are affected by a moving object or the effect of a large moving object.

When the number of motion vectors of each area obtained in the current field is less than a predetermined number, the motion vector predicted from the motion vector detected from the current field is added to the candidate vector to obtain the candidate vector. The number of set elements is set to a predetermined number or more. Then, a smaller number of motion vectors obtained before the current time are further added to form a set of candidate vectors, and a representative value in this set is output. Therefore, even if a delay time does not occur in the detected vector and the probability that the motion vector is an erroneous vector is high, an accurate motion vector can be detected with little influence of the erroneous vector.

Therefore, on the shake-corrected screen, it is possible to realize the shake correction of the image by effectively eliminating the influence of noise of the error vector and the motion vector. Further, there is no problem that the high frequency component of the shake remains without being corrected. In addition, if the number of motion vectors obtained in the current field is less than the predetermined number, a delay time may occur due to an erroneous vector. However, this is a case where a moving object is violently moving in the image or a case where noise is remarkable, and therefore the influence of the delay time due to a slight error vector hardly causes a visual unnaturalness.

[0022]

1a is a block diagram of an image motion vector detecting apparatus according to a first embodiment of the present invention. In FIG. 1a, 1 is an image signal input terminal. A representative point memory 2 stores signals of pixels corresponding to representative points in the input image signal. A difference / absolute value converter 3 outputs the absolute value of the difference between the inputs. Reference numeral 4 is a cumulative adder, 5 is a minimum point detector, and 11 is an intermediate value selecting means. Reference numeral 12 is a reliability determining means, and 30 is a memory.

The operation of the image motion vector detecting apparatus of the first embodiment of the present invention constructed as above will be described below. First, a temporally continuous image signal is input to the input terminal 1. In the representative point memory 2, as shown in FIG. 1B, first, a plurality of representative points are determined in advance in each of the detection areas 9a to 9e of the screen 8, and the signal of the pixel corresponding to the position of the representative point in the input image signal is stored. To do. The difference / absolute value converter 3 uses the absolute value | ΔL | of the difference between the signal at the position of the representative point of the previous field and the signal at the position deviated (horizontal direction i, vertical direction j) from the representative point of the current field. Find (i, j). The cumulative adder 4 has a table corresponding to the deviation (i, j) for each of the detection areas 9a to 9e,
The signals from the difference / absolute value converter 3 are cumulatively added for each deviation (i, j), and the correlation value Σ | at the deviation (i, j) is added.
Let ΔL | (i, j). The minimum point detector 5 detects the deviation (i ', j') that gives the minimum value of the correlation value, and outputs it as the motion vectors 10a to 10e of the detection areas 9a to 9e, respectively. Further, the reliability determination means 12 determines the reliability of the motion vector of each area based on the magnitude of Σ | ΔL | (i ', j'). When Σ | ΔL | (i ', j') is larger than a predetermined value, it is considered as unreliable. If it is smaller than a predetermined value, it is considered to be reliable. The vector determined to be reliable is output to the intermediate value selection unit 11. Memory 30
Stores the motion vectors of multiple outputs of the past field. The intermediate value selection means 11 obtains the motion vector of the entire screen from the outputs from the reliability determination means 12 and the memory 30 by the following method.

First, when the number of elements of the set of motion vectors (hereinafter referred to as detection vectors) obtained from each detection region which is determined to be reliable and which is input from the reliability determination means 12 in the current field is 5, Add the motion vector of the output of the field and the field before field to the element of the set of candidate vectors, set the number of elements to 7 and set the intermediate value of that set (position each element of the set in the middle of the column arranged in order of magnitude) If the number of elements to be set and the number of elements in the set are even, the average value of the values of the two elements closest to the center position of the column is output. However, the horizontal and vertical values of the vector are considered independently, and the median value of each set is used as the output vector.

If the number of detection vectors in the current field is 4, the previous field output, the output of the previous previous field, and the motion vector one field before are added to the set of candidate vectors, and the number of elements of the set is 7. Then, the intermediate value is output.

When the number of elements of the set of detection vectors in the current field is 3, the motion vectors of the previous field output and the output of the previous previous field are added to the set of candidate vectors, and the number of elements of the set is set to 5 and then Output the intermediate value.

When the number of detection vectors in the current field is 2, the motion vector output in the previous field is added to the set of candidate vectors, the number of elements in the set is set to 3, and the intermediate value is output.

Further, the number of elements of the motion vector set of the detection area determined to be reliable in the current field is 1
If, the value is output.

Further, the number of elements of the motion vector set in the detection area determined to be reliable in the current field is 0.
In the case of, the motion vector of the output of the previous field is multiplied by a coefficient of 1 or less and output.

As described above, the number of elements in the set of candidate vectors is increased more than the number of detected vectors obtained in the current field (the number to be increased is smaller than the number obtained in the current field). , And outputs the intermediate value. By doing this, even if there are a small number of false vectors in the detected vector, the elements of the remaining candidate vectors are accurate, so the correct vector value is selected and output without being affected by the false vector. it can. In particular, by increasing the number of elements in the set of candidate vectors, the probability of being unaffected by erroneous vectors increases, and stable motion vector detection is possible.

Further, with respect to the random noise included in the motion vector, it is possible to effectively reduce the random noise by increasing the number of elements of the candidate vector.

Further, even when a moving object larger than ½ of the screen enters the image and about half of the detection vectors obtained in the current field become motion vectors indicating the movement of the moving object, By adding the motion vector obtained in the field to the elements of the set of detection vectors obtained in the current field and increasing the number of elements in the candidate vector, it is possible to output the background motion vector in which the moving object has not entered.

When the number of detection vectors obtained in the current field is 5 to 2, the motion vectors output from the past fields, which are smaller than the detection vectors obtained in the current field, are added to the set of candidate vectors. , Since the number of elements of the set is increased and the value of the intermediate value in this set is output, the probability that delay time will occur in the output motion vector is small.

Considering, for example, the case where the number of detection vectors determined to be reliable in the current field is 2, this 2
When two detection vectors are both correct motion vectors in the current field, they have the same value. Therefore, the motion vector of the output of the previous field is added to this set of candidate vectors, and the intermediate value of the set is added. Even if is selected, the correct motion vector value in the current field is selected, and no delay time occurs. Further, for example, when one of the two detection vectors is an erroneous vector greatly separated from the correct motion vector in the current field, the motion vector of the output of the previous field is added to the set of the two detection vectors, and By selecting an intermediate value of the set, either the correct motion vector in the current field or the output motion vector of the previous field close to that value is output, so that the influence of the erroneous vector can be removed.

FIG. 2 is a block diagram of an image motion vector detecting apparatus according to the second embodiment of the present invention. In FIG.
Reference numeral 1 is an image signal input terminal, 2 is a representative point memory, 3 is a difference / absolute value converter, 4 is a cumulative adder, 5 is a minimum point detection circuit, 11 is an intermediate value selecting means, and 12 is reliability determining means. ,
Reference numeral 30 is a memory, and 31 is a motion vector predictor. In FIG. 2, the same elements as those in FIG. 1 are designated by the same reference numerals. The operation of the image motion vector detecting device of the present invention will be described below.

First, the configuration and operation from the image signal input terminal 1 to the representative point memory 2, the difference / absolute value converter 3, the cumulative adder 4, the minimum point detection circuit 5, and the reliability judgment means 12 are the first. It is similar to the embodiment. Next, the reliability determination means 1
2 determines the reliability of the motion vector of each area, and inputs the detection vector determined to be reliable to the intermediate value selecting means 11. (This is the first input.) When the number of detection vectors is equal to or smaller than a predetermined number (for example, 3), the motion vector of the current field is predicted using the vector predictor 31. The prediction vector obtained from the predictor 31 is also the first input. Further, similarly to the first embodiment, the motion vector obtained in the past is used as the second input of the intermediate value selecting means, both the first and second inputs are set as a candidate vector, and the intermediate value is selected from them. Select and determine the motion vector of the image.

For example, when the number of detected vectors is 5 and 4, the same method as in the first embodiment is used and the vector predictor 31 is not used. When the number of detected vectors is 3 and 2, the vector predictor 31 obtains the average value of the existing vectors and sets this as the predicted value. To the set of candidate vectors having the predicted value and the obtained vector as elements, the motion vectors obtained and output in the past less than the number of elements are added, and the intermediate value is output as the motion vector of the screen. The intermediate value is selected by the intermediate value selecting means 11, and the past motion vector is selected from the memory 30. When the number of detection vectors is 3, the number of prediction vectors is 1, and the number of past output vectors to be added is 3. Then, the intermediate value is determined as the current output vector from the seven set elements of this candidate vector. When the number of detection vectors is 2, the number of prediction vectors is 1, and the number of past output vectors to be added is 2. Then, the intermediate value of the five set elements of the candidate vector is determined as the current output vector.

As described above, according to the second embodiment of the present invention, even when the number of detection vectors obtained in the current field is small, the number of set elements of the candidate vector is increased and the past output vector is the candidate vector. The proportion contained in the elements of the set is less than the majority.

Therefore, even when the number of detected vectors is small, the number of elements of the candidate vector set can be made large, and the noise can be efficiently reduced even when the detected vector has noise.

The probability that the output motion vector obtained additionally will be delayed is very small. For example, considering the case where the number of detection vectors is 2, when these two motion vectors are both correct motion vectors in the current field, they have the same value to each other, and thus their average value is also a correct value. Become. Then, even if the motion vector of the output of the previous field is added to the candidate vector set and the intermediate value of the set is selected, the correct motion vector value in the current field is selected, and no delay time occurs. Further, for example, when one of the two candidate vectors is an erroneous vector greatly separated from the correct motion vector in the current field, the average value thereof also contains an error. Then, by adding the motion vectors of the past two outputs to the set of these three candidate vectors and selecting the intermediate value of the set, the correct motion vector in the current field or the previous field close to that value is selected. Since any one of the output motion vectors of is output, a time delay may occur, but the influence of the error vector can be removed.

As described above, in the second embodiment of the present invention, when the number of vectors obtained in the current field becomes small, the noise component of the output motion vector is suppressed and the error vector The present invention realizes a motion vector detection device capable of effectively removing the influence.

FIG. 3 is a block diagram of an image shake correcting apparatus according to a third embodiment of the present invention. In FIG. 3, 1 is an image signal input terminal, 2 is a representative point memory, 3 is a difference / absolute value converter, 4 is a cumulative adder, 5 is a minimum point detection circuit, 11 is an intermediate value selecting means, and 12 is reliability. Determination means, 30 is a memory,
Reference numeral 31 is a vector predictor, 15 is an image memory, 17 is a reading control means, and 18 is an interpolation enlargement means. In FIG. 3, elements common to those in FIGS. 1 and 2 are given the same numbers. Further, the representative point memory 2, the difference / absolute value converter 3, the cumulative adder 4, the minimum point detection circuit 5, the intermediate value selection means 11,
Reliability determination means 12, memory 30, vector predictor 3
Motion vector detecting means 1
6

The operation of the image shake correction apparatus constructed as above will be described below. First, an image signal is input from the image signal input terminal 1. The image memory 15 stores this image signal for one field. The motion vector detecting means 16 has the same configuration and operation as the motion vector detecting device of the first or second embodiment of the present invention or the first or FIG.
A motion vector of the entire screen in that field, that is, a motion vector indicating how much the entire screen has moved from the previous field to the current field is detected and output to the read position control means 17.

The read position control means 17 shifts the read position from the memory with respect to the read position of the previous field in the direction of canceling the motion of the input motion vector, that is, in the direction of eliminating the motion. Also, centering and clipping are performed so that the read position does not go out of the screen and the result is output to the interpolation enlarging means 18. Here, the read range from the image memory is a range obtained by reducing the size of the original image by a certain percentage.

The interpolation enlarging means 18 reads out the image signal of the portion designated by the read-out position from the image memory 15, further enlarges and interpolates the image signal of the read-out portion, and enlarges the reduced range. The image signal of one screen is output.

In this way, the motion vector detecting means 16
The motion (sway) of the image obtained from the image memory 15,
By using the read-out position control means 17 and the interpolation enlarging means 18, an image shake correction apparatus for canceling the image shake and eliminating the image shake is realized.

FIGS. 4a and 4b are schematic diagrams of screens for explaining the operation in more detail. 4a and 4b, 8 is a screen, 9a, 9b, 9c, 9d and 9e are a plurality of motion vector detection areas of the motion vector detecting means 16, and 10a, 10b, 10c, 10d and 10e are respectively. It is a motion vector detected from each area. 1
9a, 19b, and 19c are moving objects in the screen that move differently from the motion of the entire screen of the image.

FIG. 4a shows the case where there is no moving object on the screen. Here, in the motion vector detecting means 16,
The detection vectors 10a, 10b, 10c, 10d, and 10e from the detection regions of a, 9b, 9c, 9d, and 9e are all determined to be reliable. For these vectors,
Image motion vector 2 obtained in the past from memory 30
Then, the median value among them is output as the motion vector of the screen. Therefore, the detection vector 10
Even if up to three of a, 10b, 10c, 10d, and 10e are erroneous vectors, the influence is removed to detect a correct motion vector, and the shake of the image is canceled according to this vector, and good correction is performed. The screen is obtained.

When there is no moving object in the screen as shown in FIG. 4a, the detection vectors 10a to 10e are very unlikely to be erroneous vectors, and the probability that they are erroneous vectors at the same time is further reduced. Therefore, it can be said that there is no possibility that a delay time will occur in the motion vector output from the motion vector detection means 16, and it can be said that the correction residual due to the delay time does not occur in the shake correction screen, and good image shake correction is performed. Is possible. Moreover, since the number of elements of the candidate vector can be increased to 7, the influence of noise can be significantly reduced even when noise is superimposed on the detection vector.

FIG. 4b shows a case where many moving objects are present in the screen. Since the moving objects 19a, 19b and 19c are used here, the motion vector detecting means 16 uses 9a, 9b,
Detection vector 10 from each detection region of 9c, 9d, and 9e
Of the a to 10e, only 10d and 10e are determined to be reliable. At this time, the prediction means 31 causes 10d, 10
Let the average value of the detection vector of e be a prediction vector. From the detection vectors 10d and 10e and the prediction vectors obtained from the two, and the output motion vectors of the previous field and the previous field obtained from the memory 30, a set of candidate vectors having 5 elements is obtained, The intermediate value is obtained by the intermediate value selecting means 11 and is output as a motion vector of the screen. Even when the number of detection vectors obtained from each detection region is two as described above, it is possible to increase the number of elements of the candidate vector to 5, so that the noise component superimposed on each detection vector can be efficiently reduced. it can. Further, in this case, since the majority of the elements of the set of candidate vectors that have no time delay, the output motion vector has no time delay, and good image shake correction can be performed.

Further, under the above-described conditions of FIG. 4b,
If the detection vector 10d is an erroneous vector, it is processed as follows. Detection vectors 10d and 10
The average vector of d and 10e contains a large error. There is no error between the detection vector 10e and the past two output vectors. Therefore, by selecting an intermediate value by the intermediate value selecting means 11 and outputting it, a motion vector containing no error vector can be output, and a good correction screen can be obtained. However, under this condition, a time delay may occur, and a correction residual of the high frequency component of the shake due to the delay time may appear on the shake correction screen. However, when there are many moving objects in the screen as shown in FIG. 4B, a slight correction residual on the shake correction screen is hard to be visually perceived and rarely feels unnatural.

Under the condition of FIG. 4b, the operation will be described when the detection vectors 10a to 10c are erroneously determined to be reliable by the reliability determining means 12, for example. For the five detection vectors 10a to 10e, two output vectors (nearly 10d or 10e) from the memory 30 in the past (previous field and previous field) are output.
Is added to the set of candidate vectors, and the element is set to 7. The intermediate value selecting means 11 selects an intermediate value from the seven elements. By this selection, the detection vector 10a
The vectors 10 to 10c are significantly different from the other candidate vectors and are not selected, and one vector is selected from the detection vectors 10d and 10e or the past two output vectors close to these two vectors, and the motion vector detecting means is selected. It is output as 16 outputs. The input image is controlled in accordance with the output of the motion vector detecting means 16 to cancel the shake of the input image and realize the shake correction of the image. As described above, even when an erroneous vector (which is not suitable for image shake correction) is included in the majority of the detected vectors, it is possible to cancel the influence and cancel the shake according to the image shake (background motion). .

The probability that the output motion vector will be delayed is not limited as long as the detected vector has no error. For example, considering the case where the number of detection vectors is 2, this 2
When the two motion vectors are both correct motion vectors in the current field, they have the same value to each other, and the average value is also correct. Then, even if two motion vectors output from the previous field are added to the set and an intermediate value of the set is selected, the correct motion vector value in the current field is selected and no delay time occurs. Further, for example, when one of the two motion vectors is an erroneous vector that is far from the correct motion vector in the current field, the average value thereof also contains an error.
Then, by adding the motion vectors of the past two outputs to the set of these three motion vectors and selecting the intermediate value of the set, it is determined whether the correct motion vector in the current field.
Since one of the output motion vectors of the previous field that is close to that value is output, there is a possibility that a time delay will occur, but it is possible to remove the influence of the erroneous vector.

As described above, according to the present embodiment, when the number of detection vectors of the area determined to be reliable in the current field by the intermediate value selection means 11 is less than the predetermined number,
The number of set elements of the candidate vector is increased by using the vector predicting means, and the past output vector is also used so that the ratio of the past output vector included in the set element of the candidate vector is less than the majority. Further increase the number of set elements. Therefore, even when the number of detected vectors is small, the number of set elements of the candidate vector can be set large, and even when there is noise in the detected vector, it is possible to reduce the noise efficiently, and the fluctuation of the image is less affected by the noise. Correction can be realized.

Further, even when the majority of the detected vectors have an erroneous vector, the influence of the erroneous vector is removed,
It is possible to realize the shake correction without the influence of the error vector.

As a matter of course, the above feature can be applied even when the number of detection vectors is large. In addition, when the detection vector does not include an erroneous vector, the output vector obtained from the motion vector detecting means 16 does not include a time delay, and a sufficient shake correction can be realized even for a high frequency shake.

As described above, in the third embodiment of the present invention, even when the number of detection vectors obtained in the current field becomes small, the noise component of the output motion vector is suppressed and the error vector is erroneous. It is possible to realize an image shake correction apparatus capable of effectively removing the influence of the above, and capable of sufficiently correcting the image shake even under various conditions.

In the third embodiment, the motion vector detecting means 16 has the same structure as that of the motion vector detecting device of the second embodiment, but this is the same as the motion vector detecting device of the first embodiment. Even with the same configuration as the detection device, it is clear that the same effect can be obtained when the number of detection vectors is large.

In the first, second and third embodiments, five detection areas are provided on the screen, but the number may be any other number.

Further, in the first, second and third embodiments, as the method of obtaining the intermediate value by the intermediate value selecting means 11, one intermediate value is directly selected when the number of elements is large, but it is divided into a plurality of times. It is also possible to choose an intermediate value. For example, if the number of elements is 7, first select two intermediate elements from five elements, select three intermediate elements, then combine these three elements with the first two elements and select an intermediate value for the five elements. Is also possible.

In the first, second and third embodiments, the minimum number of elements of the set for which the intermediate value selecting means 11 determines the intermediate value is 7 and 5 or 3, respectively, but this is 3 or more. Any other number may be used as long as it is a number.

Furthermore, in the first, second and third embodiments,
The intermediate value selecting means 11 outputs the intermediate value as a representative value of the set, but this may be any one as long as it removes elements that are far from the overall tendency of the set and obtains the value from the remaining elements, for example. The number of elements in the set may be eight, and the average value of the values of the four elements near the center when arranged in order of size may be output as a representative value of the set. Further, for example, when the number of elements of a vector set is 8 and four elements are selected from them, the total sum of the distances between the four vectors is selected to be the smallest, and the average value of the four vectors is set to the set. May be output as a representative value of.

Further, in the second and third embodiments, when the number of detection vectors in the detection area determined to be reliable in the current field is 3 or less, the average value of the detection vectors is used as the prediction vector of the candidate vector. I am adding to the set,
For this, the motion vector of the current field predicted from the output of the previous field and the output of the previous field may be added to the set of candidate vectors. For example, a vector obtained by subtracting the output of the preceding field from the output of the preceding field may be added to the set. Further, the vector prediction method is not limited to this method or the average. The number of prediction vectors is not limited to one, and may be two or three.

Furthermore, in the first, second and third embodiments,
Using the representative point memory 2, the difference / absolute value converter 3, the cumulative adder 4, and the minimum point detection circuit 5, a plurality of detection areas are provided in one screen for an image signal between two fields, and the representative point matching method is used. A plurality of motion vectors are detected by using and output to the intermediate value selecting means 11 and the memory 30, but any method may be used as long as it detects a plurality of motion vectors. For example, instead of the representative point matching method, the all point matching method or the gradient method may be used.

In the third embodiment, the image memory 1
5 controls the image reading position in field units,
Although the shake is corrected by expanding the interpolation in field units, it may be in frame units or other intervals (variable).

[0066]

The image motion vector detecting device of the present invention adds motion vectors obtained in the past to the detected motion vector so as not to exceed a majority, and selects a set from among the plurality of candidate vector sets. Since the elements that are far from the overall tendency of are removed and a representative value is obtained, the effect of removing the error vector in the set is highly effective.

If there is no error vector in the motion vector detected in the current field, the number of detection vectors in the current field is large, so that no delay time occurs in the output motion vector.

Further, even when there is an erroneous vector in the majority of the detected vectors, the influence of the erroneous vector is removed,
It is possible to realize the shake correction without the influence of the error vector.

Furthermore, even when the number of detection vectors determined to be reliable due to moving objects and noise in the screen in the current field is less than a predetermined amount, the prediction vector and the motion vector output before the previous field are used. , The elements of the set of candidate vectors are supplemented, and a representative value is obtained from them, so even if there is an erroneous vector in the set or when noise is superimposed, the effect of removing the effect is high. .

Therefore, the motion vector output from the present apparatus is adjusted in accordance with variously changing image states.
Since the error is always kept very small and the delay time is suppressed as much as possible, good results are obtained and its effect is high when used for image processing such as motion compensation and coding and control of other equipment. .

Further, the image shake correcting apparatus of the present invention uses the above-described image motion vector detecting apparatus of the present invention, so that the noise does not become unstable due to the noise superimposed on the motion vector, and the The image does not shake due to the vector. Further, even when the frequency of image shake becomes high, sufficient shake correction can be realized.

Further, when there are many false vectors in the detected vector, that is, when the moving object is moving violently in the image or when noise is remarkable, the influence of this false vector can be sufficiently reduced. The time delay of shake correction that occurs at this time is difficult to be visually perceived from the state of the image, and it is unlikely to feel unnatural.

Therefore, it is possible to obtain a visually good shake-corrected image over the various states of the image that change in various ways.

[Brief description of drawings]

FIG. 1A is a block diagram of an image motion vector detecting apparatus according to a first embodiment of the present invention, and FIG. 1B is an operation explanatory diagram of an image motion vector detecting apparatus according to a first embodiment of the present invention.

FIG. 2 is a block diagram of an image motion vector detection device according to a second embodiment of the present invention.

FIG. 3 is a block diagram of an image shake correction apparatus according to a third embodiment of the present invention.

FIG. 4A is an operation explanatory view of the image shake correction apparatus of the third embodiment of the present invention. FIG. 4B is an operation explanatory view of the image shake correction apparatus of the third embodiment of the present invention.

FIG. 5 is a block diagram of a conventional image motion vector detection device.

FIG. 6 is an operation explanatory view of the conventional example.

FIG. 7 is an operation explanatory diagram when a conventional image motion vector detection device is used as a shake correction device.

[Explanation of symbols]

1 Image signal input terminal 2 Representative point memory 3 Difference / absolute value converter 4 Cumulative adder 5 Minimum point detection circuit 6 selection means 7 Motion vector smoothing means 8 screens 9 Motion vector detection area 10 Motion vector detected from each area 11 Intermediate value selection means 12 Reliability determination means 30 memory 31 Prediction means 15 Image memory 16 Motion vector detecting means 17 Read-out position control means 18 interpolation enlargement means 19 Moving objects 20 Motion vector error 21 Motion vector integration error

Claims (4)

[Claims] 1. A means for detecting a motion vector from each area having a plurality of motion vector detection areas for the entire screen or a part of an input image signal, and for each area newly detected in the current field. The detection vector is used as the first input vector of the intermediate value selecting means, and the detection vector of each area detected in the past less than the first input vector number of the intermediate value selecting means or the whole or part of the screen determined in the past. Is used as the second input of the intermediate value selecting means, the first input and the second input are added to make a candidate vector, and a representative value is selected from the set of candidate vectors to select the entire screen. Or a vector selecting means for obtaining a motion vector of a part, and obtaining a motion vector of the entire screen or a part of the input image signal It can vector detection apparatus. 2. Vector predicting means for obtaining a prediction vector predicted from the detection vector detected from the current field when the number of detection vectors newly detected in each field in the current field is equal to or smaller than a predetermined number. The motion vector detecting apparatus for an image according to claim 1, wherein the detection vector obtained in the current field and the prediction vector are combined and used as a first input of the intermediate value selecting unit. 3. A motion vector detecting device for an image according to claim 1, wherein the motion of the image is based on the motion vector of the entire screen or a part of the input image signal detected by the motion vector detecting device of the image. An image shake correction apparatus having means for controlling a position for reading an image signal from the memory in a direction for correcting the image shake. 4. A vector predicting means for obtaining a prediction vector predicted from a detection vector detected from the current field when the number of detection vectors newly detected in each field in the current field is equal to or smaller than a predetermined number. 4. The image shake correction apparatus according to claim 3, wherein the detection vector obtained in the current field and the prediction vector are combined and used as the first input of the intermediate value selection unit.