JP4337302B2 - Motion compensation circuit and method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a motion correction circuit and method for shifting the position of each detection pixel of an image signal generated by double speed conversion.
[0002]
[Prior art]
Conventionally, as a scanning system for television broadcasting, an interlace scanning system that scans every other horizontal scanning line is most widely adopted. In this interlace scanning method, one frame image is formed by a field image composed of odd-numbered scanning lines and a field image composed of even-numbered scanning lines, and surface flicker interference appears to flicker the entire screen. To prevent deterioration of screen quality.
[0003]
In addition, this interlace scanning method is adopted as a television standard method in various countries around the world. For example, in the PAL (Phase Alternation by Line) method in European television broadcasting, the field frequency is 50 [Hz] (frame The image is composed of 25 frames / second and the field image is composed of 50 fields / second).
[0004]
In particular, in this PAL system, a field frequency double speed system that converts the field frequency from 50 Hz to 100 Hz image signal by performing processing such as interpolation to further suppress surface flicker interference. Has been adopted conventionally.
[0005]
FIG. 18 shows a block configuration example of the field double speed conversion circuit 5 to which this field frequency double speed system is applied. The field double speed conversion circuit 5 is integrated in a television receiver 6 including an input terminal 61, a horizontal / vertical deflection circuit 62, and a CRT 63. The field double speed conversion circuit 5 includes a double speed conversion unit 51 and a frame memory 52.
[0006]
The double speed conversion unit 51 writes, for example, an image signal of 50 fields / second in the PAL format input from the input terminal 61 to the frame memory 52. The double speed conversion unit 51 reads the image signal written to the frame memory 52 at a speed twice as high as that at the time of writing. As a result, the frequency of the image signal of 50 fields / second can be doubled to generate an image signal of 100 fields / second.
[0007]
The double speed conversion unit 51 outputs the double-speed converted image signal to the CRT 63. The CRT 63 displays the input image signal on the screen. The horizontal and vertical deflection of the image signal in the CRT 63 is controlled based on a horizontal / vertical sawtooth wave having a frequency twice that of the input image signal generated in the horizontal / vertical deflection circuit 62.
[0008]
FIG. 19 shows the relationship between each field and pixel position in each image signal before and after double speed conversion. Here, the horizontal axis indicates time, and the vertical axis indicates the vertical position of the pixel. Further, the image signal indicated by white circles in FIG. 19A is a 50 field / second interlaced image signal before double speed conversion, and the image signal indicated by black circles in FIG. This is a field / second interlaced image signal.
[0009]
In the image signal shown in FIG. 19A, the field f1 and the field f2 are signals generated from the same frame on the film, and the field f3 and the field f4 similarly constitute the same frame. Since these image signals are interlaced image signals, the pixel positions in the vertical direction differ between adjacent fields. For this reason, one field cannot be newly generated between each field while maintaining the interlaced property.
[0010]
Therefore, as shown in FIG. 19B, two fields f2 ′ and f1 ′ are newly generated between the field f1 and the field f2. Then, no field is generated between the field f2 and the field f3, and two fields f4 ′ and f3 ′ are newly generated between the field f3 and the field f4. That is, one frame is formed by four fields and two frames.
[0011]
In the newly generated fields f1 ′, f2 ′,..., Each pixel value may be obtained as a median value of three pixels around each pixel using a median filter or the like. The newly generated fields f1 ′, f2 ′,... Have the same contents as the fields f1, f2,.
[0012]
In other words, the field double speed conversion circuit 5 alternately arranges a part that newly generates two fields and a part that does not generate at all between the fields of the image signal before the double speed conversion, thereby reducing the number of screens per unit time. Thus, the above-described surface flicker interference can be suppressed.
[0013]
By the way, in order to view a movie film composed of still images of 24 frames / second on a normal television, television cinema conversion (hereinafter referred to as telecine conversion) is performed in order to obtain an interlaced television signal. FIG. 20 shows the relationship between each field and the image position when the image moves in the horizontal direction in the image signal after the telecine conversion. Here, the horizontal axis indicates the position in the horizontal direction of the image, and the vertical axis indicates time. In the image signal before double speed conversion shown in FIG. 20 (a), the fields f1 and f2 form the same frame, so that images are displayed at the same position. This image moves in the horizontal direction (right direction) when the field f3 is entered. The field f4 is displayed at the same position as the field f3 because it forms the same frame as the field f3.
[0014]
When the image signal after the telecine conversion shown in FIG. 20 (a) is subjected to double speed conversion by the field frequency double speed method, as shown in FIG. 20 (b), fields f1, f2 ′, f1 ′, f2 constituting the same frame. Thus, the same image is displayed at the same position. Similarly, the same image is displayed at the same position in the fields f3, f4 ′, f3 ′, and f4 constituting the same frame.
[0015]
FIG. 21 (a) shows the relationship between each field and the image position when an image moves in the horizontal direction in a television signal before double speed conversion (hereinafter referred to as a TV signal). In FIG. 21 (a), fields f1, f2, f3... Form independent frames, and thus images are displayed at different positions. This image moves in the horizontal direction (right direction) every time the field f1 shifts to f2, f3,.
[0016]
When the image signal of the television signal shown in FIG. 21 (a) is subjected to double speed conversion by the field frequency double speed method, as shown in FIG. 21 (b), the same in the same position in the fields f1 and f2 ′ constituting the same frame. Is displayed. Similarly, the same image is displayed at the same position in the fields f1 ′ and f2 constituting the same frame.
[0017]
By the way, since the output image signal regularly constitutes each field with a period of 1/100 second, the time zone in which the image operates is shorter than the time zone in which the image is stationary, and the output image signal actually passes through the CRT. When watching a program, there was a problem that the movement of the image appeared discontinuous. For this reason, conventionally, in order to eliminate such discontinuity of motion, for example, based on the block matching method, the screen is divided into blocks made up of predetermined pixels, and the motion vector is obtained by evaluating the similarity in units of each block. The pixel position is shifted for each block in accordance with the obtained motion vector to correct the motion.
[0018]
Incidentally, in this block matching method, as shown in FIG. 22, the reference field is divided into a plurality of reference blocks 101, and the block having the highest similarity with the reference block 101 in the reference field 80 is searched for in the search range 104 in the reference field 90. It detects from the search block 103 which moves inside. Then, a position shift (direction and magnitude of movement) between the detected search block 103 and the reference block 101 is used as a motion vector.
[0019]
In the determination of the similarity, first, for each pixel value of the search block 103, a difference from the corresponding pixel value of the reference block 101 is calculated, and an evaluation value indicated by the difference, for example, a sum of absolute differences is obtained. Next, the above-described determination operation is performed for all the search blocks 103, and the minimum value is obtained from the obtained evaluation value sum, that is, the sum of the absolute differences. The search block 103 that gives the minimum sum of absolute differences is a block that shows the highest similarity to the reference block 101, and is specified between the origin pixel 63 of the block and the origin pixel of the reference block 101. A vector that can be used as a motion vector.
[0020]
[Problems to be solved by the invention]
However, the motion correction by the conventional block matching method efficiently eliminates the discontinuity of the image motion in various image variations in which the pixel value changes while the image moves in the horizontal direction, for example. There was a problem that could not.
[0021]
In addition, in the conventional block matching method, a motion vector is detected simply by giving priority to the minimum value of the sum of errors between blocks regardless of the motion of the original image expressed on the entire screen. Deterioration of the converted image has been a problem.
[0022]
Further, in the above conventional block matching method, even when the true motion vector 111 required because the image motion is fast can exist outside the search range 104, the error between the blocks 101 and 103 within the search range 104. In order to give the highest priority to the sum of the two, there is a problem that an irregular motion vector different from the original motion is detected.
[0023]
Furthermore, since the motion vectors detected based on the conventional block matching method are rich in variations, pixels are written in an image synthesized by shifting the pixel position for each block in accordance with the detected motion vector. Uncovered areas may occur. When this uncovered area occurs, isolated points are generated in some places as shown in FIG. 23, so that the image quality deteriorates remarkably, and processing failure occurs in units of areas, which adversely affects the operating environment.
[0024]
Accordingly, the present invention has been devised in view of the above-described problems, and an object of the present invention is to provide surface flicker in an image signal generated by double speed conversion, particularly in various image variations. The purpose is to synergistically improve the image quality by smoothing the movement of the image while suppressing the interference.
[0025]
Another object is to prevent the conversion image from deteriorating by obtaining a motion vector adapted to the motion of the original image.
[0027]
Yet another object is to fill the uncovered area, which is the result of synthesizing the image by shifting the pixel position for each block according to the detected motion vector, by filling the pixel indicating the optimal pixel value with the image quality. This is to prevent the deterioration of the process and to improve the processing failure in each area.
[0028]
[Means for Solving the Problems]
In order to solve the above-described problem, the motion correction circuit according to the present invention is an image signal in which one frame generated by double-speed conversion of a telecine-converted image is composed of four fields. Issue A reference field in which a pixel for which a motion vector is input is located; The Is it a base field Et al 2 Based on the motion vector calculated between the reference field and the frame Do Motion correction is performed on the correction field where the pixel is located. Or an image signal composed of two fields for one frame generated by double-speed conversion of a television signal, and a reference field in which a pixel for which a motion vector is obtained is located, and one frame away from the reference field Based on the motion vector obtained from the reference field, motion correction is performed on the correction field where the pixel for which motion correction is performed is located. In the motion compensation circuit, motion vector detection means for detecting a motion vector for the reference field with respect to a reference block centered on a reference pixel of the reference field, and pixels constituting the reference block based on the motion vector detected in the reference block A vector allocating unit to be allocated every time, a selection unit to select any one of the motion vectors when pixels indicated by the motion vector allocated by the vector allocating unit overlap, and a motion vector allocated to the correction field Image control means for setting the pixel value of the pixel to a pixel value at a position shifted in the vector direction within the range of the vector amount of the motion vector according to the weight assigned to each correction field, and the pixel value on the correction field Pixels for non-existent information absence area Characterized in that it comprises a pixel interpolating means for interpolating.
[0029]
In addition, in order to solve the above-described problem, the motion correction method according to the present invention is an image signal in which one frame generated by double-speed conversion of a telecine-converted image is composed of four fields. Issue The reference field where the pixel for which the motion vector is input is located and the above reference field Et al 2 Motion correction is performed based on the motion vector obtained between the reference field and the frame. Do Motion correction is performed on the correction field where the pixel is located. Or an image signal composed of two fields for one frame generated by double-speed conversion of a television signal, and a reference field in which a pixel for which a motion vector is obtained is located, and one frame away from the reference field Based on the motion vector obtained from the reference field, motion correction is performed on the correction field where the pixel for which motion correction is performed is located. In the motion correction method, a motion vector for the reference field is detected for a reference block centered on a reference pixel of the reference field, and the motion vector detected in the reference block is assigned to each pixel constituting the reference block, When any one of the pixels indicated by the assigned motion vector overlaps, one of the motion vectors is selected, and the pixel value of the pixel to which the motion vector is assigned on the correction field depends on the weight assigned to each correction field. A pixel value at a position shifted in the vector direction within the range of the vector amount of the motion vector is used, and the pixel value is interpolated with respect to an information absent region where no pixel value exists.
[0030]
In this motion correction circuit and method, a motion vector detected for a reference block centered on a reference pixel is assigned to each pixel constituting the reference block. Then, when the pixels indicated by the assigned motion vectors overlap, one of the motion vectors is selected, and the pixel value of the assigned pixel of the motion vector is calculated within the range of the vector amount of the motion vector. As a result of writing to the position shifted in the direction and shifting the pixel value, the pixel value is interpolated with respect to the information absence region where the pixel value does not exist.
[0031]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an image signal processing apparatus and method to which the present invention is applied will be described in detail with reference to the drawings.
[0032]
FIG. 1 is a block configuration diagram of a motion correction circuit 1 according to the first embodiment of the present invention. The motion correction circuit 1 is built in, for example, a television receiver using a PAL (Phase Alternation by Line) system, and receives an image signal and a television signal (hereinafter referred to as a TV signal) subjected to telecine conversion. As shown in FIG. 1, the motion correction circuit 1 includes a first image memory 11, a second image memory 12, a sequence detection unit 13, a data selection unit 14, a motion vector detection unit 15, An image shift unit 16 is provided.
[0033]
The first image memory 11 is sequentially supplied with, for example, 100 fields / second interlaced image data in which one frame generated by double-speed conversion of the telecine-converted image is composed of four fields. The first image memory 11 is sequentially supplied with an interlaced image signal of 100 fields / second, for example, in which one frame generated by double-speed conversion of the TV signal is composed of two fields.
[0034]
The first image memory 11 stores the supplied image data for one frame for each field. That is, the image data output from the first image memory 11 is one frame after the image signal supplied to the first image memory 11.
[0035]
The second image memory 12 has the same internal configuration as that of the first image memory 11 and stores the image data supplied from the first image memory 11 for one frame in each field unit. That is, the image data output from the second image memory 12 is one frame after the image data supplied to the second image memory 12 and two frames from the image data supplied to the first image memory 11. Later. The image data D1 stored in the second image memory 12 is supplied to the motion vector detection unit 15 and the image shift unit 16.
[0036]
The sequence detection unit 13 detects the image data supplied to the first image memory 11 and the image data output from the first image memory 11, compares the image signal level for each pixel, and Calculate the difference value. That is, the sequence detection unit 13 compares the image signal levels of the pixels at the same location on the screen at intervals of one frame. The sequence detection unit 13 transmits the calculation result of the difference value of the image signal level to the image shift unit 15. In addition to specifying each field as described above, the sequence detection unit 13 determines whether the signal is a telecine-converted signal or a TV signal, and transmits the determination result to the image shift unit 16 or the like as movement amount information.
[0037]
The data selection unit 14 receives image data supplied to the first image memory 11 and image data output from the first image memory 11. The data selection unit 14 selects one of the supplied image data based on the determination result received from the sequence detection unit 13. That is, when the sequence detection unit 13 determines that the signal has been telecine converted, the image data supplied to the first image memory 11 is selected. If the sequence detection unit 13 determines that the signal is a TV signal, the image data output from the first image memory 11 is selected. The image data selected by the data selection unit 14 is hereinafter referred to as image data D2. The data selection unit 14 outputs the selected image data D2 to the motion vector detection unit 15.
[0038]
Note that the data selection unit 14 can also be applied to a connection mode in which one of image data output from the first image memory 11 and image data output from the second image memory 12 is selected.
[0039]
The motion vector detection unit 15 detects the image data D1 and the image data D2, and detects a motion vector based on, for example, a block matching method. The motion vector detection unit 15 can search not only for motion vectors from both directions but also for motion vectors from a single direction. Thereby, it is possible to reduce the hardware cost.
[0040]
Further, the motion vector detection unit 15 calculates a flag F1 including error information of the detected motion vector. That is, the motion vector detection unit 15 can specify a motion vector between image signals separated by one frame or two frames by detecting a motion vector between the image data D1 and the image data D2. The motion vector detection unit 15 can also control the field interval for detecting the motion vector based on the determination result received from the sequence detection unit 13.
[0041]
The image shift unit 16 receives movement amount information including a comparison result of image signal levels from the sequence detection unit 13. In addition, the image shift unit 16 receives the motion vector detected by the motion vector detection unit 15. Further, the image shift unit 16 is supplied with the image data D1 from the second image memory 12, and is also supplied with the image data D2 from the data selection unit 14. The image shift unit 16 shifts each pixel position in the supplied image signal in the vector direction within the range of the vector amount of the received motion vector. An example of the internal configuration of the image shift unit 16 will be described later in detail.
[0042]
The motion correction circuit 1 may be integrated with a field double speed conversion circuit 3 that doubles the field frequency of the image signal. The field double speed conversion circuit 3 is integrated to prevent surface flicker interference by improving the resolution. For example, in the PAL system, by performing processing such as interpolation, image data having a field frequency of 50 Hz is obtained. The image data is doubled to 100 Hz image data.
[0043]
As shown in FIG. 1, the field frequency conversion circuit 3 includes an input terminal 31 connected to the television receiver, a double speed conversion unit 32, and a frame memory 33.
[0044]
The double speed conversion unit 32 writes the image data after the telecine conversion or the television signal input from the television receiver via the input terminal 31 into the frame memory 33. The double speed conversion unit 32 reads the image data written in the frame memory 33 at a speed twice as high as that at the time of writing. Thereby, for example, the frequency of the image signal of 50 fields / second in the PAL system can be converted to twice, and image data of 100 fields / second can be generated. The double speed conversion unit 32 supplies the double speed converted image data to the motion correction circuit 1.
[0045]
Next, an example of the internal configuration of the motion vector detection unit 15 will be described in detail with reference to FIG. The motion vector detection unit 15 includes a block matching unit 151, an out-of-range detection unit 152, and a reallocation unit 153.
[0046]
The block matching unit 151 divides the input image data D1 and image data D2 into blocks made up of predetermined pixels, and obtains a motion vector by evaluating the similarity for each block.
[0047]
The out-of-range detection unit 152 corrects the motion vector outside the search range obtained by the block matching method so that it matches the actual motion of the image.
[0048]
The reallocation unit 153 separates the motion vector obtained in units of blocks as a motion vector in units of pixels. In other words, the motion vector obtained for each block is assigned to each pixel constituting the block. Incidentally, the reallocation unit 153 selects an optimal vector value in light of the peripheral block through steps such as boundary determination within the block, detection of image edge, correlation determination with the peripheral block, and region division within the block. These are assigned for each pixel.
[0049]
Next, an example of the internal configuration of the image shift unit 16 will be described. As shown in FIG. 3, the image shift unit 16 includes a shift buffer read control unit 161, a shift buffer write control unit 162, a shift buffer 163, a data buffer read control unit 164, a first buffer 165, A second buffer 166, a data operation unit 167, and an uncover processing unit 168 are provided.
[0050]
The shift buffer read control unit 161 receives the motion vector from the motion vector detection unit 15 and the movement amount information from the sequence detection unit 13. The shift buffer read control unit 161 generates the shift buffer read control signal RS1 based on the motion vector, the movement amount information, and the built-in address calculation counter. This shift buffer read control signal RS1 includes an address signal for sequentially reading data and an enable signal. For example, when the shift buffer 163 is realized by a frame memory or the like, the shift buffer read control unit 161 calculates the X coordinate and Y coordinate address signals as absolute coordinates. On the other hand, when the shift buffer 163 is realized by a minimum necessary memory such as a line memory, the shift buffer read control unit 161 calculates the X coordinate and Y coordinate address signals as relative coordinates.
[0051]
In the built-in address calculation counter value, the X coordinate and the Y coordinate are (CX1, CY1), respectively, and in the supplied motion vector, the X coordinate and the Y coordinate are (VX, VY), respectively. The address numbers (SX, SY) of the shift buffer read control signal RS1 are expressed by the following equations.
SX = CX1 + (VX × α) (1.1)
SY = CY1 + (VY × α) (1.2)
Here, α is movement amount information, and is expressed by a number of 0 or more and 1 or less. Here, when the first field of the frame is the first field, and the subsequent fields are the second field, the third field, and the fourth field, respectively, the above α is minimized in the first field, Thereafter, it may be increased sequentially each time the field continues. Α is 0, 1/4, 2/4, 3/4 from the first field to the fourth field, respectively, and a TV signal is input when a telecine converted signal is input. In this case, it is also possible to increase linearly from 0 to 1/2 from the first field to the second field.
[0052]
As a result, the shift amount can be increased linearly with respect to time, and the motion of the image can be made smoother. Therefore, it is possible to eliminate the motion discontinuity peculiar to the double-speed converted image signal. In addition, since the motion of the image in which the surface flicker interference is suppressed can be further smoothed by the double speed conversion, the image quality can be synergistically improved.
[0053]
The shift buffer read control unit 161 supplies the generated shift buffer read control signal RS1 to the shift buffer write control unit 162 and the shift buffer 163.
[0054]
The shift buffer write control unit 162 is supplied with the flag F1 from the motion vector detection unit 15, is supplied with the flag F ′ from the shift buffer 163, and is supplied with the shift buffer read control signal RS1 from the shift buffer read control unit 161. The The shift buffer write control unit 162 determines the priority order at the time of writing based on the magnitudes of the flags F1 and F ′. Further, the shift buffer write control unit 162 obtains a write address based on the supplied shift buffer read control signal RS1, and uses the write address and the priority determined as described above as the shift buffer write control signal RS2. This is supplied to the shift buffer 163.
[0055]
The shift buffer 163 includes a motion vector buffer and a flag buffer. The motion vector buffer is a buffer for storing and supplying motion vectors, and the flag buffer is a buffer for storing and supplying flags. These buffers are written and read based on the same control signal. Incidentally, the shift buffer 163 only needs to be able to store motion vectors and flags, so that a reduction in buffer capacity can be expected.
[0056]
The shift buffer 163 may be a frame memory that accumulates data for one frame, or may be configured with a minimum necessary memory such as a line memory according to a range that a motion vector can take.
[0057]
The shift buffer 163 first initializes the flag buffer. The flag written in the flag buffer also includes mark information indicating whether or not data has been written. The mark information is represented by two types, “NM” and “OK”. “NM” indicates that data is not written at the time of initialization, and “OK” indicates that data has already been written. It shows that. The shift buffer 163 transmits a flag F ′ to the shift buffer write control unit 162 in association with the address when the enable of the shift buffer read control signal RS1 is valid. Further, when the shift buffer write control signal RS2 is enabled, the shift buffer 163 writes the motion vector and the flag F1 to the motion vector buffer and the flag buffer, respectively, according to the address value. Further, the shift buffer 163 arranges the stored motion vectors in the order of numbers (hereinafter, the motion vectors arranged in the order of the numbers are referred to as “moved motion vectors”), sequentially reads out the processing flag F2, and each of them is a data buffer control unit. 161, and supplied to the data calculation unit 167.
[0058]
The data buffer read control unit 164 is supplied with the moved motion vector from the shift buffer 163 and transmits the movement amount information from the sequence detection unit 13. The data buffer read control unit 164 calculates a buffer control signal S11 and a buffer control signal S12 based on the input motion vector. Each of these buffer control signals S11 and S12 includes an address signal for sequentially reading data and an enable signal. For example, when the first buffer 165 and the second buffer 166 are realized by a frame memory or the like, the data buffer read control unit 161 calculates each address signal of the X coordinate and the Y coordinate as an absolute coordinate. On the other hand, in the case where the first buffer 165 and the second buffer 166 are realized by the minimum necessary memory such as a line memory, the data buffer read control unit 161 uses the X coordinate and Y coordinate address signals as relative values. Calculate as coordinates.
[0059]
The data buffer read control unit 164 generates a buffer control signal S11 based on the value of the built-in address calculation counter and the motion vector. Further, the data buffer read control unit 164 generates a buffer control signal S12 based on the buffer control signal S11 and the motion vector.
[0060]
For example, when a TV signal is input, the X coordinate and the Y coordinate are (AX1, AY1) in the address of the buffer control signal S11, and the X coordinate and the Y coordinate are (( AX2, AY2), the motion vector is (VX, VY), and the internal address calculation counter value is (CX ', CY'), the address of the buffer control signal S11 is expressed by the following equation. Is done.
AX1 = CX'-INT (VX / 2) (2.1)
AY1 = CY'-INT (VY / 2) (2.2)
Here, the function INT means truncation after the decimal point.
[0061]
Further, the address (AX2, AY2) of the buffer control signal S12 is expressed by the following equation.
AX2 = AX1 + VX (2.3)
AY2 = AY1 + VY (2.4)
The data buffer read control unit 164 supplies the buffer control signal S11 including these calculated addresses to the first buffer 165. Further, the data buffer read control unit 164 supplies the second buffer 166 with a buffer control signal S12 including an address calculated in the same manner.
[0062]
The first buffer 165 sequentially stores the image data D1 transmitted from the second image memory 12. The first buffer 165 reads the stored image data D1 according to the supplied buffer control signal S11. That is, when the supplied buffer control signal S11 is enabled, the first buffer 165 reads the image data D1 stored in the first buffer 165 according to the address included in the buffer control signal S11. The read image data D1 is hereinafter referred to as shift data SD1. The first buffer 165 transmits the shift data SD1 to the data calculation unit 167.
[0063]
The first buffer 165 may be a frame memory that stores data for one frame, or may be configured with a minimum necessary memory such as a line memory based on a motion vector range. Further, the first buffer may be realized by a FIFO memory or the like in order to read data sequentially.
[0064]
The second buffer 166 sequentially accumulates the image data D2 transmitted from the data selection unit 14. The second buffer 166 reads the stored image data D2 in accordance with the supplied buffer control signal S12. That is, when the supplied buffer control signal S12 is enabled, the second buffer 166 reads the image data D2 stored in the second buffer 166 according to the address included in the buffer control signal S12. The read image data D2 is hereinafter referred to as shift data SD2. The second buffer 166 transmits the shift data SD2 to the data calculation unit 167.
[0065]
The second buffer 166 may be a frame memory that stores data for one frame, or may be configured with a minimum necessary memory such as a line memory based on a motion vector range. In such a case, a system for reading out data at random corresponding to an address given at random is constructed.
[0066]
The data calculation unit 167 calculates correction data H1 based on the supplied shift data SD1 and shift data SD2 while referring to the processing flag F2 supplied from the shift buffer 163. Note that when the processing flag F2 is “OK”, data can be determined, and the data calculation unit 167 performs a predetermined calculation. On the other hand, when the processing flag F2 is “NM”, it is assumed that the data cannot be fixed and the pixel value is not written at all from other areas (hereinafter referred to as an uncovered area). The processing is entrusted to the cover processing unit 167. The correction data H1 may be calculated by outputting the shift data SD1 and the shift data SD2 as they are, or may be an average value of the shift data SD1 and the shift data SD2. Furthermore, the movement data M1 may be calculated in the form of taking a weighted average using values such as motion vectors. The uncover processing unit 168 interpolates the pixel value with respect to the uncover area based on the received correction data H1.
[0067]
Next, the operation of the motion correction circuit 1 to which the present invention is applied will be described. FIG. 4 shows the relationship between each field and pixel position before and after double speed conversion in the field double speed conversion circuit 3. Here, the horizontal axis indicates time, and the vertical axis indicates the vertical position of the pixel.
[0068]
The image data before double speed conversion is an interlaced image of 50 fields / second in the PAL format, and forms one frame in two fields as shown in FIG.
[0069]
On the other hand, since the image data after the double speed conversion is an interlaced image of 100 fields / second, as shown in FIG. 4 (b), two fields t2 ′, t1 'is generated. Then, no field is generated between the field t2 and the field t3, and two fields t4 ′ and t3 ′ are newly generated between the field t3 and the field t4. That is, the image data forms one frame with four fields.
[0070]
In the newly generated fields t1 ′, t2 ′,..., The respective pixel values may be obtained by using a median filter or the like as an intermediate value of three pixels around each pixel. The newly generated fields t1 ′, t2 ′,... Have the same contents as the fields t1, t2,. As a result, one frame is formed in four fields, the resolution can be improved by increasing the number of screens per unit time, and surface flicker interference can be suppressed.
[0071]
FIG. 5 shows the relationship between each field and the image position when the image moves in the horizontal direction in the image data subjected to double speed conversion as described above after the telecine conversion. In FIG. 5, the horizontal axis represents the position in the horizontal direction of the image, and the vertical axis represents time. Images that have already been telecine transformed are supplied to the first image memory 11 at regular time intervals in the order of fields t1, t2 ′, t1 ′, t2, and these images are displayed at the same position. In addition, when moving to the field t3, the image moves in the horizontal direction (right direction) and is supplied to the first image memory in the order of fields t3, t4 ′, t3 ′, and t4.
[0072]
Here, for example, when the field supplied to the first image memory (hereinafter referred to as the reference field) is the field t3, two frames before the reference field output from the second image memory 12 Field (hereinafter referred to as the field two frames before) is the field t1.
[0073]
FIG. 6 shows the relationship between each field and image position when the image moves in the horizontal direction in the image data obtained by double-speed conversion of the TV signal. In the fields t1 and t2 ′ constituting the same frame, the same image is displayed at the same position. Similarly, the same image is displayed at the same position in the fields t1 ′ and t2 constituting the same frame.
[0074]
The motion vector detection unit 15 detects a motion vector for each block between the reference field and the field two frames before the signal subjected to the double-speed conversion after the telecine conversion shown in FIG. In the case of the example shown in FIG. 5, the vector direction of the motion vector is the horizontal direction (right direction) with reference to the field two frames before, and the vector amount is A. Similarly, when the reference field is t5, the field two frames before is t3, and the vector amount of the motion vector is B. By repeating this procedure, the vector direction and the vector amount of the motion vector based on the field two frames before can be obtained sequentially. The motion vector detection unit 15 sequentially transmits the obtained vector amount and vector direction of the motion vector to the image shift unit 16.
[0075]
Further, the motion vector detecting unit 15 performs a block conversion between the reference field and a field one frame before the reference field (hereinafter referred to as a field before one frame) for the signal obtained by double-speed conversion of the TV signal shown in FIG. Detect motion vectors in units. In the case of the example shown in FIG. 6, the vector direction of the motion vector is horizontal (rightward) with reference to the previous frame field, and the vector amount is C when the reference field is t1 ′. Similarly, when the reference field is t4 ′, the previous frame field is t1 ′, and the vector amount of the motion vector is D. By repeating this procedure, the vector direction and the vector amount of the motion vector based on the field one frame before can be obtained sequentially. The motion vector detection unit 74 sequentially transmits the obtained vector amount and vector direction of the motion vector to the image shift unit 15.
[0076]
The sequence detection unit 13 sequentially detects the reference field and the field one frame before output from the first image memory 11, and calculates the difference value of the pixel signal level at the same pixel position.
[0077]
That is, as shown in FIG. 7, in the case of a telecine conversion image, the reference field t1 ′ and the field t1 one frame before constitute the same frame, and thus, for example, the difference value of the pixel signal level at the pixel position a. Becomes 0. Next, when the field t2 is supplied as a reference field, the previous frame field becomes the field t2 ', and the difference value of the pixel signal level at the point a is similarly 0.
[0078]
Next, when the field t3 is supplied as a reference field, the previous frame field becomes t1 ', and both form different frames, so that the difference value of the pixel signal level at point a is other than 0 (hereinafter, "1" ”). Next, when t4 'is supplied as a reference field, the previous frame field becomes field t2, and the difference value of the pixel signal level at point a is similarly "1".
[0079]
Further, when t3 ′ is supplied as a reference field, the previous frame field becomes t3, and both form the same frame, so that the difference value of the pixel signal level at point a becomes 0 again. The same tendency is applied to the reference field supplied thereafter, and the calculated difference value is repeated in the order of “0011” in a period of 4 fields. Therefore, by detecting this sequence in units of 4 fields, it is possible to specify the context of each field.
[0080]
When this tendency is focused on the field one frame before, the difference values are in the order of “0011” from the first field of the frame. Therefore, when the difference value 0 is first calculated, the detected one frame previous field is specified as the first field (first field) of the frame. When the difference value 0 continues, the detected one frame previous field is specified as the second field. Further, when 1 is first calculated as the difference value, the detected one frame previous field is specified as the third field. When the difference value 1 continues, the detected one frame previous field is specified as the fourth field.
[0081]
Even when a TV signal is input, it is necessary to determine whether each field corresponds to the first field or the second field, but the field corresponding to the double speed conversion by the field double speed conversion circuit 3. Therefore, there is no need for sequence detection as described above. That is, when a TV signal is input from the field double speed conversion circuit 3, the first field and the second field are specified.
[0082]
Next, an exemplary operation of the block matching unit 151 in the motion vector detection unit 15 will be described in detail.
[0083]
As shown in FIG. 8, the block matching unit 151 divides the reference field 30 corresponding to the image data D1 into a plurality of reference blocks 51, and has the highest similarity with the reference block 51 in the image data D1 (reference field 30). The block shown is detected from the search block 53 moving within the search range 54 in the image data D2 (reference field 40). Then, a position shift (direction and magnitude of movement) between the detected search block 53 and the reference block 51 is used as a motion vector.
[0084]
In the determination of the similarity, first, for each pixel value of the search block 53, a difference from the corresponding pixel value of the reference block 51 is obtained, and an evaluation value indicated by the difference, for example, a sum of absolute differences is obtained. Next, the above-described determination operation is performed for all the search blocks 53, and the smallest one is obtained from the obtained evaluation value sums, that is, the sum of the absolute differences. The search block 53 that gives the minimum sum of absolute differences is a block that shows the highest similarity to the reference block 51, and is specified between the origin pixel 63 of the block and the origin pixel of the reference block 51. A vector that can be used as a motion vector.
[0085]
The block matching unit 151 may detect a motion vector suitable for the motion of the original image by applying the following detection method in addition to the motion vector detection based on the block matching method described above. .
[0086]
First, the first detection method will be described. In the first detection method, in addition to the motion correction using the block matching method described above, a motion vector is further detected from the following concept.
[0087]
FIG. 9 shows an example of the distribution tendency of the sum of absolute differences obtained for all search blocks 53 assigned within the search range 54. From FIG. 9, the region where the sum of absolute differences is high is a region where pixel values of the reference block 51 and the search block 53 are significantly different. On the other hand, a region having a low difference absolute value forming a recess is a region in which the pixel values of the reference block 51 and the search block 53 are approximate. That is, by specifying a motion vector for an area where pixel values are approximated, it is possible to realize high image quality and high resolution.
[0088]
The search range 54 is assigned a horizontal line corresponding to the vertical component (ys). Each horizontal line has a minimum value of the difference absolute value, and these minimum values form a region where the above-described difference absolute value is low. Hereinafter, the minimum value for each horizontal line is referred to as a line minimum value.
[0089]
Similarly to the normal block matching, the minimum value of the sum of absolute differences specified from the entire search range 54 is referred to as a region minimum value. That is, the difference absolute value sum that is the smallest among the line minimum values corresponds to the region minimum value. As shown in FIG. 9, the region minimum value indicated by the black circle is not much different from the other line minimum values. For this reason, even if the motion vector is specified between the pixel indicating the line minimum value and the pixel at the origin of the reference block 51 instead of the pixel indicating the region minimum value, the image quality and the resolution are hardly affected. It will be.
[0090]
For this reason, in the first detection method, paying attention to such a property, a pixel that best matches the movement of the original image is selected from the pixels having the line minimum value, and the movement between the pixel at the origin of the reference block 51 is performed. Identify the vector.
[0091]
Next, the second detection method will be described. In this second detection method, as shown in FIG. 10, paying attention to obtaining a motion vector in order from the left pixel (block) to the right pixel (block), in addition to the block matching method described above, Further, the following motion correction is performed.
[0092]
In order to identify the original image motion, the motion vector of the pixel on the left is read as a parameter for identifying the original image motion. Then, it is assumed that the motion vector obtained in the pixel 34 located to the left of the pixel 35 for obtaining the motion vector (the pixel at the origin of the reference block 51) is accurate. Actually, as shown in FIG. 11, the direction of the motion vector of the pixel 35 is matched with the motion vector of the pixel 34 located on the left side. Since the motion vector of the pixel 35 thus obtained is also accurate, the pixel 35 can be taken into account when further obtaining the motion vector of the pixel 36, and the accuracy of the motion vector can be ensured. It becomes possible.
[0093]
The block matching unit 151 may further perform a correction process shown in FIG. 12 on the motion vector detected based on the block matching method described above.
[0094]
In this correction process, a pixel for which a motion vector unrelated to the actual image motion is required (hereinafter, this pixel is referred to as a reference pixel) is extracted and adapted to the motion vector of an adjacent pixel adjacent to the reference pixel. Is.
[0095]
First, the block matching unit 151 initializes each variable in step S11, and then extracts a motion vector in the first adjacent pixel (for example, adjacent pixel A) from the adjacent pixels A to H shown in FIG. 13 in step S12. To do.
[0096]
Next, in step S13, the block matching unit 151 identifies the polarity Vx_i of the horizontal component of the motion vector in the adjacent pixel. If the polarity is positive, the process proceeds to step S14, and if the polarity is negative, step S15. Migrate to
[0097]
In step S14, the block matching unit 151 adds 1 to a counter Cnt_p (not shown), converts the horizontal component of the extracted motion vector into the sum Vx_ave_p of the horizontal components of adjacent pixels, and converts the vertical component of the motion vector into the vertical component. Is added to Vy_ave_p, and the process proceeds to step S16.
[0098]
In step S15, the block matching unit 151 adds 1 to a counter Cnt_m (not shown), converts the horizontal component of the extracted motion vector into the sum Vx_ave_m of the horizontal components of adjacent pixels, and converts the vertical component of the motion vector into the vertical component. It adds to the sum Vy_ave_m and proceeds to step S16.
[0099]
In step S16, the block matching unit 151 identifies whether or not the adjacent pixel identified in step S13 is the last adjacent pixel among the adjacent pixels A to H to be extracted. If it is the last adjacent pixel, the process proceeds to step S18. If it is not the last adjacent pixel, the process proceeds to step S17.
[0100]
In step S17, the block matching unit 151 extracts a motion vector (for example, adjacent pixel B) of the next adjacent pixel, and proceeds to step S13 again. By constructing this routine, it becomes possible to identify motion vectors for all the adjacent pixels A to H shown in FIG. In addition, when the motion vectors are identified for all the adjacent pixels A to H and the process proceeds to step S18, the counter Cnt_p determines the number of adjacent pixels in which the polarity of the horizontal component of the motion vector is positive, and the counter Cnt_m This indicates the number of adjacent pixels in which the polarity of the horizontal component of the motion vector is negative. Further, a horizontal component sum Vx_ave_p and a vertical component sum Vy_ave_p are obtained for adjacent pixels in which the polarity of the horizontal component of the motion vector is positive, and a horizontal component in the adjacent pixel in which the polarity of the horizontal component of the motion vector is negative. Sum Vx_ave_m and vertical component sum Vy_ave_m.
[0101]
Incidentally, in step S17, the order in which the adjacent pixels A to H are extracted may be clockwise, counterclockwise, or random. Further, the adjacent pixels to be extracted may not be extracted from all the adjacent pixels A to H. For example, the adjacent pixels A to C, the adjacent pixels B, D, and E, or the adjacent pixels A, B, C, D, and E may also be used. Further, the above-described processing may be executed in units of blocks composed of a plurality of pixels instead of in units of pixels.
[0102]
In step S18, the block matching unit 151 compares the numerical values of the counter Cnt_p and the counter Cnt_m. As a result, when the counter Cnt_p is larger, it is indicated that there are more adjacent pixels in which the polarity of the motion vector horizontal component is positive among the adjacent pixels, and the process proceeds to step S19. On the other hand, when the counter Cnt_m is larger, it is indicated that there are more adjacent pixels in which the polarity of the horizontal component of the motion vector is negative among the adjacent pixels, and the process proceeds to step S20.
[0103]
In step S19, a corrected motion vector at the reference pixel is obtained based on the adjacent pixel whose polarity of the horizontal component of the motion vector is positive. Each component (Vx_n, Vy_n) of the corrected motion vector is an average value of each component of adjacent pixels whose horizontal component has a positive polarity. Specifically, it is calculated based on the following formula.
Vx_n = Vx_ave_p / Cnt_p
Vy_n = Vy_ave_p / Cnt_p
[0104]
Further, in step S20, a corrected motion vector at the reference pixel is obtained based on the adjacent pixel in which the horizontal component of the motion vector has a negative polarity. Each component (Vx_n, Vy_n) of the corrected motion vector is an average value of each component of adjacent pixels in which the polarity of the horizontal component is negative. Specifically, it is calculated based on the following formula.
Vx_n = Vx_ave_m / Cnt_m
Vy_n = Vy_ave_m / Cnt_m
[0105]
That is, the block matching unit 151 counts the motion vector of the reference pixel and the vector direction of the motion vector of the adjacent pixel by following the processing procedure shown in FIG. 12, and matches the vector direction indicated by the many adjacent pixels. Can be modified as follows. In other words, the majority of the motion vector directions of neighboring pixels located around the reference pixel are determined, the direction of the larger motion vector is estimated as the actual image motion, and the motion vector vector of the reference pixel The direction can be corrected. As a result, it is possible to eliminate variations in motion vectors, and consequently, it is possible to prevent deterioration of the converted image.
[0106]
Next, an operation example of the out-of-range detection unit 152 will be described.
[0107]
The out-of-range detection unit 152 performs correction processing when the motion vector transmitted from the block matching unit 151 goes out of the search range 54 as shown in FIG.
[0108]
In the first correction process, for example, as shown in FIG. 14A, the vector direction of the motion vector to be corrected is adapted to the vector direction of the motion vector obtained by the block matching calculation unit 11. That is, only the vector amount is reduced to the contour portion of the search range 54 without changing the direction of the motion vector obtained by the block matching calculation unit 11.
[0109]
For example, as shown in FIG. 14B, the second correction process is a motion vector that adapts or corrects the vertical component of the motion vector to be corrected to the vertical component of the motion vector obtained by the block matching calculation unit 11. Are matched with the horizontal component of the motion vector obtained by the block matching calculation unit 11, and the vector amount is reduced to the contour of the search range 54.
[0110]
In the third correction process, for example, as shown in FIG. 14C, a plurality of motion vector candidates are set in advance such that the vector amount is within the search range 54, and the motion obtained by the block matching calculation unit 11 is obtained. One motion vector is selected from the motion vector candidates according to the vector direction and vector amount of the vector (in other words, according to the pixel position where the sum of absolute differences is minimized). Incidentally, the motion vector candidates to be set may be set one by one for each side of the search range 54 as shown in FIG.
[0111]
As described above, the motion vector detection device 1 according to the present invention can calculate the sum of absolute differences of the blocks 51 and 53 as in the conventional block matching method even when a true motion vector to be obtained exists outside the search range 54. The true motion vector can be easily detected without giving the highest priority to the minimum value of. Accordingly, the present invention does not require an irregular motion vector different from the original motion, and can prevent image quality degradation and processing failure in units of regions.
[0112]
Next, a specific operation example of the image shift unit 16 will be described.
[0113]
FIG. 15A shows a specific operation example of the image shift unit 16 in a one-dimensional graph when a TV signal composed of one frame and two fields is input. This operation example is for a case where a TV signal is input. A number starting from 0 is an address indicating a pixel position, and a vertical axis indicates a pixel value.
[0114]
In the present invention, with respect to the second field (motion correction field) located between the temporally different image data D1 and image data D2, the correction data is displayed so that the motion looks smooth even in various image variations. Write. That is, in the example shown in FIG. 15, when the image data D1 having the convex portion on the left side is changed to the image data D2 having the gentle convex portion in the center, an image that makes the overall movement look smooth is described above. Create in the motion compensation field.
[0115]
FIG. 16 shows a specific operation example of the image shift unit 16 shown in FIG. 15 by pixel values. FIG. 16A shows image data D1 and image data D2 input to the image shift unit 16, and the numbers are addresses indicating pixel positions. Since each pixel has luminance, a pixel value is assigned for each number in the supplied image data D1.
[0116]
That is, in the operation example shown in FIG. 16A, the image data D1 is represented by pixel values that sequentially follow 100, 100, 200,.
[0117]
Image data D2 located after image data D1 is represented by pixel values that sequentially follow 100, 100, 100,...
[0118]
The motion vector shown in FIG. 16A represents a vector amount between the image data D1 and the image data D2 with reference to the image data D1 for each pixel. For example, the pixel having a pixel value of 100 at the address of number 1 in the image data D1 is also located at the address of number 1 in the image data D2 located one field later. Therefore, the motion vector is zero. Further, for example, the pixel having the pixel value 200 at the address of number 2 in the image data D1 moves to the address of number 4 in the image data D2. Therefore, the motion vector is 2 from 4-2 = 2. Incidentally, the arrow shown in FIG. 15 indicates the motion vector for each pixel.
[0119]
A flag F1 illustrated in FIG. 16A is an example in the case where the difference absolute value between the detection pixel of the image data D1 and the detection pixel of the image data D2 is set. In the example shown in FIG. 16 (a), the pixel having the pixel value 200 at the address of number 2 in the image data D1 moves to the address of number 4 in the image data D2, and the pixel value remains the same as 200. The absolute value is zero. On the other hand, the pixel having the pixel value 110 at the address of the number 7 in the image data D1 moves to the address of the number 10 in the image data D2, and the pixel value becomes 100, so the absolute difference value becomes 10.
[0120]
FIG. 16B shows the shift buffer read control signal RS1. In the example shown in FIG. 16B, based on the equation (1.1), 0, 1, 2, 3,... In addition, a case where the movement amount information α is ½ is shown. The numerical value CX1 of the address calculation counter is output in correspondence with the number indicating the address of the image data D1.
[0121]
When the shift buffer control signal RS1 is generated, for example, when the number of the address calculation counter is “2”, the motion vector at the pixel position corresponding to the number 2 is “2” based on FIG. Therefore, based on the equation (1.1), the number of the shift buffer read control signal RS1 is “3” from 2 + 2 × 1/2 = 3. Similarly, when the number of the address calculation counter is “3”, the motion vector corresponding to the number 3 is “2” based on FIG. 16A, and is substituted into equation (1.1). From 3 + 2 × 1/2 = 4, the number of the shift buffer read control signal RS1 is “4”.
[0122]
That is, the generated number of the shift buffer read control signal RS1 indicates the number of the address to which the correction data H1 is written in the motion correction field.
[0123]
For the calculated address of the shift buffer control signal RS1, the flag F ′ is read in order to detect the writing state of the motion vector to the shift buffer 163. The flag F ′ returns “NM” when no motion vector is written in the address accessed by the shift buffer 163. On the other hand, for the address where the motion vector has already been written, the absolute value of the difference is returned.
[0124]
For example, in the example shown in FIG. 16B, it is displayed that no data is written from the shift buffer 163 via the flag F ′ at addresses 0 to 8 of the shift buffer read control signal RS1. In addition, at the address of number 9, “NM” indicating that data is not written at first is returned as the flag F ′, and the absolute difference value is returned next time. That is, it means that a plurality of motion vectors are written at the address of number 9 in the shift buffer 163. This is also shown in FIG. 15 because the motion vectors based on the numbers 6 and 9 of the image data D1 are concentrated at the address of the number 9 of the image data D2.
[0125]
Here, motion vectors are sequentially written in the address of the shift buffer 163 in which the flag F ′ is returned as “NM” according to the number of the address. When the flag F ′ has a numerical value, the flag F ′ is compared with the flag F corresponding to the address number, and the smaller numerical value is valid. As a result, it becomes possible to write a motion vector with less error before shifting from the image data D1 to the image data D2 to the shift buffer 163, and various variations such that a plurality of motion vectors are applied to a single pixel position. It is also possible to correct motion with high accuracy for images.
[0126]
The shift buffer write control unit 162 determines an address for writing the motion vector on the shift buffer 163 based on the supplied flag F ′. For example, in the number 9, the flag F is “10” and the flag F ′ is “0”. In order to give priority to a flag having a small numerical value, in this number 9, the motion vector “3” based on the address of number 6 that was originally written is continuously stored in the shift buffer 163 as it is.
[0127]
Further, the shift buffer write control unit 162 associates the number corresponding to the address with the motion vector as a shift buffer write control signal RS2, and writes this to the shift buffer 163. In the shift buffer 163 after writing, as shown in the lower part of FIG. 16B, motion vectors are stored for the addresses of the respective numbers.
[0128]
FIG. 16C shows the result of rearranging the motion vectors stored in the shift buffer 163 in the order of address numbers. The processing flag F2 indicates mark information at each address. When “OK” is output in the processing flag F2, this indicates that data is written at the address of the number, and when “NM” is output, the address of the number is displayed. Indicates that no data has been written. Incidentally, since no data is written to the address of number 2, “NM” is output as the processing flag F2.
[0129]
The moved motion vector read from the shift buffer 163 is supplied to the data buffer read control unit 164, and the processing flag F2 is supplied to the data calculation unit 167.
[0130]
The data buffer read control unit 164 generates the buffer control signals S11 and S12 using the above equations (2.1) to (2.4) based on the supplied moved motion vector. For example, when the value of the address calculation counter is a number corresponding to the address of the shift buffer 163, the moved motion vector is “2” at the address of number 3 from FIG. The signal S11 is “2” from the equation (2.1), and the buffer control signal S12 is “4” from the equation (2.3). Similarly, in the address of No. 6, the moved motion vector is “3”, so that the buffer control signal S11 is “5” and the buffer control signal S12 is “8”.
[0131]
The buffer control signal S11 calculated as described above is supplied to the first buffer 165, and the pixel value at the address corresponding to the number of the buffer control signal S11 is read from the first buffer 165. As shown in the middle part of FIG. 16C, the read pixel value is associated with the address as shift data SD1 and supplied to the data calculation unit 167 and the like.
[0132]
Similarly, the buffer control signal S12 is supplied to the second buffer 166, and the pixel value at the address corresponding to the number of the buffer control signal S12 is read from the second buffer 166. The read pixel value is associated with the address as shift data SD2 as shown in the middle part of FIG. 8C, and is supplied to the data calculation unit 167 and the like.
[0133]
The lowermost part of FIG. 16 (c) shows a case where the correction data H1 is an average value of the shift data SD1 and the shift data SD2. The data calculation unit 167 calculates correction data H1 based on the corresponding shift data SD1 and SD2 for the number for which “OK” is output as the processing flag F2 to be transmitted, and the number for which “NM” is output. At the address of 2, the pixel value at the address of the adjacent number 1 is held as it is. However, for the “NM” part, the calculation is performed again by the uncover processing unit 168, and therefore the hold value is substituted here as dummy data.
[0134]
By writing the generated correction data H1 into the motion correction field for each numbered address, an image as shown in FIG. 15A can be created.
[0135]
In other words, the image shift unit 16 selects any one of the motion vectors when the pixels indicated by the motion vectors overlap as in number 9, for example, creates the correction data H1, and writes this in the motion correction field 50. be able to. Thereby, the motion correction circuit 1 according to the present invention can efficiently eliminate the discontinuity of the motion of the image even when the pixel value changes while the image moves in the horizontal direction, for example.
[0136]
Next, an operation example of the uncover processing unit 168 will be described.
[0137]
The pixel interpolation processing by the uncover processing unit 168 is classified into three types, that is, writing processing for pixels around the uncover area, still image processing, and background processing.
[0138]
FIG. 17 is a diagram for describing a case where a writing process for pixels around the uncover area is performed. In this uncovered area, an interpolated pixel position 110 for interpolating the pixel value is set as a pixel position 120 and a pixel position 130 from the left as pixel positions adjacent to the uncovered area in the horizontal direction.
[0139]
The motion vector identification unit 11 first identifies a motion vector at the interpolation pixel position 110. Such a motion vector is specified based on, for example, a difference calculated between the standard field 30 and the reference field 40.
[0140]
Next, the pixel selection unit 12 selects a pixel adjacent in the vertical direction or the horizontal direction of the uncover area based on the vertical component extracted for the motion vector at the specified interpolation pixel position 110. In the selection of the pixels, when the vertical component of the motion vector is +2, −2, the pixels 111 and 112 adjacent in the vertical direction are selected, respectively, and when the vertical component is +1, 0, −1, the horizontal direction is selected. Pixels 120 and 130 adjacent to are selected.
[0141]
That is, when the vertical component of the motion vector at the interpolation pixel position 110 is +2, −2, it indicates that the movement toward the vertical component is more remarkable than the movement of the horizontal component in the pixels around the interpolation pixel position 110. . Therefore, in order to adapt to the pixel values of the pixels around the interpolation pixel position 110, it is preferable to write the pixel values of the pixels adjacent in the vertical direction. The shift amount is often ½ of the vector amount of the motion vector from the viewpoint of realizing smooth motion. For this reason, when +2 and -2 are multiplied by 1/2, +1 and -1 are obtained. Therefore, by writing the pixel values of the pixels 111 and 112 adjacent to each other in the vertical direction of the interpolation pixel position, it is possible to reduce the vector amount from the viewpoint of the vector amount. Matching can be made with the pixel values of the pixels around the cover area.
[0142]
On the other hand, when the vertical component of the motion vector at the interpolation pixel position 110 is +1, 0, −1, it indicates that the movement of the horizontal component is more remarkable than the movement of the vertical component in the pixels around the interpolation pixel position 110. Yes. Therefore, in order to match the pixel values of the pixels around the interpolation pixel position 110, it is preferable to write the pixel values of the pixels 120 and 130 adjacent in the horizontal direction.
[0143]
Next, a case where still image processing is used in an uncovered area processing method to which the present invention is applied will be described.
[0144]
In the still image processing, when the still condition described below is satisfied, the information writing unit 13 directly uses the pixel value at the interpolation pixel position 110 on the reference field 30 to the interpolation pixel position 110 on the motion correction field 50. Write. That is, if the image on the standard field and the image on the reference field 40 after one frame are still, it is sufficient to write the image of the standard field 30 as it is in the motion correction field.
[0145]
In the still condition A, the pixel values calculated between the reference block whose origin is the interpolation pixel position 110 cut out from the reference field 30 and the reference block whose origin is the interpolation pixel position 110 cut out from the reference field 40 are similar. In other words, the difference absolute value of these pixel values is not more than a threshold value.
[0146]
In addition, in addition to the above-described still condition A, the still condition B is similar to the pixel value of the pixel 120 adjacent to the left of the uncovered area and the pixel value of the interpolation pixel position 110 cut out from the reference field 30. In other words, the condition is that the absolute difference between these pixel values is equal to or less than a threshold value.
[0147]
Furthermore, in addition to the above-described still condition A, the still condition C is dissimilar between the pixel value of the pixel 120 adjacent to the left of the uncovered area and the pixel value of the pixel 130 adjacent to the right of the uncovered area (in other words, And the pixel value at the interpolation pixel position 110 on the reference field 30 is the pixel value at the left adjacent pixel 120 and the pixel value at the right adjacent pixel 130. As long as it is between.
[0148]
The still conditions A, B, and C can be cleared as the image on the standard field 30 and the image on the reference field 40 one frame later are closer to the stationary state. Incidentally, it is possible to freely select the stationary conditions A, B, and C according to the specifications of the built-in television receiver.
[0149]
Next, in the uncover area processing method to which the present invention is applied, a case where background processing is used will be described.
[0150]
In this background processing, the vertical component of the motion vector specified by the motion vector specifying unit 11 is identified. When the vertical component of this motion vector is −1 to +1 and each component of the motion vector in the left adjacent pixel 120 or the right adjacent pixel 130 in the uncovered area is 0, it is determined as the background. To do. Then, background processing for writing the pixel value at the interpolation pixel position 110 on the reference field 40 to the interpolation pixel position 110 on the motion correction field 50 as it is is performed. This is because it is sufficient to write the image on the reference field 40 as it is in the motion correction field 50 if it is a background image.
[0151]
The present invention is not limited to being applied to a PAL television receiver. For example, NTSC (National TV System Committee) 60 field second (30 frames / second) interlace. The present invention can also be applied to a television receiver to which an image signal is input. The present invention can also be applied to a SECAM television receiver.
[0152]
In the image shift unit 16 shown in FIG. 3, for example, the present invention eliminates the shift buffer read control unit 161, omits the shift buffer read control signal RS1, the flag F1, and the flag F ′, and does not determine the priority order. It is also applicable to. In this image shift unit 16, when the data to be written to the address overlaps, the data calculated later in time is overwritten on the already written data. Is unnecessary, and the circuit can be further simplified.
[0153]
The present invention can also be applied to, for example, a signal converter connected to a television receiver, and can be realized not only as a circuit or hardware but also as software on a processor.
[0154]
Further, the present invention can be applied to a case where an image signal transmitted over the Internet is displayed on a PC or the like, or a case where media or an image format is converted.
[0155]
【The invention's effect】
As described above in detail, the motion correction circuit and method according to the present invention is a pixel obtained by inputting an image signal composed of four fields per frame generated by double-speed conversion of a telecine-converted image. The first field is identified based on the difference value of the signal level, and the position of the detection pixel is moved in the vector direction of the motion vector so that the shift amount sequentially increases as the field moves from the identified first field to the back. Shift. For this reason, the motion correction circuit and method according to the present invention can eliminate the motion discontinuity peculiar to the image signal subjected to the double speed conversion after the telecine conversion, and the motion of the image in which the surface flicker interference is suppressed by the double speed conversion. Further smoothing can be achieved, and the image quality can be improved synergistically.
[0156]
In addition, the motion correction circuit and method according to the present invention writes optimal correction data that can smoothly move the image between the temporally different image data in the motion correction field. For example, while the image moves in the horizontal direction, Even when the pixel value changes, the discontinuity of the motion of the image can be efficiently eliminated. In addition, this motion correction circuit and method can efficiently eliminate motion discontinuity corresponding to various image variations when both a telecine-converted image signal and a TV signal are input. . As a result, it can be built in a television receiver to which both a film signal and a TV signal are input, and can be easily upgraded by newly incorporating it into a television receiver already sold. It is also possible to improve versatility.
[0157]
In addition, the motion correction circuit and method according to the present invention can obtain the motion vector suitable for the motion of the actual image, thereby eliminating the variation of the motion vector to be obtained and thus preventing the deterioration of the converted image.
[0159]
Furthermore, the motion correction circuit and method according to the present invention selects a pixel adjacent in the vertical direction or horizontal direction of the uncovered area based on the vertical component of the motion vector at the interpolation pixel position, and interpolates the selected pixel. Write to pixel location. As a result, it is possible to fill the uncovered area with pixels that show the optimum pixel value, and it is possible to eliminate isolated points on the motion correction field, thereby preventing deterioration of the converted image.
[Brief description of the drawings]
FIG. 1 is a block diagram of a motion correction circuit to which the present invention is applied.
FIG. 2 is a diagram illustrating a configuration of a motion vector detection unit.
FIG. 3 is a diagram illustrating a block configuration of an image shift unit.
FIG. 4 is a diagram showing the relationship between each field and pixel position before and after double speed conversion in a field double speed conversion circuit.
FIG. 5 is a diagram illustrating a relationship between each field and an image position when the image is moved in the horizontal direction in an image subjected to telecine conversion.
FIG. 6 is a diagram illustrating a relationship between each field and an image position when an image moves in a horizontal direction in an image of a TV signal.
FIG. 7 is a diagram for explaining a sequence detection method;
FIG. 8 is a diagram for explaining a block matching method;
FIG. 9 is a diagram for describing a case where a motion vector is detected in light of a distribution tendency of a sum of absolute differences.
FIG. 10 is a diagram for explaining a case where motion vectors are obtained in order from the left pixel to the right pixel.
FIG. 11 is a diagram for describing a case where a motion vector is obtained according to a motion vector of an adjacent pixel.
FIG. 12 is a flowchart showing a procedure of motion vector correction processing;
FIG. 13 is a diagram illustrating adjacent pixels from which motion vectors are extracted in motion vector correction processing.
FIG. 14 is a diagram for describing correction processing performed when a motion vector transmitted from a block matching unit goes out of a search range.
FIG. 15 is a diagram in which an operation processing process of the image shift unit is displayed in one dimension.
FIG. 16 is a diagram showing a specific operation example of the image shift unit displayed by pixel values.
FIG. 17 is a diagram for describing a case of interpolating pixels with respect to an uncovered area.
FIG. 18 is a block configuration diagram of a field double speed conversion circuit to which a field frequency double speed system is applied.
FIG. 19 is a diagram showing the relationship between each field and pixel position before and after double speed conversion.
FIG. 20 is a diagram illustrating a relationship between each field and an image position when the image moves in the horizontal direction.
FIG. 21 is a diagram illustrating a relationship between each field and an image position when an image moves in the horizontal direction when a TV signal is input.
FIG. 22 is a diagram for describing an example of detecting a motion vector in units of blocks.
FIG. 23 is a diagram showing a case where an uncovered area occurs.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image signal processing apparatus, 2 CRT, 3 field double speed conversion circuit, 11 1st image memory, 12 2nd image memory, 13 Sequence detection part, 14 Data selection part, 15 Motion vector detection part, 16 Image shift part, 31 input terminal, 32 double speed conversion unit, 33 frame memory, 151 block matching unit, 152 out-of-range detection unit, 153 reallocation unit, 161 shift buffer read control unit, 162 shift buffer write control unit, 163 shift buffer, 164 data Buffer read control unit, 165 first buffer, 166 second buffer, 167 data operation unit, 168 uncover processing unit

Claims (12)

One frame generated is input image signal composed of four fields by telecine converted image double-speed conversion and a reference field for the position of the pixels for obtaining the motion vector, the reference field or et 2 frames away based on the motion vector obtained between the reference field it has, had row motion correction on the correction field to the position of the pixel for motion compensation, or one frame generated by double speed conversion of the television signal 2 An image signal composed of a field is input, and a pixel for which motion correction is performed based on a motion vector obtained between a reference field in which a pixel for which a motion vector is obtained is located and a reference field one frame away from the reference field. in line cormorant motion compensation circuit the motion correction on the correction field located,
Motion vector detection means for detecting a motion vector for the reference field for a reference block centered on a reference pixel of the reference field;
Vector allocating means for allocating the motion vector detected in the reference block for each pixel constituting the reference block;
A selection unit that selects any one of the motion vectors when pixels indicated by the motion vector allocated by the vector allocation unit overlap;
On the correction field, the pixel value of the pixel to which the motion vector is assigned is shifted in the vector direction within the range of the vector amount of the motion vector according to the weight assigned to each correction field, Image control means for
A motion correction circuit comprising: pixel interpolation means for interpolating a pixel value with respect to an information absent region where no pixel value exists on the correction field.   The motion vector detection means calculates a sum of absolute differences of pixel values between the base block and a search block moving within a search range cut out from the reference field, and the sum of absolute differences is minimized. 2. The motion correction circuit according to claim 1, wherein a motion vector is detected based on a position of a search block and a position of the reference block.   2. The motion correction circuit according to claim 1, wherein the motion vector detection means detects the motion vector based on a block matching method. 4. The motion correction circuit according to claim 3 , wherein the motion vector detection means detects the motion vector based on block matching in a single direction.   2. The motion correction circuit according to claim 1, wherein the motion vector detection means detects the motion vector in accordance with a motion vector obtained in a block adjacent to a reference block where the reference pixel is located. When an image signal composed of four fields for one frame generated by double-speed conversion of the telecine-converted image is input, the pixel values at the same pixel position are spatially identical between fields one frame apart. Sequence detecting means for calculating a difference value and identifying a first field first located from an image signal composed of four fields constituting the one frame based on the difference value,
The image control means, within the range of the detected motion vector, each time a field continues from the first field to an image signal composed of four fields constituting one frame. 2. The motion correction circuit according to claim 1, wherein the shift positions are sequentially increased. One frame generated by converting the telecine converted image speed is input image signal composed of four fields, a reference field for the position of the pixels for obtaining the motion vector, the reference field or et 2 frames away based on the motion vector obtained between the reference field it has, had row motion correction on the correction field to the position of the pixel for motion compensation, or one frame generated by double speed conversion of the television signal 2 An image signal composed of a field is input, and a pixel for which motion correction is performed based on a motion vector obtained between a reference field in which a pixel for which a motion vector is obtained is located and a reference field one frame away from the reference field. in line cormorant motion compensation method the motion correction on the correction field located,
Detecting a motion vector for the reference field for a reference block centered on a reference pixel of the reference field;
The motion vector detected in the reference block is assigned to each pixel constituting the reference block,
If pixels assigned by the assigned motion vector overlap, select one of the motion vectors,
On the correction field, the pixel value of the pixel to which the motion vector is assigned is the pixel value at the position shifted in the vector direction within the range of the vector amount of the motion vector according to the weight assigned to each correction field. Also, a motion correction method characterized by interpolating a pixel value with respect to an information absent area where no pixel value exists. A difference absolute value sum of pixel values is calculated between the reference block and a search block that moves within a search range cut out from the reference field, and the position of the search block that minimizes the difference absolute value sum and the reference 8. The motion correction method according to claim 7 , wherein a motion vector is detected based on a block position. 8. The motion correction method according to claim 7, wherein the motion vector is detected based on a block matching method. The motion correction method according to claim 9 , wherein the motion vector is detected based on block matching in a single direction. The motion correction method according to claim 7 , wherein the motion vector is detected according to a motion vector obtained in a block adjacent to the reference block where the reference pixel is located. When an image signal composed of four fields for one frame generated by double-speed conversion of the telecine-converted image is input, the pixel values at the same pixel position are spatially identical between fields one frame apart. The difference value is calculated , and based on the difference value, the first field located first from the image signal composed of the four fields constituting the one frame is specified,
Each time the field continues from the first field to the image signal composed of four fields constituting the one frame, the position to be shifted is within the range of the detected motion vector. The motion correction method according to claim 7 , wherein the motion correction method sequentially increases.

Images