JP2006238372A - Coder and coding method, decoder and decoding method, image processing system and image processing method, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To decode coded data without transmitting residual information. <P>SOLUTION: A reference frame is fed to a reference frame coding section 102, wherein the reference frame is coded, and other frames are fed to a frame memory 105. A current frame fed to the frame memory 105 and a preceding frame stored in a frame memory 106 are respectively fed via high pass filters to a block matching motion searching section 108, wherein a first motion vector is obtained. Moreover, an absolute value error calculation section 109 calculates a prediction error by the first motion vector. Moreover, an evaluation value revision block matching motion searching section 111 searches a second motion vector on the basis of a weighting by the first motion vector. Further, an optimum motion determining section 112 determines which motion vector is to be used at decoding by each pixel. This invention can be applicable to an image display system, a coder, and a decoder. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an encoding apparatus and method, a decoding apparatus and method, an image processing system and an image processing method, a recording medium, and a program, and in particular, an encoding apparatus and method for preventing unauthorized copying using an analog signal, and a decoding apparatus. And an image processing system, an image processing method, a recording medium, and a program.

  2. Description of the Related Art In recent years, digital recording / reproducing apparatuses that record content such as television programs on a recording medium such as an HD (Hard Disk) or DVD (Digital Versatile Disk) as a digital signal have been rapidly spread.

  With the widespread use of digital recording / playback devices that use HD and DVD as recording media, users as viewers can easily record television programs and the like on recording media with high quality.

  On the other hand, with the widespread use of digital recording and playback devices, there is also an aspect that it becomes easier to illegally copy content such as TV programs and movies sold on DVDs.

  FIG. 1 shows an example of the configuration of an image processing system that reproduces content recorded on a recording medium, displays the content on a display, and records the reproduced content on another recording medium.

  In FIG. 1, an image processing system 1 reproduces an image signal of a content recorded on a recording medium such as an optical disc such as a DVD, and outputs a resulting analog image signal. A display 12 that displays an analog image signal as an image, and a recording device 13 that records on a recording medium such as an optical disk using the analog image signal output from the playback device 11 are configured.

  The playback device 11 includes a decoding unit 21 and a D / A (Digital-to-Analog) conversion unit 22. The decoding unit 21 decodes an encoded digital image signal read from a recording medium (not shown), and supplies the resulting digital image signal to the D / A conversion unit 22. The D / A conversion unit 22 converts the digital image signal supplied from the decoding unit 21 into an analog signal, and outputs the resulting analog image signal.

  The display 12 is composed of, for example, a CRT (Cathode-Ray Tube), an LCD (Liquid Crystal Display), or the like, and displays an analog image signal from the D / A converter 22 as an image. As a result, the user can view an image corresponding to the image signal recorded on the recording medium.

  The analog image signal output from the playback device 11 is also supplied (input) to the recording device 13.

  The recording device 13 includes an A / D (Analog-to-Digital) conversion unit 31, an encoding unit 32, and a recording unit 33, and records an input analog image signal on a recording medium (not shown) such as an optical disk. .

  An analog image signal output from the playback device 11 is input to the A / D converter 31. The A / D converter 31 converts the input analog image signal into a digital signal and supplies the resulting digital image signal to the encoder 32. The encoding unit 32 encodes the digital image signal from the A / D conversion unit 31 and supplies the encoded digital image signal obtained as a result to the recording unit 33. The recording unit 33 records the encoded digital image signal on a recording medium.

  In the image processing system 1 configured as described above, an analog image signal output from the playback device 11 can be used to record an image signal on a recording medium different from the played back recording medium. In other words, the content (image signal) may be illegally copied using the analog image signal output from the playback device 11.

  Conventionally, in order to prevent unauthorized copying using such an analog image signal, when copyright protection is provided, the analog image signal is scrambled and output, or the output of the analog image signal is prohibited. It has been proposed.

  However, the method of scrambling and outputting an analog image signal or prohibiting the output of an analog image signal can prevent unauthorized copying, but cannot display a normal image on the display 12. Will occur.

  Therefore, a method for preventing unauthorized copying using an analog image signal without causing inconvenience such as an image not being displayed or an increase in circuit scale has been proposed by the present applicant (for example, Patent Document 1).

No. 2004-289585

  In the method described in Patent Document 1, attention is paid to the phase shift of a digital image signal obtained by A / D conversion of an analog image signal, and the digital image signal is encoded by focusing on the phase shift. Although it is impossible to copy while maintaining good quality without degrading the quality of the image before copying, this prevents unauthorized copying using analog image signals, but the distribution of digital content has become common In recent years, there has been a demand for a proposal of another method for preventing unauthorized copying as described above.

  The present invention has been made in view of such a situation, and makes it possible to prevent unauthorized copying using an analog signal.

  The encoding apparatus according to the present invention estimates a first motion vector in a processing block having a plurality of processing regions in first frame data and second frame data different from the first frame data. A motion vector estimating means; a second motion vector estimating means for estimating a second motion vector different from the first motion vector in the processing block; and a first motion vector and a second motion for each processing region. Corresponding to the processing region based on one of the motion vector selection means for selecting one of the vectors, and the first motion vector and the second motion vector selected by the motion vector selection means A first motion vector selected by the motion vector selection means using the extraction means for extracting the pixels to be extracted and the pixels extracted by the extraction means When anda predictive data calculation means for calculating a prediction data for predicting the pixel data of the processing region corresponding to either one of the second motion vector.

  Image processing means for applying processing for enhancing the edge of the image to the first frame data and the second frame data can be further provided.

  The image processing means may be a high pass filter.

  A noise adding means for adding noise to the frame data can be further provided.

  An error calculating means for calculating a detection error of the first motion vector based on the first motion vector estimated by the first motion vector estimating means and the first frame data is further provided. The first motion vector estimation means can be made to estimate the first motion vector by block matching processing for the first frame data and the second frame data, and the second motion vector The estimation means includes the second motion by block matching processing weighted on the first frame data and the second frame data based on the detection error of the first motion vector calculated by the error calculation means. The vector can be estimated.

  The prediction data calculation means is based on the relationship between the sum of values obtained by multiplying the pixel values of the plurality of pixels extracted by the extraction means by predetermined coefficients and the pixel values of the pixels in the corresponding first frame data. In addition, using the least square method, the optimal value of the coefficient corresponding to the first motion vector is set as the first prediction data, and the optimal value of the coefficient corresponding to the second motion vector is set as the second prediction. Each data can be calculated.

  The prediction data calculated by the prediction data calculation means may include the first prediction data corresponding to the first motion vector and the second prediction data corresponding to the second motion vector. it can.

  An output unit that outputs an encoding result corresponding to the first frame data can be further provided, and the output unit includes a first motion vector estimated by the first motion vector estimation unit, The second motion vector estimated by the second motion vector estimation means, the data indicating the selection result by the motion vector selection means, and the prediction data calculated by the prediction data calculation means can be output. .

  The encoding method of the present invention is configured to estimate a first motion vector in a processing block having a plurality of processing regions in first frame data and second frame data different from the first frame data. A motion vector estimation step, a second motion vector estimation step for estimating a second motion vector different from the first motion vector in the processing block, and a first motion vector and a second motion for each processing region. A processing region based on one of a motion vector selection step for selecting one of the vectors and a first motion vector and a second motion vector selected by the processing of the motion vector selection step Extraction step to extract the pixel corresponding to, and motion vector selection using the pixel extracted by the processing of extraction step A prediction data calculation step of calculating prediction data for predicting pixel data in a processing region corresponding to one of the first motion vector and the second motion vector selected by the processing of the step. It is characterized by that.

  It is possible to further include an image processing step for performing processing for enhancing the edge of the image with respect to the first frame data and the second frame data.

  The processing of the image processing step can be processing for applying a high-pass filter.

  A noise adding step for adding noise to the frame data can be further included.

  An error calculation step of calculating a detection error of the first motion vector based on the first motion vector estimated by the processing of the first motion vector estimation step and the first frame data is further included. In the process of the first motion vector estimation step, the first motion vector is estimated by the block matching process for the first frame data and the second frame data, and the second motion vector estimation step In the processing, the first motion data and the second frame data are subjected to the second motion by block matching processing weighted based on the detection error of the first motion vector calculated by the processing of the error calculation step. A process for estimating a vector can be executed.

  In the processing of the prediction data calculation step, the sum of the values obtained by multiplying the pixel values of the plurality of pixels extracted by the processing of the extraction step by respective predetermined coefficients and the pixel values of the pixels in the corresponding first frame data Based on the relationship, using the least square method, the optimal value of the coefficient when corresponding to the first motion vector is the first prediction data, and the optimal value of the coefficient when corresponding to the second motion vector is the first The second prediction data can be calculated respectively.

  The prediction data calculated by the process of the prediction data calculation step may include first prediction data corresponding to the first motion vector and second prediction data corresponding to the second motion vector.

  The method further includes an output step of outputting an encoding result corresponding to the first frame data. In the processing of the output step, the first motion vector estimated by the processing of the first motion vector estimation step, and the second motion Outputting the second motion vector estimated by the vector estimation step processing, the data indicating the selection result by the motion vector selection step processing, and the prediction data calculated by the prediction data calculation step processing Can do.

  The program recorded on the first recording medium of the present invention includes the first frame data and the second frame data different from the first frame data in the first block in the processing block having a plurality of processing areas. A first motion vector estimation step for estimating a motion vector, a second motion vector estimation step for estimating a second motion vector different from the first motion vector in the processing block, and a first for each processing region A motion vector selection step for selecting one of the motion vector and the second motion vector, and any one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step Based on either of these, an extraction step for extracting pixels corresponding to the processing region, and extraction by the processing of the extraction step Using the element, the prediction data for predicting the pixel data of the processing region corresponding to one of the first motion vector and the second motion vector selected by the motion vector selection step is calculated. And a prediction data calculation step.

  The first program of the present invention is configured to estimate a first motion vector in a processing block having a plurality of processing regions in the first frame data and second frame data different from the first frame data. , A second motion vector estimation step for estimating a second motion vector different from the first motion vector in the processing block, and a first motion vector and a second motion vector for each processing region Processing based on one of a motion vector selection step for selecting one of the motion vectors, and the first motion vector and the second motion vector selected by the processing of the motion vector selection step Using the extraction step for extracting pixels corresponding to the region and the pixels extracted by the processing of the extraction step, A prediction data calculation step for calculating prediction data for predicting pixel data in a processing region corresponding to one of the first motion vector and the second motion vector selected by the processing of the selection step; The computer is caused to execute a process characterized by including.

  The decoding apparatus according to the present invention performs any one of a first motion vector in a processing block having a plurality of processing regions, a second motion vector in the processing block, and the first motion vector and the second motion vector. A motion vector determining means for receiving an input of optimal motion vector information for each processing region indicating whether it is optimal to use for decoding, and determining an optimal motion vector for each processing region of the processing block; and pixels included in the processing region And a calculation unit that receives input of prediction data for predicting data and calculates pixel data in the processing block based on the motion vector determined by the motion vector determination unit and the prediction data. To do.

  The pixel data calculated by the calculating means can be further provided with noise adding means for adding noise.

  The prediction data can include first prediction data corresponding to the first motion vector and second prediction data corresponding to the second motion vector.

  Which of the first prediction data and the second prediction data is used for the processing by the calculation means for each processing region upon receiving the input of the first prediction data, the second prediction data, and the optimal motion vector information It is possible to further include prediction data selection means for selecting.

  Prediction data is obtained by using the method of least squares based on the relationship between the sum of values obtained by multiplying the peripheral pixels corresponding to the processing region by predetermined coefficients and the pixel values of the corresponding pixels in the first frame data. It can be the calculated optimum value of the coefficient.

  Based on the motion vector determined by the motion vector determination unit, the calculation unit extracts the pixel value of the peripheral pixel corresponding to the target pixel, and multiplies the extracted pixel value by the prediction data, thereby The pixel data can be calculated.

  In the decoding method of the present invention, any one of the first motion vector in the processing block having a plurality of processing regions, the second motion vector in the processing block, and the first motion vector and the second motion vector is selected. A motion vector determining step for receiving optimal motion vector information for each processing region indicating whether it is optimal to use for decoding and determining an optimal motion vector for each processing region of the processing block; and pixels included in the processing region Including a calculation step of receiving pixel data in the processing block based on the motion vector determined by the processing of the motion vector determination step and the prediction data in response to input of prediction data for predicting data Features.

  It is possible to further include a noise adding step for adding noise to the pixel data calculated by the processing of the calculating step.

  The prediction data can include first prediction data corresponding to the first motion vector and second prediction data corresponding to the second motion vector.

  In response to the input of the first prediction data, the second prediction data, and the optimal motion vector information, for each processing region, either the first prediction data or the second prediction data is used for the processing by the calculation step. A prediction data selection step for selecting whether to use can be further included.

  Prediction data is obtained by using the method of least squares based on the relationship between the sum of values obtained by multiplying the peripheral pixels corresponding to the processing region by predetermined coefficients and the pixel values of the corresponding pixels in the first frame data. It can be the calculated optimum value of the coefficient.

  In the process of the calculation step, based on the motion vector determined by the process of the motion vector determination step, the pixel value of the peripheral pixel corresponding to the target pixel is extracted, and the extracted pixel value is multiplied by the prediction data. A process for calculating pixel data in the processing block can be included.

  The program recorded on the second recording medium of the present invention includes a first motion vector in a processing block having a plurality of processing areas, a second motion vector in the processing block, and a first motion vector and a second motion vector. Motion vector determination that receives optimal motion vector information for each processing region and determines the optimal motion vector for each processing region of the processing block. Pixel data in the processing block is received based on the motion vector determined by the processing of the motion vector determination step and the prediction data, upon receiving the input of the prediction data for predicting the pixel data included in the step and the processing region And a calculating step for calculating.

  The second program of the present invention includes a first motion vector in a processing block having a plurality of processing regions, a second motion vector in the processing block, and a first motion vector and a second motion vector. Included in the processing region is a motion vector determination step that receives the input of the optimal motion vector information indicating which is optimal for decoding for each processing region, and determines the optimal motion vector for each processing region of the processing block. A calculation step of calculating pixel data in the processing block based on the motion vector determined by the processing of the motion vector determination step and the prediction data. The computer is caused to execute a process characterized by the above.

  The first image processing system of the present invention includes an encoding unit and a decoding unit, and the encoding unit and the decoding unit repeat encoding processing and decoding processing on image data composed of a plurality of frame data. Thus, the image processing system is configured to degrade the image data, and the encoding unit includes the first frame data included in the image data and the second frame data different from the first frame data. A first motion vector estimating means for estimating a first motion vector in a processing block having a plurality of processing regions; and a second motion vector for estimating a second motion vector different from the first motion vector in the processing block. Motion vector estimation means and motion for selecting one of the first motion vector and the second motion vector for each processing region An extractor for extracting a pixel corresponding to the processing region based on one of the first motion vector and the second motion vector selected by the motion vector selector; Prediction data for predicting pixel data in a processing region corresponding to one of the first motion vector and the second motion vector selected by the motion vector selection means using the pixels extracted by And prediction data calculation means for calculating.

  Noise adding means for adding noise to the frame data can be further provided. The image data output from the decoding unit is supplied to the encoding unit after noise is added to the frame data by the noise adding unit. Can be.

  A first image processing method of the present invention includes an encoding unit and a decoding unit, and the encoding unit and the decoding unit repeat encoding processing and decoding processing on image data composed of a plurality of frame data. The image processing method of the image processing system configured to degrade the image data, wherein a plurality of processing regions are defined in the first frame data and the second frame data different from the first frame data. A first motion vector estimation step for estimating a first motion vector in a processing block, and a second motion vector estimation step for estimating a second motion vector different from the first motion vector in the processing block; A motion vector selection block for selecting one of the first motion vector and the second motion vector for each processing region. And an extraction step for extracting pixels corresponding to the processing region based on one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step; Predict pixel data of a processing region corresponding to one of the first motion vector and the second motion vector selected by the motion vector selection step processing using the pixels extracted by the step processing. And a prediction data calculation step for calculating prediction data for the purpose.

  The second image processing system of the present invention includes an encoding unit and a decoding unit, and the encoding unit and the decoding unit repeat encoding processing and decoding processing on image data composed of a plurality of frame data. The image processing system is configured to degrade the image data, and the decoding unit includes: a first motion vector in a processing block having a plurality of processing regions; a second motion vector in the processing block; Optimal motion vector information is input for each processing region indicating which of the first motion vector and the second motion vector is best used for decoding, and the optimal motion is determined for each processing region of the processing block. The motion vector determination means for determining the vector and the prediction data for predicting the pixel data included in the processing region are input, and the motion vector is received. A motion vector determined by Le determining means, based on the prediction data, characterized by comprising a calculating means for calculating the pixel data in the processing block.

  Noise adding means for adding noise to the frame data can be further provided. The image data output from the decoding unit is supplied to the encoding unit after noise is added to the frame data by the noise adding unit. Can be.

  A second image processing method of the present invention includes an encoding unit and a decoding unit, and the encoding unit and the decoding unit repeat encoding processing and decoding processing on image data composed of a plurality of frame data. According to the image processing method of the image processing system configured to degrade the image data, the first motion vector in the processing block having a plurality of processing regions, the second motion vector in the processing block, and the first Optimal motion vector information is input for each processing region indicating which of the first motion vector and the second motion vector is best used for decoding, and the optimal motion is determined for each processing region of the processing block. A motion vector determination step for determining a vector and input of prediction data for predicting pixel data included in the processing region, A motion vector determined by the processing of the vector determining step, based on the prediction data, characterized in that it comprises a calculation step of calculating the pixel data in the processing block.

  In the encoding apparatus and encoding method, the first program, and the first image processing system and image processing method of the present invention, the first frame data and the second frame different from the first frame data In the data, a first motion vector in a processing block having a plurality of processing regions is estimated, and in the processing block, a second motion vector different from the first motion vector is estimated. For each processing region, the first motion vector is estimated. Either one of the motion vector and the second motion vector is selected, and a pixel corresponding to the processing region is selected based on one of the selected first motion vector and second motion vector. A processing region corresponding to one of the first motion vector and the second motion vector selected using the extracted pixels. Prediction data for predicting the pixel data is calculated.

  In the decoding apparatus and decoding method, the second program, and the second image processing system and image processing method of the present invention, the first motion vector in the processing block having a plurality of processing regions, the second in the processing block Processing block processing is received by receiving input of motion vector and optimum motion vector information for each processing region indicating which of the first motion vector and the second motion vector is best used for decoding. An optimal motion vector is determined for each region, receives prediction data for predicting pixel data included in the processing region, and based on the motion vector determined by the processing of the motion vector determination step and the prediction data Pixel data in the processing block is calculated.

  According to the present invention, image data can be encoded, and in particular, encoding and decoding processes can be performed without residual information by searching for two motion vectors.

  In addition, according to another aspect of the present invention, image data can be decoded. In particular, by decoding image data using one of two motion vectors, encoding and decoding processing can be performed without residual information. It can be carried out.

  Embodiments of the present invention will be described below. Correspondences between constituent elements described in the claims and specific examples in the embodiments of the present invention are exemplified as follows. This description is to confirm that specific examples supporting the invention described in the claims are described in the embodiments of the invention. Accordingly, although there are specific examples that are described in the embodiment of the invention but are not described here as corresponding to the configuration requirements, the specific examples are not included in the configuration. It does not mean that it does not correspond to a requirement. On the contrary, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the embodiments of the invention are described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, a divisional application will be made. Nor does it deny the existence of an invention added by amendment.

  The encoding device according to claim 1 (for example, the encoding device 63 or the encoding unit 82 in FIG. 2) includes a plurality of first frame data and second frame data different from the first frame data. First motion vector estimation means (for example, block matching motion search in FIG. 3) for estimating a first motion vector in a processing block (for example, a block when performing motion search) having a plurality of processing regions (for example, pixels) Unit 108), a second motion vector estimation unit that estimates a second motion vector different from the first motion vector in the processing block (for example, the evaluation value change block matching motion search unit 111 in FIG. 3), Motion vector selection means (for example, FIG. 3) for selecting either the first motion vector or the second motion vector for each processing region. Based on one of the optimal motion determination unit 112) and the first motion vector and the second motion vector selected by the motion vector selection unit, the pixel corresponding to the processing region (for example, FIG. 13). Select by the motion vector selection means using the extraction means (for example, the first tap extraction section 246 and the second tap extraction section 247 in FIG. 12) and the pixels extracted by the extraction means. Prediction data calculation means (for example, a filter coefficient) for calculating pixel data in a processing region corresponding to one of the first and second motion vectors And a first neighborhood prediction filter coefficient calculation unit 248 and a second neighborhood prediction filter coefficient calculation unit 249) of FIG.

  The image processing means (for example, the high-pass filter 106 and the high-pass filter 107 in FIG. 3) that performs processing for enhancing the edge of the image with respect to the first frame data and the second frame data can be further provided.

  Noise addition means (for example, the A / D conversion unit 81 in FIG. 2) for adding noise to the frame data can be further provided.

  Based on the first motion vector estimated by the first motion vector estimation means and the first frame data, error calculation means for calculating a detection error of the first motion vector (for example, the absolute value in FIG. 3). An error calculation unit 109), and the first motion vector estimation means can estimate the first motion vector by block matching processing for the first frame data and the second frame data, The second motion vector estimation means performs block matching processing weighted on the first frame data and the second frame data based on the detection error of the first motion vector calculated by the error calculation means, A second motion vector can be estimated.

  The prediction data calculation means is a relationship between the sum of values obtained by multiplying the pixel values of the plurality of pixels extracted by the extraction means by a predetermined coefficient and the pixel values of the pixels in the corresponding first frame data (for example, Based on Equations 3 and 4), using the least square method, the optimum value of the coefficient corresponding to the first motion vector is used as the first prediction data, and the coefficient corresponding to the second motion vector is used. Can be calculated as the second prediction data.

  An output unit (for example, the code output unit 115 in FIG. 3) that outputs an encoding result corresponding to the first frame data can be further provided, and the output unit is the first estimated by the first motion vector estimation unit. 1 motion vector, the second motion vector estimated by the second motion vector estimation means, the data indicating the selection result by the motion vector selection means, and the prediction data calculated by the prediction data calculation means are output. be able to.

  The encoding method according to claim 9 is an encoding method of an encoding device (for example, the encoding device 63 or the encoding unit 82 in FIG. 2) that receives and encodes image data including a plurality of frame data. In the first frame data and the second frame data different from the first frame data, a processing block having a plurality of processing regions (for example, pixels) (for example, a block for executing motion search) ) And a second motion vector different from the first motion vector in the processing block (step S7 and step S8 in FIG. 17). A second motion vector estimation step (for example, the process of step S10 in FIG. 17) and a first motion vector for each processing region. Vector selection step (for example, the process of step S11 in FIG. 17) for selecting either the first motion vector or the second motion vector, and the first motion vector selected by the process of the motion vector selection step and the second motion vector An extraction step (for example, the process of step S83 or step S86 of FIG. 20) for extracting pixels (for example, five taps of FIG. 13) corresponding to the processing region based on any one of the motion vectors of , Pixel data of a processing area corresponding to one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step using the pixels extracted by the processing of the extraction step A prediction data calculation step (for example, the step of FIG. 20) for calculating prediction data (for example, a filter coefficient) for predicting Characterized in that it comprises a S85 or the processing in step S88) and.

  An image processing step (for example, the processing in step S6 in FIG. 17) for performing processing for enhancing the edge of the image on the first frame data and the second frame data can be further included.

  A noise adding step for adding noise to the frame data may be further included.

  Based on the first motion vector estimated by the processing of the first motion vector estimation step and the first frame data, an error calculation step for calculating a detection error of the first motion vector (for example, FIG. 17). Step S9) can be further provided. In the first motion vector estimation step, the first motion vector can be estimated by block matching processing on the first frame data and the second frame data. In the process of the second motion vector estimation step, the first frame data and the second frame data are weighted based on the detection error of the first motion vector calculated by the error calculation step. The process of estimating the second motion vector can be executed by the block matching process.

  In the processing of the prediction data calculation step, the sum of the values obtained by multiplying the pixel values of the plurality of pixels extracted by the processing of the extraction step by respective predetermined coefficients and the pixel values of the pixels in the corresponding first frame data Based on the relationship (for example, Equation 3 and Equation 4), using the least square method, the optimum value of the coefficient when corresponding to the first motion vector corresponds to the second motion vector as the first prediction data In this case, the optimum coefficient values can be calculated as the second prediction data.

  An output step (for example, the process of step S14 in FIG. 17) for outputting an encoding result corresponding to the first frame data can be further included. In the process of the output step, the process of the first motion vector estimation step is performed. The estimated first motion vector, the second motion vector estimated by the processing of the second motion vector estimation step, the data indicating the selection result by the processing of the motion vector selection step, and the processing of the prediction data calculation step The prediction data calculated by the above can be output.

  Further, in the program recorded on the recording medium according to claim 17 and the program according to claim 18, an embodiment (however, an example) to which each step corresponds corresponds to the code according to claim 9. This is the same as the conversion method.

  The decoding device according to claim 19 (for example, the decoding unit 71 or the decoding unit 84 in FIG. 2 or the decoder 114 in FIG. 3) is a processing block (for example, motion search) having a plurality of processing regions (for example, pixels). It is optimal to use any one of the first motion vector in the block for executing the second motion vector, the second motion vector in the processing block, and the first motion vector and the second motion vector for decoding. Motion vector determination means that receives the input of the optimal motion vector information for each processing region and determines the optimal motion vector for each processing region of the processing block (for example, the motion target position calculation unit 273 in FIG. 14 or FIG. 16) And input of prediction data (for example, filter coefficient) for predicting the pixel data included in the processing region, Calculation means (for example, the pixel cutout unit 274 and the filter coefficient multiplication unit 276 in FIG. 14 or 16) that calculates pixel data in the processing block based on the determined motion vector and the prediction data is provided. Features.

  Noise addition means (for example, the D / A conversion unit 72 or the D / A conversion unit 85 in FIG. 2) for adding noise to the pixel data calculated by the calculation unit can be further provided.

  Which of the first prediction data and the second prediction data is used for the processing by the calculation means for each processing region upon receiving the input of the first prediction data, the second prediction data, and the optimal motion vector information Prediction data selection means (for example, the coefficient selection unit 275 in FIG. 14 or 16) can be further provided.

  The prediction data is a relationship between the sum of values obtained by multiplying the peripheral pixels (for example, five taps in FIG. 13) corresponding to the processing region by predetermined coefficients and the pixel values of the pixels in the corresponding first frame data. Based on (for example, Formula 3 and Formula 4), it can be an optimum value of the coefficient calculated using the least square method.

  Based on the motion vector determined by the motion vector determination unit, the calculation unit extracts the pixel value of the peripheral pixel corresponding to the target pixel (for example, the processing of the pixel cutout unit 274 in FIG. 14 or FIG. 16) and is extracted. The pixel data in the processing block can be calculated by multiplying the obtained pixel value by the prediction data (for example, the processing of the filter coefficient multiplication unit 276 in FIG. 14 or 16).

  The decoding method according to claim 25 is a decoding method of a decoding device (for example, the decoding unit 71 or the decoding unit 84 in FIG. 2 or the decoder 114 in FIG. 3) which receives and decodes encoded image data. A first motion vector in a processing block (eg, a block when performing motion search) having a plurality of processing regions (eg, pixels), a second motion vector in the processing block, and a first The input of the optimal motion vector information for each processing region indicating which of the motion vector and the second motion vector is optimal for decoding is received, and the optimal motion vector is determined for each processing region of the processing block. A motion vector determination step to be determined (for example, the processing in step S115 in FIG. 21) and prediction data for predicting pixel data included in the processing region For example, a calculation step (for example, step S118 in FIG. 21) that receives input of a filter coefficient and calculates pixel data in the processing block based on the motion vector determined by the processing of the motion vector determination step and the prediction data. The processing is included.

  A noise addition step of adding noise to the pixel data calculated by the calculation step processing can be further included.

  In response to the input of the first prediction data, the second prediction data, and the optimal motion vector information, for each processing region, either the first prediction data or the second prediction data is used for the processing by the calculation step. A prediction data selection step (for example, the process of step S117 in FIG. 21) for selecting whether to use can be further included.

  The prediction data is a relationship between the sum of values obtained by multiplying the peripheral pixels (for example, five taps in FIG. 13) corresponding to the processing region by predetermined coefficients and the pixel values of the pixels in the corresponding first frame data. Based on (for example, Formula 3 and Formula 4), it can be an optimum value of the coefficient calculated using the least square method.

  In the calculation step processing, pixel values of peripheral pixels corresponding to the target pixel are extracted based on the motion vector determined by the motion vector determination step processing (for example, the processing in step S116 in FIG. 21). A process of calculating pixel data in the processing block can be included by multiplying the pixel value by the prediction data (for example, the process of step 118 in FIG. 21).

  Further, in the program recorded on the recording medium according to claim 31 and the program according to claim 32, an embodiment (however, an example) corresponding to each step is the decoding according to claim 25. It is the same as the method.

  An image processing system according to a thirty-third aspect includes an encoding unit (for example, the encoding device 63 or the encoding unit 82 in FIG. 2) and a decoding unit (for example, the decoding unit 71 or the decoding unit 84 in FIG. 3, and the encoding unit and the decoding unit repeat the encoding process and the decoding process on the image data composed of a plurality of frame data so that the image data is deteriorated. In the image processing system, the encoding unit includes a plurality of processing regions (for example, pixels) in the first frame data included in the image data and the second frame data different from the first frame data. First motion vector estimating means (for example, a first motion vector estimating means for estimating a first motion vector in a processing block (for example, a block for executing motion search) having 3, and a second motion vector estimation unit that estimates a second motion vector different from the first motion vector in the processing block (for example, the evaluation value change block in FIG. 3). A matching motion search unit 111), and a motion vector selection unit (for example, the optimal motion determination unit 112 in FIG. 3) for selecting one of the first motion vector and the second motion vector for each processing region; Based on one of the first motion vector and the second motion vector selected by the motion vector selection means, a pixel (for example, five taps in FIG. 13) corresponding to the processing region is extracted. Using the extraction means (for example, the first tap extraction unit 246 and the second tap extraction unit 247 in FIG. 12) and the pixels extracted by the extraction means, a motion vector is obtained. Prediction for calculating prediction data (for example, a filter coefficient) for predicting pixel data in a processing region corresponding to one of the first motion vector and the second motion vector selected by the toll selection means Data calculation means (for example, the first neighborhood prediction filter coefficient calculation unit 248 and the second neighborhood prediction filter coefficient calculation unit 249 in FIG. 12) are provided.

  A noise adding unit (for example, the A / D conversion unit 81 in FIG. 2) for adding noise to the frame data can be further provided, and the image data output from the decoding unit adds noise to the frame data by the noise adding unit. Then, it can be supplied to the encoding unit.

  The image processing method according to claim 35 includes an encoding unit (for example, the encoding device 63 or the encoding unit 82 in FIG. 2) and a decoding unit (for example, the decoding unit 71 or the decoding unit 84 in FIG. 3, and the encoding unit and the decoding unit repeat the encoding process and the decoding process on the image data composed of a plurality of frame data so that the image data is deteriorated. An image processing method of an image processing system, wherein a processing block (for example, a motion) having a plurality of processing regions (for example, pixels) in first frame data and second frame data different from the first frame data A first motion vector estimation step (e.g., step of FIG. 17) for estimating a first motion vector in a block when performing a search) 7 and step S8), a second motion vector estimation step for estimating a second motion vector different from the first motion vector in the processing block (for example, step S10 in FIG. 17), and processing Each region is selected by a motion vector selection step (for example, the processing in step S11 in FIG. 17) for selecting either the first motion vector or the second motion vector, and the motion vector selection step. An extraction step (for example, FIG. 20) that extracts pixels (for example, five taps in FIG. 13) corresponding to the processing region based on one of the first motion vector and the second motion vector. Step S83 or Step S86) and using the pixels extracted by the extraction step, a motion vector selection step Prediction data calculation for calculating prediction data (for example, a filter coefficient) for predicting pixel data in a processing region corresponding to one of the first motion vector and the second motion vector selected by the processing Step (for example, the process of step S85 or step S88 of FIG. 20).

  An image processing system according to a thirty-sixth aspect includes an encoding unit (for example, the encoding device 63 or the encoding unit 82 in FIG. 2) and a decoding unit (for example, the decoding unit 71 or the decoding unit 84 in FIG. 3, and the encoding unit and the decoding unit repeat the encoding process and the decoding process on the image data composed of a plurality of frame data so that the image data is deteriorated. In the image processing system, the decoding unit includes a first motion vector in a processing block having a plurality of processing regions (for example, pixels) (for example, a block for executing motion search), and a second motion in the processing block. For each processing region, the vector and which of the first motion vector and the second motion vector is best used for decoding is indicated. Motion vector determination means (for example, the motion target position calculation unit 273 in FIG. 14 or FIG. 16) that receives an input of appropriate motion vector information and determines an optimal motion vector for each processing region of the processing block, and is included in the processing region A calculation unit that receives input of prediction data (for example, a filter coefficient) for predicting pixel data and calculates pixel data in the processing block based on the motion vector determined by the motion vector determination unit and the prediction data. (For example, the pixel cutout unit 274 and the filter coefficient multiplication unit 276 in FIG. 14 or FIG. 16).

  A noise adding unit (for example, the A / D conversion unit 81 in FIG. 2) for adding noise to the frame data can be further provided, and the image data output from the decoding unit adds noise to the frame data by the noise adding unit. Then, it can be supplied to the encoding unit.

  The image processing method according to claim 38 includes an encoding unit (for example, the encoding device 63 or the encoding unit 82 in FIG. 2) and a decoding unit (for example, the decoding unit 71 or the decoding unit 84 in FIG. 3, and the encoding unit and the decoding unit repeat the encoding process and the decoding process on the image data composed of a plurality of frame data so that the image data is deteriorated. An image processing method of an image processing system, wherein a first motion vector in a processing block (for example, a block for executing motion search) having a plurality of processing regions (for example, pixels), a second motion in the processing block For each processing region, the vector and which of the first motion vector and the second motion vector is best used for decoding is indicated. In response to the input of appropriate motion vector information, a motion vector determination step (for example, the processing in step S115 in FIG. 21) for determining an optimal motion vector for each processing region of the processing block, and pixel data included in the processing region are predicted. A calculation step (e.g., calculating pixel data in the processing block based on the motion vector determined by the process of the motion vector determination step and the prediction data). 21). (Processing of step S118 in FIG. 21)

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 is a block diagram showing a configuration of an image display system 51 to which the present invention is applied.

  This image display system 51 encodes a playback device 61 that outputs analog image data, a display 62 that displays an image based on the image data output from the playback device 61, and analog image data output from the playback device 61. The encoding device 63 is configured.

  The player 61 includes a decoding unit 71 and a D / A (Digital-to-Analog) conversion unit 72. The decoding unit 71 decodes encoded image data reproduced from a recording medium such as an optical disk (not shown) and supplies the decoded image data to the D / A conversion unit 72. The D / A converter 72 converts the digital image data obtained by decoding into analog image data, and outputs the analog image data. The display 62 is composed of, for example, a CRT (Cathode-Ray Tube) display, an LCD (Liquid Crystal Display), or the like, and displays supplied analog image data.

  The encoding device 63 includes an A / D (Analog-to-Digital) conversion unit 81, an encoding unit 82, a recording unit 83, a decoding unit 84, a D / A conversion unit 85, and a display 86. Thus, it is possible to perform a process that makes it impossible to copy while maintaining a good quality without degrading the quality of the output of the data. The A / D conversion unit 81 performs A / D conversion on the analog image data output from the playback device 61 and supplies the analog image data to the encoding unit 82. The encoding unit 82 encodes digital image data and supplies the digital image data to the recording unit 83 and the decoding unit 84. The recording unit 83 records the supplied digital encoded image data on a recording medium such as an optical disk. The decoding unit 84 decodes the supplied digital encoded image data and supplies it to the D / A conversion unit 85. The D / A converter 85 converts the digital image data obtained by decoding into analog data, and outputs the analog image data to the display 86. The display 86 is composed of, for example, a CRT display, LCD, etc., and displays the supplied analog image data.

  By the way, when D / A converting digital image data in the D / A converter 72, analog image data is A / D converted in the A / D converter 81, and digital in the D / A converter 85 When D / A conversion is performed on the image data, or when analog data is transmitted from the playback device 61 to the display 62 or the encoding device 63, noise is added to the image data.

  In particular, in A / D conversion, D / A conversion, or analog data transmission, high-frequency component distortion or phase shift occurs in image data. This distortion or phase shift of the high frequency component is also referred to as analog noise.

  Among the analog noise, the distortion of the high frequency component is mainly caused by noise called white noise.

  A process in which digital image data is converted into analog image data in the D / A converter 72 and D / A converter 85, and analog image data is converted into digital image data in the A / D converter 81. In the process of conversion or the process of transmitting analog data, white noise having a substantially uniform frequency component is added to the image data. The level of white noise changes randomly in time series, and its distribution almost follows a normal distribution. That is, the level of white noise added to the analog image data corresponding to each pixel changes randomly.

  Therefore, for example, in the digital image data output from the decoding unit 71, even if a predetermined plurality of pixels that are aligned in the horizontal direction and the vertical direction have the same pixel value, D The digital image data after being D / A converted by the / A converter 72 and further A / D converted by the A / D converter 81 has the same pixel value before conversion. The pixel value of each pixel is a value dispersed within a certain range with the original value as the center value. The difference between the distributed value and the original value is a property expressed by a random number, and is not determined uniformly. As a result, distortion of high frequency components occurs in the image data. Similarly, not only the horizontal direction but also the vertical direction is distorted by high frequency components. In addition, depending on the degree of dispersion of the level of white noise added to each pixel, distortion of components other than high-frequency components may occur.

  In addition, by adding distortion due to phase shift among analog noise, the position of the pixel of the image data is shifted in the horizontal direction and the vertical direction as compared with the position of the pixel of the original image data. Note that the width of the phase shift in the horizontal direction is smaller or larger than the pixel interval in the horizontal direction, while the width of the phase shift in the vertical direction is an integral multiple of the pixel interval in the vertical direction. Further, the phase shift may occur only in one direction in the horizontal direction or the vertical direction. For example, a value based on a synchronization signal, which is not a pixel value of a pixel of original image data, is set for a pixel that has become empty due to a phase shift.

  When this phase shift occurs, the display position of the image is shifted by the width of the phase shift, and an image based on a synchronization signal other than the pixel value is displayed in the shifted portion. However, the display position shift is slight, and in many cases, it hardly affects the image quality at a level that can be visually recognized by the user.

  Needless to say, noise added in image processing and data transmission may include noise other than the noise described above.

  The D / A converter 72, the A / D converter 81, and the D / A converter 85 are each in the process of converting digital image data into analog image data or analog image data. In the process of converting to digital image data, a noise component (particularly, a noise component emphasized by a high-pass filter) may be added to the output data.

  The D / A conversion unit 72 is configured as a single component, the decoding unit 71, the A / D conversion unit 81, the encoding unit 82, and the D / A conversion unit 85, respectively, as a single component. You may do it.

  2 may be configured as independent devices. The units constituting the regenerator 61 and the encoding device 63 in FIG. That is, for example, the D / A conversion unit 72 and the decoding unit 71 may be configured as independent devices, or may be configured as a single device having two components. In addition, the encoding unit 82 and the decoding unit 84 may be configured as independent devices, or may be configured as a single device having two components of the A / D conversion unit 81 and the encoding unit 82, or D / A You may make it comprise as one apparatus with two components, the conversion part 85 and the decoding part 84. FIG.

  FIG. 3 is a block diagram showing a configuration of the encoding unit 82 of FIG.

  The encoding unit 82 uses a predetermined frame of the plurality of frame data as a reference frame, encodes the reference frame in the frame as shown in FIG. 4, and frames other than the reference frame to the previous frame. Encoding is performed by motion detection for the frame. Normally, residual compensation is performed in the coding by motion detection, but the encoding unit 82 improves prediction accuracy by performing two types of block matching without performing residual compensation. It is made to let you.

  The reference frame extraction unit 101 is supplied with the image data converted into digital data by the A / D conversion unit 81, supplies the reference frame to the reference frame encoding unit 102, and frames other than the reference frame are frame memories. It supplies to 105.

  The reference frame encoding unit 102 encodes the supplied reference frame using an intra-frame encoding method such as JPEG (Joint Photographic Experts Group) or ADRC (Adaptive Dynamic Range Coding), for example, and the reference frame decoder 103 To the decoder 114 and the code output unit 115.

  The reference frame decoder 103 decodes the supplied encoded data by a corresponding method, and supplies the decoded data to the frame memory 104.

  The frame memory 104 temporarily stores frame data supplied from the reference frame decoder 103 or a decoder 114 described later, and supplies the frame data to a high-pass filter (HPF) 107.

  The frame memory 105 temporarily stores the frame data supplied from the reference frame extraction unit 101 and supplies it to the high pass filter (HPF) 106.

  That is, the frame data temporarily stored in the frame memory 104 is the previous frame data in time with respect to the frame data temporarily stored in the frame memory 105. Hereinafter, the frame data temporarily stored in the frame memory 105 is referred to as current frame data, and the frame data temporarily stored in the frame memory 104 is referred to as previous frame data.

  The high-pass filter 106 and the high-pass filter 107 apply a high-pass filter to the supplied frame data, and perform a block matching motion search unit 108, an evaluation value change block matching motion search unit 111, an optimal motion determination unit 112, and an optimal neighborhood prediction filter minimum This is supplied to the square design unit 113. The high pass filter 106 also supplies data to the absolute value error calculation unit 109.

  That is, when the previous frame data temporarily stored in the frame memory 104 is the frame 131 in FIG. 5, the previous frame data output from the high-pass filter 107 becomes the frame 132 with the edge portion emphasized, and the frame memory 105 5 is the frame 141 in FIG. 5, the current frame data output from the high-pass filter 106 is a frame 142 in which the edge portion is emphasized.

  Instead of the high-pass filter 106 and the high-pass filter 107, for example, a filter that extracts features of the original image such as an edge and emphasizes noise, such as an edge extraction filter such as a Sobel filter, is used. Anyway.

  The block matching motion search unit 108 performs block matching using the current frame data and the previous frame data supplied from the high-pass filter 106 and the high-pass filter 107, and uses the calculated first motion vector as the code memory 110, the optimal This is supplied to the motion determination unit 112 and the optimal neighborhood prediction filter least square design unit 113.

  Block matching processing is a technique for determining the position of a moving object for each block using the difference in luminance of pixels in an image. Specifically, as shown in FIG. 6, the frame 151 that is the previous frame data and the frame 152 that is the current frame data are each divided into predetermined blocks, and the block 163 that is the search source of the frame 152 is divided into The matching process is performed within the search range 162 around the block 161 of the corresponding frame 151, and the first motion vector as shown in the frame 153 is calculated.

  The block matching motion search unit 108 further calculates a prediction frame when this frame is decoded by motion prediction using the first motion vector from the calculated first motion vector and previous frame data, The absolute value calculation unit 109 is supplied.

  The absolute value error calculation unit 109 performs a motion vector search process executed by the block matching motion search unit 108 based on the prediction frame supplied from the block matching motion search unit 108 and the current frame data supplied from the high pass filter 106. The absolute value error of the prediction based on the first motion vector obtained by the above is calculated and supplied to the evaluation value change block matching motion search unit 111.

  The processing executed by the absolute value error calculation unit 109 will be described with reference to FIG.

  For example, when a plurality of motions are mixed in the block as shown in the block 161 of FIG. 7 and the block 162 included in the next frame (current frame) including the block 161, the block matching motion search unit The first motion vector calculated by the normal block matching 108 is dragged by one of the motions and matches one motion in the block, but does not match the other motion.

  For example, when the corresponding block of the prediction frame supplied from the block matching motion search unit 108 is the block 163, one motion of the block 163 (the upward motion of the right ball) is accurate in the normal block matching process. However, the other movement (the movement of the left ball moving downward) is not accurately reflected.

  Therefore, the absolute value error calculation unit 109 compares the block 163 corresponding to the prediction frame supplied from the block matching motion search unit 108 with the block 164 included in the current frame to obtain an absolute value error. When the absolute value error obtained by the absolute value error calculation unit 109 is visualized, a block 165 is displayed. Then, when the absolute value error is digitized by using, for example, a 4 × 4 square matrix in the block, a matrix 166 indicating the magnitude and position of the absolute value error in the block can be obtained. The absolute value error calculation unit 109 supplies the matrix 166 to the evaluation value change block matching motion search unit 111 as absolute value error information.

  The code memory 110 temporarily stores the supplied code and supplies it to the decoder 114 and the code output unit 115.

  The evaluation value change block matching motion search unit 111 uses the absolute value error (see FIG. 7) calculated by the absolute value error calculation unit 109 for the current frame data and the previous frame data supplied from the high pass filter 106 and the high pass filter 107. The information corresponding to the matrix 166 described above) is weighted for each pixel position, so that the weight of the portion where the error of the first motion vector obtained by the block matching motion search unit 108 is large is increased. Matching is performed, and the calculated second motion vector is supplied to the code memory 110, the optimal motion determination unit 112, and the optimal neighborhood prediction filter least square design unit 113.

  That is, the difference square error value at the position (x, y) in the block is f (x, y), and the absolute value error of the motion search by the block matching motion search unit 108 at the position (x, y) in the block is g. In the case of (x, y), when the motion search by the block matching motion search unit 108 is expressed by the following formula (1), the motion search by the evaluation value change block matching motion search unit 111 is expressed by the following formula (2 ).

・・・（１）

... (1)

・・・（２）

... (2)

  Details of the process of the evaluation value change block matching motion search unit 111 will be described later.

  The optimal motion determination unit 112 includes the first motion vector supplied from the block matching motion search unit 108, the second motion vector supplied from the evaluation value change block matching motion search unit 111, and the high pass filter 106 and the high pass filter. Using the current frame data and previous frame data supplied from 107, an optimal motion flag indicating which of the first motion vector and the second motion vector should be used is generated for each pixel position of each block. The code memory 110 and the optimum neighborhood prediction filter least squares design unit 113 are supplied. Details of the process of the optimum motion determination unit 112 and the optimum motion flag will be described later.

  The optimal neighborhood prediction filter least squares design unit 113 includes a first motion vector supplied from the block matching motion search unit 108, a second motion vector supplied from the evaluation value change block matching motion search unit 111, and an optimal motion determination unit. 112, the optimum motion flag supplied from 112 and the current frame data and the previous frame data supplied from the high-pass filter 106 and the high-pass filter 107 are used to design an optimal prediction filter for each of the two motion vectors, and the filter coefficient Is supplied to the code memory 110. Details of the optimum neighborhood prediction filter least square design unit 113 will be described later.

  The decoder 114 includes the reference frame encoded data supplied from the reference frame encoding unit 102, the first motion vector, the second motion vector, the optimal motion flag, and the optimal prediction filter supplied from the code memory 114. Is decoded, and the decoded frame data is supplied to the frame memory 104. Details of the decoder 114 will be described later.

  The code output unit 115 outputs the first motion vector, the second motion vector, the optimal motion flag, and the optimal prediction filter supplied from the code memory 114 to the recording unit 83 or the decoding unit 84.

  Next, FIG. 8 is a functional block diagram showing further detailed functions of the evaluation value change block matching motion search unit 111.

  The evaluation value change block matching motion search unit 111 includes an absolute value error acquisition unit 171, a current frame data acquisition unit 172, a previous frame data acquisition unit 173, and a weighted block matching motion search unit 174.

  The absolute value error acquisition unit 171 acquires the absolute value error supplied from the absolute value error calculation unit 109 and supplies the absolute value error to the weighted block matching motion search unit 174.

  The current frame data acquisition unit 172 acquires the current frame data supplied from the high pass filter 106 and supplies the current frame data to the weighted block matching motion search unit 174.

  The previous frame data acquisition unit 173 acquires the previous frame data supplied from the high pass filter 107 and supplies the previous frame data to the weighted block matching motion search unit 174.

  The weighted block matching motion search unit 174 performs weighting for each pixel position with the absolute value error (information corresponding to the matrix 166 described with reference to FIG. 7) acquired by the absolute value error acquisition unit 171 to obtain the current frame data and Block matching is performed on the previous frame data, and the detected second motion vector is output.

  Specifically, as shown in FIG. 9, the frame 151 that is the previous frame data and the frame 152 that is the current frame data are each divided into predetermined blocks, and the block 163 that is the search source of the frame 152 is divided into The matching processing with the search range 162 around the block 161 of the corresponding frame 151 is performed based on the absolute value error. Therefore, as described with reference to FIG. 6, the first motion vector is a human motion detected in the horizontal direction, whereas the weighted block matching motion search unit 174 generates a frame 181. As shown in the figure, for example, a motion in the vertical direction or the oblique direction different from the first motion vector, such as a hand or ball motion, is detected as the second motion vector.

  FIG. 10 is a functional block diagram showing further detailed functions of the optimum motion determination unit 112.

  The optimal motion determination unit 112 includes a first motion search result acquisition unit 201, a second motion search result acquisition unit 202, a previous frame data acquisition unit 203, a current frame data acquisition unit 204, a first prediction data generation unit 205, The second prediction data generation unit 206, the first prediction error detection unit 207, the second prediction error detection unit 208, and the optimal motion flag detection unit 209 are configured.

  The first motion search result acquisition unit 201 acquires the first motion vector supplied from the block matching motion search unit 108 and supplies the first motion vector to the first prediction data generation unit 205.

  The second motion search result acquisition unit 202 acquires the second motion vector supplied from the evaluation value change block matching motion search unit 111 and supplies the second motion vector to the second prediction data generation unit 206.

  The previous frame data acquisition unit 203 acquires the previous frame data supplied from the high pass filter 106 and supplies the previous frame data to the first prediction data generation unit 205 and the second prediction data generation unit 206.

  The current frame data acquisition unit 204 acquires the current frame data supplied from the high pass filter 107 and supplies the current frame data to the first prediction error detection unit 207 and the second prediction error detection unit 208.

  The first prediction data generation unit 205 acquires the first motion vector supplied from the first motion search result acquisition unit 201 and the previous frame data supplied from the previous frame data acquisition unit 203. The first prediction data corresponding to the image data when the decoding process is performed using the motion vector is generated and supplied to the first prediction error detection unit 207.

  The second prediction data generation unit 206 acquires the second motion vector supplied from the second motion search result acquisition unit 202 and the previous frame data supplied from the previous frame data acquisition unit 203, and The second prediction data corresponding to the image data when the decoding process is performed using the motion vector is generated and supplied to the second prediction error detection unit 208.

  The first prediction error detection unit 207 acquires the first prediction data supplied from the first prediction data generation unit 205 and the current frame data supplied from the current frame data acquisition unit 204. A first prediction error that is a prediction error between the prediction data and the current frame data is detected and supplied to the optimum motion flag detection unit 209.

  The second prediction error detection unit 208 acquires the second prediction data supplied from the second prediction data generation unit 206 and the current frame data supplied from the current frame data acquisition unit 204, and A second prediction error that is a prediction error between the prediction data and the current frame data is detected and supplied to the optimum motion flag detection unit 209.

  The optimum motion flag detection unit 209 compares the first prediction error supplied from the first prediction error detection unit 207 and the second prediction error supplied from the second prediction error detection unit 208 for each pixel. Then, it is determined whether the first prediction data or the second prediction data has a small prediction error, and whether the first motion vector should be used for each pixel in the block. Set and output an optimal motion flag indicating whether or not to use the motion vector.

  The setting of the optimum motion flag will be described with reference to FIG.

  When a certain block included in the first prediction data generated by using the first motion vector by the first prediction data generation unit 205 is the block 221, the first prediction error detection unit 207 determines the current frame data. The block 223 corresponding to the current frame data acquired by the acquisition unit 204 is compared with the block 221 to detect the first prediction error. If a certain block included in the second prediction data generated by the second prediction data generation unit 206 using the second motion vector is the block 222, the second prediction error detection unit 208 The block 223 and the block 222 corresponding to the current frame data acquired by the frame data acquisition unit 204 are compared, and the second prediction error is detected.

  For example, when the flag 0 is set at the pixel position where the first motion vector should be used and the flag 1 is set at the pixel position where the second motion vector should be used, the optimal motion flag detection unit 209 detects the first prediction error. The first prediction error supplied from the unit 207 and the second prediction error supplied from the second prediction error detection unit 208 are compared for each pixel, and the first prediction data has a small prediction error. Flag 0 is set at the pixel position corresponding to a certain case, and flag 1 is set at the pixel position corresponding to the case where the second prediction data has a small prediction error.

  The motion flag 224 in FIG. 11 shows an example of the optimum motion flag when the number of pixels in the block is 4 × 4, for example. That is, the movement of the ball on the right side in the figure is accurately predicted by the first motion vector as shown in block 221, and the movement of the ball on the left side in the figure is shown in block 222 as shown in block 222. Since the motion vector is accurately predicted, a corresponding flag is set at each pixel position.

  In addition, although demonstrated here as what produces | generates the optimal motion flag which shows which of the 1st motion vector and the 2nd motion vector should be used for every pixel position of each block, In addition to being set in units of pixels, for example, it may be set as an optimal motion flag in a plurality of pixels within a predetermined range such as 2 × 2 or 3 × 3.

  Next, FIG. 12 is a functional block diagram showing further detailed functions of the optimum neighborhood prediction filter least square design unit 113.

  The optimal neighborhood prediction filter least squares design unit 113 includes a first motion search result acquisition unit 241, a second motion search result acquisition unit 242, an optimal motion flag acquisition unit 243, a previous frame data acquisition unit 244, and a current frame data acquisition unit. 245, a first tap extraction unit 246, a second tap extraction unit 247, a first neighborhood prediction filter coefficient calculation unit 248, and a second neighborhood prediction filter coefficient calculation unit 249.

  The first motion search result acquisition unit 241 acquires the first motion vector supplied from the block matching motion search unit 108 and supplies the first motion vector to the first tap extraction unit 246.

  The second motion search result acquisition unit 242 acquires the second motion vector supplied from the evaluation value change block matching motion search unit 111 and supplies the second motion vector to the second tap extraction unit 247.

  The optimal motion flag acquisition unit 243 acquires the optimal motion flag supplied from the optimal motion determination unit 112 and supplies the optimal motion flag to the first neighborhood prediction filter coefficient calculation unit 248 and the second neighborhood prediction filter coefficient calculation unit 249.

  The previous frame data acquisition unit 244 acquires the previous frame data supplied from the high-pass filter 107 and supplies the previous frame data to the first tap extraction unit 246 and the second tap extraction unit 247.

  The current frame data acquisition unit 245 acquires the current frame data supplied from the high pass filter 106 and supplies the current frame data to the first neighborhood prediction filter coefficient calculation unit 248 and the second neighborhood prediction filter coefficient calculation unit 249.

  The first tap extraction unit 246 is supplied with the first motion vector from the first motion search result acquisition unit 241, and performs the first tap on the target pixel specified by the first neighborhood prediction filter coefficient calculation unit 248. A plurality of pixels around a pixel position predicted based on one motion vector is defined as a first tap, information on each position of the first tap and a pixel value corresponding to the first tap are extracted. To the neighborhood prediction filter coefficient calculation unit 248.

  The second tap extraction unit 247 is supplied with the second motion vector from the second motion search result acquisition unit 242 and applies the second pixel to the target pixel designated by the second neighborhood prediction filter coefficient calculation unit 249. A plurality of pixels around the pixel position predicted based on the motion vector of the second as a second tap, information on the position of each of the second tap and a pixel value corresponding to the second tap are extracted, To the neighborhood prediction filter coefficient calculation unit 249.

  For example, when five pixels are extracted as the first tap and the second tap, as shown in FIG. 13, the pixel corresponds to the target pixel according to the first motion vector or the second motion vector. Five pixels of the pixel and its surrounding pixels are extracted as the first tap or the second tap, and the position information and the pixel value corresponding to the position of the tap are the first neighborhood prediction filter coefficients. The calculation unit 248 or the second neighborhood prediction filter coefficient calculation unit 249 is supplied.

  In FIG. 13, the pixel position predicted corresponding to the target pixel and the five pixels adjacent to the top, bottom, left, and right are described as being selected as peripheral pixels for calculating the prediction filter. The peripheral pixels for calculating the filter are not limited to this. For example, a number of pixels other than five may be extracted, pixels adjacent to an oblique position, or two adjacent pixels may be extracted. The above continuous pixels or neighboring pixels that are not adjacent to each other may be selected as neighboring pixels for calculating a prediction filter.

  The first neighborhood prediction filter coefficient calculation unit 248 receives the current frame from the current frame data acquisition unit 245 and uses the first motion flag based on the optimal motion flag supplied from the optimal motion flag acquisition unit 243. A pixel to be processed is a pixel of interest. Then, the first neighborhood prediction filter coefficient calculation unit 248 requests the first tap corresponding to the target pixel from the first tap extraction unit 246, and calculates the pixel value at each position corresponding to the first tap. get.

  And the 1st neighborhood prediction filter coefficient calculation part 248 calculates | requires following Formula (3) in each pixel which should use a 1st motion flag.

・・・（３）

... (3)

Here, f _t (x, y) is the pixel value of the pixel of interest in the current frame, and f _t−1 (x, y) is the pixel value at each position of the corresponding first tap in the previous frame. Indicates. In Equation (3), a _{1 to} a ₅ are prediction coefficients corresponding to the first motion vector.

Then, the first neighborhood prediction filter coefficient calculation unit 248 calculates the optimum values of the prediction coefficients a _{1 to} a ₅ for the first motion vector using the least square method based on the equation (3), Output as the optimum filter coefficient.

  The second neighborhood prediction filter coefficient calculation unit 249 receives the current frame from the current frame data acquisition unit 245 and uses the second motion flag based on the optimal motion flag supplied from the optimal motion flag acquisition unit 243. A pixel to be processed is a pixel of interest. Then, the second neighborhood prediction filter coefficient calculation unit 249 requests the second tap extraction unit 247 for the second tap corresponding to the target pixel, and calculates the pixel value at each position corresponding to the second tap. get.

  Then, the second neighborhood prediction filter coefficient calculation unit 249 obtains the following expression (4) for each pixel that should use the second motion flag.

・・・（４）

... (4)

Here, f _t (x, y) is the pixel value of the pixel of interest in the current frame, and f _t−1 (x, y) is the pixel value at each position of the corresponding second tap in the previous frame. Indicates. In Equation (3), b _{1 to} b ₅ are prediction coefficients corresponding to the second motion vector.

The second neighborhood prediction filter coefficient calculation unit 249 calculates the optimum values of the prediction coefficients b _{1 to} b ₅ for the second motion vector using the least square method based on Expression (4), and sets the optimum filter Output as a coefficient.

As described above, in the encoding unit 82, the first motion vector, the second motion vector, the optimum motion flag, the optimum filter coefficients a _{1 to} a ₅ corresponding to the first motion flag, and the second motion vector. The optimum filter coefficients b _{1 to} b ₅ corresponding to the motion flags of the current are generated, temporarily stored in the code memory 110, output from the code output unit 115, decoded by the decoder 114, and used for the next frame encoding. .

  Next, FIG. 14 is a functional block diagram showing further detailed functions of the decoder 114 of FIG.

  The decoder 114 includes a reference frame decoder 271, a frame memory 272, a motion target position calculation unit 273, a pixel cutout unit 274, a coefficient selection unit 275, a filter coefficient multiplication unit 276, and a frame memory 227.

  The reference frame decoder 271 acquires the code of the reference frame, decodes and outputs the reference frame, and supplies it to the frame memory 272.

  The frame memory 272 receives a decoded reference frame supplied from the reference frame decoder 271 or a frame other than the decoded reference frame supplied from the frame memory 277 described later, and temporarily holds this. , And supplied to the pixel cutout unit 274.

That is, in the decoder 114, the first motion vector, the second motion vector, the optimum motion flag, and the first motion vector, which are the motion components of the intra-frame-encoded reference frame and the other frames, The decoding process is performed by receiving the optimum filter coefficients a _{1 to} a ₅ corresponding to the motion flag and the optimum filter coefficients b _{1 to} b ₅ corresponding to the second motion flag. Therefore, as shown in FIG. 15, the reference frame is decoded in the frame by the reference frame decoder 271, and frames other than the reference frame supplied with the motion component as a code are the same as the previous frame. Decoding processing is performed based on the decoding result.

  The motion target position calculation unit 273 is supplied with the first motion vector, the second motion vector, and the optimal motion flag, and uses either the first motion vector or the second motion vector for each pixel. Is calculated, the movement target position of the target pixel is calculated, and supplied to the pixel cutout unit 274.

  The pixel cutout unit 274 reads the previous decoded image data held in the frame memory 272, and the neighboring prediction filter coefficient at the time of encoding the surrounding pixels of the movement target position supplied from the movement target position calculation unit 273. Are set and cut out in the same manner as the taps used when setting (for example, 5 taps shown in FIG. 13), and supplied to the filter coefficient multiplier 276.

The coefficient selection unit 275 is supplied with the optimal motion flag and the optimal filter coefficient, and uses the optimal filter coefficients a _{1 to} a ₅ corresponding to the first motion flag or the second motion as filter coefficients for predicting the target pixel. Which of the optimum filter coefficients b _{1 to} b ₅ corresponding to the flag is to be used is determined and supplied to the filter coefficient multiplier 276.

  The filter coefficient multiplication unit 276 receives supply of peripheral pixels (for example, 5 taps) extracted based on the optimal motion vector from the pixel cutout unit 274, and multiplies the filter coefficient supplied from the coefficient selection unit 275. The pixel value of the target pixel is obtained. The filter coefficient multiplication unit 276 supplies the pixel value for one frame thus obtained, that is, the decoded image data, to the frame memory 227.

  The frame memory 227 holds the decoded image data supplied from the filter coefficient multiplication unit 276, outputs it as a decoding result, and supplies it to the frame memory 272 for use in decoding the next frame.

  The decoding result output from the decoder 114 is supplied to the frame memory 104 and used for encoding the next frame.

  As described with reference to FIGS. 3 to 15, the encoding unit 82 encodes the frame data.

  Next, FIG. 16 is a functional block diagram showing a more detailed configuration of the decoding unit 84 of FIG.

  Note that portions corresponding to those in FIG. 14 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  That is, the decoding unit 84 in FIG. 16 has basically the same configuration and function as the decoder 114 in FIG. 14, and executes the same processing.

  In the case where the player 61 in FIG. 2 is configured to be able to reproduce the image data encoded by the encoding unit 82 and recorded on a predetermined recording medium by the recording unit 83, the decoding unit 71 is also configured. 16 has basically the same configuration and function as the decoding unit 84 shown in FIG. 16, and can execute the same processing.

  Next, the encoding process executed by the encoding unit 82 will be described with reference to the flowchart of FIG.

  In step S1, the reference frame extraction unit 101 determines whether the supplied frame data is a reference frame. If it is determined in step S1 that the frame is not a reference frame, the process proceeds to step S6 described later.

  The frame image data supplied at this time is decoded by the decoding unit 71 of the playback device 61, converted into analog data by the D / A conversion unit 72, and converted from the D / A conversion unit 72 to the A / D conversion unit 81. The image data is transmitted and converted into digital image data by the A / D converter 81. Therefore, the frame data supplied in step S1 includes, for example, the above-described analog noise in the course of the data conversion process of the D / A converter 72 or A / D converter 81 or on the transmission path therebetween. A noise component is added.

  Note that, as described above, in some cases, a noise component is positively added in the D / A converter 72 or the A / D converter 81. In such a case, in the stage before step S1, the A / D conversion unit 81 converts the supplied analog image data into digital image data, and executes a process of adding a noise component, and the noise component The digital image data added with is supplied to the encoding unit 82.

  If it is determined in step S1 that the frame is a reference frame, in step S2, the reference frame extraction unit 101 supplies the supplied frame data to the reference frame encoding unit 102. Therefore, the reference frame encoding unit 102 For example, the supplied reference frame is encoded using an intra-frame encoding method such as JPEG or ADRC, and supplied to the reference frame decoder 103, the decoder 114, and the code output unit 115.

  In step S3, the code output unit 115 outputs the encoded reference frame.

  In step S <b> 4, the reference frame decoder 103 decodes the supplied reference frame and supplies it to the frame memory 104.

  In step S5, the frame memory 104 caches the reference frame decoded by the reference frame decoder 103, and the process proceeds to step S17 described later.

  If it is determined in step S <b> 1 that the frame is not a reference frame, the reference frame extraction unit 101 supplies the supplied frame data to the frame memory 105 in step S <b> 6. The frame memory 105 supplies the supplied frame data to the high-pass filter 106, and the frame memory 104 supplies the stored frame data to the high-pass filter 107. The high-pass filter 106 applies a high-pass filter to the current frame. The block matching motion search unit 108, the absolute value error calculation unit 109, the evaluation value change block matching motion search unit 111, the optimum motion determination unit 112, and the optimum neighborhood prediction filter least square design unit 113 are supplied to the high pass filter. A high-pass filter 107 is applied to the previous frame and supplied to the block matching motion search unit 108, the evaluation value changed block matching motion search unit 111, the optimal motion determination unit 112, and the optimal neighborhood prediction filter least square design unit 113.

  In step S7, the block matching motion search unit 108 selects one of unprocessed blocks from the supplied frame.

  In step S8, the block matching motion search unit 108 performs a first motion vector search process. Specifically, as described with reference to FIG. 6, the block matching motion search unit 108 performs a corresponding previous frame with respect to the block 163 that is the selected search source in the frame 152 that is the current frame data. A matching process with the search range 162 around the data block 161 is executed, a first motion vector is calculated, and the calculated first motion vector is used as the code memory 110, the optimal motion determination unit 112, and the optimal neighborhood. This is supplied to the prediction filter least square design unit 113.

  In step S9, the block matching motion search unit 108 calculates a prediction frame from the calculated first motion vector and the previous frame data and supplies the prediction frame to the absolute value calculation unit 109. Therefore, the absolute value error calculation unit 109 Based on the prediction frame supplied from the block matching motion search unit 108 and the current frame data supplied from the high-pass filter 106, the absolute value error of the first motion vector search process executed by the block matching motion search unit 108 ( For example, information corresponding to the matrix 166 described with reference to FIG. 7 is calculated and supplied to the evaluation value change block matching motion search unit 111.

  In step S10, an evaluation value change block matching motion search process described later with reference to FIG. 18 is executed.

  In step S11, the optimum motion determination process described later with reference to FIG. 19 is executed.

  In step S12, the optimum filter coefficient calculation process described later with reference to FIG. 20 is executed.

  In step S <b> 13, the decoder 114 or the code output unit 115 determines whether or not the processing of all the blocks in the frame has been completed and the code memory 110 has recorded a code for one frame. If it is determined in step S13 that the processing of all the blocks in the frame has not been completed, the processing returns to step S7, and the subsequent processing is repeated.

  If it is determined in step S13 that the processing of all blocks in the frame has been completed, the code output unit 115 reads the code for one frame from the code memory 110 and outputs the code data in step S14.

  In step S15, a decoding process, which will be described later with reference to FIG.

  In step S16, the frame memory 104 caches the decoded frame data supplied from the decoder 114.

  In step S <b> 17, the reference frame extraction unit 101 determines whether or not the last frame has been processed. If it is determined in step S17 that the process for the last frame has not been completed, the process returns to step S1 and the subsequent processes are repeated. If it is determined in step S17 that the process for the last frame has been completed, the process ends.

  By such processing, the encoding unit 82 encodes the frame data.

  Next, the evaluation value change block matching motion search process executed in step S10 of FIG. 17 will be described with reference to the flowchart of FIG.

  In step S31, the absolute value error acquisition unit 171 of the evaluation value change block matching motion search unit 111 calculates the absolute value of the search result of the first motion vector supplied from the absolute value error calculation unit 109 and the image to be obtained. A value error (for example, information corresponding to the matrix 166 described with reference to FIG. 7) is acquired and supplied to the weighted block matching motion search unit 174.

  In step S32, the current frame data acquisition unit 172 acquires the current frame data supplied from the high pass filter 106, and the previous frame data acquisition unit 173 acquires the previous frame data supplied from the high pass filter 107, respectively. The weighted block matching motion search unit 174 is supplied.

  In step S33, the weighted block matching motion search unit 174 performs weight matching corresponding to the absolute value error acquired by the absolute value error acquisition unit 171 to perform block matching on the current frame data and the previous frame data, and The second motion vector search process is executed, and a second motion vector is output as a search result. Specifically, as described with reference to FIG. 9, the weighted block matching motion search unit 174 applies the frame 151 of the corresponding previous frame data to the block 163 that is the search source of the frame 152 that is the current frame data. The matching process with the search range 162 around the block 161 is executed based on the weighting based on the absolute value error. After the process of step S33 is completed, the process returns to step S10 in FIG. 17 and proceeds to step S11.

  By such processing, block matching is performed by a method different from normal block matching, and a second motion vector different from the first motion vector obtained by normal block matching is searched.

  Next, the optimal motion determination process executed in step S11 of FIG. 17 will be described with reference to the flowchart of FIG.

  In step S51, the first motion search result acquisition unit 201 of the optimal motion determination unit 112 acquires the first motion vector search result supplied from the block matching motion search unit 108, and the first prediction data generation unit 205, the second motion search result acquisition unit 202 acquires the second motion vector search result supplied from the evaluation value change block matching motion search unit 111, and sends it to the second prediction data generation unit 206. Supply.

  In step S <b> 52, the previous frame data acquisition unit 203 acquires the previous frame data supplied from the high pass filter 107 and supplies the previous frame data to the first prediction data generation unit 205 and the second prediction data generation unit 206.

  In step S <b> 53, the first prediction data generation unit 205 includes the first motion vector that is the first motion vector search result supplied from the first motion search result acquisition unit 201 and the previous frame data acquisition unit 203. The first prediction data corresponding to the image data when the decoding process is performed using the first motion vector is generated and supplied to the first prediction error detection unit 207 To do.

  In step S54, the second prediction data generation unit 206 receives the second motion vector that is the second motion vector search result supplied from the second motion search result acquisition unit 202, and the previous frame data acquisition unit 203. The previous frame data supplied from is acquired, second prediction data corresponding to the image data when decoding processing is performed using the second motion vector is generated, and supplied to the second prediction error detection unit 208 To do.

  In step S55, the current frame data acquisition unit 204 acquires the current frame data supplied from the high-pass filter 106, and supplies the current frame data to the first prediction error detection unit 207 and the second prediction error detection unit 208.

  In step S <b> 56, the first prediction error detection unit 207 acquires the first prediction data supplied from the first prediction data generation unit 205 and the current frame data supplied from the current frame data acquisition unit 204. The first prediction error, which is the prediction error between the first prediction data and the current frame data, is detected for each pixel and supplied to the optimum motion flag detection unit 209.

  In step S57, the second prediction error detection unit 208 acquires the second prediction data supplied from the second prediction data generation unit 206 and the current frame data supplied from the current frame data acquisition unit 204. The second prediction error, which is the prediction error between the second prediction data and the current frame data, is detected for each pixel and supplied to the optimum motion flag detection unit 209.

  In step S58, the optimal motion flag detection unit 209 calculates the first prediction error supplied from the first prediction error detection unit 207 and the second prediction error supplied from the second prediction error detection unit 208. It should be determined whether the first prediction data or the second prediction data has a small prediction error compared with each pixel, and the first motion vector should be used for each pixel in the block. Or the optimal motion flag indicating whether the second motion vector should be used is detected and output, and the process returns to step S11 in FIG. 17 and proceeds to step S12.

  By such a process, an optimal motion flag indicating whether the first motion vector should be used or the second motion vector should be used in decoding each pixel in the block, Is output.

  Next, the optimum filter coefficient calculation process executed in step S12 of FIG. 17 will be described with reference to the flowchart of FIG.

  In step S81, the first motion search result acquisition unit 241 of the optimal neighborhood prediction filter least square design unit 113 acquires the first motion vector supplied from the block matching motion search unit 108, and the first tap extraction unit. The second motion search result acquisition unit 242 acquires the second motion vector supplied from the evaluation value change block matching motion search unit 111 and supplies the second motion vector to the second tap extraction unit 247.

  In step S82, the previous frame data acquisition unit 244 acquires the previous frame data supplied from the high pass filter 107, supplies the previous frame data to the first tap extraction unit 246 and the second tap extraction unit 247, and the current frame data acquisition unit. 245 acquires the current frame data supplied from the high-pass filter 106 and supplies the current frame data to the first neighborhood prediction filter coefficient calculation unit 248 and the second neighborhood prediction filter coefficient calculation unit 249.

  In step S83, the first neighborhood prediction filter coefficient calculation unit 248 uses the optimal motion flag supplied from the optimal motion flag acquisition unit 243 as a pixel of interest to use the first motion flag as the target pixel. The first tap extraction unit 246 is requested for the first tap corresponding to the target pixel. Then, the first tap extraction unit 246 extracts the previous frame data acquisition unit 244 based on the first motion vector that is the first motion vector search result supplied from the first motion search result acquisition unit 241. The pixel values corresponding to the positions of the predetermined number of first taps for filter detection are extracted from the previous frame data and supplied to the first neighborhood prediction filter coefficient calculation unit 248.

  In step S84, the first neighborhood prediction filter coefficient calculation unit 248 is a relational expression between the pixel value that is the target of the current frame data and the pixel value of the prediction tap based on the extracted first tap. Formula (3) described above is created.

  In step S85, the first neighborhood prediction filter coefficient calculation unit 248 calculates and outputs the first neighborhood prediction filter coefficient using the least square method.

  In step S <b> 86, the second neighborhood prediction filter coefficient calculation unit 249 uses the optimal motion flag supplied from the optimal motion flag acquisition unit 243 as a pixel of interest and uses the second motion flag as the target pixel. Request the first tap corresponding to the pixel of interest. Then, the second tap extraction unit 247 extracts the previous frame data acquisition unit 244 based on the second motion vector that is the second motion vector search result supplied from the second motion search result acquisition unit 242. Pixel values corresponding to the positions of a predetermined number of first taps for filter detection are extracted from the previous frame data, and supplied to the second neighborhood prediction filter coefficient calculation unit 249.

  In step S87, the second neighborhood prediction filter coefficient calculation unit 249 is a relational expression between the pixel value that is the target of the current frame data and the pixel value of the prediction tap based on the extracted second tap. Formula (4) described above is created.

  In step S88, the second neighborhood prediction filter coefficient calculation unit 249 calculates and outputs the second neighborhood prediction filter coefficient using the method of least squares, and the process returns to step S12 in FIG. Proceed to

Through such processing, the optimum neighborhood prediction filter least squares design unit 113 uses the optimum values of the prediction coefficients a _{1 to} a _{5 for} the _first motion vector and the prediction coefficients b _{1 to} b ₅ for the second motion vector. Can be calculated and output as the optimum filter coefficient.

  Next, the decoding process executed in step S15 of FIG. 17 and executed in the decoding unit 84 or the decoding unit 71 will be described with reference to the flowchart of FIG.

  In step S111, the decoder 114, the decoding unit 84, or the decoding unit 71 determines whether or not the supplied encoded frame is a reference frame. If it is determined in step S111 that the frame is not a reference frame, the process proceeds to step S114.

  When the supplied encoded frame is frame data encoded by the above-described encoding process, the frame data includes D / A conversion unit 72, A / D conversion unit 81, or D / For example, a noise component such as the above-described analog noise is added during the data conversion process of the A conversion unit 85 or on the transmission path between them.

  If it is determined in step S111 that the frame is a reference frame, in step S112, the decoder 114, the decoding unit 84, or the reference frame decoder 271 of the decoding unit 71 decodes the supplied reference frame.

  In the decoding process, since the same process is executed in any of the decoder 114, the decoding unit 84, and the decoding unit 71 in the following processing, it is referred to as the decoder 114, the decoding unit 84, or the decoding unit 71. Description is omitted as necessary.

  In step S113, the reference frame decoder 271 outputs the decoded reference frame and supplies it to the frame memory 272. The frame memory 272 caches the supplied frame data, and the process proceeds to step S122 described later.

  If it is determined in step S111 that the frame is not a reference frame, in step S114, the motion target position calculation unit 273 selects one of the unprocessed blocks from the supplied encoded data of frames other than the reference frame. To do.

  In step S115, the motion target position calculation unit 273 performs the first motion vector and the second motion vector for each pixel based on the one motion vector, the second motion vector, and the optimum motion flag. Are used, the movement target position of the pixel of interest is calculated, and supplied to the pixel cutout unit 274.

  In step S116, the pixel cutout unit 274 uses the tap used when the neighborhood prediction filter coefficient is set at the time of encoding from the previous decoded image data (that is, the previous frame data) held in the frame memory 272. Similarly to (for example, 5 taps shown in FIG. 13), the peripheral pixels of the target pixel are set, the pixel values are acquired, and supplied to the filter coefficient multiplication unit 276.

  In step S <b> 117, the coefficient selection unit 275 selects, for each pixel, the optimum filter coefficient used for the calculation of the filter multiplier 276 based on the supplied optimum motion flag, and supplies the selected filter coefficient to the filter coefficient multiplier 276.

  In step S118, the filter coefficient multiplication unit 276 multiplies the pixel values of the peripheral pixels supplied from the pixel cutout unit 274 by the filter coefficients supplied from the coefficient selection unit 275, and adds up the pixel values of the target pixel. Ask. The filter coefficient multiplication unit 276 supplies the pixel value obtained by summing the multiplication result of the pixel value and the filter coefficient, that is, the pixel value for one pixel of the decoded image data to the frame memory 227.

  In step S119, the decoder 114, the decoding unit 84, or the decoding unit 71 determines whether or not the processing of all the pixels in the block has been completed. If it is determined in step S119 that the processing of all the pixels in the block has not been completed, the processing returns to step S115, and the subsequent processing is repeated.

  If it is determined in step S119 that the processing of all the pixels in the block has been completed, in step S120, the decoder 114, the decoding unit 84, or the decoding unit 71 has completed the processing of all the blocks in the frame. Judge whether or not. If it is determined in step S120 that the processing of all blocks in the frame has not been completed, the processing returns to step S114, and the subsequent processing is repeated.

  If it is determined in step S120 that the processing of all blocks in the frame has been completed, the frame memory 277 outputs the decoding result for one frame and supplies it to the frame memory 272 in step S121. The frame memory 272 caches the supplied decoding result.

  After the process of step S113 or step S121 is completed, in step S122, the decoder 114, the decoding unit 84, or the decoding unit 71 determines whether or not the last frame has been processed. If it is determined in step S121 that the process of the last frame has not been completed, the process returns to step S111, and the subsequent processes are repeated. When it is determined in step S121 that the processing of the last frame has been completed, the processing of the decoding unit 84 or the decoding unit 71 is ended, and when the decoder 114 is executing the decoding processing, the processing is as shown in FIG. Returning to step S15, the process proceeds to step S16.

By such processing, the intra-frame-encoded reference frame and the motion components of the other frames, that is, the first motion vector, the second motion vector, the optimum motion flag, and the first motion described above. It is possible to decode an encoded frame including the optimum filter coefficients a _{1 to} a ₅ corresponding to the flag and the optimum filter coefficients b _{1 to} b ₅ corresponding to the second motion flag.

  As described above, in the D / A conversion unit 72 or the D / A conversion unit 85, for example, a noise component such as the above-described analog noise may be positively added. In such a case, after the decoding process is performed in the decoding unit 71 or the decoding unit 84, in other words, in the stage after step S122, the D / A conversion unit 72 or the D / A conversion unit 85 is supplied. The digital image data is converted into analog image data, and a process for adding a noise component is executed.

  By the processing described above, encoding and decoding processing can be performed without including residual information in the encoded frame data. In the image display system 51 described with reference to FIG. 2, the image data decoded and output by the decoding unit 71 is converted into an analog image signal by the D / A conversion unit 72 and transmitted. The transmitted analog data is displayed on the display 62 or converted into digital data by the A / D converter 81 and supplied to the encoding device 63.

  That is, the image supplied to the encoding device 63 is an image to which noise has been added by the conversion processing of the D / A conversion unit 72 or the A / D conversion unit 81 or during the data transmission.

  As in the case of conventional encoding, when motion residuals are included in the encoded information, by the conversion process of the D / A conversion unit 72 or the A / D conversion unit 81 or during the transmission of data, Even if noise is added, image degradation can be prevented based on residual information.

  However, when the present invention is applied, since encoding and decoding processing can be performed without including residual information in the encoded frame data, conversion processing of the D / A conversion unit 72 and the A / D conversion unit 81 is possible. Or by adding noise during data transmission, the analog image signal output from the player 61 is recorded (dubbed or copied) on a recording medium while maintaining good image quality. Can not be.

  Further, as described with reference to FIG. 3, the encoding device 63 is provided with a high-pass filter 106 and a high-pass filter 107, which extract features of the original image such as edges and emphasize noise. ing. Therefore, when an image is recorded on a recording medium (so-called dubbing or copying is performed) and reproduced, noise is emphasized and the image is greatly deteriorated by repeating the encoding process and the decoding process.

  That is, according to the image display system to which the present invention is applied, it becomes impossible to record (dubbing or copying) the analog image signal output from the playback device 61 on a recording medium while maintaining good image quality. .

  Therefore, the image display system 51 of FIG. 2 to which the present invention is applied is a still image that can make copying impossible while maintaining good quality without degrading the quality of output by the data before copying. The image is encoded and decoded.

  The series of processes described above can also be executed by software. The software is a computer in which the program constituting the software is incorporated in dedicated hardware, or various functions can be executed by installing various programs, for example, a general-purpose personal computer For example, it is installed from a recording medium. In this case, for example, all or part of the image display system described with reference to FIG. 2 includes a personal computer 301 as shown in FIG.

  In FIG. 22, a CPU (Central Processing Unit) 311 performs various processes according to a program stored in a ROM (Read Only Memory) 312 or a program loaded from a storage unit 318 to a RAM (Random Access Memory) 313. Execute. The RAM 313 also appropriately stores data necessary for the CPU 311 to execute various processes.

  The CPU 311, ROM 312, and RAM 313 are connected to each other via a bus 314. An input / output interface 315 is also connected to the bus 314.

  The input / output interface 315 includes an input unit 316 including a keyboard and a mouse, an output unit 317 including a display and a speaker, a storage unit 318 including a hard disk, and a communication unit 319 including a modem and a terminal adapter. It is connected. The communication unit 319 performs communication processing via a network including the Internet.

  A drive 320 is connected to the input / output interface 315 as necessary, and a magnetic disk 331, an optical disk 332, a magneto-optical disk 333, a semiconductor memory 334, or the like is appropriately mounted, and a computer program read from them is loaded. Installed in the storage unit 318 as necessary.

  When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a network or a recording medium into a general-purpose personal computer or the like.

  As shown in FIG. 22, this recording medium is distributed to supply a program to a user separately from the apparatus main body, and includes a magnetic disk 331 (including a floppy disk) on which a program is stored, an optical disk 332 ( Package media including CD-ROM (compact disk-read only memory), DVD (including digital versatile disk), magneto-optical disk 333 (including MD (mini-disk) (trademark)), or semiconductor memory 334 In addition to being configured, it is configured by a ROM 312 in which a program is stored and a hard disk included in the storage unit 318, which is supplied to the user in a state of being incorporated in the apparatus main body in advance.

  Further, in the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but may be performed in parallel or It also includes processes that are executed individually.

  In the present specification, the term “system” represents the entire apparatus constituted by a plurality of apparatuses.

It is a block diagram which shows the structure of the conventional image display system. It is a block diagram which shows the structure of the image display system to which this invention is applied. FIG. 3 is a block diagram showing a more detailed configuration of an encoding unit 82 in FIG. 2. It is a figure for demonstrating a reference | standard frame and another frame. It is a figure for demonstrating the process of a high pass filter. It is a figure for demonstrating a 1st motion vector. It is a figure for demonstrating an absolute value error. It is a block diagram which shows the further detailed structure of the evaluation value change block matching motion search part of FIG. It is a figure for demonstrating a 2nd motion vector. It is a block diagram which shows the further detailed structure of the optimal motion determination part of FIG. It is a figure for demonstrating the optimal motion flag. It is a block diagram which shows the further detailed structure of the optimal neighborhood prediction filter least squares design part of FIG. It is a figure for demonstrating the relationship between the tap and motion vector for calculating the optimal neighborhood prediction filter coefficient. FIG. 4 is a block diagram showing a more detailed configuration of the decoder of FIG. 3. It is a figure for demonstrating a reference | standard frame and decoding. FIG. 3 is a block diagram showing a more detailed configuration of a decoding unit 84 in FIG. 2. It is a flowchart for demonstrating an encoding process. It is a flowchart for demonstrating an evaluation value change block matching motion search process. It is a flowchart for demonstrating optimal motion determination processing. It is a flowchart for demonstrating the optimal filter coefficient calculation process. It is a flowchart for demonstrating a decoding process. It is a block diagram which shows the structure of a personal computer.

Explanation of symbols

  51 image display system, 61 player, 62 encoding device, 71 decoding unit, 72 D / A conversion unit, 81 A / D conversion unit, 82 encoding unit, 83 recording unit, 84 decoding unit, 85 D / A conversion , 86 display, 101 reference frame extraction unit, 102 reference frame encoding unit, 103 reference frame decoder, 104, 105 frame memory, 106, 107 high-pass filter, 108 block matching motion search unit, 109 absolute value error calculation unit, 110 Code memory, 111 evaluation value change block matching motion search unit, 112 optimal motion determination unit, 113 optimal neighborhood prediction filter least square design unit, 114 decoder, 115 code output unit, 171 absolute value error acquisition unit, 172 current frame data acquisition unit , 17 Previous frame data acquisition unit, 174 Weighted block matching motion search unit, 201 First motion search result acquisition unit, 202 Second motion search result acquisition unit, 203 Previous frame data acquisition unit, 204 Current frame data acquisition unit, 205 1 prediction data generation unit, 206 second prediction data generation unit, 207 first prediction error detection unit, 208 second prediction error detection unit, 241 first motion search result acquisition unit, 242 second motion search Result acquisition unit, 243 optimal motion flag acquisition unit, 244 previous frame data acquisition unit, 245 current frame data acquisition unit, 246 first tap extraction unit, 247 second tap extraction unit, 248 first neighborhood prediction filter coefficient calculation 249 second neighborhood prediction filter coefficient calculation unit 271 reference frame deco 272 frame memory, 273 motion target position detection unit, 274 pixel segmentation unit, 275 coefficient selection unit, 276 filter coefficient multiplication unit, 277 frame memory

Claims (38)

In an encoding apparatus that receives and encodes image data consisting of a plurality of frame data,
First motion vector estimation means for estimating a first motion vector in a processing block having a plurality of processing regions in the first frame data and second frame data different from the first frame data;
A second motion vector estimating means for estimating a second motion vector different from the first motion vector in the processing block;
Motion vector selection means for selecting one of the first motion vector and the second motion vector for each processing region;
Extraction means for extracting pixels corresponding to the processing region based on one of the first motion vector and the second motion vector selected by the motion vector selection means;
A pixel in the processing region corresponding to one of the first motion vector and the second motion vector selected by the motion vector selection unit using the pixel extracted by the extraction unit. An encoding device comprising: prediction data calculation means for calculating prediction data for predicting data. The encoding apparatus according to claim 1, further comprising: an image processing unit that performs processing for enhancing an edge of an image with respect to the first frame data and the second frame data. The encoding apparatus according to claim 2, wherein the image processing means is a high-pass filter. The encoding apparatus according to claim 1, further comprising noise adding means for adding noise to the frame data. Error calculating means for calculating a detection error of the first motion vector based on the first motion vector estimated by the first motion vector estimating means and the first frame data;
The first motion vector estimation means estimates the first motion vector by block matching processing for the first frame data and the second frame data,
The second motion vector estimation unit weights the first frame data and the second frame data based on the detection error of the first motion vector calculated by the error calculation unit. The encoding apparatus according to claim 1, wherein the second motion vector is estimated by block matching processing. The prediction data calculation means includes a sum of values obtained by multiplying pixel values of the plurality of pixels extracted by the extraction means by respective predetermined coefficients, and pixel values of pixels in the corresponding first frame data. Based on the relationship, using the least square method, the optimal value of the coefficient in the case of corresponding to the first motion vector is used as the first prediction data, and the coefficient of the coefficient in the case of corresponding to the second motion vector is determined. The encoding device according to claim 1, wherein the optimum value is calculated as the second prediction data. The prediction data calculated by the prediction data calculation means is
First prediction data corresponding to the first motion vector;
The encoding apparatus according to claim 6, further comprising: second prediction data corresponding to the second motion vector. Output means for outputting an encoding result corresponding to the first frame data;
The output means includes
The first motion vector estimated by the first motion vector estimation means;
The second motion vector estimated by the second motion vector estimation means;
Data indicating a selection result by the motion vector selection means;
The encoding apparatus according to claim 1, wherein the prediction data calculated by the prediction data calculation means is output. In an encoding method of an encoding apparatus that receives and encodes image data consisting of a plurality of frame data,
A first motion vector estimation step for estimating a first motion vector in a processing block having a plurality of processing regions in the first frame data and second frame data different from the first frame data;
A second motion vector estimating step for estimating a second motion vector different from the first motion vector in the processing block;
A motion vector selection step for selecting one of the first motion vector and the second motion vector for each processing region;
An extraction step for extracting a pixel corresponding to the processing region based on one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step;
The pixel corresponding to any one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step using the pixels extracted by the processing of the extraction step. A prediction data calculation step of calculating prediction data for predicting pixel data of a processing region. The encoding method according to claim 9, further comprising an image processing step of performing processing for enhancing an edge of an image on the first frame data and the second frame data. The encoding method according to claim 10, wherein the process of the image processing step is a process of applying a high-pass filter. The encoding method according to claim 9, further comprising a noise adding step of adding noise to the frame data. An error calculating step of calculating a detection error of the first motion vector based on the first motion vector estimated by the processing of the first motion vector estimating step and the first frame data; Including
In the processing of the first motion vector estimation step, the first motion vector is estimated by block matching processing for the first frame data and the second frame data,
In the process of the second motion vector estimation step, based on the detection error of the first motion vector calculated by the process of the error calculation step with respect to the first frame data and the second frame data. The encoding method according to claim 9, wherein a process of estimating the second motion vector is executed by block matching process weighted in the same manner. In the process of the predicted data calculation step, the sum of values obtained by multiplying the pixel values of the plurality of pixels extracted by the process of the extraction step by a predetermined coefficient, and the corresponding pixels in the first frame data When the optimum value of the coefficient corresponding to the first motion vector corresponds to the second motion vector as the first prediction data using the method of least squares based on the relationship with the pixel value The encoding method according to claim 9, wherein the optimum value of the coefficient is calculated as second prediction data. The prediction data calculated by the processing of the prediction data calculation step is
First prediction data corresponding to the first motion vector;
The encoding method according to claim 14, further comprising: second prediction data corresponding to the second motion vector. An output step of outputting an encoding result corresponding to the first frame data;
In the process of the output step,
The first motion vector estimated by the processing of the first motion vector estimation step;
The second motion vector estimated by the processing of the second motion vector estimation step;
Data indicating a selection result obtained by the motion vector selection step;
The encoding method according to claim 9, wherein the prediction data calculated by the processing of the prediction data calculation step is output. A program for causing a computer to execute a process of receiving and encoding image data including a plurality of frame data,
A first motion vector estimation step for estimating a first motion vector in a processing block having a plurality of processing regions in the first frame data and second frame data different from the first frame data;
A second motion vector estimating step for estimating a second motion vector different from the first motion vector in the processing block;
A motion vector selection step for selecting one of the first motion vector and the second motion vector for each processing region;
An extraction step for extracting a pixel corresponding to the processing region based on one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step;
The pixel corresponding to any one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step using the pixels extracted by the processing of the extraction step. A computer-readable recording medium on which a program is recorded, comprising: a prediction data calculation step of calculating prediction data for predicting pixel data of a processing region. A program for causing a computer to execute a process of receiving and encoding image data including a plurality of frame data,
A first motion vector estimation step for estimating a first motion vector in a processing block having a plurality of processing regions in the first frame data and second frame data different from the first frame data;
A second motion vector estimating step for estimating a second motion vector different from the first motion vector in the processing block;
A motion vector selection step for selecting one of the first motion vector and the second motion vector for each processing region;
An extraction step for extracting a pixel corresponding to the processing region based on one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step;
The pixel corresponding to any one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step using the pixels extracted by the processing of the extraction step. A prediction data calculation step of calculating prediction data for predicting pixel data of a processing region. A program for causing a computer to execute processing characterized by comprising: In a decoding device that receives and decodes encoded image data,
Any one of the first motion vector in the processing block having a plurality of processing regions, the second motion vector in the processing block, and the first motion vector and the second motion vector is used for decoding. Motion vector determination means for receiving optimal motion vector information indicating whether each processing region is optimal and determining an optimal motion vector for each processing region of the processing block;
Receiving input of prediction data for predicting pixel data included in the processing region, pixel data in the processing block is converted based on the motion vector determined by the motion vector determination unit and the prediction data. And a calculating means for calculating. The decoding apparatus according to claim 19, further comprising noise adding means for adding noise to the pixel data calculated by the calculating means. The prediction data is
First prediction data corresponding to the first motion vector;
The decoding apparatus according to claim 19, further comprising: second prediction data corresponding to the second motion vector. The first prediction data, the second prediction data, and the optimal motion vector information are input, and the first prediction data and the second prediction are subjected to processing by the calculation unit for each processing region. The decoding apparatus according to claim 21, further comprising prediction data selection means for selecting which of the data to use. The prediction data is based on a least square method based on the relationship between the sum of values obtained by multiplying the peripheral pixels corresponding to the processing region by predetermined coefficients and the pixel values of the corresponding pixels in the first frame data. The decoding device according to claim 21, wherein the decoding value is an optimal value of the coefficient calculated by using. The calculating means includes
Based on the motion vector determined by the motion vector determination means, extract pixel values of peripheral pixels corresponding to the target pixel;
The decoding apparatus according to claim 19, wherein pixel data in the processing block is calculated by multiplying the extracted pixel value by the prediction data. In a decoding method of a decoding device for receiving and decoding encoded image data,
Any one of the first motion vector in the processing block having a plurality of processing regions, the second motion vector in the processing block, and the first motion vector and the second motion vector is used for decoding. A motion vector determination step for receiving optimal motion vector information indicating whether each processing region is optimal and determining an optimal motion vector for each processing region of the processing block;
Pixels in the processing block are received on the basis of the motion vector determined by the processing of the motion vector determination step and the prediction data upon receiving prediction data for predicting pixel data included in the processing region. And a calculation step for calculating data. 26. The decoding method according to claim 25, further comprising a noise adding step of adding noise to the pixel data calculated by the processing of the calculating step. The prediction data is
First prediction data corresponding to the first motion vector;
The decoding method according to claim 25, further comprising: second prediction data corresponding to the second motion vector. In response to the input of the first prediction data, the second prediction data, and the optimal motion vector information, the first prediction data and the second are subjected to processing by the calculation step for each processing region. 28. The decoding method according to claim 27, further comprising: a prediction data selection step of selecting which of the prediction data to be used. The prediction data is based on a least square method based on the relationship between the sum of values obtained by multiplying the peripheral pixels corresponding to the processing region by predetermined coefficients and the pixel values of the corresponding pixels in the first frame data. The decoding method according to claim 27, wherein the coefficient is an optimum value calculated using The process of the calculation step is as follows:
Based on the motion vector determined by the processing of the motion vector determination step, extract pixel values of peripheral pixels corresponding to the target pixel;
The decoding method according to claim 25, further comprising: calculating pixel data in the processing block by multiplying the extracted pixel value by the prediction data. A program for causing a computer to execute a process of receiving and decoding encoded image data,
Any one of the first motion vector in the processing block having a plurality of processing regions, the second motion vector in the processing block, and the first motion vector and the second motion vector is used for decoding. A motion vector determination step for receiving optimal motion vector information indicating whether each processing region is optimal and determining an optimal motion vector for each processing region of the processing block;
Pixels in the processing block are received on the basis of the motion vector determined by the processing of the motion vector determination step and the prediction data upon receiving prediction data for predicting pixel data included in the processing region. A computer-readable recording medium having a program recorded thereon, comprising: a calculation step for calculating data. A program for causing a computer to execute a process of receiving and decoding encoded image data,
Any one of the first motion vector in the processing block having a plurality of processing regions, the second motion vector in the processing block, and the first motion vector and the second motion vector is used for decoding. A motion vector determination step for receiving optimal motion vector information indicating whether each processing region is optimal and determining an optimal motion vector for each processing region of the processing block;
Pixels in the processing block are received on the basis of the motion vector determined by the processing of the motion vector determination step and the prediction data upon receiving prediction data for predicting pixel data included in the processing region. A program for causing a computer to execute a process characterized by including a calculation step for calculating data. An encoding unit and a decoding unit, and the encoding unit and the decoding unit repeat the encoding process and the decoding process on the image data composed of a plurality of frame data so that the image data is deteriorated. In the image processing system made in
The encoding unit includes:
A first motion for estimating a first motion vector in a processing block having a plurality of processing regions in first frame data included in the image data and second frame data different from the first frame data Vector estimation means;
A second motion vector estimating means for estimating a second motion vector different from the first motion vector in the processing block;
Motion vector selection means for selecting one of the first motion vector and the second motion vector for each processing region;
Extraction means for extracting pixels corresponding to the processing region based on one of the first motion vector and the second motion vector selected by the motion vector selection means;
A pixel in the processing region corresponding to one of the first motion vector and the second motion vector selected by the motion vector selection unit using the pixel extracted by the extraction unit. An image processing system comprising: prediction data calculation means for calculating prediction data for predicting data. Noise adding means for adding noise to the frame data;
The image processing system according to claim 33, wherein the image data output from the decoding unit is supplied to the encoding unit after noise is added to the frame data by the noise adding unit. . An encoding unit and a decoding unit, and the encoding unit and the decoding unit repeat the encoding process and the decoding process on the image data composed of a plurality of frame data so that the image data is deteriorated. In the image processing method of the image processing system made in
A first motion vector estimation step for estimating a first motion vector in a processing block having a plurality of processing regions in the first frame data and second frame data different from the first frame data;
A second motion vector estimating step for estimating a second motion vector different from the first motion vector in the processing block;
A motion vector selection step for selecting one of the first motion vector and the second motion vector for each processing region;
An extraction step for extracting a pixel corresponding to the processing region based on one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step;
The pixel corresponding to any one of the first motion vector and the second motion vector selected by the processing of the motion vector selection step using the pixels extracted by the processing of the extraction step. A prediction data calculation step of calculating prediction data for predicting pixel data of a processing area. An image processing method comprising: An encoding unit and a decoding unit, and the encoding unit and the decoding unit repeat the encoding process and the decoding process on the image data composed of a plurality of frame data so that the image data is deteriorated. In the image processing system made in
The decoding unit
Any one of the first motion vector in the processing block having a plurality of processing regions, the second motion vector in the processing block, and the first motion vector and the second motion vector is used for decoding. Motion vector determination means for receiving optimal motion vector information indicating whether each processing region is optimal and determining an optimal motion vector for each processing region of the processing block;
Receiving input of prediction data for predicting pixel data included in the processing region, pixel data in the processing block is converted based on the motion vector determined by the motion vector determination unit and the prediction data. An image processing system comprising: a calculating means for calculating. Noise adding means for adding noise to the frame data;
The image processing system according to claim 36, wherein the image data output from the decoding unit is supplied to the encoding unit after noise is added to the frame data by the noise adding unit. . An encoding unit and a decoding unit, and the encoding unit and the decoding unit repeat the encoding process and the decoding process on the image data composed of a plurality of frame data so that the image data is deteriorated. In the image processing method of the image processing system made in
Any one of the first motion vector in the processing block having a plurality of processing regions, the second motion vector in the processing block, and the first motion vector and the second motion vector is used for decoding. A motion vector determination step for receiving optimal motion vector information indicating whether each processing region is optimal and determining an optimal motion vector for each processing region of the processing block;
Pixels in the processing block are received on the basis of the motion vector determined by the processing of the motion vector determination step and the prediction data upon receiving prediction data for predicting pixel data included in the processing region. An image processing method comprising: a calculation step for calculating data.

Images