JP4748113B2 - Learning device, learning method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology of learning tap coefficients capable of decoding coded data into an image with high image quality and sound with high sound quality. <P>SOLUTION: A teacher data generating section 161 generates teacher data being a teacher for learning the tap coefficients from learning data, a student data generating section 163 generates student data being a student for learning the tap coefficients from the learning data, and a coding section 12 codes the learning data and outputs learning coded data including characteristic data as to the learning data to a mismatch detection section 13. The mismatch detection section 13 judges correctness of the characteristic data included in the learning coded data and outputs mismatch information denoting a result of the judgement to an adaptive learning section 160, and the adaptive learning section 160 uses the teacher data and the student data to learn the tap coefficients on the basis of the mismatch information. The technology above can be applied to, e.g. learning apparatuses. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to a learning apparatus, a learning method, a program, and a recording medium, and particularly, for example, decodes encoded data obtained by encoding an image, sound, or the like into a high-quality (high image quality or high sound quality) image or sound. The present invention relates to a learning device and a learning method for learning a tap coefficient so that the program can be performed, a program and a recording medium.

  For example, MPEG (Moving Picture Experts Group) method is known as a high-efficiency encoding method for image (moving image) data. In MPEG method, image data is a block unit of 8 × 8 pixels in horizontal × vertical. Then, two-dimensional DCT (Discrete Cosine Transform) transformation is performed in two directions, horizontal and vertical, and further quantized.

  As described above, in the MPEG system, image data is subjected to two-dimensional DCT conversion. For example, in MPEG2, the DCT type of a block to be subjected to two-dimensional DCT conversion is changed into a frame DCT mode and a field DCT mode in macroblock units. You can switch to In the frame DCT mode, a block is composed of pixels of the same frame, and the pixel values of such a block are two-dimensionally DCT transformed. In the field DCT mode, a block is composed of pixels in the same field, and pixel values of such a block are two-dimensionally DCT transformed.

  Whether the DCT type is the frame DCT mode or the field DCT mode is basically determined based on the characteristics of the image such as the motion of the image and continuity with the surrounding macroblocks. Is determined so as to reduce block distortion mosquito noise and the like. That is, for example, the field DCT mode is selected for an image with large motion, and the frame DCT mode is selected for an image with little motion.

  Here, the encoded data obtained by MPEG-encoding an image includes a DCT type and the like in addition to a two-dimensional DCT coefficient obtained by two-dimensional DCT transform and quantizing the image. As described above, the DCT type is determined based on the motion of the image and the like, and can be said to represent the characteristics of the image.

  By the way, in MPEG encoding, the data rate of encoded data is limited so that overflow and underflow do not occur on the decoder side. In order to limit the data rate of the encoded data, the DCT type that should originally be set to the frame DCT mode or the field DCT mode may be inappropriately set to the field DCT mode or the frame DCT mode. is there.

  However, even when such an inappropriate DCT type is set, the decoder side must decode the encoded data in accordance with the inappropriate DCT type, and the quality of the decoded image is deteriorated. There was a problem to do.

  The present invention has been made in view of such a situation, and is designed to learn tap coefficients that can decode encoded data into high-quality images and sounds.

The learning device according to the present invention is encoded data obtained by encoding data, and is used for prediction calculation with encoded data, which is performed in order to decode at least encoded data including characteristic data representing characteristics of the data. A learning device for learning tap coefficients, learning data generating means for generating teacher data from a learning data and outputting the teacher data, and learning tap coefficients from the learning data Student data generating means for generating and outputting student data to be a student, encoding means for encoding learning data, and outputting learning encoded data including characteristic data about the data, and learning data Information obtained by comparing the characteristic data included in the encoded data with the actual characteristic that is the actual characteristic of the learning data corresponding to the encoded data for learning A comparison output means for outputting, and a learning means for learning tap coefficients using teacher data and student data based on mismatch information, and the learning means outputs a plurality of attention teacher data as attention teacher data. Based on the class tap extraction means for extracting the class tap used for classifying into any one of the classes from the student data with the class tap extraction pattern previously associated with the mismatch information, and the class tap The class classification unit that classifies the teacher data of interest and outputs the class code of the corresponding class, and the prediction tap used for the prediction calculation with the tap coefficient of the teacher data of interest are associated with the mismatch information in advance. in extraction pattern of prediction tap, the prediction tap extracting means for extracting from the student data, the prediction tap The learning data includes tap coefficient calculation means for obtaining, for each class, a tap coefficient that statistically minimizes the prediction error of the predicted value of the teacher data obtained by performing the prediction calculation using the tap coefficient. Is a speech data, the encoding means encodes the learning speech data by a CELP (Code Excited Liner Prediction coding) method and outputs the learning encoded data, and the comparison output means provides the learning data. Output mismatch information representing a difference value between the characteristic data included in the encoded data and the actual characteristic that is the actual characteristic of the speech data for learning corresponding to the encoded data for learning; The student data generation means outputs the learning voice data as it is as teacher data, and the student data generation means outputs the learning voice data as CE data. Encoding and decoding by the P method, and outputting the decoding result as student data, or the teacher data generation means performs linear prediction analysis on the speech data for learning and uses the resulting linear prediction coefficient as a filter coefficient The prediction filter is driven by the speech data for learning to generate a residual signal, the residual signal is output as teacher data, and the student data generating means encodes the speech data for learning by the CELP method. The residual signal that drives the speech synthesis filter obtained as a result is output as student data, or the teacher data generation means performs linear predictive analysis on the speech data for learning and obtains the result. Output linear prediction coefficients as teacher data, and the student data generation means encodes and decodes the speech data for learning by the CELP method Then, the linear prediction coefficient that is the filter coefficient of the speech synthesis filter obtained as a result is output as student data, and when the learning data is image data, the encoding means converts the learning image data into MPEG ( Encoding by Moving Picture Experts Group) method and output encoded data for learning.Comparison output means is for learning data corresponding to characteristic data included in encoded data for learning and encoded data for learning. The mismatch information representing the combination with the actual characteristic which is the actual characteristic of the image data is output, and the teacher data generating means and the student data generating means are the teacher data generating means, and the learning image data is directly used as the teacher data. The student data generation means encodes and decodes the learning image data by the MPEG method, and outputs the decoding result as student data. Alternatively, the teacher data generation means outputs the learning image data as it is as teacher data, and the student data generation means encodes and decodes the learning image data by the MPEG method, and the result is obtained. The two-dimensional DCT coefficient is output as student data, or the teacher data generation means performs two-dimensional DCT conversion on the image data for learning, and outputs the resulting two-dimensional DCT coefficient as teacher data. The data generation means is a learning device that encodes and decodes learning image data by the MPEG method, and outputs the resulting two-dimensional DCT coefficients as student data.

The learning method of the present invention is encoded data obtained by encoding data, and is used for prediction calculation with encoded data performed to decode at least encoded data including characteristic data representing the characteristics of the data. A learning method of a learning device for learning a tap coefficient, wherein a teacher data generation step for generating and outputting teacher data serving as a teacher for learning a tap coefficient from learning data, and a tap from the learning data A student data generating step for generating and outputting student data to be a student of the coefficient learning, an encoding step for encoding the learning data and outputting encoded data for learning including characteristic data for the data; Compare the characteristic data included in the encoded data for learning with the actual characteristic that is the actual characteristic of the learning data corresponding to the encoded data for learning. A comparison output step for outputting the mismatch information, and a learning step for learning the tap coefficient using the teacher data and the student data based on the mismatch information, wherein the learning step is the teacher data of interest. A class tap extraction step of extracting the class tap used for classifying the data into any one of a plurality of classes from the student data with a class tap extraction pattern previously associated with the mismatch information; Based on the class tap, classifying the attention teacher data and outputting the class code of the corresponding class, and the prediction tap used for prediction calculation with the tap coefficient for the attention teacher data in advance corresponding to the mismatch information in the extraction pattern of prediction tap is attached, or the student data A tap coefficient for each class to obtain a tap coefficient that statistically minimizes the prediction error of the predicted value of the teacher data obtained by performing the prediction calculation using the prediction tap extraction step to be extracted and the prediction tap and the tap coefficient. When the learning data is speech data, the encoding step encodes the learning speech data by a CELP (Code Excited Liner Prediction coding) method, The comparison output step represents a difference value between the characteristic data included in the learning encoded data and the actual characteristic that is the actual characteristic of the learning speech data corresponding to the learning encoded data. The mismatch information is output. In the teacher data generation step and the student data generation step, the teacher data generation step directly converts the learning voice data into the teaching data. The data is output as data, and the student data generation step encodes and decodes the learning voice data by the CELP method and outputs the decoding result as student data, or the teacher data generation step determines the learning voice data. Is used to generate a residual signal by driving a prediction filter that uses the resulting linear prediction coefficient as a filter coefficient with learning speech data, and outputs the residual signal as teacher data In the student data generation step, the learning speech data is encoded and decoded by the CELP method, and the resulting residual signal for driving the speech synthesis filter is output as student data, or the teacher data The generation step performs linear prediction analysis on the speech data for learning, and uses the resulting linear prediction coefficients as teacher data. And the student data generation step encodes and decodes the speech data for learning by the CELP method, and outputs the linear prediction coefficients, which are the filter coefficients of the speech synthesis filter, obtained as a result, as student data, When the image data is image data, the encoding step encodes the learning image data by the MPEG (Moving Picture Experts Group) method and outputs the learning encoded data, and the comparison output step performs the learning. Output mismatch information representing a combination of characteristic data included in the encoded data for learning and actual characteristics that are actual characteristics of the learning image data corresponding to the encoded data for learning, and a teacher data generation step; In the student data generation step, the teacher data generation step outputs the image data for learning as teacher data as it is, The data generation step encodes and decodes the learning image data by the MPEG method, and outputs the decoding result as student data, or the teacher data generation step leaves the learning image data as it is as the teacher data. The student data generation step outputs the two-dimensional DCT coefficients obtained as a result of encoding and decoding the learning image data by the MPEG method as student data, or the teacher data generation step The learning image data is subjected to two-dimensional DCT transformation, and the two-dimensional DCT coefficient obtained as a result is output as teacher data. The student data generation step encodes and decodes the learning image data by the MPEG method. In this learning method, the two-dimensional DCT coefficient obtained as a result is output as student data.

The program of the present invention is encoded data obtained by encoding data, and is used for a prediction operation with encoded data, which is performed to decode at least encoded data including characteristic data representing the characteristics of the data. A program for causing a computer to perform a learning process for learning tap coefficients, a teacher data generation step for generating and outputting teacher data serving as a teacher for learning tap coefficients from learning data, and a learning data A student data generation step for generating and outputting student data to be a student of tap coefficient learning from the data, and encoding the learning data and outputting encoded learning data including characteristic data about the data Encoding step, characteristic data included in the encoded learning data, and learning data corresponding to the encoded learning data A comparison output step that outputs mismatch information obtained by comparing actual characteristics that are actual characteristics, and a learning step that learns tap coefficients using teacher data and student data based on the mismatch information. In the step, a class tap extraction pattern in which class taps used for classifying attention teacher data as attention teacher data into one of a plurality of classes is previously associated with mismatch information The class tap extracting step for extracting from the student data, the class classification step for classifying the attention teacher data based on the class tap and outputting the class code of the corresponding class, and the tap coefficient for the attention teacher data predicting a prediction tap used for prediction calculation, it is related in advance to the mismatch information Extraction pattern-up, the prediction tap extraction step of extracting from the student data, the prediction error of the prediction value of the teacher data obtained by performing prediction calculation is statistically minimized by using the prediction tap and the tap coefficient A tap coefficient calculation step for obtaining a tap coefficient for each class, and when the learning data is speech data, the encoding step uses the CELP (Code Excited Liner Prediction coding) method to convert the learning speech data. Encode and output the encoded data for learning, and the comparison output step includes the characteristic data included in the encoded data for learning and the actual characteristics of the speech data for learning corresponding to the encoded data for learning Mismatch information representing a difference value from the actual characteristic is output, and the teacher data generation step and the student data generation step include The audio data for learning is output as teacher data as it is, and the student data generation step encodes and decodes the audio data for learning by the CELP method, and outputs the decoding result as student data. The data generation step generates a residual signal by performing linear prediction analysis of the speech data for learning and driving a prediction filter using the resulting linear prediction coefficient as a filter coefficient by the speech data for learning, The residual signal is output as teacher data, and the student data generation step encodes and decodes the speech data for learning by the CELP method, and obtains the residual signal that drives the speech synthesis filter obtained as a result. Output as data, or the teacher data generation step performs linear predictive analysis of the speech data for learning and obtains the result. The linear prediction coefficient is output as teacher data, and the student data generation step encodes and decodes the speech data for learning using the CELP method, and obtains the linear prediction coefficient that is the filter coefficient of the speech synthesis filter obtained as a result. When the learning data is output as student data, and the learning data is image data, the encoding step encodes the learning image data by the MPEG (Moving Picture Experts Group) method, and converts the learning encoded data into the learning data. Output and the comparison output step is a mismatch information representing a combination of the characteristic data included in the learning encoded data and the actual characteristic that is the actual characteristic of the learning image data corresponding to the learning encoded data In the teacher data generation step and the student data generation step, the teacher data generation step outputs the learning image data as it is. Also, it is output as teacher data, and the student data generation step encodes and decodes the learning image data by the MPEG method and outputs the decoding result as student data, or the teacher data generation step Are directly output as teacher data, and the student data generation step encodes and decodes the learning image data by the MPEG method, and outputs the resulting two-dimensional DCT coefficients as student data. Alternatively, the teacher data generation step performs two-dimensional DCT transformation on the learning image data, and outputs the two-dimensional DCT coefficient obtained as a result as teacher data, and the student data generation step converts the learning image data into , Encode and decode by MPEG method, and output the resulting 2D DCT coefficients as student data That the learning process, is a program for causing a computer.

The recording medium of the present invention is encoded data obtained by encoding data, and is used for prediction calculation with encoded data, which is performed to decode at least encoded data including characteristic data representing the characteristics of the data. Is a recording medium on which a program for causing a computer to perform a learning process for learning a tap coefficient is recorded, and teacher data serving as a teacher for learning the tap coefficient is generated from the learning data and output. Including a teacher data generation step, a student data generation step for generating and outputting student data as students for learning tap coefficients from the learning data, encoding the learning data, and including characteristic data about the data The encoding step for outputting the encoded data for learning, the characteristic data included in the encoded data for learning, and the encoded data for learning The comparison output step for outputting mismatch information obtained by comparing the actual characteristics, which are the actual characteristics of the corresponding learning data, and learning the tap coefficient using the teacher data and the student data based on the mismatch information A learning step, and in the learning step, class taps used for classifying attention teacher data, which is teacher data of interest, into one of a plurality of classes are associated with mismatch information in advance. A class tap extraction step for extracting from student data with a class tap extraction pattern, a class classification step for classifying attention teacher data based on the class tap, and outputting a class code of the corresponding class, and a attention teacher for data, the prediction tap used to predict operation of the tap coefficients, advance the mismatch information In extraction pattern of prediction tap is attached response, the prediction tap extraction step of extracting from the student data, the prediction error of the prediction value of the teacher data obtained by performing prediction calculation by using the prediction tap and tap coefficients Statistics When the learning data is speech data, the encoding step converts the learning speech data into CELP (Code Excited Liner). Prediction coding) is used to output learning encoded data, and the comparison output step includes characteristic data included in the learning encoded data and learning speech corresponding to the learning encoded data. Mismatch information representing the difference between the actual characteristics of the data and the actual characteristics is output. The teacher data generation step and the student data generation step The data generation step outputs the learning voice data as it is as teacher data, and the student data generation step encodes and decodes the learning voice data by the CELP method, and the decoding result is used as student data. Or the teacher data generation step performs linear predictive analysis on the speech data for learning and drives a prediction filter using the resulting linear prediction coefficient as a filter coefficient by the speech data for learning. A signal is generated, the residual signal is output as teacher data, and the student data generation step encodes and decodes the speech data for learning by the CELP method, and obtains a residual signal for driving the speech synthesis filter obtained as a result. The difference signal is output as student data, or the teacher data generation step linearizes the audio data for learning. A linear prediction coefficient obtained by measurement and analysis is output as teacher data, and a student data generation step encodes and decodes learning speech data by the CELP method, and obtains a filter of a speech synthesis filter obtained as a result When the linear prediction coefficient that is a coefficient is output as student data and the learning data is image data, the encoding step encodes the learning image data by the MPEG (Moving Picture Experts Group) method, The learning encoded data is output, and the comparison output step is an actual characteristic that is the actual characteristic of the characteristic data included in the encoded learning data and the actual image data corresponding to the encoded learning data. Mismatch information representing the combination of the teacher data generation step and the student data generation step, the teacher data generation step The image data is output as teacher data as it is, and the student data generation step encodes and decodes the learning image data by the MPEG method, and outputs the decoding result as student data or generates teacher data. The step outputs the learning image data as teacher data as it is, and the student data generation step encodes and decodes the learning image data by the MPEG method, and the two-dimensional DCT coefficient obtained as a result is The data is output as student data, or the teacher data generation step performs two-dimensional DCT conversion on the learning image data, and the resulting two-dimensional DCT coefficients are output as teacher data. The student data generation step performs learning Image data is encoded and decoded by the MPEG method, and the resulting two-dimensional DCT coefficients are The learning process of outputting as student data, a recording medium having a program recorded for causing a computer.

In the learning device, the learning method, the program, and the recording medium of the present invention, teacher data serving as a teacher for learning tap coefficients and student data serving as students are generated and output from learning data. Further, learning data is encoded, and encoded learning data including characteristic data for the data is output. Based on the mismatch information obtained by comparing the characteristic data included in the learning encoded data and the actual characteristic that is the actual characteristic of the learning data corresponding to the learning encoded data , the teacher data The tap coefficient is learned using the student data.

  According to the learning device, the learning method, the program, and the recording medium of the present invention, it is possible to learn a tap coefficient that enables decoding of encoded data into high-quality data.

  FIG. 1 shows a configuration example of an embodiment of a decoding device to which the present invention is applied.

  The decoding apparatus includes encoded data reproduced from a recording medium (not shown) (for example, an optical disk, a magneto-optical disk, a phase change disk, a magnetic tape, a semiconductor memory, etc.) or a transmission medium (for example, the Internet or a CATV network). Encoded data transmitted via a satellite line, terrestrial wave, etc.) is input as a decoding target. Here, the encoded data is obtained by encoding predetermined data with a predetermined encoding method, and includes at least characteristic data representing characteristics of the predetermined data.

  As the encoded data, for example, as described later, audio data encoded by CELP (Code Excited Linear Prediction coding) method, image data encoded by MPEG2 method, or the like may be employed. it can.

  Here, when the encoded data is audio data encoded by the CELP method, the encoded data includes an L code representing a lag. Since this lag corresponds to the pitch period of the encoded voice data, and thus represents the characteristic of the voice data called the pitch period, it can be called characteristic data.

  In addition, when the encoded data is obtained by encoding image data by the MPEG2 system, as described above, the encoded data includes a DCT type. Therefore, it represents the characteristics of the image, and can also be referred to as characteristic data.

  Note that the encoded data to be decoded in the decoding device is not limited to audio data encoded by the CELP method as described above or image data encoded by the MPEG2 method.

  The encoded data input to the decoding device is supplied to the mismatch detection unit 1 and the decoding processing unit 2.

  The mismatch detection unit 1 detects mismatch information from the encoded data. That is, the mismatch detection unit 1 determines the correctness of the characteristic data included in the encoded data, and outputs mismatch information representing the determination result to the decoding processing unit 2. The decoding processing unit 2 decodes the encoded data based on the mismatch information supplied from the mismatch detection unit 1, and outputs decoded data obtained as a result.

  Next, processing (decoding processing) of the decoding device in FIG. 1 will be described with reference to the flowchart in FIG.

  Encoded data is supplied to the mismatch detection unit 1 and the decoding processing unit 2, and the mismatch detection unit 1 first detects mismatch information from the encoded data and supplies it to the decoding processing unit 2 in step S1. Then, the process proceeds to step S2. In step S2, the decoding processing unit 2 decodes the encoded data in which the mismatch information is detected based on the mismatch information supplied from the mismatch detection unit 1, outputs the decoded data, and proceeds to step S3. In step S3, the mismatch detection unit 1 or the decoding processing unit 2 determines whether there is still encoded data to be decoded. If it is determined in step S3 that encoded data to be decoded still exists, the process returns to step S1, and the same processing is repeated thereafter.

  If it is determined in step S3 that there is no encoded data to be decoded, the process ends.

  Next, FIG. 3 shows a configuration example of another embodiment of a decoding device to which the present invention is applied. In the figure, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the decoding apparatus in FIG. 3 is basically configured in the same manner as the decoding apparatus in FIG. 1 except that the parameter storage unit 3 is newly provided.

  The parameter storage unit 3 stores parameters obtained by learning by a learning device, which will be described later, and the decoding processing unit 2 uses the parameters stored in the parameter storage unit 3 and supplies encoded data supplied thereto. Is decrypted.

  Therefore, in the decoding device of FIG. 3, the decoding processing unit 2 performs the same processing as the decoding device of FIG. 1 except that the encoded data is decoded using the parameters stored in the parameter storage unit 3. Therefore, the description of the process is omitted.

  Next, FIG. 4 shows a configuration example of an embodiment of a learning device that learns parameters to be stored in the parameter storage unit 3 of FIG.

  The learning data storage unit 11 stores learning data that is data used for parameter learning.

  The encoding unit 12 reads the learning data stored in the learning data storage unit 11 and encodes the learning data with the same encoding method as the encoded data to be decoded by the decoding device in FIG. To do. Encoded data obtained by encoding the learning data (hereinafter referred to as learning encoded data as appropriate) is supplied from the encoding unit 12 to the mismatch detection unit 13.

  The mismatch detection unit 13 is configured in the same manner as the mismatch detection unit 1 in FIG. 3, detects mismatch information from the encoded data supplied from the encoding unit 12, and supplies the mismatch information to the learning processing unit 14.

  The learning processing unit 14 reads out the learning data stored in the learning data storage unit 11 and, from the learning data, the teacher data serving as a learning teacher about the parameters and the student data serving as the learning student. Generate. Further, the learning processing unit 14 learns parameters using the generated teacher data and student data based on the mismatch information supplied from the mismatch detection unit 13.

  Next, processing (learning processing) of the learning device in FIG. 4 will be described with reference to the flowchart in FIG.

  First, in step S <b> 11, the encoding unit 12 reads and encodes the learning data stored in the learning data storage unit 11, and sends the learning encoded data obtained as a result to the mismatch detection unit 13. Then, the process proceeds to step S12. In step S12, the mismatch detection unit 13 detects mismatch information from the encoded data supplied from the encoding unit 12, supplies the mismatch information to the learning processing unit 14, and proceeds to step S13.

  In step S13, the learning processing unit 14 reads the learning data from the learning data storage unit 11, and generates teacher data and student data from the learning data. Further, the learning processing unit 14 learns parameters using the generated teacher data and student data based on the mismatch information supplied from the mismatch detection unit 13.

  That is, the learning processing unit 14 performs a process (learning) for calculating an optimum parameter for enabling the corresponding teacher data to be obtained from the student data based on the mismatch information. Do.

  In step S14, the encoding unit 12 or the learning processing unit 14 determines whether learning data that has not yet been processed is stored in the learning data storage unit 11. If it is determined in step S14 that learning data that has not yet been processed is stored in the learning data storage unit 11, the process returns to step S11, and the learning data that has not yet been processed is targeted. Thereafter, the same processing is repeated.

  If it is determined in step S14 that the learning data not yet processed is not stored in the learning data storage unit 11, that is, all the learning data stored in the learning data storage unit 11 are stored. When learning is performed using the learning process, the process proceeds to step S15, where the learning processing unit 14 calculates parameters based on the learning result of step S13, and ends the process.

  Next, details of the decoding device and the learning device in the case where the encoded data is audio data encoded by the CELP method will be described. In the present embodiment, the decoding device and the learning device utilize the class classification adaptation process previously proposed by the applicant.

  The class classification adaptation process includes a class classification process and an adaptation process. By the class classification process, data is classified based on its property, and the adaptation process is performed for each class.

  Here, the adaptive processing will be described by taking, as an example, a case where low-quality sound (hereinafter, appropriately referred to as low-quality sound) is converted into high-quality sound (hereinafter, appropriately referred to as high-quality sound).

  In this case, in the adaptive processing, a high sound quality in which the sound quality of the low sound quality sound is improved by linear combination of a sound sample constituting the low sound quality sound (hereinafter referred to as a low sound quality sound sample as appropriate) and a predetermined tap coefficient. By obtaining the predicted value of the voice sample of the voice, it is possible to obtain a voice in which the quality of the low-quality voice is improved.

Specifically, for example, a certain high sound quality voice data is used as teacher data, and a low sound quality sound data obtained by degrading the sound quality of the high sound quality sound is used as student data, and a sound sample (hereinafter referred to as a high sound quality sound) The predicted value E [y] of y (referred to as a high-quality sound sample, as appropriate), a set of several low-quality sound samples (sound samples constituting low-quality sound) x ₁ , x ₂ ,. Suppose that it is obtained by a linear linear combination model defined by linear combination of tap coefficients w ₁ , w ₂ ,. In this case, the predicted value E [y] can be expressed by the following equation.

E [y] = w ₁ x ₁ + w ₂ x ₂ +...
... (1)

で定義すると、次のような観測方程式が成立する。 In order to generalize equation (1), a matrix W composed of a set of tap coefficients w _j , a matrix X composed of a set of student data x _ij , and a matrix Y ′ composed of a set of predicted values E [y _j ]

Then, the following observation equation holds.

XW = Y '
... (2)

Here, the component x _ij of the matrix X means the j-th student data in the i-th set of student data (the set of student data used for prediction of the _i-th teacher data y _i ). A component w _j of W represents a tap coefficient by which a product with the jth student data in the set of student data is calculated. Y _i represents the i-th teacher data, and therefore E [y _i ] represents the predicted value of the i-th teacher data. Note that y on the left side of the equation (1) is obtained by omitting the suffix i of the component y _i of the matrix Y, and x ₁ , x ₂ ,. The suffix i of the component x _ij is omitted.

で定義すると、式（２）から、次のような残差方程式が成立する。 Consider that the least square method is applied to the observation equation of Equation (2) to obtain a predicted value E [y] close to the high-quality sound sample y. In this case, it is composed of a matrix Y composed of a set of true values y of high sound quality speech samples as teacher data and a set of residuals (errors relative to the true value y) e of predicted values E [y] of the high sound quality speech samples y. Matrix E

From the equation (2), the following residual equation is established.

XW = Y + E
... (3)

を最小にすることで求めることができる。 In this case, the tap coefficient w _j for obtaining the predicted value E [y] close to the high-quality sound sample y is a square error.

Can be obtained by minimizing.

Accordingly, when the above-mentioned square error differentiated by the tap coefficient w _j is 0, that is, the tap coefficient w _j satisfying the following equation is optimal for obtaining the predicted value E [y] close to the high-quality sound sample y. It will be value.

・・・（４）

... (4)

Therefore, first, the following equation is established by differentiating the equation (3) by the tap coefficient w _j .

・・・（５）

... (5)

  From equations (4) and (5), equation (6) is obtained.

・・・（６）

... (6)

Further, considering the relationship among the student data x _ij , the tap coefficient w _j , the teacher data y _i , and the residual e _{i in} the residual equation of Equation (3), the following normal equation is obtained from Equation (6): Obtainable.

・・・（７）

... (7)

で定義するとともに、ベクトルＷを、数１で示したように定義すると、式
ＡＷ＝ｖ
・・・（８）で表すことができる。 In addition, the normal equation shown in Expression (7) has a matrix (covariance matrix) A and a vector v,

And when the vector W is defined as shown in Equation 1, the equation AW = v
(8)

Each normal equation in equation (7) can be set to the same number as the number J of tap coefficients w _{j to} be obtained by preparing a certain number of sets of student data x _ij and teacher data y _i. Therefore, by solving the equation (8) with respect to the vector W (however, to solve the equation (8), the matrix A in the equation (8) needs to be regular), the optimal tap coefficient w _j is _calculated . Can be sought. In solving the equation (8), for example, a sweeping method (Gauss-Jordan elimination method) or the like can be used.

As described above, the optimum tap coefficient using the student data and the teacher data (in this case, when the predicted value of the teacher data is obtained from the student data, the tap coefficient that minimizes the sum of the square errors of the predicted values) ) leave the learning for determining the w _j, further using the tap coefficient w _j, the equation (1), an adaptive processing determine the closest prediction value E [y] to the teacher data y.

  Note that adaptive processing is not included in low-quality sound, but differs from simple interpolation in that a component included in high-quality sound is reproduced. That is, in the adaptive processing, as long as only the expression (1) is seen, it looks the same as simple interpolation using a so-called interpolation filter, but the tap coefficient w corresponding to the tap coefficient of the interpolation filter uses the teacher data y. In other words, since it is obtained by learning, it is possible to reproduce components included in high-quality sound. From this, it can be said that the adaptive process is a process having a voice creation action.

  In the above-described case, high-quality sound data is used as teacher data, and sound data having low sound quality as teacher data is used as student data. High-quality image data was used as the data, and as the student data, the image data as the teacher data was thinned out, added with noise, or filtered with a low-pass filter to reduce the image quality. It is possible to use one. In this case, a tap coefficient for converting a low-quality image into a high-quality image (predicted value thereof) can be obtained.

  Furthermore, for example, high-quality image data is used as the teacher data, and two-dimensional DCT coefficients obtained by performing two-dimensional DCT conversion on the image data as the teacher data, and further quantizing and dequantizing the student data are used. It is also possible to do so. In this case, a tap coefficient for converting the two-dimensional DCT coefficient into a high-quality image (predicted value thereof) can be obtained.

  In the above-described case, the predicted value of high-quality sound is linearly predicted, but the predicted value can also be predicted by a quadratic or higher formula.

  FIG. 6 shows a configuration example of an audio data processing device that converts low-quality sound data into high-quality sound data by the class classification adaptive processing as described above.

  The low sound quality voice data is supplied to the pitch detection unit 21 and the tap extraction units 22 and 23.

  The pitch detection unit 21 detects the pitch period of the low-quality sound data supplied thereto and supplies the detected pitch period to the tap extraction units 22 and 23.

  The tap extraction unit 22 sequentially extracts voice samples of the high-quality sound data as attention data, and further extracts some sound samples of the low-quality sound data used for predicting the attention data as prediction taps. Further, the tap extraction unit 23 extracts some audio samples of the low sound quality audio data used for classifying the attention data as class taps.

  Here, the tap extraction unit 22 extracts, as prediction taps, some voice samples located at positions close to the voice sample corresponding to the data of interest among the voice samples of the low sound quality voice data. Further, the tap extraction unit 22 changes the structure of the prediction tap according to the pitch period of the position corresponding to the attention data supplied from the pitch detection unit 21. That is, the tap extraction unit 22 changes the sound sample of the low-quality sound data that is the prediction tap according to the pitch period. Specifically, for example, when the pitch period is long, the tap extraction unit 22 selects a predetermined number of audio samples over a relatively wide range from the audio samples corresponding to the data of interest among the audio samples of the low-quality audio data. Are extracted as prediction taps. For example, when the pitch period is short, the tap extraction unit 22 predicts a predetermined number of audio samples over a relatively narrow range from the audio samples corresponding to the data of interest among the audio samples of the low-quality audio data. Extract as a tap.

  The tap extraction unit 23 also extracts class taps from the low-quality sound data in the same manner as the tap extraction unit 22.

  Here, the prediction tap and the class tap have the same tap structure in order to simplify the description. However, the prediction tap and the class tap can have different tap structures.

  The prediction tap obtained by the tap extraction unit 22 is supplied to the prediction unit 26, and the class tap obtained by the tap extraction unit 23 is supplied to the class classification unit 24.

  The class classification unit 24 classifies the data of interest based on the class tap from the tap extraction unit 23, and outputs a class code corresponding to the class obtained as a result to the coefficient memory 25.

  Here, as a method of classifying, for example, ADRC (Adaptive Dynamic Range Coding) or the like can be employed.

  In the method using ADRC, voice samples constituting a class tap are subjected to ADRC processing, and the class of data of interest is determined according to the ADRC code obtained as a result.

In the K-bit ADRC, for example, the maximum value MAX and the minimum value MIN of the audio samples constituting the class tap are detected, and DR = MAX-MIN is set as the local dynamic range of the set, and the dynamic range DR Based on this, the speech samples that make up the class tap are requantized to K bits. That is, from each voice sample forming the class taps, the minimum value MIN is subtracted, and the subtracted value is divided (quantized) by DR / 2 ^K. Then, a bit string obtained by arranging the K-bit audio samples constituting the class tap in a predetermined order, which is obtained as described above, is output as an ADRC code. Therefore, when a class tap is subjected to, for example, 1-bit ADRC processing, each audio sample constituting the class tap is an average value of the maximum value MAX and the minimum value MIN after the minimum value MIN is subtracted. Division (rounded down after the decimal point) is performed, whereby each audio sample is converted into one bit (binarized). A bit string in which the 1-bit audio samples are arranged in a predetermined order is output as an ADRC code.

Note that the class classification unit 24 can output, for example, the level distribution pattern of the audio sample constituting the class tap as it is as the class code. However, in this case, if the class tap is composed of N speech samples, and K bits are assigned to each speech sample, the number of class codes output by the class classification unit 24 is (2 ^N ) ^K, which is a huge number that is exponentially proportional to the number of bits K of the audio sample.

  Therefore, the class classification unit 24 preferably performs class classification by compressing the information amount of the class tap by the above-described ADRC processing or vector quantization.

  The coefficient memory 25 stores the tap coefficient of the class corresponding to the class code at the address corresponding to each class code, and the tap stored at the address corresponding to the class code supplied from the class classification unit 24. The coefficient is supplied to the prediction unit 26.

  The prediction unit 26 acquires the prediction tap output from the tap extraction unit 22 and the tap coefficient output from the coefficient memory 25, and uses the prediction tap and the tap coefficient to perform the linear prediction calculation shown in Expression (1). I do. Thereby, the prediction unit 26 obtains (outputs a predicted value) of high-quality sound data as attention data and outputs it.

  Next, FIG. 7 shows a configuration example of a learning device that learns tap coefficients to be stored in the coefficient memory 25 of FIG.

  High-quality sound data is input to the learning device as learning sound data. The learning sound data is supplied to the time thinning filter 31 and is added as a teacher data to the adder 36. To be supplied.

  The time decimation filter 31 decimates audio samples of high sound quality audio data as learning audio data at a predetermined decimation rate, thereby generating low sound quality audio data, and pitch detection unit 32 and taps as student data It supplies to the extraction parts 33 and 34.

  The pitch detection unit 32 detects the pitch period of the low-quality sound data as the student data supplied thereto, and supplies it to the tap extraction units 33 and 34.

  The tap extraction unit 33 sequentially uses voice samples of high-quality sound data as teacher data as attention data, and the prediction data having the same structure as the tap extraction unit 22 of FIG. It is constructed by extracting some sound samples from the low sound quality sound data as the student data supplied to. The tap extraction unit 34 also selects a class tap having the same structure as the tap extraction unit 23 shown in FIG. 6 for the data of interest, and extracts some audio samples from the low-quality sound data as student data supplied thereto. Configure by extracting.

  Note that the tap extraction units 33 and 34, respectively, in the same manner as the tap extraction units 22 and 23 in FIG. 6, are predicted taps according to the pitch period of the position corresponding to the data of interest supplied from the pitch detection unit 32. The tap structure of the class tap is changed.

  The prediction tap obtained by the tap extraction unit 33 is supplied to the addition unit 36, and the class tap obtained by the tap extraction unit 34 is supplied to the class classification unit 35.

  Similar to the case of the class classification unit 24 of FIG. 6, the class classification unit 35 classifies the attention data based on the class tap from the tap extraction unit 33, and adds the class code corresponding to the class obtained as a result. Output to the insertion unit 36.

  The adding unit 36 classifies the adding of the teacher data serving as attention data among the teacher data supplied thereto and the student data constituting the prediction tap supplied from the tap extracting unit 33 into the class. This is performed for each class code supplied from the classification unit 35.

That is, the adding unit 36 uses the prediction tap (student data) for each class corresponding to the class code supplied from the class classifying unit 35, and is a student in the matrix A of Expression (8). An operation corresponding to multiplication (x _in x _im ) between data and summation (Σ) is performed.

Furthermore, the adding unit 36 also uses the prediction tap (student data) and the attention data (teacher data) for each class corresponding to the class code supplied from the class classification unit 35, and uses the vector v of Expression (8). The calculation corresponding to multiplication (x _in y _i ) of student data and teacher data and summation (Σ), which are the components _in FIG.

In other words, the adding unit 36 stores the component of the matrix A and the component of the vector v in the formula (8) obtained for the teacher data, which was previously regarded as the attention data, in a built-in memory (not shown). Corresponding to each component of the matrix A or vector v, the teacher data newly set as the attention data is calculated using the teacher data y _i and the student data x _in (x _im ) Add component x _in x _im or x _in y _i (addition represented by summation in matrix A, vector v).

  Then, the adding unit 36 sets the normal equation shown in the equation (8) for each class by performing the above addition using all the teacher data supplied thereto as the data of interest, and calculates the tap coefficient. To the unit 37.

  The tap coefficient calculation unit 37 finds and outputs the tap coefficient for each class by solving the normal equation for each class supplied from the addition unit 36. The coefficient memory 25 in FIG. 6 stores the tap coefficients for each class determined in this way.

  In addition, there may be a class in which the number of normal equations necessary for obtaining tap coefficients cannot be obtained due to an insufficient number of samples of input learning speech data. For the class, the tap coefficient calculation unit 37 outputs a default tap coefficient, for example.

  Next, encoding and decoding of audio data by the CELP method will be described with reference to FIGS. As CELP methods, there are VSELP (Vector Sum Excited Linear Prediction), PSI-CELP (Pitch Synchronous Innovation CELP), CS-ACELP (Conjugate Structure Algebraic CELP), etc., but here, for example, VSELP A method will be described as an example.

  FIG. 8 shows a configuration example of a VSELP encoding apparatus that encodes audio data by the VSELP method.

  The audio to be encoded is input to a microphone (microphone) 41, where it is converted into an audio signal as an electrical signal, and supplied to an A / D (Analog / Digital) converter 42. The A / D converter 42 samples the analog audio signal from the microphone 41 at a sampling frequency such as 8 kHz to perform A / D conversion into a digital audio signal, and further performs quantum quantization with a predetermined number of bits. Then, the data is supplied to a calculator 43 and an LPC (Liner Prediction Coefficient) analysis unit 44.

The LPC analysis unit 44 performs LPC analysis on the audio signal from the A / D conversion unit 42 for each frame having a length of 160 samples, for example, and P-order linear prediction coefficients α ₁ , α ₂ ,. Find α _P. Then, the LPC analysis unit 44 supplies a vector having the P-th order linear prediction coefficient α _p (p = 1, 2,..., P) as an element to the vector quantization unit 45 as a speech feature vector. To do.

  The vector quantization unit 45 stores a code book in which a code vector having a linear prediction coefficient as an element and a code are associated with each other, and based on the code book, the feature vector α from the LPC analysis unit 44 is converted into a vector quantum. And a code obtained as a result of the vector quantization (hereinafter referred to as A code (A_code) as appropriate) is supplied to the code determination unit 55.

Further, the vector quantizing unit 45 linear predictive coefficients α ₁ ′, α ₂ ′,..., Α _P that are elements constituting the code vector α ′ corresponding to the A code output to the code determining unit 55. 'Is supplied to the speech synthesis filter 46.

The speech synthesis filter 46 is, for example, an IIR (Infinite Impulse Response) type digital filter, which converts the linear prediction coefficient α _p ′ (p = 1, 2,..., P) from the vector quantization unit 45 into the IIR filter. In addition to the filter coefficient (tap coefficient), speech synthesis is performed using the residual signal e supplied from the computing unit 54 as an input signal.

That, LPC analysis performed by the LPC analysis section 44, the sample value s _n-1 of the audio signal (sample value) s _n, and past adjacent to P number of the current time n, s _n-2, · ..., to s _nP, formula _{s n + α 1 s n-} 1 + α 2 s n-2 + ··· + α P s nP = e n
(9) Assuming that the linear primary combination shown in (9) is established, the predicted value (linear predicted value) s _n ′ of the sample value s _n at the current time n is used as the past P sample values s _n−1. , S _n-2 ,..., S _nP , the expression s _n ′ = − (α ₁ s _n−1 + α ₂ s _n−2 +... + Α _P s _nP )
... (10)
The linear prediction coefficient α _p that minimizes the square error between the actual sample value s _n and the linear prediction value s _n ′ when linear prediction is performed by the above-described method is obtained.

Here, in equation (9), {e _n } (..., E _n−1 , e _n , e _{n + 1} ,...) Has an average value of 0 and a variance of the predetermined value σ ² . They are random variables that are uncorrelated with each other.

Equation (9), the sample value s _n has the formula _{s n = e n - (α} 1 s n-1 + α 2 s n-2 + ··· + α P s nP)
(11) When this is converted to Z, the following equation is established.

S = E / (1 + α ₁ z ⁻¹ + α ₂ z ⁻² +... + Α _P z ^−P )
(12)
However, in the equation (12), S and E, the Z-transform of s _n and e _n in the equation (11) represents, respectively.

Here, from equation (9) and (10), e _n is the formula e _n = s _n -s _n '
(13), which is called a residual signal between the actual sample value s _n and the linear prediction value s _n ′.

Therefore, from equation (12), the linear prediction coefficient alpha _p with the tap coefficients of the IIR filter, by the residual signal e _n as an input signal of the IIR filter, it is possible to obtain the speech signal s _n.

Therefore, as described above, the speech synthesis filter 46 uses the linear prediction coefficient α _p ′ from the vector quantization unit 45 as a tap coefficient and the residual signal e supplied from the computing unit 54 as an input signal. (12) is calculated (the residual signal e is filtered) to obtain a voice signal (synthesized sound signal) ss.

Note that in the speech synthesis filter 46, not the linear prediction coefficient α _p obtained as a result of the LPC analysis by the LPC analysis unit 44 but the linear prediction coefficient α _p ′ as a code vector corresponding to the code obtained as a result of the vector quantization. Therefore, the synthesized sound signal output from the speech synthesis filter 46 is not basically the same as the speech signal output from the A / D converter 42.

  The synthesized sound signal ss output from the speech synthesis filter 46 is supplied to the computing unit 43. The computing unit 43 subtracts the speech signal s output from the A / D conversion unit 42 from the synthesized sound signal ss from the speech synthesis filter 46 and supplies the subtraction value to the square error computation unit 47. The square error calculation unit 47 calculates the square sum of the subtraction values from the calculator 43 (the square sum of the sample values of the k-th frame), and supplies the square error obtained as a result to the square error minimum determination unit 48. .

  The square error minimum determination unit 48 is associated with the square error output by the square error calculation unit 47, an L code (L_code) as a code representing lag, a G code (G_code) as a code representing gain, and a code word I code (I_code) is stored as a code representing, and L code, G code, and L code corresponding to the square error output by the square error calculation unit 47 are output. The L code is supplied to the adaptive codebook storage unit 49, the G code is supplied to the gain decoder 50, and the I code is supplied to the excitation codebook storage unit 51. Further, the L code, the G code, and the I code are also supplied to the code determination unit 55.

  The adaptive codebook storage unit 49 stores an adaptive codebook in which, for example, a 7-bit L code is associated with a predetermined delay time (lag), and the residual signal e supplied from the computing unit 54 is The result is delayed by the delay time associated with the L code supplied from the square error minimum determination unit 48 and output to the computing unit 52.

  Here, since the adaptive codebook storage unit 49 outputs the residual signal e with a delay corresponding to the time corresponding to the L code, the output signal is a signal close to a periodic signal whose period is the delay time. Become. This signal mainly serves as a drive signal for generating a synthesized sound of voiced sound in speech synthesis using a linear prediction coefficient. Therefore, the time corresponding to the L code represents the pitch period of the voiced sound.

  The gain decoder 50 stores a table in which G codes are associated with predetermined gains β and γ, and gains β and γ associated with G codes supplied from the square error minimum determination unit 48 are stored. Output. Gains β and γ are supplied to computing units 52 and 53, respectively.

  The excitation code book storage unit 51 stores, for example, an excitation code book in which a 9-bit I code is associated with a predetermined excitation signal, and is associated with the I code supplied from the square error minimum determination unit 48. The excited signal is output to the computing unit 53.

  Here, the excitation signal stored in the excitation codebook is, for example, a signal close to white noise or the like, and in speech synthesis using a linear prediction coefficient, mainly a drive signal for generating unvoiced synthesized sound and Become.

  The computing unit 52 multiplies the output signal of the adaptive codebook storage unit 49 and the gain β output from the gain decoder 50, and supplies the multiplication value l to the computing unit 54. The computing unit 53 multiplies the output signal of the excitation codebook storage unit 51 by the gain γ output from the gain decoder 50 and supplies the multiplication value n to the computing unit 54. The computing unit 54 adds the multiplication value l from the computing unit 52 and the multiplication value n from the computing unit 53, and supplies the addition value to the speech synthesis filter 46 as a residual signal e.

In the speech synthesis filter 46, as described above, the residual signal e supplied from the computing unit 54 is filtered by the IIR filter using the linear prediction coefficient α _p ′ supplied from the vector quantization unit 45 as a tap coefficient. The synthesized sound signal obtained as a result is supplied to the calculator 43. Then, the calculator 43 and the square error calculation unit 47 perform the same process as described above, and the square error obtained as a result is supplied to the square error minimum determination unit 48.

  The square error minimum determination unit 48 determines whether or not the square error from the square error calculation unit 47 is minimized (minimum). When the square error minimum determination unit 48 determines that the square error is not minimized, the L error code, the G code, and the L code corresponding to the square error are output as described above. The process is repeated.

  On the other hand, when the square error minimum determination unit 48 determines that the square error is minimized (for example, when the square error is equal to or smaller than a predetermined threshold value), the square error minimum determination unit 48 outputs a determination signal to the code determination unit 55. The code determination unit 55 latches the A code supplied from the vector quantization unit 45, and sequentially latches the L code, G code, and I code supplied from the square error minimum determination unit 48. When the confirmation signal is received from the square error minimum determination unit 48, the A code, L code, G code, and I code latched at that time are supplied to the channel encoder 56. The channel encoder 56 multiplexes the A code, L code, G code, and I code from the code determination unit 55 and outputs the multiplexed data.

  Hereinafter, in order to simplify the description, it is assumed that the A code, the L code, the G code, and the I code are obtained for each frame. However, for example, one frame can be divided into four subframes, and the L code, G code, and I code can be obtained for each subframe.

  Here, in FIG. 8 (the same applies to FIGS. 9 to 11 to be described later), [k] is added to each variable, which is an array variable. Although k represents the number of frames, the description thereof is omitted as appropriate in the specification.

  Next, FIG. 9 shows a configuration example of a VSELP decoding apparatus that decodes encoded data output from the VSELP encoding apparatus of FIG. 8 by the VSELP method.

  The encoded data output from the VSELP encoding apparatus in FIG. 8 is supplied to the channel decoder 61. The channel decoder 61 separates the L code, G code, I code, and A code from the encoded data, and each of them is an adaptive codebook storage unit 62, gain decoder 63, excitation codebook storage unit 64, filter coefficient decoding Supply to the container 65.

  The adaptive codebook storage unit 62, the gain decoder 63, the excitation codebook storage unit 64, and the calculators 66 to 68 are the adaptive codebook storage unit 49, the gain decoder 50, the excitation codebook storage unit 51, and the calculator of FIG. 52 to 54 are configured in the same manner, and the same processing as described in FIG. 8 is performed, whereby the L code, the G code, and the I code are decoded into the residual signal e. This residual signal e is given as an input signal to the speech synthesis filter 69.

The filter coefficient decoder 65 stores the same codebook as that stored in the vector quantization unit 45 of FIG. 8, decodes the A code into the linear prediction coefficient α _p ′, and generates a speech synthesis filter 69. To supply.

The speech synthesis filter 69 is configured in the same manner as the speech synthesis filter 46 in FIG. 8, and uses the linear prediction coefficient α _p ′ from the filter coefficient decoder 65 as a filter coefficient (tap coefficient) and is supplied from the computing unit 68. Equation (12) is calculated using the residual signal e to be input as an input signal, thereby generating a synthesized sound signal when the square error minimum determination unit 48 in FIG. Output as audio data.

  As described above, since the residual signal and the linear prediction coefficient given to the speech synthesis filter 69 of the VSELP decoding apparatus in FIG. 9 are encoded and transmitted in the VSELP encoding apparatus in FIG. 8, the VSELP in FIG. In the decoding device, the code is decoded into a residual signal and a linear prediction coefficient, and given to the speech synthesis filter 69.

  However, the decoded residual signal and linear prediction coefficient (hereinafter, appropriately referred to as a decoded residual signal or a decoded linear prediction coefficient) include errors such as quantization error (error due to vector quantization). Therefore, the residual signal obtained by LPC analysis of speech does not match the linear prediction coefficient.

  For this reason, the decoded speech data output from the speech synthesis filter 69 of the VSELP decoding apparatus in FIG. 9 has distortion and deteriorated sound quality.

  Therefore, by performing the above-described class classification adaptive processing in the VSELP decoding device, it is possible to obtain decoded speech data with improved sound quality.

  FIG. 10 shows a configuration example of such a VSELP decoding device. In the figure, portions corresponding to those in FIG. 9 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  The tap extraction unit 81 is supplied with the decoded voice data output from the voice synthesis filter 69, and the tap extraction unit 81 is similar to the tap extraction unit 22 in FIG. What is used as a prediction tap (sample value) is extracted and supplied to the prediction unit 85.

  The tap extraction unit 82 is also supplied with the decoded voice data output from the voice synthesis filter 69. The tap extraction unit 82, like the tap extraction unit 23 in FIG. A class tap (sample value) is extracted and supplied to the class classification unit 83.

  Similar to the class classification unit 24 in FIG. 6, the class classification unit 83 performs class classification based on the class tap supplied from the tap extraction unit 82, and stores the class code as the class classification result in the coefficient memory 84. Supply.

  The coefficient memory 84 stores tap coefficients for each class obtained by performing a learning process in the learning device of FIG. 11 described later, and is stored at an address corresponding to the class code output by the class classification unit 83. The tap coefficient is supplied to the prediction unit 85.

  The prediction unit 85 acquires the prediction tap output from the tap extraction unit 81 and the tap coefficient output from the coefficient memory 84, and uses the prediction tap and the tap coefficient, similarly to the prediction unit 26 in FIG. The linear prediction calculation shown in Formula (1) is performed. As a result, the prediction unit 85 outputs high-quality sound data in which the low-quality decoded sound data output from the sound synthesis filter 69 is improved.

  Note that the tap extraction unit 81 is supplied with an L code, a G code, an I code, and an A code for each frame (or subframe) output from the channel decoder 61. The tap extraction unit 81 can extract a prediction tap from the L code, G code, I code, or A code. Furthermore, the tap extraction unit 81 can also change the tap structure of the prediction tap based on the L code, G code, I code, or A code.

  The tap extraction unit 82 is also supplied with the L code, G code, I code, and A code output from the channel decoder 61, and the tap extraction unit 82 is also the same as in the tap extraction unit 81. Class taps are also extracted from L code, G code, I code, or A code, and further, the tap structure of the class tap is changed based on the L code, G code, I code, or A code Is possible.

  Next, FIG. 11 shows a configuration example of a learning device that performs learning processing of tap coefficients stored in the coefficient memory 84 of FIG.

  The computing units 93 to code determination unit 105 are configured in the same manner as the computing units 43 to code determination unit 45 of FIG. A learning speech signal is input to the computing unit 93. Therefore, the computing unit 93 to the code determination unit 105 perform the same processing as that in FIG. 8 on the learning speech signal. Applied.

  The tap extraction units 111 and 112 are supplied with the decoded speech data output from the speech synthesis filter 96 when the square error minimum determination unit 98 determines that the square error is minimized as student data. Further, the learning audio signal is supplied as it is to the adding unit 114 as teacher data.

  The tap extraction unit 111 extracts a prediction tap having the same structure as the tap extraction unit 81 in FIG. 10 from the speech sample of the decoded speech data output from the speech synthesis filter 96 and supplies the prediction tap to the addition unit 114.

  The tap extraction unit 112 also extracts a class tap having the same structure as the tap extraction unit 82 in FIG. 10 from the speech sample of the decoded speech data output from the speech synthesis filter 96 and supplies the class tap to the class classification unit 113.

  Based on the class tap from the tap extraction unit 112, the class classification unit 113 performs the same class classification as in the class classification unit 83 of FIG. 10 and supplies the resulting class code to the addition unit 114. .

  The adding unit 114 receives the learning speech signal as teacher data, receives the prediction tap from the tap extraction unit 111 as student data, and receives the teacher data and student data from the class classification unit 113 as targets. For each class code, the same addition as in the addition unit 36 of FIG. 7 is performed, whereby the normal equation shown in Expression (8) is established for each class.

  Similar to the tap coefficient calculation unit 37 in FIG. 7, the tap coefficient calculation unit 115 calculates and outputs a tap coefficient for each class by solving the normal equation generated for each class in the addition unit 114.

  The coefficient memory 84 in FIG. 10 stores the tap coefficient for each class output from the tap coefficient calculation unit 115 as described above.

  Therefore, the tap coefficients stored in the coefficient memory 84 of FIG. 10 are learned so that the prediction error (square error) of the predicted value of the high-quality sound obtained by performing the linear prediction calculation is statistically minimized. Therefore, the audio data output by the prediction unit 85 in FIG. 10 is of high sound quality.

  The tap extraction units 111 and 112 are supplied with the L code, the G code, the I code, and the A code that are output when the code determination unit 105 receives the confirmation signal from the square error minimum determination unit 98. When the tap extraction unit 81 or 82 in FIG. 10 uses the L code, the G code, the I code, or the A code to configure the prediction tap or the class tap, the tap extraction unit 111 or 112 is used. However, prediction taps and class taps are configured using L codes, G codes, I codes, or A codes.

  Next, FIG. 12 shows a detailed configuration example of the decoding device of FIG.

  The encoded characteristic information extracting unit 121 is supplied with encoded data to be decoded. The encoded characteristic information extracting unit 121 extracts characteristic data included in the encoded data from the encoded data. Extracted and supplied to the determination unit 123.

  The actual characteristic extraction unit 122 is also supplied with the encoded data to be decoded, and the actual characteristic extraction unit 122 extracts the actual characteristic that is the actual characteristic of the original data corresponding to the encoded data. And supplied to the determination unit 123.

  Here, for example, when the encoded data is encoded audio data, the actual characteristic extraction unit 122 determines, for example, the pitch period of the audio data as the actual characteristic. For example, when the encoded data is obtained by encoding image data, the actual characteristic extraction unit 122 obtains, for example, an evaluation value for evaluating the movement of the image data as the actual characteristic.

  The determination unit 123 determines the correctness of the characteristic data by comparing the characteristic data supplied from the encoding characteristic information extraction unit 121 and the actual characteristic supplied from the actual characteristic extraction unit 122. Then, the determination unit 123 outputs mismatch information as a determination result of the correctness of the characteristic data to the decoding processing unit 2.

  The encoding characteristic information extraction unit 121, the actual characteristic extraction unit 122, and the determination unit 123 described above constitute the mismatch detection unit 1.

  The preprocessing unit 131 is supplied with encoded data to be decoded. The preprocessing unit 131 performs predetermined preprocessing on the encoded data, and the preprocessed data obtained as a result thereof. Is supplied to the class classification adaptive processing unit 132.

  The class classification adaptive processing unit 132 configures prediction taps and class taps from the preprocess data supplied from the preprocessing unit 131, and performs the class classification adaptive processing as described above by referring to the coefficient memory 141. Then, the class classification adaptive processing unit 132 outputs data obtained by performing the class classification adaptive processing (hereinafter referred to as adaptive processing data as appropriate) to the post-processing unit 133.

  Here, the class classification adaptation processing unit 132 is supplied with the mismatch information output from the determination unit 123 of the mismatch detection unit 1, and the class classification adaptation processing unit 132 determines the class based on the mismatch information. Classification adaptation processing is performed.

  The post-processing unit 133 performs predetermined post-processing on the data output from the class classification adaptation processing unit 132, thereby obtaining and outputting the encoded data decoded into high-quality decoded data.

  The above preprocessing unit 131, class classification adaptive processing unit 132, and postprocessing unit 133 constitute the decoding processing unit 2.

  The coefficient memory 141 stores tap coefficients for each class used by the class classification adaptation processing unit 132 for performing the class classification adaptation process.

  Note that the parameter storage unit 3 is configured by the coefficient memory 141.

  Next, FIG. 13 shows a configuration example of the class classification adaptation processing unit 132 of FIG.

  The preprocessing data output from the preprocessing unit 131 is supplied to the tap extraction units 151 and 152.

  The tap extraction unit 151 extracts the adaptive processing data to be obtained as attention data, and further extracts some of the preprocessing data used for predicting the attention data as prediction taps. Further, the tap extraction unit 152 extracts some of the preprocess data used for classifying the attention data as class taps.

  Here, mismatch information output from the determination unit 123 (FIG. 12) is also supplied to the tap extraction units 151 and 152. And the tap extraction parts 151 and 152 change the structure of a prediction tap and a class tap, respectively based on mismatch information.

  Here, in order to simplify the description, it is assumed that the prediction tap and the class tap have the same tap structure. However, the prediction tap and the class tap can have different tap structures.

  The prediction tap obtained by the tap extraction unit 151 is supplied to the prediction unit 154, and the class tap obtained by the tap extraction unit 152 is supplied to the class classification unit 153.

  In addition to the class tap, mismatch information is also supplied to the class classification unit 153. The class classification unit 153 classifies the data of interest based on the class tap and the mismatch information from the tap extraction unit 152. The class code corresponding to the class obtained as a result is supplied to the coefficient memory 141.

  The coefficient memory 141 stores the tap coefficient of the class corresponding to the class code at the address corresponding to each class code, and the tap stored at the address corresponding to the class code supplied from the class classification unit 153 The coefficient is supplied to the prediction unit 154.

  The prediction unit 154 acquires the prediction tap output from the tap extraction unit 151 and the tap coefficient output from the coefficient memory 141, and uses the prediction tap and the tap coefficient to perform the linear prediction calculation shown in Expression (1). I do. As a result, the prediction unit 154 obtains and outputs the adaptive process data (predicted value thereof).

  Next, processing (decoding processing) of the decoding device in FIG. 12 will be described with reference to the flowchart in FIG.

  In the tap extraction unit 151 of the class classification adaptive processing unit 132 (FIG. 13), the adaptive processing data to be obtained is the attention data. In step S21, the mismatch detection unit 1 encodes the encoded data corresponding to the attention data. Mismatch information is generated from (hereinafter, referred to as encoded data of interest as appropriate).

  That is, in the mismatch detection unit 1, the encoding characteristic information extraction unit 121 extracts characteristic data included in the target encoded data from the target encoded data, supplies the characteristic data to the determination unit 123, and also the actual characteristic extraction unit 122. However, an actual characteristic that is an actual characteristic of the original data corresponding to the encoded data of interest is extracted and supplied to the determination unit 123. Then, the determination unit 123 determines the correctness of the characteristic data by comparing the characteristic data supplied from the encoding characteristic information extraction unit 121 and the actual characteristic supplied from the actual characteristic extraction unit 122, The mismatch information as the determination result is supplied to the class classification adaptation processing unit 132.

  Then, the process proceeds to step S22, and the preprocessing unit 131 performs preprocessing on the encoded data for obtaining preprocessing data necessary for configuring the prediction tap and the class tap for the data of interest, and the result The obtained preprocessing data is supplied to the class classification adaptive processing unit 132.

  In the class classification adaptive processing unit 132 (FIG. 13), in step S23, the tap extraction units 151 and 152 use the preprocessing data supplied from the preprocessing unit 131 and use the tap structure based on the mismatch information from the mismatch detection unit 1. Each of the prediction tap and class tap is configured. Then, the prediction tap is supplied from the tap extraction unit 151 to the prediction unit 154, and the class tap is supplied from the tap extraction unit 152 to the class classification unit 153.

  The class classification unit 153 receives the class tap for the data of interest from the tap extraction unit 152, and classifies the data of interest based on the class tap and the mismatch information supplied from the mismatch detection unit 1 in step S24. The class code representing the class of the data of interest is output to the coefficient memory 141.

  The coefficient memory 141 reads and outputs the tap coefficient stored at the address corresponding to the class code supplied from the class classification unit 153. In step S25, the prediction unit 154 acquires the tap coefficient output from the coefficient memory 141, and proceeds to step S26.

  In step S <b> 26, the prediction unit 154 performs the linear prediction calculation shown in Expression (1) using the prediction tap output from the tap extraction unit 151 and the tap coefficient acquired from the coefficient memory 141. As a result, the prediction unit 154 obtains (predicted value) of adaptive processing data as the attention data and supplies it to the post-processing unit 133.

  In step S27, the post-processing unit 133 (FIG. 12) performs predetermined post-processing on the data of interest from the class classification adaptive processing unit 132 (prediction unit 154 thereof), thereby obtaining decoded data for output. To do.

  Thereafter, the process proceeds to step S28, where it is determined whether there is any adaptive processing data that has not yet been set as the data of interest. If it is determined in step S28 that there is adaptation processing data that has not yet been set as attention data, one of the adaptation processing data that has not yet been set as attention data is newly set as attention data, and the process returns to step S21. Thereafter, the same processing is repeated.

  If it is determined in step S28 that there is no adaptive process data that has not yet been set as attention data, the process ends.

  Next, FIG. 15 shows a detailed configuration example of the learning device in FIG. 4 when learning tap coefficients to be stored in the coefficient memory 141 in FIG.

  In the embodiment of FIG. 15, the mismatch detection unit 13 includes an encoding characteristic information extraction unit 171, an actual characteristic extraction unit 172, and a determination unit 173. The encoded data output from the encoding unit 12 is The encoding characteristic information extraction unit 171 and the actual characteristic extraction unit 172 are supplied. The encoding characteristic information extraction unit 171, the actual characteristic extraction unit 172, or the determination unit 173 is configured similarly to the encoding characteristic information extraction unit 121, the actual characteristic extraction unit 122, or the determination unit 123 of FIG. As in the case described with reference to FIG. 12, mismatch information is obtained from encoded data corresponding to attention teacher data to be described later and supplied to the learning processing unit 14.

  The learning processing unit 14 includes an adaptive learning unit 160, a teacher data generation unit 161, and a student data generation unit 163.

  The adaptive learning unit 160 includes a teacher data storage unit 162, a student data storage unit 164, tap extraction units 165 and 166, a class classification unit 167, an addition unit 168, and a tap coefficient calculation unit 169, and a teacher data generation unit 161. Includes a reverse post-processing unit 161A, and the student data generation unit 163 includes an encoding unit 163A and a pre-processing unit 163B.

  The reverse post-processing unit 161A reads the learning data from the learning data storage unit 11, and performs processing complementary to the processing performed by the post-processing unit 133 in FIG. 12 (hereinafter referred to as reverse post-processing as appropriate). . That is, for example, if the learning data is y and the post-processing unit 133 in FIG. 12 performs post-processing on the adaptive processing data x by a function f (x), the reverse post-processing unit 161A Then, the processing represented by the function f-1 (y) (f-1 () represents the inverse function of the function f ()) is applied to the learning data y as a reverse post-processing, and the result is obtained. The data is output to the adaptive learning unit 160 as teacher data. Note that the teacher data output by the reverse post-processing unit 161A corresponds to adaptive data supplied from the class classification adaptive processing unit 132 to the post-processing unit 133 in FIG.

  The teacher data storage unit 162 temporarily stores the teacher data output from the teacher data generation unit 161 (the inverse post-processing unit 161A).

  The encoding unit 163A reads the learning data from the learning data storage unit 11, encodes it with the same encoding method as the encoding unit 12, and outputs the encoded data. Therefore, the encoding unit 163A outputs the same encoded data that the encoding unit 12 outputs. Note that the encoding units 12 and 163A can be shared by a single encoding unit.

  The preprocessing unit 163B performs the same preprocessing as that performed by the preprocessing unit 131 in FIG. 12 on the encoded data output from the encoding unit 163A, and uses the preprocessed data obtained as a result as student data. To the adaptive learning unit 160. Note that the student data output by the preprocessing unit 163B corresponds to the preprocessing data supplied from the preprocessing unit 131 to the class classification adaptive processing unit 132 in FIG.

  The student data storage unit 164 temporarily stores the student data output from the student data generation unit 163 (preprocessing unit 163B).

  The tap extraction unit 165 sequentially uses the teacher data stored in the teacher data storage unit 162 as attention teacher data, and extracts the student data stored in the student data storage unit 164 for the attention teacher data. A prediction tap having the same tap structure as that of the 13 tap extraction units 151 is configured and output. The tap extraction unit 165 is supplied with mismatch information output from the mismatch detection unit 13 (the determination unit 173), and the tap extraction unit 165 is similar to the tap extraction unit 151 of FIG. The tap structure of the prediction tap is changed based on the mismatch information about the attention teacher data.

  The tap extraction unit 166 configures a class tap having the same tap structure as the tap extraction unit 152 of FIG. 13 by extracting the student data stored in the student data storage unit 164 for the teacher data of interest. Output. Note that the tap extraction unit 166 is supplied with mismatch information output from the mismatch detection unit 13, and the tap extraction unit 166 is similar to the tap extraction unit 152 in FIG. The tap structure of the class tap is changed based on the mismatch information.

  The class classification unit 167 is supplied with the class tap output from the tap extraction unit 166 and the mismatch information output from the mismatch detection unit 13. The class classification unit 167 performs the same class classification as the class classification unit 153 of FIG. 13 based on the class tap and mismatch information about the teacher data of interest, and adds the class code corresponding to the resulting class to the addition unit 168. Output to.

  The adding unit 168 reads the attention teacher data from the teacher data storage unit 162, and targets the attention teacher data and the student data constituting the prediction tap configured for the attention teacher data supplied from the tap extraction unit 165. The addition is performed for each class code supplied from the class classification unit 167.

In other words, the adding unit 168 uses the prediction tap (student data) for each class corresponding to the class code supplied from the class classification unit 167, and is a component in the matrix A of Expression (8). An operation corresponding to multiplication (x _in x _im ) between data and summation (Σ) is performed.

Furthermore, the adding unit 168 again uses the prediction tap (student data) and the teacher data for each class corresponding to the class code supplied from the class classification unit 167, and uses each component in the vector v of the equation (8) The calculation corresponding to multiplication (x _in y _i ) of student data and teacher data and summation (Σ) is performed.

In other words, the adding unit 168 stores the component of the matrix A and the component of the vector v in the formula (8) obtained for the teacher data that was previously set as the teacher data of interest in a built-in memory (not shown). For each component of the matrix A or vector v, the teacher data newly set as the attention teacher data is calculated using the teacher data y _i and the student data x _in (x _im ). Add the corresponding component x _in x _im or x _in y _i (addition represented by summation in matrix A, vector v).

  Then, the addition unit 168 performs the above-described addition using all the teacher data stored in the teacher data storage unit 162 as attention teacher data, thereby obtaining the normal equation shown in the equation (8) for each class. Then, the normal equation is supplied to the tap coefficient calculation unit 169.

  The tap coefficient calculation unit 169 calculates and outputs the tap coefficient for each class by solving the normal equation for each class supplied from the addition unit 168.

  Next, processing (learning processing) of the learning device in FIG. 15 will be described with reference to the flowchart in FIG.

  First, in step S31, the teacher data generation unit 161 and the student data generation unit 163 generate teacher data and student data from the learning data stored in the learning data storage unit 11, respectively. The teacher data is supplied from the teacher data generation unit 161 to the teacher data storage unit 162 and stored therein, and the student data is supplied from the student data generation unit 163 to the student data storage unit 164 and stored therein.

  After that, the tap extraction unit 165 sets the teacher data stored in the teacher data storage unit 162 as notable teacher data that has not yet been noted teacher data. In step S32, the encoding unit 12 encodes the learning data stored in the learning data storage unit 11, thereby encoding data corresponding to the attention teacher data (the learning data corresponding to the attention teacher data). Is obtained) and supplied to the mismatch detection unit 13.

  The mismatch detection unit 13 generates mismatch information about the teacher data of interest from the encoded data supplied from the encoding unit 12 and supplies the mismatch information to the tap extraction units 165 and 166 and the class classification unit 167 of the learning processing unit 14. .

  In step S 34, the tap extraction unit 165 reads the student data stored in the student data storage unit 164 for the teacher data of interest based on the mismatch information, forms a prediction tap, and supplies the prediction tap to the addition unit 168. At the same time, the tap extraction unit 166 also reads out the student data stored in the student data storage unit 164 for the teacher data of interest based on the mismatch information, forms a class tap, and supplies the class tap to the class classification unit 167.

  In step S35, the class classification unit 167 performs class classification on the attention teacher data based on the class tap and mismatch information regarding the attention teacher data, and outputs the class code corresponding to the class obtained as a result to the addition unit 168. To do.

  In step S36, the adding unit 168 reads the attention teacher data from the teacher data storage unit 162, uses the attention teacher data and the prediction tap from the tap extraction unit 165, and uses the matrix A of Expression (8) and the vector v. Calculate the component. Furthermore, the adding unit 168 applies the matrix A obtained from the attention data and the prediction tap to the component corresponding to the class code from the class classification unit 167 among the components of the matrix A and the vector v that have already been obtained. And the vector v component are added, and the process proceeds to step S37.

  In step S <b> 37, the tap extraction unit 165 determines whether teacher data that has not yet been set as the teacher data of interest is stored in the teacher data storage unit 162. If it is determined in step S37 that the teacher data that is not the attention teacher data is still stored in the teacher data storage unit 162, the tap extraction unit 165 newly adds the teacher data that is not yet the attention teacher data. As attention teacher data, the process returns to step S32, and the same processing is repeated thereafter.

  If it is determined in step S37 that the teacher data that is not the attention teacher data is not stored in the teacher data storage unit 162, the adding unit 168 determines that the matrix A for each class obtained by the processing so far is performed. Then, the normal equation of Expression (8) composed of the components of the vector v is supplied to the tap coefficient calculation unit 169, and the process proceeds to Step S38.

  In step S38, the tap coefficient calculation unit 169 calculates and outputs a tap coefficient for each class by solving the normal equation for each class supplied from the addition unit 168, and ends the process.

  It should be noted that due to the fact that the number of learning data stored in the learning data storage unit 11 is not sufficient, a class in which the number of normal equations necessary for obtaining tap coefficients cannot be obtained may arise. Although possible, for such a class, the tap coefficient calculation unit 169 outputs a default tap coefficient, for example.

  Next, FIG. 17 illustrates a first detailed configuration example of the decoding device in FIG. 12 when the encoded data is audio data encoded by the CELP method.

  In the embodiment of FIG. 17, the encoding characteristic information extraction unit 121 includes a channel decoder 181. For example, the channel decoder 181 is configured in the same manner as the channel decoder 61 in FIG. 9, extracts an L code from the encoded data, and supplies the L code as characteristic data to the determination unit 123.

  The actual characteristic extraction unit 122 includes a VSELP decoding device 182 and a pitch detection unit 183. The VSELP decoding device 182 is configured in the same manner as the VSELP decoding device shown in FIG. 9, decodes encoded data by the VSELP method, and supplies decoded speech data obtained as a result to the pitch detection unit 183.

  The pitch detection unit 183 detects the pitch period of the decoded speech data output from the VSELP decoding device 182. That is, for example, the pitch detection unit 183 calculates the autocorrelation of the decoded speech data, detects the pitch period based on the autocorrelation, and supplies it to the determination unit 123 as an actual characteristic.

  The determination unit 123 includes a difference calculation unit 184. The difference calculation unit 184 calculates the difference between the time corresponding to the L code from the channel decoder 181 (the time representing the pitch period of the voice) and the pitch period of the actually obtained decoded voice data, and calculates the difference value. The mismatch information is supplied to the class classification adaptation processing unit 132.

  On the other hand, the preprocessing unit 131 includes a VSELP decoding device 185. Similarly to the VSELP decoding device 182, the VSELP decoding device 185 decodes the encoded data by the VSELP method, and outputs the decoded speech data to the class classification adaptation processing unit 132 as preprocessing data.

  In the class classification adaptation processing unit 132, class classification adaptation processing is performed on the decoded speech data output from the VSELP decoding device 185 of the preprocessing unit 131, and the adaptive processing data obtained as a result is output to the post-processing unit 133. Is done. The post-processing unit 133 outputs the adaptive processing data from the class classification adaptive processing unit 132 as it is as high-quality sound data.

  Therefore, in the embodiment of FIG. 17, the class classification adaptation processing unit 132 performs the class classification adaptation process, thereby decoding the encoded data output from the VSELP decoding device 185 of the preprocessing unit 131 using the VSELP method. The decoded audio data is converted into high sound quality audio data and output.

  That is, in the class classification adaptive processing unit 132 (FIG. 13), the decoded speech data output from the VSELP decoding device 185 of the preprocessing unit 131 is supplied to the tap extraction units 151 and 152.

  The tap extraction unit 151 extracts, as prediction taps, some speech samples of decoded speech data used to predict the attention data, using the high-quality sound data that is not yet the attention data as the attention data. The tap extraction unit 152 also extracts some audio samples of the decoded audio data used for classifying the data of interest as class taps.

  Here, as described above, mismatch information is also supplied from the determination unit 123 to the tap extraction units 151 and 152, and the tap extraction units 151 and 152 are configured based on the mismatch information and predicted taps and classes. Each tap structure is changed.

  That is, in the channel decoder 181 of the encoding characteristic information extraction unit 121 (FIG. 17), for example, the L code of the subframe (or frame) including the decoded audio data at the position corresponding to the data of interest is extracted. The difference calculation unit 184 is supplied.

  In addition, in the VSELP decoding device 182 of the actual characteristic extraction unit 122, for example, several tens of frames before and after a frame including decoded audio data at a position corresponding to the data of interest (hereinafter, referred to as interest decoded audio data as appropriate) are decoded. The decoded audio data obtained as a result is supplied to the pitch detector 183. The pitch detector 183 calculates the autocorrelation of the decoded speech data supplied from the VSELP decoding device 182 and detects the pitch period near the decoded speech data of interest based on the autocorrelation. The pitch period is supplied to the difference calculation unit 184. The difference calculation unit 184 calculates a difference between the time T1 corresponding to the L code supplied from the channel decoder 181 and the pitch period T2 supplied from the pitch detection unit 183, and the difference value ΔT (= T1−T2). ) As mismatch information for the data of interest.

When the tap extraction unit 151 (FIG. 13) receives the difference value ΔT as mismatch information for the data of interest as described above, for example, the absolute value of the difference value ΔT is compared with a predetermined threshold value TH _T. To do.

Then, when the absolute value of the difference value ΔT is equal to or less than the threshold value TH _T (ie, the time corresponding to the L code of the subframe including the focused decoded speech data), the tap extracting unit 151 performs the focused decoding. When the pitch period of audio data is correctly expressed, for example, all audio samples of a subframe including target decoded audio data (hereinafter, appropriately referred to as a target subframe) and one subframe immediately preceding the target subframe are included. A speech sample every other sample and every other sample in the subframe immediately after the target subframe are extracted as prediction taps.

Further, when the absolute value of the difference value ΔT is greater than (or greater than) the threshold value TH _T , the tap extraction unit 151, that is, the time corresponding to the L code of the subframe including the target decoded speech data When the pitch period of the decoded audio data is not correctly expressed, for example, all the audio samples of the target subframe, audio samples every two samples of the first and second subframes of the target subframe, and the target subframe Are extracted as prediction taps for every two samples of subframes after the first and second subframes.

  Similarly to the tap extraction unit 151, the tap extraction unit 152 also extracts a class tap whose tap structure has been changed based on the mismatch information from the decoded speech data.

  Here, based on the mismatch information, only the position of the speech sample extracted as the prediction tap is changed, and the number of speech samples constituting the prediction tap is not changed. However, in the tap extraction unit 151, the mismatch is performed. It is also possible to change the number of audio samples of the decoded audio data constituting the prediction tap based on the information.

  Further, in the tap extraction unit 151, as in the case described with reference to FIG. 10, the L code, G code, I code, or A code obtained in the VSELP decoding device 185 can be extracted as a prediction tap. Also in this case, it is possible to change the position and number of subframes of the L code, G code, I code, or A code that are prediction taps based on the mismatch information.

  Further, the mismatch information includes not only the difference value ΔT but also the L code used to obtain the difference value ΔT, the pitch period T2 of the decoded audio data, that is, the L code output by the channel decoder 181; It is possible to include the pitch period T2 output by the pitch detector 183. In this case, the tap extraction unit 151 changes the tap structure of the prediction tap as described above based on not only the difference value ΔT but also the L code and the pitch period T2 of the decoded speech data. Is possible.

  The tap extraction unit 152 can also configure class taps in the same manner as in the tap extraction unit 151.

  The prediction tap obtained by the tap extraction unit 151 is supplied to the prediction unit 154, and the class tap obtained by the tap extraction unit 152 is supplied to the class classification unit 153.

  In addition to the class tap, mismatch information about the attention data is also supplied to the class classification unit 153. As described above, the class classification unit 153 classifies the attention data based on the class tap and the mismatch information.

  That is, the class classification unit 153 obtains a class code by performing the above-described ADRC processing based on, for example, a class tap for attention data. Here, the class code obtained from the class tap is hereinafter appropriately referred to as a class tap code.

Furthermore, the classification unit 153, for example, the absolute value of the difference value △ T as mismatch information for the target data, by comparing with a predetermined threshold value TH _T, determination of the 1-bit class code.

That is, the class classification unit 153 determines that the time corresponding to the L code of the subframe including the target decoded speech data is equal to the pitch of the target decoded speech data when the absolute value of the difference value ΔT is equal to or less than the threshold value TH _T. When the period is correctly expressed, for example, 1 of 0 or 1 is set as a class code. Further, the classification unit 153, the absolute value of the difference value △ T is greater than the threshold value TH _T, i.e., the time corresponding to the L code of the subframe including the target decoded audio data, the pitch period of interest decoded audio data Is not represented correctly, for example, 0 or 1 is set as the class code. Here, the class code obtained from the mismatch information is hereinafter referred to as a mismatch code as appropriate.

  Thereafter, the class classification unit 153, for example, adds the mismatch code obtained for the attention data as the upper bits of the class tap code obtained for the attention data, and generates a code composed of this class tap code and the mismatch code. , And output as the final class code for the data of interest.

  This class code is supplied to the coefficient memory 141. In the coefficient memory 141, the tap coefficient corresponding to the class code is read and supplied to the prediction unit 154.

  As described above, not only the difference value ΔT but also the L code used to obtain the difference value ΔT or the pitch period T2 of the decoded audio data, that is, the channel decoder 181 outputs the mismatch information as described above. When including the L code and the pitch period T2 output from the pitch detection unit 183, the class classification unit 153 performs class classification based on the L code and the pitch period T2 included in the mismatch information. Is possible.

Further, in the above case, in response to the magnitude relationship between the absolute value and the threshold value TH _T of the difference value △ T, has been to determine the mismatched code 1 bit, as the mismatched code, other, for example, It is possible to adopt a 2's complement display of the difference value ΔT.

  The prediction unit 154 performs the linear prediction calculation shown in Expression (1) using the prediction tap output from the tap extraction unit 151 and the tap coefficient acquired from the coefficient memory 141. Accordingly, the prediction unit 154 obtains attention data (predicted value thereof), that is, high-quality sound data, and supplies it to the post-processing unit 133.

  In the post-processing unit 133, as described above, the output of the class classification adaptive processing unit 132 (the prediction unit 154), that is, the high-quality sound data is output as it is.

  Next, FIG. 18 illustrates a detailed configuration example of the learning device in FIG. 15 when learning tap coefficients to be stored in the coefficient memory 141 of the decoding device in FIG.

  In the embodiment of FIG. 18, high-quality sound data (learning sound data) is stored as learning data in the learning data storage unit 11.

  The encoding unit 12 includes a VSELP encoding device 191. The VSELP encoding device 191 is configured similarly to the VSELP encoding device illustrated in FIG. 8, for example. However, the VSELP encoding device 191 is not provided with the microphone 41 and the A / D conversion unit 42 of the VSELP encoding device of FIG.

  The VSELP encoding device 191 reads out the learning speech data from the learning data storage unit 11 and encodes it by the VSELP method, and the encoded data obtained as a result is encoded characteristic information extracting unit 171 and actual characteristic extracting unit 172. To supply.

  The encoding characteristic information extraction unit 171 includes a channel decoder 192, the actual characteristic extraction unit 172 includes a VSELP decoding device 193 and a pitch detection unit 194, and the determination unit 173 includes a difference calculation unit 195. The channel decoder 192, VSELP decoding device 193, pitch detection unit 194, or difference calculation unit 195 performs the same processing as the channel decoder 181, VSELP decoding device 182, pitch detection unit 183, or difference calculation unit 184 of FIG. As a result, the difference value ΔT described with reference to FIG. 17 is obtained as mismatch information for the attention teacher data, and is output to the adaptive learning unit 160.

  The reverse post-processing unit 161A reads the learning speech data from the learning data storage unit 11, and outputs the learning speech data as it is to the adaptive learning unit 160 as teacher data. In adaptive learning section 160 (FIG. 15), teacher data from post-processing section 161A is stored in teacher data storage section 162.

  The encoding unit 163A is configured by a VSELP encoding device 196. The VSELP encoding device 196 reads the learning speech data from the learning data storage unit 11 and encodes it using the VSELP method, similarly to the VSELP encoding device 191. The encoded data obtained as a result is output to the preprocessing unit 163B.

  The preprocessing unit 163B is configured by a VSELP decoding device 197 configured similarly to the VSELP decoding device in FIG. 9, and the VSELP decoding device 197 decodes the encoded data from the VSELP encoding device 196 by the VSELP method, The decoded speech data obtained as a result is output to the adaptive learning unit 160 as student data. In the adaptive learning unit 160 (FIG. 15), the student data from the VSELP decoding device 197 is stored in the student data storage unit 164.

  Then, the adaptive learning unit 160 uses the teacher data and the student data, and predicts the predicted value of the teacher data obtained by performing the linear prediction calculation of Expression (1) from the prediction tap and tap coefficient extracted from the student data. Learning is performed to obtain a tap coefficient that statistically minimizes the error.

  That is, in the adaptive learning unit 160 (FIG. 15), the tap extraction unit 165 sets the teacher data stored in the teacher data storage unit 162 that has not yet been set as the attention teacher data as the attention teacher data, and the attention teacher data. Is configured from the student data stored in the student data storage unit 164 and supplied to the adding unit 168. Further, the tap extraction unit 166 configures class taps from the student data stored in the student data storage unit 164 for the teacher data of interest and supplies the class taps to the class classification unit 167.

  Here, the channel decoder 192, the VSELP decoding device 193, the pitch detection unit 194, or the difference calculation unit 195 is the same as the channel decoder 181, the VSELP decoding device 182, the pitch detection unit 183, or the difference calculation unit 184 of FIG. As a result, the difference value ΔT as mismatch information for the teacher data of interest is supplied to the tap extraction units 165 and 166 and the class classification unit 167.

  Then, in the tap extraction unit 165 or 166, as in the case of the tap extraction unit 151 or 152 (FIG. 13) described with reference to FIG. 17, the predicted tap or class tap whose tap structure has been changed based on the mismatch information is determined by the student. It consists of decoded audio data as student data stored in the data storage unit 164.

  In tap extracting section 165 or 166, a prediction tap or a class tap having the same tap structure as that in tap extracting section 151 or 152 (FIG. 13) described in FIG. 17 is configured. Therefore, when the tap extraction unit 151 or 152 uses the L code, the G code, the I code, or the A code obtained by the VSELP decoding device 185 to configure a prediction tap or a class tap, the tap extraction unit Also in 165 or 166, a prediction tap or a class tap having the same tap structure as that in the tap extraction unit 151 or 152 is configured by using the L code, G code, I code, or A code obtained by the VSELP decoding device 197. Is done.

  Further, in each of the tap extraction units 165 and 166, when the mismatch information includes not only the difference value ΔT but also the L code used to obtain the difference value ΔT and the pitch period T2 of the decoded speech data. In the same manner as in the tap extraction unit 151 or 152 (FIG. 13) described in FIG. 17, the change of the tap structure of the prediction tap or class tap is not only the difference value ΔT but also the L code or decoded voice data. This is also performed based on the pitch period T2.

  After that, the class classification unit 167 performs class classification similar to that in the class classification unit 153 (FIG. 13) described in FIG. 17 on the attention teacher data based on the class tap and mismatch information regarding the attention teacher data. The class code corresponding to the class obtained as a result is output to the adding unit 168.

  The adding unit 168 reads the attention teacher data from the teacher data storage unit 162, and calculates the components of the matrix A and the vector v in Expression (8) using the attention teacher data and the prediction tap from the tap extraction unit 165. . Further, the adding unit 168 calculates the matrix obtained from the attention teacher data and the prediction tap for the components corresponding to the class code from the class classification unit 167 among the components of the matrix A and the vector v already obtained. Add the components of A and vector v.

  When the above processing is performed on all the teacher data stored in the teacher data storage unit 162 as attention teacher data, the adding unit 168 adds the matrix A and the vector v for each class obtained by the above processing. The normal equation of the equation (8) composed of components is supplied to the tap coefficient calculation unit 169, and the tap coefficient calculation unit 169 solves the normal equation for each class, thereby obtaining the tap coefficient for each class. Find and output.

  Next, FIG. 19 shows a second detailed configuration example of the decoding device in FIG. 12 when the encoded data is audio data encoded by the CELP method. In the figure, portions corresponding to those in FIG. 17 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  That is, the decoding apparatus of FIG. 19 is basically the same as the decoding apparatus of FIG. 17 except that the post-processing unit 133 is configured by the speech synthesis filter 201 configured similarly to the speech synthesis filter 69 of FIG. It is constituted similarly.

  However, in FIG. 9, the VSELP decoding device 185 of the preprocessing unit 131 does not use the decoded speech data output from the speech synthesis filter 69 but the linear prediction coefficient output from the filter coefficient decoder 65 and the remaining output from the computing unit 68 in FIG. The difference signal is output to the class classification adaptive processing unit 132 as preprocessed data.

  In the class classification adaptive processing unit 132, class classification adaptive processing is performed on the residual signal (decoded residual signal) and the linear prediction coefficient (decoded linear prediction coefficient) output from the VSELP decoding device 185 of the preprocessing unit 131. Thus, in the speech synthesis filter 201, a high-quality sound data (predicted value thereof) from which a residual signal and a linear prediction coefficient (hereinafter appropriately referred to as a high-quality sound residual signal and a high-quality sound linear prediction coefficient, respectively) are obtained. Is required as adaptive processing data.

  That is, in the class classification adaptive processing unit 132 (FIG. 13), the decoding residual signal output from the VSELP decoding device 185 of the preprocessing unit 131 is supplied to the tap extraction units 151 and 152.

  The tap extraction unit 151 extracts, as prediction data, some samples of the decoded residual signal that are used to predict the attention data, using the samples of the high-quality sound residual signal that are not yet the attention data as the attention data. The tap extraction unit 152 also extracts some samples of the decoded residual signal used for classifying the data of interest as class taps.

  As described with reference to FIG. 17, the tap extraction units 151 and 152 are supplied with mismatch information about the data of interest, and the tap extraction unit 151 or 152 is based on the mismatch information. Each of the prediction taps or class taps having the tap structure as described in FIG. 17 is configured.

  The prediction tap obtained by the tap extraction unit 151 is supplied to the prediction unit 154, and the class tap obtained by the tap extraction unit 152 is supplied to the class classification unit 153.

  In addition to the class tap, mismatch information about the attention data is also supplied to the class classification unit 153. The class classification unit 153 receives the attention data based on the class tap and the mismatch information in the same manner as described in FIG. The class is classified and the class code for the data of interest is supplied to the coefficient memory 141. In the coefficient memory 141, the tap coefficient corresponding to the class code for the data of interest is read and supplied to the prediction unit 154.

  The prediction unit 154 performs the linear prediction calculation shown in Expression (1) using the prediction tap output from the tap extraction unit 151 and the tap coefficient acquired from the coefficient memory 141. As a result, the prediction unit 154 obtains attention data (predicted value thereof), that is, a high sound quality residual signal, and supplies it to the post-processing unit 133.

  In the embodiment of FIG. 19, two classes of class classification adaptive processing unit 132 and coefficient memory 141 are provided. In the class classification adaptive processing unit 132 and coefficient memory 141 of one system, the decoding residual signal is Is processed as follows. Then, the class classification adaptation processing unit 132 and the coefficient memory 141 of the other system perform the same processing as in the case of the decoded residual signal for the decoded linear prediction coefficient output from the VSELP decoding device 185 of the preprocessing unit 131. As a result, a high sound quality linear prediction coefficient is obtained and supplied to the post-processing unit 133.

  In the post-processing unit 133, the speech synthesis filter 201 filters the high-quality decoded residual signal from the class classification adaptive processing unit 132 using the high-quality linear prediction coefficient from the class classification adaptive processing unit 132 as a filter coefficient. Thus, high-quality sound data is obtained and output.

  Next, FIG. 20 and FIG. 21 illustrate a detailed configuration example of the learning device in FIG. 15 when learning the tap coefficients to be stored in the coefficient memory 141 of the decoding device in FIG. In the figure, portions corresponding to those in FIG. 18 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  FIG. 20 shows a configuration example of a learning device that learns tap coefficients for converting a decoded residual signal into a high sound quality residual signal, and FIG. 21 shows a tap for converting a decoded linear prediction coefficient into a high sound quality linear prediction coefficient. The example of a structure of the learning apparatus which learns a coefficient is shown.

  In the embodiment of FIG. 20, the inverse post-processing unit 161A includes an LPC analysis unit 211 and a prediction filter 212, and the VSELP decoding device 197 configuring the pre-processing unit 163B includes a decoded residual signal (FIG. 9) is supplied to the adaptive learning unit 160 as student data.

  The LPC analysis unit 211 reads the learning speech data from the learning data storage unit 11 and performs the LPC analysis on the learning speech data in the same manner as in the LPC analysis unit 44 in FIG. Is obtained and supplied to the prediction filter 212.

  The prediction filter 212 reads the learning data that the LPC analysis unit 211 has performed the LPC analysis from the learning data storage unit 11, and uses the learning data and the linear prediction coefficient supplied from the LPC analysis unit 211, For example, a residual signal is obtained by performing an operation according to Equation (9), and supplied to the adaptive learning unit 160 as teacher data.

Here, the Z transform of the audio data (audio signal) s _n and the residual signal e _n in the equation (9), when expressed respectively S and E, the formula (9) can be expressed as: .

E = (1 + α ₁ z ⁻¹ + α ₂ z ⁻² +... + Α _P z ^−P ) S
(14)

From equation (14), the residual signal e can be obtained by the product-sum operation of the speech data s and the linear prediction coefficient α _P, and therefore the prediction filter 212 for obtaining the residual signal e is FIR (Finite Impulse Response). ) Type digital filter.

  In the adaptive learning unit 160 (FIG. 15), the teacher data storage unit 162 stores a residual signal (corresponding to the above-mentioned high-quality residual signal) as teacher data supplied from the prediction filter 212, and also a student. In the data storage unit 164, a decoding residual signal as student data supplied from the VSELP decoding device 197 is stored.

  Then, as in the case described with reference to FIG. 18, the adaptive learning unit 160 uses the teacher data and the student data, and performs the linear prediction calculation of Expression (1) from the prediction tap and the tap coefficient extracted from the student data. Learning is performed to obtain tap coefficients that statistically minimize the prediction error of the predicted value of the teacher data obtained by the above, and thereby, tap coefficients for each class for converting the decoded residual signal into a high-quality residual signal are obtained. It is done.

  Next, in the embodiment of FIG. 21, the inverse post-processing unit 161A is configured by an LPC analysis unit 221, and the VSELP decoding device 197 that configures the pre-processing unit 163B is configured to decode decoded linear prediction coefficients (FIG. 9). The linear prediction coefficient output from the filter coefficient decoder 65 is supplied to the adaptive learning unit 160 as student data.

  The LPC analysis unit 221 reads the learning speech data from the learning data storage unit 11, and performs the LPC analysis on the learning speech data in the same manner as in the LPC analysis unit 44 of FIG. Is supplied to the adaptive learning unit 160 as teacher data.

  In the adaptive learning unit 160 (FIG. 15), in the teacher data storage unit 162, linear prediction coefficients (corresponding to the above-described high sound quality linear prediction coefficients) as teacher data supplied from the LPC analysis unit 221 are stored. The student data storage unit 164 stores decoded linear prediction coefficients as student data supplied from the VSELP decoding device 197.

  Then, as in the case described with reference to FIG. 18, the adaptive learning unit 160 uses the teacher data and the student data, and performs the linear prediction calculation of Expression (1) from the prediction tap and the tap coefficient extracted from the student data. Learning to find the tap coefficient that statistically minimizes the prediction error of the prediction value of the teacher data obtained by the above is performed, and the tap coefficient for each class for converting the decoded linear prediction coefficient into the high-quality linear prediction coefficient is obtained. It is done.

  Next, FIG. 22 shows a first detailed configuration example of the decoding device of FIG. 12 when the encoded data is obtained by encoding image data by the MPEG2 system.

  In the embodiment of FIG. 17, the encoding characteristic information extraction unit 121 includes an inverse VLC unit 231. The inverse VLC unit 231 is configured, for example, in the same manner as the inverse VLC unit 241 (FIG. 23) that configures an MPEG decoder 232 described later. To supply.

  The actual characteristic extraction unit 122 includes an MPEG decoder 232 and a correlation calculation unit 233. The MPEG decoder 232 decodes the encoded data by the MPEG method, and supplies the decoded image data obtained as a result to the correlation calculation unit 233.

  Here, FIG. 23 shows a configuration example of the MPEG decoder 232.

  The encoded data is supplied to the inverse VLC unit 241. The inverse VLC unit 241 includes a VLC code (quantized two-dimensional DCT coefficient) VLC code (quantized DCT coefficient variable-length encoded) included in the encoded data, a quantization step, a motion vector, Separate picture type, temporal reference, and other information.

  Then, the inverse VLC unit 241 performs inverse VLC processing on the VLC code of the quantized DCT coefficient to decode the quantized DCT coefficient, and supplies the quantized DCT coefficient to the inverse quantization unit 242. Further, the inverse VLC unit 241 supplies the quantization step to the inverse quantization unit 242, the motion vector to the motion compensation unit 246, the picture type to the memory 245, and the temporal reference to the picture selection unit 247.

  The inverse quantization unit 242 inversely quantizes the quantized DCT coefficient supplied from the inverse VLC unit 241 in the quantization step similarly supplied from the inverse VLC unit 242, and converts the obtained two-dimensional DCT coefficient into an inverse DCT. This is supplied to the conversion unit 242. The inverse DCT transform unit 243 performs a two-dimensional inverse DCT transform on the two-dimensional DCT coefficient supplied from the inverse quantization unit 242, and supplies the result to the calculation unit 244.

  The calculation unit 244 is supplied with the output of the motion compensation unit 246 in addition to the output of the inverse DCT conversion unit 243. The calculation unit 244 performs motion compensation on the output of the inverse DCT conversion unit 243. The output of the unit 246 is added as necessary to obtain and output decoded image data.

  That is, in MPEG encoding, three picture types, I, P, and B, are defined, and each picture is two-dimensionally DCT-converted in units of 8 × 8 pixels in width × length. The I picture block is intra-coded, the P picture block is intra-coded or forward-predicted, and the B-picture block is intra-coded, forward-predicted and backward-predicted. Or bi-directional predictive coding.

  Here, in forward predictive coding, an image of a frame (or field) temporally preceding the frame (or field) of the block to be coded is used as a reference image, and the reference image is obtained by motion compensation. The difference between the prediction image of the block to be encoded and the block to be encoded is obtained, and the difference value (hereinafter referred to as a residual image as appropriate) is subjected to two-dimensional DCT transform.

  Further, in backward predictive coding, a predicted image of a block to be encoded, which is obtained by performing motion compensation on the reference image using a frame image temporally following the frame of the block to be encoded as a reference image. And the difference from the block to be encoded are obtained, and the difference value (residual image) is subjected to two-dimensional DCT transform.

  Furthermore, in bi-directional predictive coding, two frames (or fields) of a frame temporally preceding and following a frame of a block to be encoded are used as reference images, and the reference image is subjected to motion compensation. The obtained difference between the prediction image of the encoding target block and the encoding target block is obtained, and the difference value (residual image) is subjected to two-dimensional DCT transform.

  Therefore, when the block is non-intra coded (forward prediction coding, backward prediction coding, or bidirectional prediction coding), the output of the inverse DCT transform unit 243 is a residual image (original The difference between the image and the predicted image is decoded. The arithmetic unit 244 decodes the residual image (hereinafter referred to as a decoded residual image as appropriate) and the motion compensation unit 246. Are added to the prediction image supplied from the non-intra-coded block, and the decoded image data obtained as a result is output.

  On the other hand, when the block output from the inverse DCT transform unit 243 is an intra-coded block, the output from the inverse DCT transform unit 243 is obtained by decoding the original image, and the computation unit 244. Outputs the output of the inverse DCT conversion unit 243 as decoded image data as it is.

  The decoded image data output from the calculation unit 244 is supplied to the memory 245 and the picture selection unit 247.

  When the decoded image data supplied from the calculation unit 244 is I-picture or P-picture image data, the memory 245 temporarily stores the decoded image data as a reference image of encoded data to be decoded thereafter. Here, in MPEG2, since the B picture is not a reference image, when the decoded image supplied from the calculation unit 244 is a B picture image, the memory 245 does not store the decoded B picture image. Note that the memory 245 determines whether the decoded image supplied from the calculation unit 244 is a picture of I, P, or B by referring to the picture type supplied from the inverse VLC unit 241. .

  The picture selection unit 247 selects and outputs the decoded image output from the calculation unit 244 or the frame (or field) of the decoded image stored in the memory 245 in the display order. That is, in the MPEG2 system, since the display order of the frame (or field) of the image does not match the decoding order (encoding order), the picture selection unit 247 allows the frame (or field) of the decoded image obtained in the decoding order. Are output in the order of display. Note that the picture selection unit 247 determines the display order by referring to the temporal reference supplied from the reverse VLC unit 241.

  On the other hand, the motion compensation unit 246 receives the motion vector output from the inverse VLC unit 241 and reads out a frame (or field) serving as a reference image from the memory 245 and outputs the reference image from the inverse VLC unit 241. The motion compensation according to the motion vector is performed, and the predicted image obtained as a result is supplied to the calculation unit 244. As described above, the calculation unit 244 adds the prediction image from the motion compensation unit 246 and the residual image output from the inverse DCT conversion unit 243, thereby decoding the non-intra coded block.

  Returning to FIG. 22, the correlation calculation unit 233 calculates the correlation between lines for each block of the decoded image data output from the MPEG decoder 232.

  That is, the correlation calculation unit 233 calculates a correlation between lines constituting a frame in a block (hereinafter referred to as frame line correlation as appropriate) and a correlation between lines constituting a field (hereinafter referred to as field line correlation as appropriate). To do.

  Specifically, as shown in FIG. 24, the correlation calculation unit 233 calculates the correlation P (i, i + 1) between the adjacent i-th line (i-th line from the top) and i + 1-th line in the block. For example, it calculates | requires according to following Formula.

P (i, i + 1) = 1 / (Σ (x (i, j) −x (i + 1, j))
... (15)

  However, x (i, j) represents the pixel value of the j-th (j-th column) pixel from the left of the i-th line. Σ represents a summation with j changed from 1 to 8.

  Then, for example, the correlation calculation unit 233 calculates the average value of the correlations P (i, i + 1) ((P (1,2) + P (2,3) + P (3,4)) + P (4,5) + P (5 6) + P (6,7) + P (7,8)) / 7) is obtained, and this average value is output as a frame line correlation.

  Further, as shown in FIG. 24, the correlation calculation unit 233 calculates the correlation P (i, i + 2) between the i-th line and the i + 2 line adjacent to every other line in the block, for example, in the equation (15). Therefore seek.

  Then, for example, the correlation calculation unit 233 calculates the average value of the correlation P (i, i + 2) ((P (1,3) + P (2,4) + P (3,5) + P (4,6) + P (5 7) + P (6,8)) / 6) is obtained, and this average value is output as a field line correlation.

  The frame line correlation and field line correlation output from the correlation calculation unit 233 are supplied to the determination unit 123 as actual characteristics.

  Here, when the motion of an image in a certain block is relatively small, generally, the frame line correlation becomes large and the field line correlation becomes small. When the motion of the image in the block is relatively large, generally, the field line correlation becomes large and the frame line correlation becomes small. Therefore, it can be said that the frame line correlation and the field line correlation represent the actual characteristics (actual characteristics) of the image.

  The determination unit 123 includes a block characteristic determination unit 234 and a comparison unit 235. Based on the frame line correlation and the field line correlation of a block (hereinafter referred to as “target block” as appropriate) including a pixel corresponding to the target data in the class classification adaptation processing unit 132, the block characteristic determination unit 234 determines whether the target block is in frame DCT mode or It is determined which of the field DCT modes has a characteristic to be encoded, and the determination result (hereinafter referred to as an actual characteristic type as appropriate) is supplied to the comparison unit 235.

  That is, for example, when the frame line correlation of the block of interest is smaller than (or less than) the field line correlation, the block characteristic determination unit 234 has the characteristic that the block of interest should be encoded in the field DCT mode. The actual characteristic type is supplied to the comparison unit 235. In addition, the block characteristic determination unit 234 determines the actual characteristic type that the target block has a characteristic to be encoded in the frame DCT mode when the frame line correlation of the target block is not smaller than the field line correlation. 235.

  The comparison unit 235 includes the DCT type of the target block (DCT type of the macroblock including the target block) supplied from the inverse VLC unit 231 of the coding characteristic information extraction unit 121 and the target block supplied from the block characteristic determination unit 234. And a set of flags representing the DCT type of the block of interest and a flag representing the actual characteristic type are supplied to the class classification adaptation processing unit 132 as mismatch information.

  On the other hand, the preprocessing unit 131 includes an MPEG decoder 236. Similar to the MPEG decoder 232, the MPEG decoder 236 decodes the encoded data by the MPEG method, and outputs the decoded image data to the class classification adaptive processing unit 132 as preprocessed data.

  In the class classification adaptive processing unit 132, class classification adaptive processing is performed on the decoded image data output from the MPEG decoder 236 of the preprocessing unit 131, and the adaptive processing data obtained as a result is output to the post-processing unit 133. The The post-processing unit 133 outputs the adaptive processing data from the class classification adaptive processing unit 132 as it is as high-quality image data (high-quality image data).

  Therefore, in the embodiment of FIG. 22, the class classification adaptive processing unit 132 performs the class classification adaptive processing, and thus the encoded data output from the MPEG decoder 236 of the preprocessing unit 131 is decoded by the MPEG method. The decoded image data is converted into high-quality image data and output.

  That is, in the class classification adaptive processing unit 132 (FIG. 13), the decoded image data output from the MPEG decoder 236 of the preprocessing unit 131 is supplied to the tap extraction units 151 and 152.

  The tap extraction unit 151 predicts some of the decoded image data (pixels) used to predict the attention data (the pixel value thereof) using the pixels of the high-quality image data that are not yet the attention data as the attention data. Extract as a tap. The tap extraction unit 152 also extracts some of the decoded image data used for classifying the attention data as class taps.

  Here, as described above, mismatch information is also supplied from the determination unit 123 to the tap extraction units 151 and 152, and the tap extraction units 151 and 152 are configured based on the mismatch information and predicted taps and classes. Each tap structure is changed.

  That is, as described above, the set of the DCT type and the actual characteristic type for the block of interest is supplied as mismatch information for the data of interest from the determination unit 123 (the comparison unit 235) to the class classification adaptation processing unit 132. The

  When the tap extraction unit 151 receives a set of the DCT type and the actual characteristic type for the block of interest as mismatch information, the tap extraction unit 151 uses, for example, a tap structure as shown in FIG. 25 from the decoded image data supplied from the MPEG decoder 236. Extract a tap with a tap structure according to the setting table.

  That is, when both the DCT type and the actual characteristic type as mismatch information are in the field DCT mode, the tap extraction unit 151 configures a prediction tap having a pattern A tap structure including only field taps to be described later. Further, when the DCT type and the actual characteristic type as mismatch information are the field DCT mode and the frame DCT mode, respectively, the tap extraction unit 151 includes taps of the pattern B in which the number of field taps is larger than the number of frame taps described later. Configure the structure prediction tap. Further, when the DCT type and the actual characteristic type as mismatch information are the frame DCT mode and the field DCT mode, respectively, the tap extraction unit 151 has the tap structure of the pattern C in which the number of frame taps is larger than the number of field taps. Configure the prediction tap. Further, when both the DCT type and the actual characteristic type as mismatch information are in the frame DCT mode, the tap extraction unit 151 configures a prediction tap having a tap structure of a pattern D including only frame taps.

  Here, FIG. 26 shows a tap structure of patterns A to D. In FIG. 26, the circles represent the pixels of the decoded image data. In addition, a circle mark with a hatched line represents a pixel that is a field tap, and a mark ● represents a pixel that is a frame tap.

  FIG. 26A shows the tap structure of pattern A. FIG. The tap structure of the pattern A is adjacent to the pixel of the decoded image data corresponding to the target data (hereinafter referred to as the target pixel as appropriate), two pixels adjacent to the left and right of the target pixel, and one pixel above the target pixel. Pixel, 2 pixels adjacent to the left and right of the pixel, 3 pixels above the pixel of interest adjacent to each other, 2 pixels adjacent to the left and right of the pixel, and 1 pixel below the pixel of interest The pixel is composed of a total of 25 pixels: two pixels adjacent to the left and right sides of the pixel, pixels adjacent to each other in the downward direction of the pixel of interest, and two pixels adjacent to the left and right sides of the pixel.

  Here, the field tap means a pixel in which two adjacent pixels above and below are not taps (here, prediction taps or class taps). In the tap structure of the pattern A in FIG. 26A, all the taps are field taps because the adjacent pixels above and below the taps are not taps.

  FIG. 26B shows a tap structure of the pattern B. The tap structure of the pattern B includes the target pixel, two pixels adjacent to the left and right of the target pixel, two pixels adjacent to the left and right of the adjacent pixel in the upper direction of the target pixel, and three upwards of the target pixel. 1 pixel adjacent to the left and right of each adjacent pixel in the pixel, 2 pixels adjacent to the left and right of each adjacent pixel in the downward direction of the target pixel, and 3 pixels adjacent in the downward direction of the target pixel 1 pixel adjacent to each of the left and right, 4 pixels adjacent above the target pixel, and 4 pixels adjacent below the target pixel, for a total of 25 pixels.

  Here, the frame tap means a pixel in which at least one of the adjacent pixels above or below is a tap. In the tap structure of the pattern B in FIG. 26B, a total of nine pixels of the target pixel and four pixels adjacent to the top and bottom of the target pixel are frame taps, and the remaining 16 pixels are field taps. .

  FIG. 26C shows the tap structure of the pattern C. The tap structure of the pattern C includes a pixel of interest, two pixels adjacent to the left and right of the pixel of interest, two pixels adjacent to the left and right of each adjacent pixel in the upper direction of the pixel of interest, and 1 downward of the pixel of interest. 2 pixels adjacent to the left and right of the adjacent pixel, 4 pixels adjacent to the upper and lower sides of the target pixel, 1 pixel adjacent to the left and right of the adjacent pixel above the target pixel, and adjacent to the lower side of the target pixel It consists of a total of 25 pixels, one pixel adjacent to the left and right of each pixel.

  In the tap structure of pattern C, the pixel of interest, the four pixels adjacent to the top and bottom of the pixel of interest, the pixel adjacent to the left of the pixel of interest, the two pixels adjacent to the top and bottom of the pixel, the pixel adjacent to the right of the pixel of interest, A total of 19 pixels, which are two adjacent pixels above and below the pixel, are frame taps, and the remaining 6 pixels are field taps.

  FIG. 26D shows the tap structure of the pattern D. The tap structure of the pattern D is composed of a total of 25 pixels, with 5 × 5 pixels in the horizontal and vertical directions that are adjacent to each other with the pixel of interest at the center.

  In the tap structure of the pattern D, all the taps are frame taps because at least one pixel above or below is a tap.

  Based on the mismatch information, the tap extraction unit 151 (FIG. 13) configures a prediction tap having a tap structure of any one of the patterns A to D illustrated in FIG.

  Similarly to the tap extraction unit 151, the tap extraction unit 152 configures a class tap having a tap structure based on mismatch information.

  Here, based on the mismatch information, only the pixel position of the decoded image data extracted as the prediction tap is changed, and the number of pixels constituting the prediction tap remains 25 pixels. The extraction unit 151 can change the number of pixels of the decoded image data constituting the prediction tap based on the mismatch information.

  Also, in the MPEG decoder 236 of the pre-processing unit 131, the encoded data is motion vector other than the quantized DCT coefficient included in the encoded data, DCT type, quantization step and other information for controlling decoding (hereinafter, referred to as “decoded data”). The tap extraction unit 151 can also include such decoding control information in the prediction tap. Further, in this case, it is also possible to change the decoding control information used as the prediction tap based on the mismatch information. Furthermore, the tarp extraction unit 151 can also include the quantized DCT coefficient included in the encoded data and the two-dimensional DCT coefficient obtained by inverse quantization of the quantized DCT coefficient in the prediction tap. .

  The tap extraction unit 152 can also configure class taps in the same manner as in the tap extraction unit 151.

  The prediction tap obtained by the tap extraction unit 151 is supplied to the prediction unit 154, and the class tap obtained by the tap extraction unit 152 is supplied to the class classification unit 153.

  In addition to the class tap, mismatch information about the attention data is also supplied to the class classification unit 153. As described above, the class classification unit 153 classifies the attention data based on the class tap and the mismatch information.

  That is, the class classification unit 153 obtains a class code (class tap code) by performing the above-described ADRC processing based on, for example, the class tap for the data of interest.

  Furthermore, the class classification unit 153 obtains a 2-bit class code (mismatch code) based on, for example, a set of a DCT type and an actual characteristic type as mismatch information for the data of interest.

  That is, the class classification unit 153 sets the 2-bit mismatch code to, for example, “00” when the DCT type and the actual characteristic type are both in the field DCT mode. Further, the class classification unit 153 sets the 2-bit mismatch code to “01”, for example, when the DCT type and the actual characteristic type are the field DCT mode and the frame DCT mode, respectively. Furthermore, the class classification unit 153 sets the 2-bit mismatch code to, for example, “10” when the DCT type and the actual characteristic type are the frame DCT mode and the field DCT mode, respectively. Further, the class classification unit 153 sets the 2-bit mismatch code to, for example, “11” when the DCT type and the actual characteristic type are both in the frame DCT mode.

  Thereafter, the class classification unit 153, for example, adds the mismatch code obtained for the attention data as the upper bits of the class tap code obtained for the attention data, and generates a code composed of this class tap code and the mismatch code. , And output as the final class code for the data of interest.

  In addition, the class classification unit 153 can perform class classification based on, for example, decoding control information.

  The class code output from the class classification unit 153 is supplied to the coefficient memory 141. In the coefficient memory 141, the tap coefficient corresponding to the class code is read and supplied to the prediction unit 154.

  The prediction unit 154 performs the linear prediction calculation shown in Expression (1) using the prediction tap output from the tap extraction unit 151 and the tap coefficient acquired from the coefficient memory 141. Thereby, the prediction unit 154 obtains attention data (predicted value thereof), that is, high-quality image data, and supplies it to the post-processing unit 133.

  As described above, the post-processing unit 133 outputs the output of the class classification adaptive processing unit 132 (prediction unit 154 thereof), that is, the high-quality image data as it is.

  In the embodiment of FIG. 22, the block characteristic determination unit 234 outputs an actual characteristic type representing only one of the frame DCT mode and the field DCT mode. However, as the actual characteristic type, In addition, for example, the frame line correlation and the field line correlation of the block of interest can be used as they are. In this case, the comparison unit 235 evaluates how appropriate the DCT type of the target block output from the inverse VLC unit 231 is for the target block based on the frame line correlation and the field line correlation of the target block. It is possible to obtain a value and output the evaluation value as mismatch information. Here, if the frame line correlation and the field line correlation of the block of interest are expressed as F1 and F2, respectively, when the DCT type of the block of interest is the frame DCT mode, the evaluation value is, for example, F1 / (F1 + F2) When the DCT type of the target block is the field DCT mode, for example, F2 / (F1 + F2) can be adopted as the evaluation value.

  Furthermore, the tap extraction units 151 and 152 may compare the evaluation value as mismatch information with one or more threshold values, and change the tap structure of the prediction tap or the class tap based on the comparison result. Is possible.

  The class classification unit 153 can quantize the evaluation value as mismatch information and use the quantized value as a mismatch code.

  Further, in the embodiment of FIG. 22, the actual characteristic type of the target block is determined from the frame line correlation and the field line correlation of the target block. It is possible to make a determination based on blocks around the block. That is, the final actual characteristic type of the target block is, for example, an actual characteristic type determined from the frame line correlation and field line correlation of the target block, and the frame line correlation and field line of one or more blocks adjacent to the target block. It can be determined by majority voting with the actual characteristic type of each block, determined from the correlation.

  Next, in the embodiment of FIG. 22, the actual characteristic extraction unit 122 decodes the encoded data by the MPEG method, obtains the frame line correlation and the field line correlation from the decoded image data obtained as a result, and determines the determination unit 123. In FIG. 5, the actual characteristic type is obtained from the frame line correlation and the field line correlation. However, in the determination unit 123, for example, the actual characteristic type can be obtained from the two-dimensional DCT coefficient included in the encoded data. Is possible.

  That is, in the real characteristic extraction unit 122, for example, as shown in FIG. 27, among the two-dimensional DCT coefficients of the block obtained from the encoded data, those based on horizontal horizontal stripes, that is, at the left end of the block, Seven 7-dimensional DCT coefficients excluding DC (Direct Current) coefficients (hereinafter referred to as horizontal stripe 2-dimensional DCT coefficients as appropriate) (parts indicated by hatching in FIG. 27) are obtained as actual characteristics, and the determination unit 123 The actual characteristic type can be determined based on the horizontal stripe two-dimensional DCT coefficient as the actual characteristic.

  Further, in the actual characteristic extraction unit 122, for example, of the two-dimensional DCT coefficients of the block obtained from the encoded data, an arbitrary horizontal stripe two-dimensional DCT coefficient and an arbitrary AC (Alternating Current) excluding the horizontal stripe two-dimensional DCT coefficient The difference from the coefficient (hereinafter referred to as the coefficient difference as appropriate), or the power of any horizontal stripe two-dimensional DCT coefficient (for example, the square of the two-dimensional DCT coefficient) and any AC coefficient excluding the horizontal stripe two-dimensional DCT coefficient The determination unit 123 can obtain the actual characteristic type based on the coefficient difference or the power difference.

  Therefore, FIG. 28 illustrates a configuration example of the actual characteristic extraction unit 122 that obtains the coefficient difference or the power difference as the complete characteristic type.

  The encoded data is supplied to the inverse VLC unit 251 and the MPEG decoder 254.

  The inverse VLC unit 251 separates the VLC code of the quantized DCT coefficient, the quantization step, the motion vector, and other information included in the encoded data. Then, the inverse VLC unit 251 performs inverse VLC processing on the VLC code of the quantized DCT coefficient to decode the quantized DCT coefficient, and supplies the quantized DCT coefficient to the inverse quantization unit 252. Further, the inverse VLC unit 251 supplies the quantization step to the inverse quantization unit 252 and the motion vector to the motion compensation unit 256, respectively.

  The inverse quantization unit 252 inversely quantizes the quantized DCT coefficient supplied from the inverse VLC unit 251 in the quantization step similarly supplied from the inverse VLC unit 251, and 2 of the resulting 8 × 8 pixel block. The dimensional DCT coefficient is supplied to the calculation unit 253.

  On the other hand, in the MPEG decoder 254, the encoded data is encoded by the MPEG method, and decoded image data is output. Of the decoded images output from the MPEG decoder 254, I and P pictures that can be used as reference images are supplied to the memory 255 and stored therein.

  Then, the motion compensation unit 256 reads the reference image stored in the memory 255, performs motion compensation on the reference image according to the motion vector supplied from the inverse VLC unit 251, and thereby the inverse quantization unit. A predicted image of the block supplied from 252 to the calculation unit 253 is generated and supplied to the DCT conversion unit 257. The DCT conversion unit 257 performs two-dimensional DCT conversion on the prediction image supplied from the motion compensation unit 256 and supplies the two-dimensional DCT coefficient obtained as a result to the calculation unit 253.

  The calculation unit 253 adds each two-dimensional DCT coefficient of the block supplied from the inverse quantization unit 252 and the corresponding two-dimensional DCT coefficient supplied from the DCT conversion unit 257 as necessary. A two-dimensional DCT coefficient obtained by two-dimensional DCT transform of the pixel value of the block is obtained.

  That is, when the block supplied from the inverse quantization unit 252 is intra-coded, the two-dimensional DCT coefficient of the block supplied from the inverse quantization unit 252 converts the original pixel value into a two-dimensional DCT transform. Therefore, the calculation unit 253 outputs the two-dimensional DCT coefficient of the block supplied from the inverse quantization unit 252 as it is.

  When the block supplied from the inverse quantization unit 252 is non-intra coded, the two-dimensional DCT coefficient of the block supplied from the inverse quantization unit 252 is the original pixel value, the predicted image, Since the difference value (residual image) is two-dimensional DCT transformed, the calculation unit 253 is supplied from the DCT coefficient of the block supplied from the inverse quantization unit 252 and the DCT conversion unit 257. Then, by adding the corresponding two-dimensional DCT coefficient obtained by two-dimensional DCT conversion of the predicted image, a two-dimensional DCT coefficient obtained by two-dimensional DCT conversion of the original pixel value is obtained and output.

  The two-dimensional DCT coefficient of the block output from the calculation unit 253 is supplied to the DCT coefficient difference calculation unit 258.

  In the DCT coefficient difference calculation unit 258, the coefficient difference and the power difference as described above are obtained using the two-dimensional DCT coefficient of the block, and supplied to the determination unit 123 as actual characteristics.

  In this case, the determination unit 123 refers to, for example, the coefficient difference or power difference of the block of interest, and the magnitude relationship between the horizontal stripe two-dimensional DCT coefficient used to obtain the coefficient difference or power difference and the AC coefficient. Is determined. Further, in the determination unit 123, for example, when the horizontal stripe two-dimensional DCT coefficient used for obtaining the coefficient difference or power difference of the block of interest is smaller than (or less than) the AC coefficient, the actual characteristic type is the field DCT mode. If the horizontal stripe two-dimensional DCT coefficient is not smaller than the AC coefficient, the actual characteristic type is recognized as the frame DCT mode. When the horizontal stripe two-dimensional DCT coefficient used for obtaining the coefficient difference or power difference of the block of interest is smaller than the AC coefficient, the image of the block of interest should be encoded in the field DCT mode. In addition to this, it also represents an image with many horizontal stripes.

  Here, the determination unit 123 can output the coefficient difference or power difference and further include the two-dimensional DCT coefficient used to obtain the coefficient difference or power difference in mismatch information. is there. In this case, for example, in the class classification adaptation processing unit 132 (FIG. 13), the tap extraction units 151 and 152 perform prediction based on the coefficient difference or power difference included in the mismatch information and the two-dimensional DCT coefficient. The tap structure of taps and class taps is changed, and the class classification unit 153 also performs class classification based on the coefficient difference or power difference included in the mismatch information and the two-dimensional DCT coefficient. Is possible.

  Next, the frame line correlation and the field line correlation of the block of interest can be obtained from, for example, the one-dimensional DCT coefficient of the block of interest.

  Here, the one-dimensional DCT coefficient will be described with reference to FIGS. 29 and 30. FIG.

  In an image encoding method using DCT conversion such as MPEG or JPEG (Joint Photographic Experts Group), image data is converted into two-dimensional DCT conversion (two-dimensional DCT conversion) / inverse DCT conversion (2 Dimensional inverse DCT transform) is performed.

  A pixel value in an 8 × 8 pixel block as shown in FIG. 29A is represented by a matrix X of 8 rows × 8 columns, and a two-dimensional DCT in an 8 × 8 block as shown in FIG. If the coefficient is represented by a matrix F of 8 rows × 8 columns, the two-dimensional DCT transformation / two-dimensional inverse DCT transformation can be represented by the following equation.

CXC ^T = F
... (16)
C ^T FC = X
... (17)

Here, the superscript T represents transposition. C is a DCT transformation matrix of 8 rows × 8 columns, and a component c _ij of the (i + 1) th row and the (j + 1) th column is expressed by the following equation.

c _ij = A _i × cos ((2j + 1) × i × π / 16)
... (18)

However, in Expression (18), when i = 0, A _i = 1 / (2√2), and when i ≠ 0, A _i = 1/2. I and j are integer values ranging from 0 to 7.

  Equation (16) represents a two-dimensional DCT transformation that transforms the pixel value X into a two-dimensional DCT coefficient F, and Equation (17) represents a two-dimensional inverse DCT transformation that transforms the two-dimensional DCT coefficient F into a pixel value X. Represents.

Therefore, according to equation (17), two-dimensional DCT coefficients F, as well as applying a matrix C ^T from the left, by multiplying the matrix C from the right side, is converted into the pixel value X, two-dimensional DCT coefficients against F, or just make a matrix C ^T from the left side, or, by either simply multiplying the matrix C from the right side, it is possible to obtain the one-dimensional DCT coefficients.

That is, the two-dimensional DCT coefficients F, when applying only the left from the matrix C ^T, as shown in FIG. 29 (C), the vertical direction in the two-dimensional DCT coefficients F is converted to the spatial domain, horizontal The vertical one-dimensional inverse DCT transformation that remains in the frequency domain is performed, and as a result, a horizontal one-dimensional DCT coefficient vXhF that represents the spatial frequency component in the horizontal direction can be obtained.

  Also, when only the matrix C is applied to the two-dimensional DCT coefficient F from the right side, the horizontal direction in the two-dimensional DCT coefficient F is converted into a spatial domain, and the vertical direction is the frequency as shown in FIG. The horizontal one-dimensional inverse DCT transformation that remains in the region is performed, and as a result, the vertical one-dimensional DCT coefficient hXvF representing the vertical spatial frequency component can be obtained.

  In addition, when a two-dimensional DCT coefficient F of horizontal × vertical 8 × 8 is subjected to a vertical one-dimensional inverse DCT transform, eight sets (eight lines) of 8 × 1 horizontal one-dimensional DCT coefficients are obtained. (FIG. 29C). Further, when the two-dimensional DCT coefficient F is subjected to the horizontal one-dimensional inverse DCT transform, 8 sets (for eight columns) of 1 × 8 vertical one-dimensional DCT coefficients are obtained (FIG. 29D). .

  Then, for an 8 × 1 horizontal one-dimensional DCT coefficient in a certain row, the DCT coefficient at the left end represents the direct current component (DC component) of the pixel values of the eight pixels in the row (average value of the pixel values of eight pixels). And the other seven DCT coefficients represent the horizontal AC component of the row. For a 1 × 8 vertical one-dimensional DCT coefficient in a certain column, the DCT coefficient in the uppermost row represents the DC component of the pixel value of the eight pixels in the column, and the other seven DCT coefficients are in the column. Represents the AC component in the vertical direction.

Here, according to the equation (16), the horizontal one-dimensional DCT coefficients, the pixel value X corresponding to the two-dimensional DCT coefficients F, by performing a horizontal one-dimensional DCT transform to apply a matrix C ^T from the right Can also be sought. The vertical one-dimensional DCT coefficient can also be obtained by performing a vertical one-dimensional DCT transform that applies the matrix C from the left side to the pixel value X corresponding to the two-dimensional DCT coefficient F.

  FIG. 30 shows an actual image and a two-dimensional DCT coefficient, a horizontal one-dimensional DCT coefficient, and a vertical one-dimensional DCT coefficient for the image.

  FIG. 30 shows an 8 × 8 block image, a two-dimensional DCT coefficient, a horizontal one-dimensional DCT coefficient, and a vertical one-dimensional DCT coefficient for the image. 30A shows an actual image, FIG. 30B shows a two-dimensional DCT coefficient, FIG. 30C shows a horizontal one-dimensional DCT coefficient, and FIG. 30D shows a vertical 1 The dimensional DCT coefficients are shown respectively.

  Here, the image in FIG. 30A has an 8-bit pixel value, and the DCT coefficient obtained from such a pixel value can take a negative value. However, in the embodiment shown in FIGS. 30B to 30D, 128 (= 2 7) is added to the obtained DCT coefficient, and the addition value less than 0 is clipped to 0 In addition, the DCT coefficients in the range of 0 to 255 are shown by clipping to 255 when the added value is 256 or more.

  Since the information of the entire block of 8 × 8 pixels is reflected in the two-dimensional DCT coefficient, it is difficult to grasp local information such as information on specific pixels in the block from the two-dimensional DCT coefficient. It is. On the other hand, since the horizontal one-dimensional DCT coefficient or the vertical one-dimensional DCT coefficient reflects information of only one row or one column of the block, it is compared with the two-dimensional DCT coefficient. The local information of can be easily grasped.

  That is, the feature of a certain row of a block can be grasped from the 8 × 1 horizontal one-dimensional DCT coefficient of the row, and the feature of a certain column can be grasped from the 1 × 8 vertical one-dimensional DCT coefficient of the column. be able to. Furthermore, the characteristics of a pixel with a block can be grasped from the 8 × 1 horizontal one-dimensional DCT coefficient of the row where the pixel is located and the 1 × 8 vertical one-dimensional DCT coefficient of the column where the pixel is located. it can.

  In addition, for the state of the boundary between adjacent blocks on the left and right, a vertical one-dimensional DCT coefficient representing a spatial frequency component in the vertical direction of the boundary portion of the block is used rather than a two-dimensional DCT coefficient reflecting the information of the entire block. It is possible to grasp more accurately. Further, for the state of the boundary between adjacent blocks, a horizontal one-dimensional DCT coefficient representing a horizontal spatial frequency component of the boundary part of the block is used rather than a two-dimensional DCT coefficient reflecting the information of the entire block. It is possible to grasp more accurately.

  In the actual characteristic extraction unit 122, the calculation of the frame line correlation and the field line correlation of the block of interest using the one-dimensional DCT coefficient as described above is performed as follows, for example.

  That is, as shown in FIG. 31, the actual characteristic extraction unit 122 calculates the correlation Q (i, i + 1) between the i-th line (i-th line from the top) and the i + 1-th line in the block, for example, Obtained according to the following equation.

Q (i, i + 1) = 1 / (Σ (d _H (i, j) −d _H (i + 1, j))
... (19)

Here, d _H (i, j) represents the j-th (j-th column) horizontal one-dimensional DCT coefficient from the left of the i-th line. Σ represents a summation with j changed from 1 to 8.

  Then, the actual characteristic extraction unit 122, for example, calculates the average value of the correlation Q (i, i + 1) ((Q (1,2) + Q (2,3) + Q (3,4) + Q (4,5) + Q (5 , 6) + Q (6,7) + Q (7,8)) / 7), and outputs this average value as a frame line correlation.

  Further, as shown in FIG. 31, the actual characteristic extraction unit 122 calculates the correlation Q (i, i + 2) between the i-th line and the i + 2 line adjacent to every other line in the block, for example, Equation (19) According to

  Then, the actual characteristic extraction unit 122, for example, calculates the average value of the correlation Q (i, i + 2) ((Q (1,3) + Q (2,4) + Q (3,5) + Q (4,6) + Q (5 7) + Q (6,8)) / 6), and the average value is output as a field line correlation.

  Next, FIG. 32 illustrates a configuration example of the actual characteristic extraction unit 122 that obtains the frame line correlation and the field line correlation using the one-dimensional DCT coefficient as described above. In the figure, portions corresponding to those in FIG. 28 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the actual characteristic extraction unit 122 in FIG. 32 is the same as in FIG. 28 except that a vertical one-dimensional inverse DCT conversion unit 261 and a correlation calculation unit 262 are provided instead of the DCT coefficient difference calculation unit 258. It is configured.

  The vertical one-dimensional inverse DCT transform unit 261 is supplied with the two-dimensional DCT coefficient of the block output from the calculation unit 253. The vertical one-dimensional inverse DCT transform unit 261 obtains a horizontal one-dimensional DCT coefficient block by performing a vertical one-dimensional inverse DCT transform on the two-dimensional DCT coefficient block from the computation unit 253, and supplies the block to the correlation computation unit 262. . The correlation calculation unit 262 calculates and outputs the frame line correlation and the field line correlation from the horizontal one-dimensional DCT coefficient from the vertical one-dimensional inverse DCT conversion unit 261 as described in FIG.

  In the embodiment of FIGS. 28 and 32, a predicted image is generated from the decoded image data output from the MPEG decoder 254, the predicted image is converted into a two-dimensional DCT coefficient, and the encoded data is calculated in the arithmetic unit 253. The two-dimensional DCT coefficient of the residual image obtained from the above and the two-dimensional DCT coefficient of the predicted image are added to obtain the two-dimensional DCT coefficient of the original image. For example, the decoded image data output from the MPEG decoder 254 is subjected to two-dimensional DCT transform, and the resulting two-dimensional DCT coefficient is used as the two-dimensional DCT coefficient of the original image. Processing can be performed in the vertical one-dimensional DCT inverse DCT transform unit 261 in FIG.

  Also, in the actual characteristic extraction unit 122 of FIGS. 28 and 32, the DCT coefficient difference calculation unit 258 and the vertical one-dimensional inverse DCT conversion unit 261 use the code instead of the two-dimensional DCT coefficient of the original image output from the calculation unit 253. It is possible to perform processing using the two-dimensional DCT coefficient of the residual image obtained from the quantized data (output of the inverse quantization unit 252).

  Next, FIG. 33 shows a detailed configuration example of the learning device in FIG. 15 when learning the tap coefficients to be stored in the coefficient memory 141 in FIG.

  In the embodiment of FIG. 33, high-quality image data (learning image data) is stored in the learning data storage unit 11 as learning data.

  In the embodiment of FIG. 33, the encoding unit 12 includes an MPEG encoder 271. The MPEG encoder 271 reads the learning image data from the learning data storage unit 11, encodes it using the MPEG2 method, and Output the resulting encoded data.

  That is, FIG. 34 shows a configuration example of the MPEG encoder 271 of FIG.

  The learning image data is supplied to the motion vector detection unit 321 and the calculation unit 323. The motion vector detection unit 321 detects the motion vector of the learning image data, for example, by performing block matching on the learning image data, and supplies the motion vector to the motion compensation unit 322.

  Further, the calculation unit 323 subtracts the prediction image supplied from the motion compensation unit 322 from the learning image data (original image) as necessary, and the resulting residual image is sent to the DCT conversion unit 324. Supply. The DCT conversion unit 324 performs two-dimensional DCT conversion on the residual image from the calculation unit 323 and supplies the two-dimensional DCT coefficient obtained as a result to the quantization unit 325. The quantization unit 325 obtains a quantized DCT coefficient by quantizing the two-dimensional DCT coefficient supplied from the DCT transform unit 324 in a predetermined quantization step, and sends the quantized DCT coefficient to the VLC unit 326 and the inverse quantization unit 327. Supply.

  The VLC unit 326 variable-length-encodes the quantized DCT coefficient supplied from the quantization unit 325 into a VLC code, and further performs necessary decoding control information (for example, the motion vector detected by the motion vector detection unit 321, the quantum The quantizing step used in the encoding unit 325 is multiplexed to obtain encoded data and output it.

  On the other hand, in the inverse quantization unit 327, the quantized DCT coefficient output from the quantization unit 325 is inversely quantized, and a two-dimensional DCT coefficient is obtained and supplied to the inverse DCT transform unit 328. The inverse DCT transform unit 328 decodes the two-dimensional DCT coefficient from the inverse quantization unit 327 into a residual image by performing two-dimensional inverse DCT transform, and supplies the residual image to the arithmetic unit 329.

  The arithmetic unit 329 is supplied with the residual image from the inverse DCT transform unit 328, and also receives the same predicted image used in the arithmetic unit 323 to obtain the residual image from the motion compensation unit 322. The arithmetic unit 329 decodes the original image (local decoding) by adding the residual image and the predicted image. This decoded image is supplied to the memory 330 and stored as a reference image.

  Then, the motion compensation unit 322 reads the reference image stored in the memory 330 and performs motion compensation according to the motion vector supplied from the motion vector detection unit 321 to generate a predicted image. This predicted image is supplied from the motion compensation unit 322 to the calculation units 323 and 329.

  As described above, the computation unit 323 obtains a residual image using the prediction image from the motion compensation unit 322, and the computation unit 329 uses the prediction image from the motion compensation unit 322 to obtain the original image. The image is decoded.

  Returning to FIG. 33, the encoded data output from the MPEG decoder 271 is supplied to the encoded characteristic information extraction unit 171 and the actual characteristic extraction unit 172.

  The encoding characteristic information extraction unit 171 is configured by an inverse VLC unit 272, and the actual characteristic extraction unit 172 is configured by an MPEG decoder 273 and a correlation calculation unit 274. The inverse VLC unit 272, the MPEG decoder 273, or the correlation calculation unit 274 performs the same processing as the inverse VLC unit 231, the MPEG decoder 232, or the correlation calculation unit 233 in FIG. The correlation calculation unit 274 supplies the DCT type of the block of interest to the determination unit 173, respectively, the frame line correlation and the field line correlation of the block of interest.

  The determination unit 173 includes a block characteristic determination unit 275 and a comparison unit 276. In the block characteristic determination unit 275 and the comparison unit 276, the DCT type of the target block supplied thereto, the frame line correlation, and the field line correlation are calculated. By using the same processing as in the case of the block characteristic determination unit 234 and the comparison unit 235 in FIG. 22, mismatch information is generated for the teacher data that is the attention teacher data in the adaptive learning unit 160. . This mismatch information is supplied from the comparison unit 276 to the adaptive learning unit 160.

  In the case where mismatch information is obtained in the encoding characteristic information extraction unit 121, the actual characteristic extraction unit 122, and the determination unit 123 in the decoding device of FIG. 22 as described with reference to FIGS. Similarly, the encoding characteristic information extraction unit 171, the actual characteristic extraction unit 172, and the determination unit 173 in the 33 learning devices also obtain mismatch information.

  The reverse post-processing unit 161A reads the learning image data from the learning data storage unit 11, and outputs the learning image data as it is to the adaptive learning unit 160 as teacher data. In adaptive learning section 160 (FIG. 15), teacher data from post-processing section 161A is stored in teacher data storage section 162.

  The encoding unit 163A includes an MPEG encoder 277. The MPEG encoder 277 reads the learning image data from the learning data storage unit 11 and encodes it using the MPEG2 system, as with the MPEG encoder 271, and obtains the result. The encoded data is output to the preprocessing unit 163B.

  The preprocessing unit 163B includes an MPEG decoder 278 configured in the same manner as the MPEG decoder 232 in FIG. 23. The MPEG decoder 278 decodes the encoded data from the MPEG encoder 277 using the MPEG2 method, and is obtained as a result. The decoded image data is output to the adaptive learning unit 160 as student data. In the adaptive learning unit 160 (FIG. 15), the student data from the MPEG decoder 278 is stored in the student data storage unit 164.

  Then, the adaptive learning unit 160 uses the teacher data and the student data, and predicts the predicted value of the teacher data obtained by performing the linear prediction calculation of Expression (1) from the prediction tap and tap coefficient extracted from the student data. Learning is performed to obtain a tap coefficient that statistically minimizes the error.

  That is, in the adaptive learning unit 160 (FIG. 15), the tap extraction unit 165 sets the teacher data stored in the teacher data storage unit 162 that has not yet been set as the attention teacher data as the attention teacher data, and the attention teacher data. Is configured from the student data stored in the student data storage unit 164 and supplied to the adding unit 168. Further, the tap extraction unit 166 configures class taps from the student data stored in the student data storage unit 164 for the teacher data of interest and supplies the class taps to the class classification unit 167.

  Here, mismatch information is supplied to the tap extraction units 165 and 166, and the tap extraction unit 165 or 166 uses the class classification adaptation described with reference to FIG. 22 for the attention teacher data based on the mismatch information. A prediction tap or a class tap having the same tap structure as the tap extraction unit 151 or 152 (FIG. 13) of the processing unit 132 is configured.

  Therefore, for example, in the tap extraction unit 151 or 152, when the prediction tap or the class tap is configured using the decoding control information as described with reference to FIG. In the extraction unit 165 or 166 (FIG. 15), the prediction tap or the class tap is configured using the decoding control information.

  Thereafter, in the class classification unit 167 (FIG. 15), based on the class tap and mismatch information for the attention teacher data, the same class classification as that in the class classification unit 153 described in FIG. The class code corresponding to the class obtained as a result is output to the adding unit 168.

  The adding unit 168 reads the attention teacher data from the teacher data storage unit 162, and calculates the components of the matrix A and the vector v in Expression (8) using the attention teacher data and the prediction tap from the tap extraction unit 165. . Further, the adding unit 168 calculates the matrix obtained from the attention teacher data and the prediction tap for the components corresponding to the class code from the class classification unit 167 among the components of the matrix A and the vector v already obtained. Add the components of A and vector v.

  When the above processing is performed on all the teacher data stored in the teacher data storage unit 162 as attention teacher data, the adding unit 168 adds the matrix A and the vector v for each class obtained by the above processing. The normal equation of the equation (8) composed of components is supplied to the tap coefficient calculation unit 169, and the tap coefficient calculation unit 169 solves the normal equation for each class, thereby obtaining the tap coefficient for each class. Find and output.

  In the learning apparatus of FIG. 33, for example, before the learning image data is MPEG-encoded by the MPEG encoder 277 of the encoding unit 163A, the number of pixels of the learning image data is thinned to 1 / N. As a result, the adaptive learning unit 160 can obtain tap coefficients of the MPEG decoded image data with high image quality and N-times the number of pixels (increasing the resolution).

  Next, FIG. 35 shows a second detailed configuration example of the decoding device of FIG. 12 when the encoded data is obtained by encoding image data by the MPEG2 system. In the figure, portions corresponding to those in FIG. 22 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  In the embodiment of FIG. 35, the preprocessing unit 131 includes an inverse VLC unit 281, an inverse quantization unit 282, a calculation unit 283, an MPEG decoder 284, a memory 285, a motion compensation unit 286, and a DCT conversion unit 287. Yes.

  The inverse VLC unit 281, inverse quantization unit 282, calculation unit 283, MPEG decoder 284, memory 285, motion compensation unit 286, or DCT conversion unit 287 is the same as the inverse VLC unit 251, inverse quantization unit 252, calculation unit in FIG. 28. 253, MPEG decoder 254, memory 255, motion compensation unit 256, or DCT conversion unit 257, respectively, and the encoded data supplied to the preprocessing unit 131 is described with reference to FIG. 28. As a result, the preprocessing unit 131 obtains the two-dimensional DCT coefficient of the original image and supplies it to the class classification adaptation processing unit 132 as preprocessing data.

  The class classification adaptive processing unit 132 performs class classification adaptive processing on the two-dimensional DCT coefficients output from the preprocessing unit 131, thereby obtaining high-quality image data (predicted values thereof) as adaptive processing data. It is done.

  That is, in the class classification adaptive processing unit 132 (FIG. 13), the two-dimensional DCT coefficient output from the preprocessing unit 131 is supplied to the tap extraction units 151 and 152.

  The tap extracting unit 151 extracts, as prediction taps, two-dimensional DCT coefficients as preprocessing data used for predicting the attention data using pixels of high-quality image data that are not yet attention data as attention data. To do. The tap extraction unit 152 also extracts some of the two-dimensional DCT coefficients as preprocessing data used for classifying the data of interest as class taps.

  Note that the tap extraction unit 151 or 152 changes the tap structure of the prediction tap or the class tap based on the mismatch information about the data of interest.

  That is, for example, the tap extraction unit 151 extracts, in addition to all the two-dimensional DCT coefficients of the block of attention data (target block), two-dimensional DCT coefficients in blocks adjacent to the upper, lower, left, and right of the target block according to the mismatch information. Then, a prediction tap is configured. The tap extraction unit 151 also forms a class tap in the same manner as the tap extraction unit 151.

  Then, the prediction tap obtained by the tap extraction unit 151 is supplied to the prediction unit 154, and the class tap obtained by the tap extraction unit 152 is supplied to the class classification unit 153.

  The class classification unit 153 classifies the attention data based on the class tap and the mismatch information regarding the attention data, and supplies the class code for the attention data to the coefficient memory 141 in the same manner as described with reference to FIG. Is done. In the coefficient memory 141, the tap coefficient corresponding to the class code for the data of interest is read and supplied to the prediction unit 154.

  The prediction unit 154 performs the linear prediction calculation shown in Expression (1) using the prediction tap output from the tap extraction unit 151 and the tap coefficient acquired from the coefficient memory 141. Thereby, the prediction unit 154 obtains attention data (predicted value thereof), that is, high-quality image data, and supplies it to the post-processing unit 133.

  The post-processing unit 133 outputs the high-quality image data from the class classification adaptation processing unit 132 as it is.

  Therefore, in the embodiment of FIG. 35, the class classification adaptation processing unit 132 converts the two-dimensional DCT coefficients into high-quality image data.

  Next, FIG. 36 illustrates a detailed configuration example of the learning device in FIG. 15 when learning tap coefficients to be stored in the coefficient memory 141 of the decoding device in FIG. In the figure, portions corresponding to those in FIG. 33 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  In the embodiment of FIG. 36, the preprocessing unit 163B includes an inverse VLC unit 291, an inverse quantization unit 292, a calculation unit 293, an MPEG decoder 294, a memory 295, a motion compensation unit 296, and a DCT conversion unit 297. The inverse VLC unit 291 to DCT conversion unit 297 are configured in the same manner as the inverse VLC unit 281 to DCT conversion unit 287 of FIG.

  Accordingly, in the preprocessing unit 163B, the same processing as in the preprocessing unit 131 in FIG. 35 is performed on the encoded data output from the MPEG encoder 277 of the encoding unit 163A, and the two-dimensional DCT obtained thereby is obtained. The coefficient is supplied to the adaptive learning unit 160 as student data.

  In the adaptive learning unit 160 (FIG. 15), the student data storage unit 164 stores the two-dimensional DCT coefficients supplied from the preprocessing unit 163B as student data, and in the same manner as described with reference to FIG. A tap coefficient that statistically minimizes the prediction error of the predicted value of the teacher data obtained by performing the linear prediction calculation of Equation (1) from the prediction tap and tap coefficient extracted from the student data using the student data. Learning to be obtained is performed, and thereby, a tap coefficient for each class for converting a two-dimensional DCT coefficient as student data into high-quality image data is obtained.

  However, in the embodiment of FIG. 36, in the adaptive learning unit 160 (FIG. 15), the tap extraction unit 151 or 152 in the class classification adaptive processing unit 132 (FIG. 13) of FIG. A prediction tap or a class tap having the same tap structure as that configured by is configured based on the mismatch information. Furthermore, the class classification unit 167 in the adaptive learning unit 160 (FIG. 15) in FIG. 36 performs the same class classification as the class classification unit 153 in the class classification adaptation processing unit 132 (FIG. 13) in FIG.

  Next, FIG. 37 shows a third detailed configuration example of the decoding device of FIG. 12 when the encoded data is obtained by encoding image data by the MPEG2 system. In the figure, portions corresponding to those in FIG. 35 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  The decoding apparatus in FIG. 37 is configured in the same manner as in FIG. 35 except that the post-processing unit 133 is configured by an inverse DCT transform unit 301.

  In the embodiment of FIG. 37, the class classification adaptive processing unit 132 performs the class classification adaptive processing on the two-dimensional DCT coefficients output from the preprocessing unit 131, thereby performing the two-dimensional inverse DCT transform. In this case, a two-dimensional DCT coefficient (hereinafter, appropriately referred to as a high-quality two-dimensional DCT coefficient) (predicted value) from which high-quality image data can be obtained is obtained as adaptive processing data.

  That is, in the class classification adaptive processing unit 132 (FIG. 13), the two-dimensional DCT coefficient as the preprocessing data output from the preprocessing unit 131 is supplied to the tap extraction units 151 and 152.

  The tap extraction unit 151 extracts high-quality two-dimensional DCT coefficients that are not yet used as attention data as attention data, and extracts some of the two-dimensional DCT coefficients as preprocessing data used to predict the attention data as prediction taps. To do. That is, the tap extraction unit 151 configures a prediction tap having the same tap structure as that in FIG. 35 for the data of interest based on the mismatch information. Based on the mismatch information, the tap extraction unit 152 also configures class taps having the same tap structure as in FIG.

  Then, the prediction tap obtained by the tap extraction unit 151 is supplied to the prediction unit 154, and the class tap obtained by the tap extraction unit 152 is supplied to the class classification unit 153.

  The class classification unit 153 classifies the attention data based on the class tap and the mismatch information regarding the attention data, and supplies the class code for the attention data to the coefficient memory 141 in the same manner as in FIG. . In the coefficient memory 141, the tap coefficient corresponding to the class code for the data of interest is read and supplied to the prediction unit 154.

  The prediction unit 154 performs the linear prediction calculation shown in Expression (1) using the prediction tap output from the tap extraction unit 151 and the tap coefficient acquired from the coefficient memory 141. As a result, the prediction unit 154 calculates attention data (predicted value thereof), that is, a high-quality two-dimensional DCT coefficient, and supplies it to the post-processing unit 133.

  In the post-processing unit 133, in the inverse DCT conversion unit 301, the high-quality two-dimensional DCT coefficient output from the class classification adaptation processing unit 132 is subjected to two-dimensional inverse DCT conversion, whereby high-quality image data is obtained and output. The

  Next, FIG. 38 shows a detailed configuration example of the learning device in FIG. 15 when learning tap coefficients to be stored in the coefficient memory 141 of the decoding device in FIG. In the figure, portions corresponding to those in FIG. 36 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  The learning device in FIG. 38 is configured in the same manner as in FIG. 36 except that the reverse post-processing unit 161A is configured by a DCT conversion unit 311.

  Therefore, in the reverse post-processing unit 161A, the DCT conversion unit 311 performs high-dimensional image data as learning image data read from the learning data storage unit 11 on a two-dimensional DCT basis in units of blocks, and obtains the result. The high-quality two-dimensional DCT coefficient to be obtained is supplied to the adaptive learning unit 160 as teacher data.

  In the adaptive learning unit 160 (FIG. 15), the high-quality two-dimensional DCT coefficient supplied from the inverse post-processing unit 161A is stored as teacher data in the teacher data storage unit 162, and the teacher data and student data storage unit 164 are stored. The prediction value of the teacher data obtained by performing the linear prediction calculation of Expression (1) from the prediction tap extracted from the student data and the tap coefficient using the two-dimensional DCT coefficient as the student data stored in Learning for obtaining a tap coefficient that statistically minimizes the error is performed, whereby a tap coefficient for each class that converts the two-dimensional DCT coefficient as student data into a high-quality two-dimensional DCT coefficient is obtained.

  That is, in this case, the two-dimensional DCT coefficient that is student data is obtained from the encoded data in the pre-processing unit 163B and includes a quantization error. An image obtained by the dimensional inverse DCT transform has a low image quality having so-called block distortion or the like.

  Therefore, in the adaptive learning unit 160, as described above, teacher data (high-quality two-dimensional DCT coefficient obtained by two-dimensional DCT conversion of learning image data) obtained by performing the linear prediction calculation of Expression (1). By performing learning for obtaining a tap coefficient that statistically minimizes the prediction error of the predicted value, a tap coefficient for each class that converts the two-dimensional DCT coefficient being student data into a high-quality two-dimensional DCT coefficient is obtained. Desired.

  38, in the adaptive learning unit 160 (FIG. 15), the tap extraction unit 151 or 152 in the class classification adaptation processing unit 132 (FIG. 13) in FIG. A prediction tap or a class tap having the same tap structure as that configured by is configured based on the mismatch information. Further, the class classification unit 167 in the adaptive learning unit 160 (FIG. 15) in FIG. 38 performs the same class classification as the class classification unit 153 in the class classification adaptation processing unit 132 (FIG. 13) in FIG.

  As described above, the correctness of the characteristic data included in the encoded data is determined, and based on the mismatch information indicating the determination result, decoding of the encoded data, learning of tap coefficients used for the decoding, and the like are performed. Therefore, for example, even if the characteristic data included in the encoded data does not correctly represent the characteristics of the original data, the encoded data can be decoded into high-quality data.

  Next, the series of processes described above can be performed by hardware or software. When a series of processing is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

  Therefore, FIG. 39 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.

  The program can be recorded in advance on a hard disk 405 or a ROM 403 as a recording medium built in the computer.

  Alternatively, the program is stored temporarily on a removable recording medium 411 such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored permanently (recorded). Such a removable recording medium 411 can be provided as so-called package software.

  The program is installed in the computer from the removable recording medium 411 as described above, or transferred from the download site to the computer wirelessly via a digital satellite broadcasting artificial satellite, LAN (Local Area Network), The program can be transferred to a computer via a network such as the Internet, and the computer can receive the program transferred in this way by the communication unit 408 and install it in the built-in hard disk 405.

  The computer includes a CPU (Central Processing Unit) 402. An input / output interface 410 is connected to the CPU 402 via the bus 401, and the CPU 402 operates the input unit 407 including a keyboard, a mouse, a microphone, and the like by the user via the input / output interface 410. When a command is input by the equalization, a program stored in a ROM (Read Only Memory) 403 is executed accordingly. Alternatively, the CPU 402 may be a program stored in the hard disk 405, a program transferred from a satellite or a network, received by the communication unit 408 and installed in the hard disk 405, or a removable recording medium 411 mounted on the drive 409. The program read and installed in the hard disk 405 is loaded into a RAM (Random Access Memory) 404 and executed. Thereby, the CPU 402 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 402 outputs the processing result from the output unit 406 configured by an LCD (Liquid Crystal Display), a speaker, or the like via the input / output interface 410, or from the communication unit 408 as necessary. Transmission and further recording on the hard disk 405 are performed.

  Here, in this specification, the processing steps for describing a program for causing a computer to perform various types of processing do not necessarily have to be processed in time series according to the order described in the flowchart, but in parallel or individually. This includes processing to be executed (for example, parallel processing or processing by an object).

  Further, the program may be processed by a single computer, or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

  In this embodiment, the case where the image data is encoded by the MPEG system and the case where the audio data is encoded by the CELP system have been described. However, the present invention is limited to these encoding systems. Instead, for example, the present invention can also be applied to encoded data obtained by encoding audio data using the MP3 (MPEG-1 Audio Layer 3) method.

  In addition, according to the decoding device and decoding method, the first program, and the first recording medium to which the present invention is applied, the correctness of the characteristic data is determined, and mismatch information representing the determination result is output. Then, the encoded data is decoded based on the mismatch information. Therefore, the encoded data can be decoded into high quality data.

  Furthermore, according to the learning device and the learning method, the second program, and the second recording medium to which the present invention is applied, the teacher data serving as a teacher for learning the tap coefficient and the student serving as the student from the learning data. Data is generated and output. Further, learning data is encoded, and encoded learning data including characteristic data for the data is output. Then, the correctness of the characteristic data included in the learning encoded data is determined, and the tap coefficient is learned using the teacher data and the student data based on the mismatch information representing the determination result. Therefore, the encoded data can be decoded into high quality data by the tap coefficient.

It is a block diagram which shows the structural example of one Embodiment of the decoding apparatus to which this invention is applied. It is a flowchart explaining the process of a decoding apparatus. It is a block diagram which shows the structural example of other one Embodiment of the decoding apparatus to which this invention is applied. It is a block diagram which shows the structural example of one Embodiment of the learning apparatus to which this invention is applied. It is a flowchart explaining the process of a learning apparatus. It is a block diagram which shows the structural example of the audio | voice data processing apparatus which converts audio | voice data into audio | voice data of high sound quality by a class classification adaptive process. It is a block diagram which shows the structural example of the learning apparatus which learns the tap coefficient memorize | stored in the coefficient memory. It is a block diagram which shows the structural example of the VSELP encoding apparatus which encodes audio | voice data by a VSELP system. It is a block diagram which shows the structural example of the VSELP decoding apparatus which decodes coding data by a VSELP system. It is a block diagram which shows the structural example of the VSELP decoding apparatus to which the class classification adaptive process is applied. 3 is a block diagram illustrating a configuration example of a learning device that learns tap coefficients stored in a coefficient memory 84. FIG. It is a block diagram which shows the more detailed structural example of the decoding apparatus to which this invention is applied. 3 is a block diagram illustrating a configuration example of a class classification adaptation processing unit 132. FIG. It is a flowchart explaining the process of a decoding apparatus. It is a block diagram which shows the more detailed structural example of the learning apparatus to which this invention is applied. It is a flowchart explaining the process of a learning apparatus. It is a block diagram which shows the 1st structural example of the decoding apparatus which decodes the coding data encoded by the VSELP system. It is a block diagram which shows the 1st structural example of the learning apparatus which learns the tap coefficient used in decoding the coding data encoded by the VSELP system. It is a block diagram which shows the 2nd structural example of the decoding apparatus which decodes the coding data encoded by the VSELP system. It is a block diagram which shows the 2nd structural example of the learning apparatus which learns the tap coefficient used in decoding the encoding data encoded by the VSELP system. It is a block diagram which shows the 3rd structural example of the learning apparatus which learns the tap coefficient used in decoding the encoding data encoded by the VSELP system. It is a block diagram which shows the 1st structural example of the decoding apparatus which decodes the encoding data encoded by the MPEG system. 3 is a block diagram illustrating a configuration example of an MPEG decoder 232. FIG. It is a figure for demonstrating the method of calculating | requiring frame line correlation and field line correlation from image data. It is a figure which shows a tap structure setting table. It is a figure which shows the tap structure of the pattern A thru | or D. FIG. It is a figure which shows the DCT coefficient based on a horizontal stripe. 3 is a block diagram illustrating a configuration example of an actual characteristic extraction unit 122. FIG. It is a figure for demonstrating a one-dimensional DCT coefficient. It is the photograph of the intermediate gradation displayed on the display explaining a one-dimensional DCT coefficient. It is a figure for demonstrating the method of calculating | requiring a frame line correlation and a field line correlation from a one-dimensional DCT coefficient. 12 is a block diagram illustrating another configuration example of the actual characteristic extraction unit 122. FIG. It is a block diagram which shows the 1st structural example of the learning apparatus which learns the tap coefficient used in decoding the encoding data encoded by the MPEG system. 2 is a block diagram illustrating a configuration example of an MPEG encoder 271. FIG. It is a block diagram which shows the 2nd structural example of the decoding apparatus which decodes the encoding data encoded by the MPEG system. It is a block diagram which shows the 2nd structural example of the learning apparatus which learns the tap coefficient used in decoding the encoding data encoded by the MPEG system. It is a block diagram which shows the 3rd structural example of the decoding apparatus which decodes the encoding data encoded by the MPEG system. It is a block diagram which shows the 3rd structural example of the learning apparatus which learns the tap coefficient used in decoding the encoding data encoded by the MPEG system. It is a block diagram which shows the structural example of one Embodiment of the computer to which this invention is applied.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Mismatch detection part, 2 Decoding processing part, 3 Parameter storage part, 11 Learning data storage part, 12 Coding part, 13 Mismatch detection part, 14 Learning processing part, 21 Pitch detection part, 22, 23 Tap extraction part, 24 Class classification unit, 25 coefficient memory, 26 prediction unit, 31 time decimation filter, 32 pitch detection unit, 33, 34 tap extraction unit, 35 class classification unit, 36 addition unit, 37 tap coefficient calculation unit, 41 microphone, 42 A / D conversion unit, 43 arithmetic unit, 44 LPC analysis unit, 45 vector quantization unit, 46 speech synthesis filter, 47 square error calculation unit, 48 square error minimum determination unit, 49 adaptive codebook storage unit, 50 gain decoder, 51 Excitation code book storage unit, 52 to 54 computing unit, 55 Code determination unit, 56 channel encoder, 61 channel decoder, 62 adaptive codebook storage unit, 63 gain decoder, 64 excitation codebook storage unit, 65 filter coefficient decoder, 66 to 68 arithmetic unit, 69 speech synthesis filter, 81, 82 tap extraction unit, 83 class classification unit, 84 coefficient memory, 85 prediction unit, 92 A / D conversion unit, 93 calculator, 94 LPC analysis unit, 95 vector quantization unit, 96 speech synthesis filter, 97 square error calculation unit , 98 square error minimum determination unit, 99 adaptive codebook storage unit, 100 gain decoder, 101 excitation codebook storage unit, 102 to 104 arithmetic unit, 105 code determination unit, 111, 112 tap extraction unit, 113 class classification unit, 114 addition part, 115 tap coefficient calculation unit, 121 encoding characteristic information extraction unit, 122 actual characteristic extraction unit, 123 determination unit, 131 preprocessing unit, 132 class classification adaptive processing unit, 133 post processing unit, 141 coefficient memory, 151, 152 tap extraction 153 class classification unit, 154 prediction unit, 160 adaptive learning unit, 161 teacher data generation unit, 161A inverse post-processing unit, 162 teacher data storage unit, 163 student data generation unit, 163A encoding unit, 163B pre-processing unit, 164 Student data storage unit, 165, 166 tap extraction unit, 167 class classification unit, 168 addition unit, 169 tap coefficient calculation unit, 171 encoding characteristic information extraction unit, 172 actual characteristic extraction unit, 173 determination unit, 181 channel decoder , 182 VSELP decoding Device, 183 pitch detector, 184 difference calculator, 185 VSELP decoder, 191 VSELP encoder, 192 channel decoder, 193 VSELP decoder, 194 pitch detector, 195 difference calculator, 196 VSELP encoder, 197 VSELP Decoding apparatus, 201 speech synthesis filter, 211 LPC analysis unit, 212 prediction filter, 221 LPC analysis unit, 231 inverse VLC unit, 232 MPEG decoder, 233 correlation operation unit, 234 block characteristic determination unit, 235 comparison unit, 236 MPEG decoder, 241 inverse VLC unit, 242 inverse quantization unit, 243 inverse DCT transform unit, 244 calculation unit, 245 memory, 246 motion compensation unit, 247 picture selection unit, 251 inverse VLC unit, 252 Inverse quantization unit, 253 calculation unit, 254 MPEG decoder, 255 memory, 256 motion compensation unit, 257 DCT conversion unit, 258 DCT coefficient difference calculation unit, 261 vertical one-dimensional inverse DCT conversion unit, 262 correlation calculation unit, 271 MPEG Encoder, 272 Inverse VLC section, 273 MPEG decoder, 274 Correlation calculation section, 275 Block characteristic determination section, 276 comparison section, 277 MPEG encoder, 278 MPEG decoder, 281 Inverse VLC section, 282 Inverse quantization section, 283 calculation section, 284 MPEG decoder, 285 memory, 286 motion compensation unit, 287 DCT conversion unit, 291 inverse VLC unit, 292 inverse quantization unit, 293 arithmetic unit, 294 MPEG decoder, 295 memory, 296 motion Compensation unit, 297 DCT conversion unit, 301 inverse DCT conversion unit, 311 DCT conversion unit, 321 motion vector detection unit, 322 motion compensation unit, 323 calculation unit, 324 DCT conversion unit, 325 quantization unit, 326 VLC unit, 327 inverse Quantization unit, 328 inverse DCT conversion unit, 329 operation unit, 330 memory, 401 bus, 402 CPU, 403 ROM, 404 RAM, 405 hard disk, 406 output unit, 407 input unit, 408 communication unit, 409 drive, 410 I / O Interface, 411 removable recording media

Claims (4)

Learning tap coefficients used for predictive calculation with encoded data, which is encoded data obtained by encoding data and is performed to decode at least encoded data including characteristic data representing the characteristics of the data Learning device
Teacher data generation means for generating and outputting teacher data serving as a teacher for learning the tap coefficient from learning data;
Student data generation means for generating and outputting student data to be students of learning of the tap coefficient from the learning data;
Encoding means for encoding the learning data and outputting encoded learning data including the characteristic data for the data;
Outputs mismatch information obtained by comparing the characteristic data included in the learning encoded data and the actual characteristic that is an actual characteristic of the learning data corresponding to the learning encoded data. Comparison output means;
Learning means for learning the tap coefficient using the teacher data and student data based on the mismatch information;
The learning means includes
The class tap extraction pattern in which class taps used for classifying the teacher data of interest which is the teacher data of interest into a class of any of a plurality of classes are previously associated with the mismatch information And a class tap extracting means for extracting from the student data,
Classifying means for classifying the teacher data of interest based on the class tap, and outputting a class code of the corresponding class;
A prediction tap extracting means for extracting a prediction tap used for the prediction calculation with the tap coefficient from the student data with the extraction pattern of the prediction tap associated in advance with the mismatch information for the attention teacher data;
Tap coefficient calculation means for obtaining, for each class, the tap coefficient that statistically minimizes the prediction error of the predicted value of the teacher data obtained by performing the prediction calculation using the prediction tap and the tap coefficient. Have
When the learning data is voice data,
The encoding means encodes the learning speech data by a CELP (Code Excited Linear Prediction coding) method, and outputs the learning encoded data,
The comparison output means calculates a difference value between the characteristic data included in the learning encoded data and an actual characteristic that is an actual characteristic of the learning speech data corresponding to the learning encoded data. Output the mismatch information that represents
The teacher data generation means and the student data generation means are
The teacher data generation means outputs the learning voice data as it is as the teacher data, and the student data generation means encodes and decodes the learning voice data by a CELP method, The result is output as the student data,
The teacher data generation means performs linear prediction analysis on the learning speech data, and drives a prediction filter using the resulting linear prediction coefficient as a filter coefficient by the learning speech data. And the residual signal is output as the teacher data, and the student data generation means encodes and decodes the learning speech data by the CELP method, and drives a speech synthesis filter obtained as a result The residual signal to be output as the student data,
Alternatively, the teacher data generation means performs linear prediction analysis on the learning voice data, and outputs a linear prediction coefficient obtained as a result thereof as the teacher data, and the student data generation means outputs the learning voice data. Is encoded and decoded by the CELP method, and a linear prediction coefficient that is a filter coefficient of a speech synthesis filter obtained as a result is output as the student data,
When the learning data is image data,
The encoding means encodes the image data for learning by an MPEG (Moving Picture Experts Group) method, and outputs the encoded data for learning.
The comparison output means represents a combination of the characteristic data included in the learning encoded data and an actual characteristic that is an actual characteristic of the learning image data corresponding to the learning encoded data. Output the mismatch information;
The teacher data generation means and the student data generation means are
The teacher data generation means outputs the learning image data as the teacher data as it is, and the student data generation means encodes and decodes the learning image data by the MPEG method, and the decoding The result is output as the student data,
The teacher data generation means outputs the learning image data as the teacher data as it is, and the student data generation means encodes and decodes the learning image data by the MPEG method, and the result Outputting the obtained two-dimensional DCT coefficient as the student data;
Alternatively, the teacher data generation unit performs two-dimensional DCT conversion on the learning image data, and outputs a two-dimensional DCT coefficient obtained as a result thereof as the teacher data, and the student data generation unit outputs the learning data. A learning device that encodes and decodes image data according to the MPEG method, and outputs the resulting two-dimensional DCT coefficients as the student data. Learning tap coefficients used for predictive calculation with encoded data, which is encoded data obtained by encoding data and is performed to decode at least encoded data including characteristic data representing the characteristics of the data In the learning method of the learning device to
A teacher data generation step for generating and outputting teacher data to be a teacher for learning the tap coefficient from the learning data;
A student data generation step of generating and outputting student data to be students of learning of the tap coefficient from the learning data;
An encoding step of encoding the learning data and outputting encoded learning data including the characteristic data for the data;
Outputs mismatch information obtained by comparing the characteristic data included in the learning encoded data and the actual characteristic that is an actual characteristic of the learning data corresponding to the learning encoded data. A comparison output step;
A learning step of learning the tap coefficient using the teacher data and student data based on the mismatch information;
The learning step includes
The class tap extraction pattern in which class taps used for classifying the teacher data of interest which is the teacher data of interest into a class of any of a plurality of classes are previously associated with the mismatch information And a class tap extraction step for extracting from the student data;
A class classification step of classifying the teacher data of interest based on the class tap and outputting a class code of a corresponding class;
A prediction tap extraction step of extracting a prediction tap used for the prediction calculation with the tap coefficient with respect to the attention teacher data from the student data with an extraction pattern of the prediction tap previously associated with the mismatch information;
A tap coefficient calculation step for determining, for each class, the tap coefficient that statistically minimizes the prediction error of the predicted value of the teacher data obtained by performing the prediction calculation using the prediction tap and the tap coefficient; Have
When the learning data is voice data,
In the encoding step, the speech data for learning is encoded by a CELP (Code Excited Liner Prediction coding) method, and the encoded data for learning is output.
The comparison output step calculates a difference value between the characteristic data included in the learning encoded data and an actual characteristic that is an actual characteristic of the learning speech data corresponding to the learning encoded data. Output the mismatch information that represents
The teacher data generation step and the student data generation step include:
The teacher data generation step outputs the learning voice data as it is as the teacher data, and the student data generation step encodes and decodes the learning voice data by a CELP method, The result is output as the student data,
The teacher data generation step performs linear prediction analysis on the learning speech data, and drives a prediction filter using a linear prediction coefficient obtained as a result as a filter coefficient by the learning speech data. And the residual signal is output as the teacher data, and the student data generation step encodes and decodes the learning speech data by a CELP method, and drives a speech synthesis filter obtained as a result The residual signal to be output as the student data,
Alternatively, the teacher data generation step performs linear prediction analysis on the learning voice data, and outputs a linear prediction coefficient obtained as a result thereof as the teacher data, and the student data generation step includes the learning voice data. Is encoded and decoded by the CELP method, and a linear prediction coefficient that is a filter coefficient of a speech synthesis filter obtained as a result is output as the student data,
When the learning data is image data,
In the encoding step, the image data for learning is encoded by an MPEG (Moving Picture Experts Group) method, and the encoded data for learning is output.
The comparison output step represents a combination of the characteristic data included in the learning encoded data and an actual characteristic that is an actual characteristic of the learning image data corresponding to the learning encoded data. Output the mismatch information;
The teacher data generation step and the student data generation step include:
The teacher data generation step outputs the learning image data as it is as the teacher data, and the student data generation step encodes and decodes the learning image data by the MPEG method, The result is output as the student data,
The teacher data generation step outputs the learning image data as it is as the teacher data, and the student data generation step encodes and decodes the learning image data by the MPEG method, and the result Outputting the obtained two-dimensional DCT coefficient as the student data;
Alternatively, the teacher data generation step performs two-dimensional DCT transformation on the learning image data, and outputs a two-dimensional DCT coefficient obtained as a result thereof as the teacher data, and the student data generation step includes the learning data A learning method in which image data is encoded and decoded by the MPEG method, and a two-dimensional DCT coefficient obtained as a result is output as the student data. Learning tap coefficients used for predictive calculation with encoded data, which is encoded data obtained by encoding data and is performed to decode at least encoded data including characteristic data representing the characteristics of the data In a program for causing a computer to perform learning processing
A teacher data generation step for generating and outputting teacher data to be a teacher for learning the tap coefficient from the learning data;
A student data generation step of generating and outputting student data to be students of learning of the tap coefficient from the learning data;
An encoding step of encoding the learning data and outputting encoded learning data including the characteristic data for the data;
Outputs mismatch information obtained by comparing the characteristic data included in the learning encoded data and the actual characteristic that is an actual characteristic of the learning data corresponding to the learning encoded data. A comparison output step;
A learning step of learning the tap coefficient using the teacher data and student data based on the mismatch information;
The learning step includes
The class tap extraction pattern in which class taps used for classifying the teacher data of interest which is the teacher data of interest into a class of any of a plurality of classes are previously associated with the mismatch information And a class tap extraction step for extracting from the student data;
A class classification step of classifying the teacher data of interest based on the class tap and outputting a class code of a corresponding class;
A prediction tap extraction step of extracting a prediction tap used for the prediction calculation with the tap coefficient with respect to the attention teacher data from the student data with an extraction pattern of the prediction tap previously associated with the mismatch information;
A tap coefficient calculation step for determining, for each class, the tap coefficient that statistically minimizes the prediction error of the predicted value of the teacher data obtained by performing the prediction calculation using the prediction tap and the tap coefficient; Have
When the learning data is voice data,
In the encoding step, the speech data for learning is encoded by a CELP (Code Excited Liner Prediction coding) method, and the encoded data for learning is output.
The comparison output step calculates a difference value between the characteristic data included in the learning encoded data and an actual characteristic that is an actual characteristic of the learning speech data corresponding to the learning encoded data. Output the mismatch information that represents
The teacher data generation step and the student data generation step include:
The teacher data generation step outputs the learning voice data as it is as the teacher data, and the student data generation step encodes and decodes the learning voice data by a CELP method, The result is output as the student data,
The teacher data generation step performs linear prediction analysis on the learning speech data, and drives a prediction filter using a linear prediction coefficient obtained as a result as a filter coefficient by the learning speech data. And the residual signal is output as the teacher data, and the student data generation step encodes and decodes the learning speech data by a CELP method, and drives a speech synthesis filter obtained as a result The residual signal to be output as the student data,
Alternatively, the teacher data generation step performs linear prediction analysis on the learning voice data, and outputs a linear prediction coefficient obtained as a result thereof as the teacher data, and the student data generation step includes the learning voice data. Is encoded and decoded by the CELP method, and a linear prediction coefficient that is a filter coefficient of a speech synthesis filter obtained as a result is output as the student data,
When the learning data is image data,
In the encoding step, the image data for learning is encoded by an MPEG (Moving Picture Experts Group) method, and the encoded data for learning is output.
The comparison output step represents a combination of the characteristic data included in the learning encoded data and an actual characteristic that is an actual characteristic of the learning image data corresponding to the learning encoded data. Output the mismatch information;
The teacher data generation step and the student data generation step include:
The teacher data generation step outputs the learning image data as it is as the teacher data, and the student data generation step encodes and decodes the learning image data by the MPEG method, The result is output as the student data,
The teacher data generation step outputs the learning image data as it is as the teacher data, and the student data generation step encodes and decodes the learning image data by the MPEG method, and the result Outputting the obtained two-dimensional DCT coefficient as the student data;
Alternatively, the teacher data generation step performs two-dimensional DCT transformation on the learning image data, and outputs a two-dimensional DCT coefficient obtained as a result thereof as the teacher data, and the student data generation step includes the learning data A program for causing a computer to perform a learning process in which image data is encoded and decoded by the MPEG method, and a two-dimensional DCT coefficient obtained as a result is output as the student data. Learning tap coefficients used for predictive calculation with encoded data, which is encoded data obtained by encoding data and is performed to decode at least encoded data including characteristic data representing the characteristics of the data In a recording medium on which a program for causing a computer to perform learning processing is recorded,
A teacher data generation step for generating and outputting teacher data to be a teacher for learning the tap coefficient from the learning data;
A student data generation step of generating and outputting student data to be students of learning of the tap coefficient from the learning data;
An encoding step of encoding the learning data and outputting encoded learning data including the characteristic data for the data;
Outputs mismatch information obtained by comparing the characteristic data included in the learning encoded data and the actual characteristic that is an actual characteristic of the learning data corresponding to the learning encoded data. A comparison output step;
A learning step of learning the tap coefficient using the teacher data and student data based on the mismatch information;
The learning step includes
The class tap extraction pattern in which class taps used for classifying the teacher data of interest which is the teacher data of interest into a class of any of a plurality of classes are previously associated with the mismatch information And a class tap extraction step for extracting from the student data;
A class classification step of classifying the teacher data of interest based on the class tap and outputting a class code of a corresponding class;
A prediction tap extraction step of extracting a prediction tap used for the prediction calculation with the tap coefficient with respect to the attention teacher data from the student data with an extraction pattern of the prediction tap previously associated with the mismatch information;
A tap coefficient calculation step for determining, for each class, the tap coefficient that statistically minimizes the prediction error of the predicted value of the teacher data obtained by performing the prediction calculation using the prediction tap and the tap coefficient; Have
When the learning data is voice data,
In the encoding step, the speech data for learning is encoded by a CELP (Code Excited Liner Prediction coding) method, and the encoded data for learning is output.
The comparison output step calculates a difference value between the characteristic data included in the learning encoded data and an actual characteristic that is an actual characteristic of the learning speech data corresponding to the learning encoded data. Output the mismatch information that represents
The teacher data generation step and the student data generation step include:
The teacher data generation step outputs the learning voice data as it is as the teacher data, and the student data generation step encodes and decodes the learning voice data by a CELP method, The result is output as the student data,
The teacher data generation step performs linear prediction analysis on the learning speech data, and drives a prediction filter using a linear prediction coefficient obtained as a result as a filter coefficient by the learning speech data. And the residual signal is output as the teacher data, and the student data generation step encodes and decodes the learning speech data by a CELP method, and drives a speech synthesis filter obtained as a result The residual signal to be output as the student data,
Alternatively, the teacher data generation step performs linear prediction analysis on the learning voice data, and outputs a linear prediction coefficient obtained as a result thereof as the teacher data, and the student data generation step includes the learning voice data. Is encoded and decoded by the CELP method, and a linear prediction coefficient that is a filter coefficient of a speech synthesis filter obtained as a result is output as the student data,
When the learning data is image data,
In the encoding step, the image data for learning is encoded by an MPEG (Moving Picture Experts Group) method, and the encoded data for learning is output.
The comparison output step represents a combination of the characteristic data included in the learning encoded data and an actual characteristic that is an actual characteristic of the learning image data corresponding to the learning encoded data. Output the mismatch information;
The teacher data generation step and the student data generation step include:
The teacher data generation step outputs the learning image data as it is as the teacher data, and the student data generation step encodes and decodes the learning image data by the MPEG method, The result is output as the student data,
The teacher data generation step outputs the learning image data as it is as the teacher data, and the student data generation step encodes and decodes the learning image data by the MPEG method, and the result Outputting the obtained two-dimensional DCT coefficient as the student data;
Alternatively, the teacher data generation step performs two-dimensional DCT transformation on the learning image data, and outputs a two-dimensional DCT coefficient obtained as a result thereof as the teacher data, and the student data generation step includes the learning data A recording medium on which is recorded a program for causing a computer to perform learning processing for encoding and decoding image data by the MPEG method and outputting the resulting two-dimensional DCT coefficient as the student data.

Images