JP2005151603A - Coefficient generation apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To calculat a coefficient for each class by performing proper class classification adaptive processing in consideration of various signal characteristics of the input image signal. <P>SOLUTION: A plurality of pixels in a first image signal for learning are selected on the basis of the movement around a target pixel in the first image signal for learning, the class of the target pixel is decided on the basis of the patterns of the plurality of selected pixels, and an operation is performed using the plurality of pixels around the target pixel in the pixel for learning and a pixel which is corresponds to the first image signal for learning and to which a second image signal for learning different from the first image signal for learning. By doing this, since a coefficient calculated for each class can be stored, pixels having the similar features are collected and the coefficient for each class can be calculated highly accurately. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a coefficient generation apparatus and a coefficient generation method. For example, a standard resolution signal such as NTSC (hereinafter referred to as an SD (Standard Definition) image signal) is referred to as a high resolution signal such as a high definition (hereinafter referred to as HD (High Definition)). Difinition) (referred to as an image signal) is suitable for application when generating coefficients for image signal conversion.

  With the advent of various digital devices, a signal conversion device that realizes signal conversion between devices is required for connection between devices having different signal formats. For example, when SD image data is displayed on an HD monitor, an up-converter that converts the SD image data into HD image data is necessary.

  Conventionally, in this type of up-converter, HD image data is formed by subjecting SD image data to frequency interpolation processing using an interpolation filter to perform pixel interpolation. This interpolation processing will be described with reference to a spatial arrangement example of SD pixels and HD pixels in FIG. In the figure, the solid line represents the first field, and the dotted line represents the second field. In FIG. 38, for simplicity of explanation, the number of HD pixels is doubled in the horizontal and vertical directions with respect to the number of SD pixels. If attention is paid to the “◎” SD pixel in the figure, there are four types of HD pixels, mode1, mode2, mode3, and mode4 in the vicinity thereof. Therefore, the up-converter generates these four types of HD pixels mode1, mode2, mode3, and mode4 by frequency interpolation processing.

  As a simple configuration example of the up-converter, there is one that generates the above four types of HD pixels mode1, mode2, mode3, and mode4 from the field data of SD image data. As the configuration of the interpolation filter used at this time, there are the in-space two-dimensional non-separable filter 1 shown in FIG. 39 and the horizontal / vertical separable filter 5 shown in FIG.

  The two-dimensional non-separable filter 1 generates four types of HD pixels mode1, mode2, mode3, and mode1 by independently performing interpolation processing using the two-dimensional filters 2A to 2D, respectively. Then, the interpolation result is serialized in the selection unit 3 to generate HD image data. The horizontal / vertical separable interpolation filter 5 performs processing for the HD pixel mode1 and mode3 by the vertical interpolation filter 6A, and performs processing for the HD pixel mode2 and mode4 by the vertical interpolation filter 6B. Form data. Next, the final four types of HD pixels mode1, mode2, mode3 and mode1 are generated by using the horizontal interpolation filters 7A and 7B for each scanning line, and HD image data is serialized in the selection unit 8 to thereby generate HD image data. Is generated.

  However, in the conventional up-converter as described above, even if an ideal filter is used as an interpolation filter, the spatial resolution is the same as that of the input SD image although the number of pixels increases to match the HD format. In addition, since an ideal filter cannot actually be used, there is a problem that it is only possible to generate an HD image having a resolution lower than that of an SD image.

  As a method for solving such a problem, the input SD image data is classified into classes according to the distribution of the signal levels, and a prediction calculation is performed using a prediction coefficient obtained by learning in advance for each class. An image signal conversion apparatus and method using so-called class classification adaptive processing for obtaining an HD pixel close to a value has been proposed. For example, such a method is proposed in the specification and drawings of Japanese Patent Application Publication No. 5-328185 (publication date: December 10,1993) by the present applicant. The US application corresponding to this Japanese application is Serial No. 08 / 061,730 filed May 17, 1993.

  The up-converter using this class classification adaptation process is configured as shown in FIG. The up-converter 10 supplies the input SD image data D1 to the class classification unit 11 and the prediction calculation unit 12, respectively. The class classification unit 11 classifies a plurality of peripheral pixels centered on the target pixel as shown in FIG. 42A for input SD image data D1 (for example, 8-bit PCM (Pulse Code Modulation) data). Pixels (hereinafter referred to as class classification taps), and class data D2 is generated based on the waveform characteristics (level distribution pattern). In the figure, the solid line represents the first field, and the dotted line represents the second field.

  At this time, as a method of forming the class data D2 by the class classification unit 11, the PCM data is directly used (that is, the PCM data is directly used as the class data D2), ADRC (Adaptive Dynamic Range Coding), DPCM (Differential Pulse). A method of reducing the number of classes using a compression technique such as Code Modulation) or VQ (Vector Quantization) is considered. Of these methods, the method of directly setting the PCM data as the class data D2 is, for example, when a class classification tap consisting of 7 pixels is used as shown in FIG. It becomes a huge number of 256 and becomes a practical problem.

  Therefore, the number of classes is actually reduced by using a compression method such as ADRC. For example, consider a case in which 1-bit ADRC for compressing each pixel to 1 bit is applied to 7 pixels set as a class classification tap. That is, the 1-bit ADRC adaptively quantizes each pixel value of each tap after removing the minimum value of 7 pixels based on the dynamic range defined from the data of 7 pixels. Therefore, the number of classes can be reduced to 128 classes. ADRC was developed as a signal compression method for image signals, but is suitable for expressing the waveform characteristics of an input signal with a small number of classes.

  A prediction coefficient ROM (Read Only Memory) 13 outputs prediction coefficient data D3 obtained by learning in advance using the class data D2 as a read address, and the prediction coefficient data D3 is supplied to the prediction calculation unit 12. For example, as shown in FIG. 42B, the prediction calculation unit 12 sets a plurality of peripheral pixels centered on the target pixel as pixels for prediction calculation (hereinafter, referred to as a prediction tap), and the prediction tap Using the respective pixel values and prediction coefficient data D3 constituting the following equation (1)

  Here, the prediction coefficient data D3 for each class stored in the prediction coefficient ROM 13 is acquired in advance by learning using HD image data. This learning procedure will be described with reference to FIG. When the learning process procedure is started in step SP1, first, learning data is formed using an already known HD image in order to learn a prediction coefficient in step SP2.

  In step SP3, it is determined whether sufficient learning data necessary for obtaining the prediction coefficient has been collected. If it is determined that more learning data is necessary, the process proceeds to step SP4. If it is determined that sufficient learning data is obtained, the process proceeds to step SP6. In step SP4, the learning data is classified. This class classification executes the same process as the process of the class classification unit 11 (FIG. 41) of the up-converter 10 described above. At this time, generally, the influence of noise is eliminated by excluding the data change activity having a small activity from the learning target.

  Next, in step SP5, a normal equation is formed for each class based on the learning data classified into classes. The processing at step SP6 will be specifically described. Here, for generalization, a case where one interpolation target pixel (HD pixel) is represented by n SD pixels will be described. First, the relationship between each pixel level x1,..., Xn of the SD image data and the pixel level y of the target interpolation pixel is expressed by a linear linear combination expression of n taps using prediction coefficients w1,. The following formula (2)

Form.

  What is necessary is just to obtain | require the prediction coefficient w1, ..., wn in this (2) Formula. As a method for obtaining the prediction coefficients w1,. In this solution, first, X is SD pixel data, W is a prediction coefficient, and Y is target interpolation pixel data.

Collect data to create an observation equation. In Equation (3), m represents the number of learning data, and n is the number of prediction taps (ie, n = 13) set by the prediction calculation unit 12 (FIG. 41) described above.

  Next, based on the observation equation (3), the following equation (4)

Set up the residual equation of. From equation (4), the most probable value for each wi is the following equation (5):

It is considered that the condition for minimizing is satisfied. That is, the following formula (6)

These conditions should be taken into consideration. (6) Consider n conditions based on i in the equation, and calculate w1,..., Wn that satisfy this condition. Therefore, the residual equation (4) to the following equation (7)

Further, from the formulas (6) and (7), the following formula (8) is obtained.

Get. From the equations (4) and (8), the following equation (9)

The normal equation can be obtained. Since the normal equation (8) is a simultaneous equation with n unknowns, the most probable value of each wi can be obtained. In practice, the simultaneous equations are solved using the sweep-out method (Gauss-Jordan elimination method).

  In the predictive learning processing procedure of FIG. 43, the loop of steps SP2-SP3-SP4-SP5-SP2 is executed until the same number of normal equations as the unknown number n are formed in order to obtain the undetermined coefficients w1,. repeat. When the necessary number of normal equations is obtained in this way, an affirmative result is obtained at step SP3, and the processing shifts to determination of a prediction coefficient at step SP6.

In step SP6, the normal coefficient of equation (9) is solved to calculate the prediction coefficient w1 for each class.
, ..., determine wn. The prediction coefficient obtained in this way is registered in a storage unit such as a ROM (that is, the prediction coefficient ROM 13 (FIG. 41)) that is address-divided for each class in the next step SP7. Prediction coefficient data used in the class classification adaptation process is generated by the above learning, and the prediction learning process procedure is terminated in the next step SP8.

  As a hardware configuration for realizing this predictive learning process, a learning circuit 20 as shown in FIG. 44 can be considered. The learning circuit 20 converts the HD image data into SD image data via the vertical thinning filter 21 and the horizontal thinning filter 22, and supplies the SD image data to the class classification unit 23. Here, the class classification unit 23 has the same configuration as the class classification unit 11 of the above-described up converter 10 (FIG. 41), sets a class classification tap from SD image data, and class data based on its waveform characteristics. D2 'is generated. The class classification unit 23 is a part that executes the class determination process (step SP4) of FIG. The class classification unit 23 sends the generated class data D2 ′ to the coefficient calculation unit 24.

  The coefficient calculation unit 24 is a part that forms a normal equation and further determines a prediction coefficient. That is, the coefficient calculation unit 24 uses the SD image data and the HD image data to establish a normal equation of equation (9) for each class represented by the class data D2 ′, and obtains a prediction coefficient for each class from the normal equation. The obtained prediction coefficient is stored in the corresponding class address of the prediction coefficient ROM 13.

  However, in the case of generating HD image data using the class classification adaptive process, if an appropriate class classification process is not performed according to the characteristics of the input SD image data when generating a prediction coefficient by learning, an up converter There was a problem that the prediction accuracy due to. That is, in order to predict HD image data closer to the true value, it is important to collect only SD image data with similar characteristics, generate each class, and learn the corresponding HD image data as a teacher signal. It becomes.

  However, if the class classification capability is insufficient, the HD image data that should originally be classified into another class is classified into the same class. For this reason, in such a case, the prediction coefficient obtained by learning predicts the average value of HD image data having different properties. As a result, the up-converter that predicts an HD image signal using this prediction coefficient has a problem in that the ability to create resolution decreases.

  The present invention has been made in consideration of the above points, and by performing appropriate class classification adaptive processing in consideration of various signal characteristics of an input image signal, a low-resolution image signal is further improved. It is an object of the present invention to propose a coefficient generation device and a coefficient generation method capable of calculating with high accuracy the coefficients for each class for conversion into.

  In order to solve such a problem, in the present invention, in a coefficient generation device that generates a coefficient for image signal conversion for converting a first input image signal into a second image signal different from the first input image signal. , A motion detection unit that detects motion around the target pixel in the first learning image signal, and a class classification that selects a plurality of pixels in the first learning image signal based on the motion detected by the motion detection unit A pixel selection unit, a class determination unit that determines a class of the pixel of interest based on a pattern of a plurality of pixels selected by the class classification pixel selection unit, and for each class, a region around the pixel of interest in the first learning image signal Calculation is performed using a plurality of pixels and corresponding pixels of the second learning image signal corresponding to the first learning image signal and different from the first learning image signal, and a coefficient is calculated for each class. You A coefficient calculation unit, to be provided and a coefficient storage unit for storing the coefficients calculated in the coefficient calculating unit for each class.

  Accordingly, a plurality of pixels in the first learning image signal are selected based on the movement around the target pixel in the first learning image signal, and the class of the target pixel is determined based on the pattern of the selected plurality of pixels. For each class, a plurality of pixels around the target pixel in the first learning image signal and a second learning image corresponding to the first learning image signal and different from the first learning image signal Coefficients calculated for each class can be stored by performing calculations using the corresponding pixels of the signal, so that pixels with similar characteristics are collected together to calculate the coefficients for each class with high accuracy Can do.

  According to the present invention, a plurality of pixels in the first learning image signal are selected based on the movement around the pixel of interest in the first learning image signal, and the pixel of interest is based on the pattern of the selected plurality of pixels. For each class, a plurality of pixels around the target pixel in the first learning image signal and a second corresponding to the first learning image signal and different from the first learning image signal. Coefficients calculated for each class can be stored by performing calculations using corresponding pixels of the image signal for learning, so that pixels with similar characteristics can be collected and the coefficients for each class can be calculated with high accuracy. A coefficient generation device that can be calculated can be realized.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

(1) First Embodiment (1-1) Overall Configuration FIG. 1 shows an up-converter 30 according to a first embodiment to which the present invention is applied. The up-converter 30 supplies the input SD image data D1 to the class classification unit 31 and the prediction calculation unit 12, respectively. The class classification unit 31 detects the motion amount of the input image based on the input SD image data D1, performs class classification processing in consideration of the motion amount, and generates class data corresponding to the class classification processing. That is, the class classification unit 31 can classify the input SD image signal more accurately according to the feature. The class classification unit 31 sends the class data D32 obtained thereby to the prediction coefficient ROM 32.

  The prediction coefficient ROM 32 stores the prediction coefficient acquired by learning for each class classified in consideration of the amount of motion of the image as in the class classification process in the class classification unit 31. The prediction coefficient for each class is stored corresponding to each of the four types of generated HD pixels mode1, mode2, mode3, and mode4 shown in FIG. The prediction coefficient ROM 32 is created by a learning circuit 40 as shown in FIG. That is, in FIG. 2 in which the same reference numerals are assigned to corresponding parts to FIG. 44, the learning circuit 40 has a class classification unit 41 having the same configuration as the class classification unit 31 of the up converter 30 (FIG. 1). The class classification unit 41 classifies the SD image data in consideration of the amount of motion. The coefficient calculation unit 24 calculates a prediction coefficient for each class classified by the class classification unit 41 using the normal equation described above. At this time, the prediction coefficient for each class is calculated corresponding to each of the four types of HD pixels mode1, mode2, mode3, and mode4. The prediction coefficient is stored in the prediction coefficient ROM 32.

  Thus, the learning circuit 40 classifies the SD image in consideration of the motion of the image, and obtains the prediction coefficient for each class, so that the SD image is simply based on the level distribution pattern shape of the SD image. Compared with the case of class classification, it is possible to perform class classification processing more reflecting the characteristics of the image. As a result, it is possible to obtain the prediction coefficient for each class by gathering together those with similar characteristics, and thus the accuracy of the prediction coefficient can be improved.

  Further, the up-converter 30 reads the accurate prediction coefficient obtained in this way in correspondence with the class data D32 from the class classification unit 31. Then, prediction coefficients for generating the four types of read HD pixels mode1, mode2, mode3, and mode4 are supplied to the prediction calculation unit 12. The input SD image data supplied via a delay unit (not shown) is also supplied to the prediction calculation unit 12. Then, the prediction calculation unit 12 performs prediction calculation processing using the respective prediction coefficients and the input SD image data, and the four types of HD pixels mode1, mode2, mode3, and mode4 that are closer to true values. Data D33 is generated respectively. Then, the HD interpolation pixel data D33 is supplied to the conversion unit 14, and the conversion unit 14 converts it into time-series HD image data and displays it on the screen. Therefore, it is possible to obtain HD image data D34 with a higher resolution than in the conventional method. Although the method using the learning circuit 40 has been described as a method for generating the prediction coefficient, the present invention is not limited to this, and the prediction coefficient is generated by a computer or the like using a flowchart as shown in FIG. Also good. The same applies to the embodiments described later.

(1-2) Configuration of Class Classification Unit The class classification units 31 and 41 are configured as shown in FIG. Since the class classification unit 41 has the same configuration as the class classification unit 31, the class classification unit 31 will be described below. The class classification unit 31 supplies the SD image data D1 input via the input terminal IN to the frame memory 51 and the motion vector detection class classification unit 52 included in the first class classification unit 50, and class classification pixels. The data is supplied to a plurality of pattern setting units 54A to 54H of the selection unit 53.

  The motion vector detection class classification unit 52 detects the motion vector around the pixel of interest in the SD image data D1 using the SD image data of the previous frame and the SD image data of the current frame given from the frame memory 51, and the detection result The motion class code CLASS0 based on is output. The motion vector detection class classifying unit 52 classifies the motion vector into eight types according to which of the regions AR0 to AR6 or AR7 shown in FIG. To detect. Then, a motion class code CLASS0 representing the type to which the detected motion vector belongs is formed. Therefore, the motion class code CLASS0 is expressed by 3-bit data. Further, the embodiment of the motion vector detection class classification unit 52 uses a block matching method as a motion vector detection method.

  The motion class code CLASS0 is sent to the subsequent prediction coefficient ROM 32 as it is, and is also sent to the cluster selection unit 55. The cluster tap selection unit 55 inputs a plurality of types of class classification taps from the plurality of pattern setting units 54A to 54H, and selects a class classification tap corresponding to the motion class code CLASS0 from the plurality of types of class classification taps. Then, the data is output to the ADRC class classification unit 56 which is the second class classification unit.

  For example, with respect to an input SD image in which the amount of motion is almost zero, the ADRC class classification unit 56 as the second class classification unit can perform class classification based on signal changes in the vicinity of the spatio-temporal of the target pixel as much as possible. Select the tap. On the other hand, for an input SD image with a large amount of motion, a class classification tap is selected so that class classification can be performed by a large signal change in a wide area. For an SD image with an intermediate motion amount, a class classification tap is selected so that class classification with increased class classification sensitivity in the direction of motion can be performed.

  The class classification tap data output from the cluster top selection unit 55 is provided to the ADRC class classification unit 56 for classifying the target pixel according to the level distribution pattern of the surrounding pixels. The ADRC class classification unit 56 adaptively requantizes the class classification tap data in accordance with the dynamic range thereof, thereby forming an ADRC class code CLASS1 with a reduced number of classes. Further, the ADRC class classification unit 54 further changes the waveform of the SD image by ADRC by switching the number of pixels used when defining the dynamic range (DR) according to the motion class code CLASS0 during requantization. It can be reflected in the class code CLASS1.

  The ADRC class classification unit 56 according to the first embodiment performs a 1-bit ADRC process on the 8-pixel class classification tap data (8 bits per pixel) given from the cluster-type selection unit 55. The pixel value is adaptively requantized to 1 bit. The ADRC class classification unit 56 forms an ADRC class code CLASS1 of 256 classes (8 bits).

  In this way, the class classification unit 31 detects a motion vector using a plurality of pixels around the target pixel to be classified, classifies the target pixel based on the motion vector, and obtains a motion class code CLASS0. . Then, a combination of the motion class code CLASS0 and the ADRC class code CLASS1, which is a class classification result based on the level distribution pattern around the target pixel, is output as the final class code D32. In the case of this first embodiment, since the motion class code CLASS0 is 3 bits and the ADRC class code CLASS1 is 8 bits, the final class code D32 is expressed by 11 bits (2048 classes). This is the address number of the prediction coefficient ROM 32 (FIG. 1).

  Next, the pattern setting units 54A to 54H of the class classification pixel selection unit 53 will be described. Each pattern setting unit 54A to 54H sets a pixel pattern corresponding to the type of motion vector. In the case of the first embodiment, eight pattern setting sections 54A to 54H are provided in order to set eight types of pixel patterns corresponding to the eight types of movement. That is, each of the pattern setting units 54A to 54H sets a pixel pattern corresponding to each motion vector for the eight areas AR0 to AR7 shown in FIG.

  The pattern setting unit 54A sets a pixel pattern corresponding to the area AR0 within the dotted line at the center of FIG. 4, that is, when the motion vector is completely zero. At this time, since the input signal is in a completely stationary state, class classification that reflects the characteristics of the input signal is possible by using a tap pattern for spatiotemporal class classification near the target pixel when classifying based on the level distribution pattern It becomes. Therefore, the pattern setting unit 54A sets a class classification cluster top pattern as shown in FIG. 5 to 12, the HD pixel indicated by a small circle represents the target HD pixel to be interpolated.

  The pattern setting unit 54B sets a pixel pattern corresponding to a so-called quasi-still state. That is, the pixel pattern corresponding to the case where a motion vector exists in the area AR1 sandwiched between the center dotted line and the second dotted line from the center in FIG. 4 is set. In this case, there is a slight movement in the image. If an intermediate pixel pattern between the stationary part and the moving picture part is set, class classification reflecting the characteristics of the input signal becomes possible. For this reason, the pattern setting unit 54B sets a class classification tap pattern as shown in FIG. Compared to the completely stationary case of FIG. 5, by increasing the number of pixels in the same field, the spatial pixels are more important than the spatiotemporal pixels.

  The pattern setting unit 54C sets a pixel pattern corresponding to an intermediate motion amount. That is, the pixel pattern corresponding to the case where the motion vector exists in the area AR2 excluding the areas AR0 and AR1 and the areas AR4 to AR7 which are characteristic motion directions in the outermost dotted line in FIG. Set. In practice, the pattern setting unit 54C reflects a waveform change in a somewhat wide area in the class by setting pixels in a slightly wide area in the same field as shown in FIG.

  The pattern setting unit 54D sets a pixel pattern corresponding to a large amount of motion. That is, a pixel pattern corresponding to the case where a motion vector exists in the area AR3 outside the outermost dotted line in FIG. 4 is set. For this reason, the pattern setting unit 54D classifies waveform changes in a wider region than the pattern setting unit 54C by setting pixels in a wider region in the same field than in the case of FIG. 7 as shown in FIG. To reflect.

  The pattern setting unit 54E sets a pixel pattern corresponding to the upward tilt movement when a motion vector exists in the area AR4 in the upward dotted line in FIG. For this reason, the pattern setting unit 54E can perform class classification corresponding to upward movement in the ADRC class classification unit 56 by setting a pixel pattern extending upward (that is, the direction of movement) as shown in FIG. Like that. The pattern setting unit 54F sets a pixel pattern corresponding to the downward tilt movement when there is a movement in the direction of the area AR5 within the dotted line in the downward direction of FIG. For this reason, the pattern setting unit 54F sets a pixel pattern extending downward as shown in FIG. 10 so that the ADRC class classification unit 56 can perform class classification corresponding to the downward movement.

  The pattern setting unit 54G sets a pixel pattern corresponding to the right panning motion when a motion vector exists in the area AR6 within the dotted line in the right direction in FIG. Therefore, the pattern setting unit 54G sets the pixel pattern extending in the right direction as shown in FIG. 11 so that the characteristics of the image moving to the right side can be reflected in the class classification. Here, since it is considered that there is little movement in the vertical direction, the number of pixels in the vertical direction is set to be small. The pattern setting unit 54H sets a pixel pattern corresponding to the movement of the left panning when there is a movement in the direction of the area AR7 within the dotted line in the left direction of FIG. For this reason, the pattern setting unit 54H sets the pixel pattern extending in the left direction as shown in FIG. 12 so that the characteristics of the image moving to the left can be reflected in the class classification.

  Further, in each of the pattern setting units 54A to 54H, as shown in FIGS. 5 to 12, in order to detect a local dynamic range (DR) in consideration of the class classification by the ADRC class classification unit 56 in the subsequent stage. This pixel area is made wider than the set pixel pattern. Therefore, the tap data supplied from the cluster tap selection unit 55 is the tap data for determining the class classification tap data and the dynamic range DR. As a result, the waveform change around the pixel of interest can be further reflected in the class in the class classification process by the ADRC class classification unit 56.

  The ADRC class classification unit 56 may switch the number of quantization bits according to the motion class code CLASS0.

(1-3) Operation of First Example In the above configuration, the class classification units 31 and 41 first classify the input SD image in the first class classification unit 50 when classifying the target pixel. A motion vector of the data D1 is detected, and a motion class code CLASS0 is formed based on the detection result. In the class classification units 31 and 41, the class classification pixel selection unit 53 sets a class classification tap pattern corresponding to each motion class in advance from the input SD image data D1.

  The class classification units 31 and 41 are second class classification units for selecting a class classification tap pattern corresponding to the detected motion class and expressing the selected tap pattern with a small amount of information as a level distribution pattern. The data is sent to the ADRC class classification unit 56. Thus, in the ADRC class classification unit 56, since the ADRC class code CLASS1 based on the level distribution pattern of the SD image data D1 is formed using the class classification tap pattern corresponding to the movement of the SD image data D1, the ADRC class code CLASS1 Can sufficiently reflect the characteristics of the image.

  The class classification units 31 and 41 send the motion class code CLASS0 and ADRC class code CLASS1 thus formed as the class code D32 to the prediction coefficient ROM 32 (FIG. 1) or the coefficient calculation unit 24 (FIG. 2) at the subsequent stage. . Thereby, in the coefficient calculation part 24, those with similar characteristics can be collected to obtain the prediction coefficient for each class, so that the accuracy of the prediction coefficient can be improved. Further, the up-converter 30 can perform the prediction calculation process using the accurate prediction coefficient obtained in this way, so that an HD interpolation pixel closer to the true value can be obtained.

(1-4) Effects of First Example According to the above configuration, in addition to the class based on the level distribution pattern of the peripheral pixels of the target pixel, a motion class based on the motion of the peripheral pixels of the target pixel is formed. By using the combination of classes as the final classification result, the input SD image can be classified more accurately. In addition, the class classification tap pattern used when performing class classification based on the level distribution pattern according to the motion class is adaptively selected. It can be well reflected.

  As a result, if a prediction coefficient is calculated according to this class, a more accurate prediction coefficient can be obtained for each class. Furthermore, if the prediction calculation is performed using this prediction coefficient to obtain the interpolated pixel value, the prediction performance of the HD interpolated pixel data D33 can be improved, so that the SD image data D1 is further improved in HD image data. An up-converter 30 that can convert to D34 can be realized.

(2) Second Embodiment In the first embodiment described above, eight types of motion are detected from the input image signal to form a motion class code CLASS0 and set in advance corresponding to these eight types of motion. The class classification tap pattern corresponding to the motion class code CLASS0 is selected from the eight types of class classification tap patterns. In this second embodiment, only the presence or absence of motion is detected to form a motion class code, A tap pattern for class classification is selected according to the presence or absence of. Actually, when the vicinity of the target pixel is a moving image portion, an in-field pixel is selected as the class classification tap pattern, and when it is a still portion, an intra-frame pixel is selected.

  That is, when there is a motion in the image to be classified, a pixel in the field is selected, and class classification is performed in accordance with a signal change in the space in which time characteristics are purely excluded. On the other hand, when there is no movement in the image signal to be classified, a pixel having a spread in the time direction is selected so that the signal change in the vicinity of the target pixel is reflected in the class classification as much as possible in the spatiotemporal structure. For this reason, for example, pixels within the frame, and further, pixels over several fields are selected.

  In practice, the class classification unit of the second embodiment is configured as shown in FIG. That is, the class classification unit 60 supplies the SD image data D1 supplied via the input terminal IN to the frame memory 62 and the motion detection unit 63 constituting the first class classification unit 61, respectively, and class selection pixel selection. This is supplied to the in-field tap selection unit 65 and the in-frame tap selection unit 66 of the unit 64.

  The motion detection unit 63 directly detects the presence or absence of a motion amount based on the supplied current frame SD image data D1 and the past frame SD image data D1 supplied via the frame memory 62. At this time, the motion amount detection unit 63 calculates the absolute value sum of the difference values between the image data of the current frame and the image data of the past frame for each block centered on the target pixel. If the sum of absolute values of the difference values is less than a predetermined threshold, the block is determined to be a moving image portion, and if the sum is greater than the predetermined threshold, the block is determined to be a stationary portion. The motion detector 63 outputs the determination result as a 1-bit motion class code CLASS2.

  As shown in FIG. 14A, the in-field tap selection unit 65 sets the in-field tap centered on the target pixel (“◎” in the figure), and sends the in-field tap data to the selection unit 67. As shown in FIG. 14B, the in-frame tap selection unit 66 sets an in-frame tap centered on the pixel of interest, and sends the in-frame tap data to the selection unit 67. The selection unit 67 alternatively selects and outputs the input in-field tap data or in-frame tap data according to the motion class code CLASS2. That is, if the motion class code CLASS2 represents a moving image portion, the in-field tap data is output. If the motion class code CLASS2 represents a still portion, the in-frame tap data is output.

  The class classification tap data output from the selection unit 67 is given to the ADRC class classification unit 68. The ADRC class classification unit 68 adaptively requantizes the class classification tap data in accordance with the dynamic range thereof to form an ADRC class code CLASS3 with a reduced number of classes. In the second embodiment, since data for seven pixels is sent to the ADRC class classification unit 68 as class classification tap data, for example, when 1-bit ADRC processing is performed on the class classification tap data. The class classification tap data is classified into 128 types of classes. Finally, the class classification unit 60 outputs a class classification result of a total of 8 bits (256 types) of 1 bit as the motion class code CLASS2 and 7 bits as the ADRC class code CLASS3.

  In practice, the class classification unit 60 is used as the class classification unit 31 of the up-converter 30 (FIG. 1) and the class classification unit 41 of the learning circuit 40 (FIG. 2) described above in the first embodiment. The class code CLASS2 and the ADRC class code CLASS3 are sent to the prediction coefficient ROM 32 and the coefficient calculation unit 24.

  According to the above configuration, in addition to the class based on the level distribution pattern of the peripheral pixels of the target pixel, a motion class corresponding to the presence or absence of motion in the peripheral pixels of the target pixel is formed, and the combination of these classes is finally determined. By using the class classification result, it is possible to classify the input SD image more accurately. Further, the cluster classification pattern used when performing class classification based on the level distribution pattern is adaptively selected according to the presence or absence of motion, so that the class classification result based on the level distribution pattern can be obtained from the input SD image. The characteristics can be well reflected.

  As a result, if a prediction coefficient is calculated according to this class, a more accurate prediction coefficient can be obtained for each class. Furthermore, if the prediction calculation is performed using this prediction coefficient to obtain the interpolated pixel value, the prediction performance of the HD interpolated pixel data can be improved, so that the SD image data D1 is further converted into HD image data with higher resolution. An up-converter that can be converted can be realized.

  In addition, since the motion class is formed based only on the presence or absence of motion, the configuration of the first class classification unit 61 and the class classification pixel selection unit 64 can be simplified as compared with the first embodiment. .

In addition, in the ADRC class classifying unit 68 as the second class classifying unit, as in the first embodiment, the pixel area for detecting the dynamic range may be set wider than the pixel pattern, Further, the pixel region for detecting the dynamic range may be determined by the motion class code CLASS2.
Furthermore, the ADRC class classification unit 68 may switch the number of quantization bits according to the motion class code CLASS2.

(3) Third Embodiment In the third embodiment, class classification performance is improved by performing multi-stage class classification processing in consideration of image motion. That is, after coarse class classification is performed on the input image at the first stage, a class classification tap corresponding to the result at the first stage is adaptively selected at the next stage, and fine class classification different from the first stage is performed. As a result, it is possible to obtain a classification result that more accurately represents the characteristics of the input image.

  In the first and second embodiments described above, the first class classification units 50 and 61 are considered as the first class classification process, and the class classification taps adaptively selected according to the class classification result are used. If the ADRC processing is considered as the next class classification process, it can be said that the multi-class classification process is performed similarly. However, in the third embodiment, the method of forming the motion class and the method of setting the class classification tap are different from those in the first and second embodiments.

  That is, in the third embodiment, the first class classification unit has a plurality of coefficients selected in advance in consideration of movements of a plurality of types of images, and the prediction coefficient and the pixel data around the pixel of interest A prediction value calculation unit that calculates a plurality of prediction values for the target pixel, a difference value calculation unit that calculates a difference value between the plurality of prediction values calculated by the prediction value calculation unit and the pixel value of the target pixel, And a first class code generation unit that determines a motion around the target pixel by detecting a minimum value of the difference value and outputs a first class code representing the motion.

  FIG. 15 shows a specific configuration of the class classification unit of the third embodiment. FIG. 15 shows an example in which class classification is performed in two stages. The class classification unit 70 corresponds to the class classification unit 31 of FIG. 1, performs a rough class classification based on image movement in the first-stage class classification unit 71 as the first class classification unit, and in the next-stage class classification unit 72, A class classification tap corresponding to the classification result of the first class classification unit is selected to perform fine class classification processing. Here, the next-stage class classification unit 72 has both the function of the class classification pixel selection unit 53 of the first embodiment and the function of the second class classification unit (that is, the ADRC class classification unit) 56.

  For example, when there is a motion in the input image, the first-stage class classification unit 71 determines from which direction the pixel of interest has moved, and classifies the input image according to the determination result. Therefore, the first class classification unit 71 calculates the difference between the pixel value of the target pixel and the linear prediction value for the target pixel using a plurality of pixels around the target pixel (in the case of the example, calculates nine difference values). ) Then, a plurality of coarse classifications are first performed by comprehensively comparing and determining the difference results.

  Here, the coefficient used in the linear prediction is created by learning an input image using an image that is artificially moved in a certain direction with an arbitrary size. The method for learning the coefficient is the same as the method for learning the prediction coefficient in the prediction coefficient ROM, and the description thereof is omitted here. For example, nine types of directions as shown in FIG. 17 are learned as rough classification indices, and nine types of coefficient groups are obtained. This coefficient set is used as coefficients for nine types of linear prediction. In FIG. 17, for example, the central area “0” indicates a stationary or quasi-static area where there is little movement, and for example, the upper area “1” indicates a case where there is movement in the upward direction.

  Actually, the first-stage class classification unit 71 supplies the input SD image data D1 supplied via the input terminal IN to the cluster-type setting unit 73 that sets a class classification tap for coarse class classification in the first stage. As shown in FIG. 16, the cluster setting setting unit 73 classifies the neighboring pixels in the time space of the pixel of interest from the frame in which the pixel of interest exists, the frame in the past by one frame, and the frame in the past by two frames. (In this example, 16 pixels including the target pixel are set), and among these, the spatiotemporal neighboring pixel data excluding the target pixel is sent to each of the prediction units 75A to 75I.

  As described above, each of the prediction units 75A to 75I has a prediction coefficient group suitable for each direction of motion, and predicts a target pixel value by linear prediction of the prediction coefficient group and a cluster group. The difference units 76A to 76I calculate the difference between the pixel value of the target pixel input via the delay unit 74 and the predicted value obtained by each of the prediction units 75A to 75I, and each difference value is the minimum value determination unit 77. To be supplied.

  Here, for example, when the input image is moving from right to left, the prediction result calculated using the prediction coefficient obtained by learning assuming such movement is the most compared to other cases. The value should be close to the target pixel. For this reason, the minimum value determining unit 77 classifies the direction of motion by detecting the nine types of difference results having the smallest absolute value. In this way, the rough classification of the first stage can be executed. Thus, the first class classification unit 71 outputs nine types (four bits) of motion class codes CLASS4.

  When the direction of movement is detected by the first-stage class classification unit 71, the next-stage class classification unit 72 further extracts features from such an image and performs fine class classification. Here, the next class classification unit 72 does not perform class classification using a uniform cluster structure, but specializes in the direction of motion based on the motion class classified by the first class classification unit 71. A classification pattern is selected and a classification pattern is selected. As a result, the next-stage class classification unit 72 can perform class classification more suitable for the characteristics of the input image.

  For this reason, the next-stage class classification unit 72 has a circuit configuration in which the class classification tap pattern can be made variable according to the first-class classification result. The next class classification unit 72 supplies the output of the cluster group setting unit 73 to the plurality of selectors 79A to 79I via the register array 78. Each of the selectors 79A to 79I sets nine types of class classification taps corresponding to the motion class code CLASS4 as shown in FIGS. Then, any one of the selectors 79A to 79H or 79I is alternatively selected according to the motion class code CLASS4 output from the minimum value determination unit 77 of the first stage class classification unit 71, and the selected selectors 79A to 79H or The 79I cluster group is sent to the ADRC class classification unit 80 which is the second class classification unit. As a result, the ADRC class classification unit 80 is supplied with the optimum class classification tap corresponding to the first class classification result.

  For example, when the first class classification unit 71 determines that the vicinity of the pixel of interest has moved from left to right (that is, when there is movement in the direction of the region “4” in FIG. 17), the class classification tap is It is advantageous to extend in the horizontal direction in terms of space, and the selectors 79A to 79I are controlled so that such 12 pixels are selected (class 4 in FIG. 19). In the same way in the other directions, a classification tap corresponding to the movement class code CLASS4 is selected.

  The ADRC class classification unit 80 compresses the number of bits in the level direction of the selected class classification tap in the same manner as in the first and second embodiments described above by using an adaptive dynamic range, thereby providing an ADRC class code. Form CLASS5. In this embodiment, as shown in FIGS. 18 to 20, since a 12-pixel class classification tap is used, each pixel is adaptively re-quantized to 1 bit by the ADRC class classification unit 80. The ADRC class after classification is 4096.

  In this way, the class classification tap of the next class classification unit 72 is selected according to the class classification result of the first class classification unit 71, and the 4096 class of 1-bit ADRC is selected using the selected class classification tap. Pattern classification is realized. As a result, together with the class code CLASS4 from the first stage, 36864 classes can be classified for the pixel of interest.

  The motion class code CLASS4 and ADRC class code CLASS5 obtained in this way are output as read addresses of the prediction coefficient ROM 82. Incidentally, the delay unit 81 delays the motion class code CLASS4 by the processing time of the next class classification unit 72. Here, the prediction coefficient in the prediction coefficient ROM 82 can be created by using the class classification unit 70 of the third embodiment instead of the class classification unit 41 of the learning circuit of FIG. 2, and the flowchart of FIG. It can be created by applying the algorithm of the class classification unit 70 of the third embodiment. That is, for each class formed by combining the motion class code CLASS4 and ADRC class code CLASS5, the prediction corresponding to mode1, mode2, mode3, and mode4 is performed by learning using the normal equations corresponding to mode1, mode2, mode3, and mode4. A coefficient is obtained, and the prediction coefficient may be stored at the address of the class.

  According to the motion class code CLASS4 and ADRC class code CLASS5, the prediction coefficients for mode1, mode2, mode3, and mode4 read from the prediction coefficient ROM 82 are predicted together with the prediction type set by the prediction type setting unit 83. This is given to the calculation unit 84. Similar to the above-described prediction calculation unit 12 (FIG. 1), the prediction calculation unit 84 linearly combines the prediction taps and the respective prediction coefficients to obtain HD interpolation pixel data corresponding to mode1, mode2, mode3, and mode4. Ask for. Then, the HD interpolation pixel data is converted into time-series data and output. FIG. 21 shows an example of the prediction tap set by the prediction tap setting unit 83.

  According to the above configuration, the first class classification unit 71 performs the coarse class classification based on the motion of the image, and then the second class classification unit 72 performs the fine class classification. The classification accuracy can be improved.

  In addition, when classifying the input image in the first stage, a plurality of linear predictions are performed on the pixel of interest using a plurality of prediction coefficients obtained by learning in advance for each direction of movement, and the most true value among them is performed. The movement code CLASS4 can be easily formed by obtaining the movement class code CLASS4 using the direction in which the predicted value close to is obtained as the movement direction.

In the ADRC class classifying unit 80, which is the next class classifying unit, the pixel area for detecting the dynamic range may be set wider than the pixel pattern in the same manner as in the above-described embodiment. The pixel area for detecting the dynamic range may be determined by the motion class code CLASS4.
Further, the ADRC class classification unit 80 may switch the number of quantization bits according to the motion class code CLASS2.

(4) Fourth Example In the fourth example, the first class classification unit is a first class forming a block having a predetermined size centered on the target pixel in the current frame or field of the input image data. And a second blocking unit for forming a plurality of blocks arranged in a plurality of directions around the position of the block formed in the current frame or field in the past frame or field of the input image data. A difference value calculating unit for calculating a difference value between pixels in the block between the block formed by the first blocking unit and the plurality of blocks formed by the second blocking unit; and a plurality of blocks For each, the absolute value sum calculation unit that calculates the sum of the absolute values of the difference values and the minimum value of the absolute value sum are detected to determine the movement around the target pixel. And a first class code generation unit that outputs a first class code that represents motion.

  FIG. 22 shows a specific configuration of the class classification unit of the fourth embodiment. In FIG. 22, in which parts corresponding to those in FIG. 15 are assigned the same reference numerals, the class classification unit 90 of the fourth embodiment is centered on pixel-of-interest data in the current frame or field in the first-stage class classification unit 91. A rough classification is performed by calculating a difference between a block and a plurality of blocks cut out at a plurality of different positions in the past frame or field and comparing the sum of absolute values of the differences. .

  Similar to the first class classification unit 71 of the third embodiment, the first class classification unit 91 determines, for example, from which direction the pixel of interest has moved when there is a motion in the input image, and the determination This is for classifying the target pixel into large groups according to the result. In order to perform this with high accuracy, it is necessary to perform motion detection. However, if this motion detection is performed by, for example, the blotching method, the amount of calculation increases, so that the amount of hardware becomes too heavy for rough classification in the first stage.

  Therefore, in the fourth embodiment, a block to be referred to in the past is divided into nine types of areas as shown in FIG. The hardware is reduced by performing simple block scrambling, which calculates the sum of absolute values of pixel unit differences between each area and the current frame or field block, and detects the minimum value. . For example, when the image moves from right to left, the absolute value sum of the difference from the past region “3” tends to be the smallest. Thus, the first-stage coarse class classification can be performed with a small amount of calculation, and as a result, the configuration of the first-stage class classification unit 91 can be simplified.

  Actually, in the first class classification unit 91 of this embodiment, the input SD image is sent to each of the region division units 95A to 95I via the frame delay unit 93. Then, in the area dividing units 95A to 95I, past blocks having different positions are set as shown in FIG. The plurality of past block data and the current block data centered on the pixel of interest obtained through the blocking unit 94 are sent to the difference units 96A to 96I. The difference values for the corresponding pixels obtained by the difference units 96A to 96I are sent to the absolute value sum units 97A to 97I. Then, the sum of absolute differences obtained as a result is sent to the minimum value determination unit 98. The minimum value determination unit 98 outputs the one having the smallest difference absolute value sum as the motion class code CLASS6 representing the direction of motion.

  The next-stage class classification unit 92 has the same configuration as the next-stage motion class classification unit 72 described above in the third embodiment, and a plurality of classes for setting the class classification taps in the next stage as shown in FIGS. Selectors 79A to 79I. Then, among the selectors 79A to 79I, 79A to 79H or 79I corresponding to the motion class code CLASS6 obtained by the first stage class classification unit 91 is alternatively selected, so that the next class classification tap is selected. select.

  According to the above configuration, when the motion class of the first stage is obtained, the motion class can be easily obtained by performing simple block matching in which a plurality of reference blocks at different positions are set in advance. The configuration of the first class classification unit 91 (first class classification means) can be simplified.

(5) Fifth Embodiment In the fifth embodiment, basically, as in the first to fourth embodiments described above, a motion class is first formed based on the motion of the input SD image, and then The class classification with high accuracy is realized by switching the class classification tap in the next stage according to the motion class. However, in this embodiment, the method of forming the motion class is different from the first to fourth embodiments.

  In the fifth embodiment, in the first class classification unit, a block having a predetermined size cut out from a frame in which the pixel of interest exists and a position spatially the same as the block cut out from a frame adjacent to the frame. The inter-frame difference value is calculated with the block located at. Then, the average value of the absolute values of the inter-frame difference values is compared with a predetermined threshold value, and the movement around the target pixel is determined based on the comparison result, thereby forming a class code representing the movement class.

  FIG. 24 shows a specific configuration of the up-converter according to the fifth embodiment. The up-converter 100 is roughly divided into a class classification unit 101 that classifies each target pixel of the input SD image data D1, a prediction coefficient ROM 103 that outputs a prediction coefficient according to the classification result, and the output prediction coefficient and the input SD image. The prediction calculation unit 102 generates HD interpolation pixel data in the vertical direction by performing prediction calculation using the data D1.

  Here, the positional relationship between the SD pixel and the HD pixel to be interpolated in the fifth embodiment is as shown in FIG. That is, there are two types of HD pixels to be interpolated: an HD pixel y1 that exists near the SD pixel and a y2 that exists far from the SD pixel when viewed in the same field. Hereinafter, a mode for estimating an HD pixel existing at a position close to the SD pixel is referred to as mode 1, and a mode for estimating an HD pixel existing at a position far from the SD pixel is referred to as mode 2.

  The up-converter 100 inputs the SD image data D1 supplied from the input terminal to the area cutout unit 104. The area cutout unit 104 cuts out pixels necessary for class classification (motion class) for representing the degree of movement. In this embodiment, ten SD pixels m1 to m5 and n1 to n5 existing at positions shown in FIG. 26 are cut out from the supplied SD image with respect to HD pixels y1 and y2 to be interpolated.

  The data cut out by the area cutout unit 104 is supplied to the motion class determination unit 105. The motion class determination unit 105 calculates a motion parameter by calculating a difference between frames of the supplied SD pixel data and determining an average of the absolute values as a threshold. Specifically, the motion class determination unit 105 calculates the following equation (10)

The average value param of the absolute value of the difference is calculated from the supplied SD pixel data. However, in the example, n = 5.

  The motion class determination unit 105 obtains the motion class code CLASS8 by comparing the average value param calculated in this way with a preset threshold value. Here, for example, the case where the average value param is “2” or less is class “0”, the case where the average value param is greater than “2” and “4” or less is class “1”, and the average value param is “ If it is greater than 4 ”and less than or equal to“ 8 ”, the class is“ 2 ”. If the average param is greater than“ 8 ”, the class is“ 3 ”. If param is less than“ 2 ”, the class is“ 3 ”. A motion class code CLASS8 consisting of four classes can be formed. Here, this threshold value may be set by, for example, dividing the histogram of the absolute value of the difference of the SD pixel data into n equal parts.

  The motion class code CLASS8 formed in this way is sent to the area cutout unit 107 and the class code generation unit 109. The region cutout unit 107 is supplied with the input SD image data D1 through a delay unit (DL) 106 that delays the input SD image data D1 by the processing time of the region cutout unit 104 and the motion class determination unit 105. The SD pixel data cut out by the region cutout unit 107 is supplied to the ADRC unit 108, and the ADRC unit 108 patterns the SD pixel data into a small number of bits (in-space class classification). For example, SD pixel data of 8 bits per pixel is compressed into SD pixel data of 2 bits per pixel.

  Here, in the class classification in space, in the case of an image with small motion, a method using pixels with the number of fields corresponding to two fields or more is effective and efficient. On the other hand, in the case of an image with large motion, a method using pixels within one field is effective and efficient. Therefore, when the motion class code CLASS8 is the class “0” or the class “1”, the region cutout unit 107, for example, converts the five SD pixels k1 to k5 located at positions as shown in FIG. Cut out as pixels to be used for classification (that is, taps for class classification). On the other hand, when the motion class code CLASS8 is class “2” or class “3”, the region cutout unit 107 moves the five SD pixels k1 to k5 located at positions as shown in FIG. Cut out as a pixel used for classification.

  The pixel data compressed by the ADRC unit 108 is supplied to the class code generation unit 109 as an ADRC class code CLASS9. Based on the ADRC class code CLASS9 and the motion class code CLASS8, the class code generation unit 109 calculates the following formula (11)

By performing the above operation, the final class to which the block belongs is detected. Then, the class code CLASS10 indicating the detected class is output as a read address of the prediction coefficient ROM 103. However, qi in the equation (11) represents each pixel data requantized by ADRC, p represents bit allocation at the time of ADRC, n is 5, and p is 2. Incidentally, in the prediction coefficient ROM 103 of the fifth embodiment, a prediction coefficient for mode 1 corresponding to the HD pixel y1 and a prediction coefficient for mode 2 corresponding to the HD pixel y2 are prepared independently.

  On the other hand, the input SD image data D1 supplied from the input terminal is also given to the area cutout unit 110. The region cutout unit 110 cuts out SD pixel data (that is, a prediction tap) used for prediction calculation from input data. In the case of the fifth embodiment, 17 pixels x1 to x17 located at positions as shown in FIG. 29 are cut out as prediction taps. The output of the region cutout unit 110 is supplied to the prediction calculation unit 112 via the delay unit 111 prepared for the purpose of timing adjustment. The prediction calculation unit 112 performs a linear primary calculation using the prediction coefficient of mode 1 corresponding to the supplied HD pixel y1, the prediction coefficient of mode 2 corresponding to the HD pixel y2, and the prediction tap, respectively. HD interpolation pixel data y1 and y2 corresponding to the SD pixel data are calculated.

  The HD interpolation pixel data output from the prediction calculation unit 112 is supplied to the horizontal interpolation filter 113. The horizontal interpolation filter 113 doubles the number of pixels in the horizontal direction by interpolation processing. Finally, HD interpolation pixel data such as mode1, mode2, mode3 and mode4 in FIG. 1 is obtained by this horizontal interpolation processing. Is generated and supplied to the conversion unit 115. In the conversion unit 115, the supplied HD interpolation pixel data is converted into time-series data, and the output is supplied to an HD television receiver, an HD video tape recorder, or the like.

  In the ADRC unit 108, as in the above-described embodiment, the pixel range for detecting the dynamic range may be set wider than the set pixel pattern, and the dynamic range is determined by the motion class code CLASS8. A pixel region for detecting the image may be determined.

  Further, the ADRC unit 108 may switch the number of quantization bits in accordance with the motion class code CLASS8.

  Next, the configuration of a learning circuit for creating a prediction coefficient stored in the prediction coefficient ROM 103 of this embodiment will be described. As shown in FIG. 30, in which parts corresponding to those in FIG. 24 are denoted by the same reference numerals, the learning circuit 120 first corresponds to an already known HD image and has an SD of 1/4 pixel number of the HD image. Form an image. Specifically, the vertical pixel of the HD image data supplied via the input terminal is thinned by the vertical thinning filter 121 so that the vertical frequency becomes 1/2, and further, the horizontal thinning filter is used. SD image data is obtained by thinning out horizontal pixels of the HD image data in 122.

  The SD image data is supplied to the class classification unit 101 and also to the coefficient calculation unit 123 corresponding to the coefficient calculation unit 24 of FIG. Then, the class code CLASS10 formed by the class classification unit 101 is supplied to the normal equation forming unit 124 of the coefficient calculation unit 123. The normal equation forming unit 124 forms normal equation data based on the above-described equations (2) to (9) for each class represented by the class code CLASS10. At this time, normal equation data is generated for mode 1 and mode 2, respectively.

  After the input of all the learning data is completed, the normal equation forming unit 124 outputs the mode 1 and mode 2 normal equation data to the prediction coefficient determining unit 125, respectively. The prediction coefficient determination unit 125 solves the prediction coefficients of mode 1 and mode 2 by using a general matrix solving method such as a sweeping method, and outputs the prediction coefficients of mode 1 and mode 2 to the memory 126. As a result, the memory 126 stores mode 1 and mode 2 prediction coefficients that can be estimated statistically closest to the true value when the target HD interpolation pixel data y1 and y2 are estimated for each class. The table stored in the memory 126 may be used as the prediction coefficient ROM 103 of the up converter 100.

  According to the above configuration, the class classification result based on the level distribution pattern can be obtained by adaptively selecting the class classification tap pattern used when performing class classification based on the level distribution pattern according to the motion class. The feature of the input image can be well reflected. In addition, since the motion class is obtained by calculating the inter-frame difference of the SD image and determining the average value of the absolute value as a threshold, the motion class can be easily obtained. Can be simplified.

(6) Sixth Embodiment In the first to fifth embodiments described above, a motion class is formed and a class classification tap used for class classification based on the level distribution pattern is adaptively selected according to the motion class. As described above, in the sixth embodiment, the prediction tap used for the prediction calculation is adaptively selected according to the motion class.

  In practice, the up-converter of the sixth embodiment is configured as shown in FIG. In FIG. 31, in which parts corresponding to those in FIG. 24 are denoted by the same reference numerals, the motion class determination unit 144 provided in the class classification unit 141 of the up converter 140 performs the motion obtained by the same method as in the fifth embodiment. The class code CLASS8 is sent to the class code generation unit 109 and also sent to the area cutout unit 147 of the prediction calculation unit 142.

  The region cutout unit 147 cuts out a prediction tap corresponding to the motion class code CLASS8 from the SD image data given through the delay unit 146 for timing adjustment. That is, when the motion class code CLASS8 indicates that the motion is small, such as class “0” or class “1”, for example, nine SD pixels x1 at positions as shown in FIG. Cut out ~ x9 as a prediction tap. On the other hand, when the motion class CLASS8 indicates that the motion is large as in the class “2”, for example, nine SD pixels x1 to x9 at positions as shown in FIG. 34 are predicted. Cut out as a tap. When the motion class CLASS8 indicates that the motion is very large as in the class “3”, nine SD pixels x1 to x9 located at positions as shown in FIG. 35 are cut out as prediction taps. . As a result, the prediction calculation unit 112 is supplied with an optimal prediction tap based on the motion of the input SD image to be classified.

  Here, in the conventional prediction calculation, the same pixel is always used as a prediction tap regardless of the class classified. For this reason, when the prediction performance is emphasized, the number of pixels constituting the prediction tap is increased, resulting in an increase in hardware scale. On the other hand, when the hardware scale is reduced, there is a drawback in that the prediction performance is lowered.

  On the other hand, in the up converter 140 of the sixth embodiment, since the prediction tap is adaptively selected according to the motion class code CLASS8, the prediction performance can be improved with a simple configuration. . That is, as described above, when the motion is small, an SD pixel spatially close to the HD pixel to be interpolated is selected as the prediction tap. When the motion is large, an SD pixel that is temporally close to the HD pixel to be interpolated is selected. Therefore, only the prediction tap suitable for the prediction calculation is set without increasing the number of prediction taps unnecessarily.

  In the sixth embodiment, the area cutout unit 145 for cutting out the class classification tap is, for example, five SD pixels k1 located in the vicinity of the HD pixels y1 and y2 to be interpolated as shown in FIG. Cut out ~ k5. That is, the area cutout unit 145 is different from the fifth embodiment in that a fixed class classification tap is cut out regardless of movement.

  Next, a learning circuit for creating the prediction coefficient ROM 143 (FIG. 31) of the sixth embodiment will be described. As shown in FIG. 32 in which parts corresponding to those in FIG. 31 are denoted by the same reference numerals, the prediction coefficient calculation unit 151 of the learning circuit 150 performs SD according to the motion class code CLASS8 determined by the motion class determination unit 105. Pixels are cut out by the area cutout unit 147 and sent to the normal equation forming unit 133. Also, the normal equation forming unit 133 receives the class code CLASS12 obtained by the class code generating unit 109 based on the motion class CLASS8 and the ADRC class code CLASS11 obtained from the fixed class classification tap.

  As described above, the normal equation forming unit 133 establishes normal equations for the mode 1 and the mode 2 for each class code CLASS12 and sends the normal equations to the prediction coefficient determination unit 134. The prediction coefficients for mode 1 and mode 2 obtained by the prediction coefficient determination unit 134 are stored in the memory 135. Thus, according to the learning circuit 150, it is possible to generate an accurate prediction coefficient by adaptively selecting a prediction tap used for learning according to a motion class.

According to the above configuration, an up-converter with a reduced hardware scale and good prediction performance can be realized by forming a prediction tap with only pixels that are important in prediction calculation according to the motion class. Can do.
In the ADRC unit 108, as in the above-described embodiment, the pixel range for detecting the dynamic range may be set wider than the set pixel pattern, and the dynamic range is determined by the motion class code CLASS8. A pixel region for detecting the image may be determined.

  Further, the ADRC unit 108 may switch the number of quantization bits in accordance with the motion class code CLASS8.

(7) Seventh Embodiment In the first to fifth embodiments described above, a class classification tap is adaptively selected according to the motion class, and in the sixth embodiment, a prediction tap is selected according to the motion class. In the seventh embodiment, both the class classification tap and the prediction tap are adaptively selected according to the motion class.

  The configurations of the up-converter and the learning circuit of the seventh embodiment are shown in FIGS. In FIG. 36, in which parts corresponding to those in FIG. 24 and FIG. 31 are assigned the same reference numerals, the class determining unit 164 of the upconverter 105 extracts a region classifying unit 107 for extracting the determined motion class code CLASS8 from the class classification tap. To the region cutout unit 147 for cutting out the predicted tap. The area cutout unit 107 cuts out a class classification tap corresponding to the movement class code CLASS8 as described in the fifth embodiment. Further, the area cutout unit 147 cuts out a prediction tap corresponding to the motion class code CLASS8 as described above in the sixth embodiment.

  Thus, the learning circuit of the seventh embodiment can obtain a prediction coefficient with higher accuracy, and the up-converter can obtain an HD interpolation pixel closer to the true value.

(8) Other Embodiments In the above-described embodiments, the case where the present invention is applied to an up-converter that converts SD image data into HD image data has been described. The present invention can be widely applied when converting high resolution image data to high resolution image data. That is, in the above-described embodiment, the present invention is applied when creating a pixel that is not included in the SD image. For example, each pixel expressed by 8 bits is expressed by 10 bits having a higher resolution. It can also be applied to. In this case, prediction coefficient data for each class is obtained in advance by learning using 10-bit pixels, and prediction calculation processing is performed using the prediction coefficient data corresponding to the result of class classification using 8-bit pixels. You can do that.

  The present invention can also be applied to an interpolation method for subsampled image signals, and can also be applied to an interpolation method for enlargement such as electronic zoom.

  In the above-described embodiment, a prediction coefficient storage unit in which prediction coefficient data is stored for each class code obtained by combining the first and second class codes is provided, and a prediction coefficient output from the prediction coefficient storage unit Although a case has been described in which image data with high resolution is generated according to a classification result by providing a prediction calculation unit that performs prediction calculation using data and input image data, the present invention is not limited to this, and prediction is performed. Instead of the coefficient storage unit and the prediction calculation unit, the present invention can also be applied to a case where a prediction value storage unit in which a prediction value is stored for each class code obtained by combining the first and second class codes is provided. Here, the predicted value stored in the predicted value storage unit corresponds to the estimated interpolation pixel value output from the prediction calculation unit. Such a method is proposed, for example, in the above-mentioned Japanese Patent Application Publication No. 5-328185 specification and drawings by the present applicant. The US application corresponding to this Japanese application is Serial No. 08 / 1,730 filed May 17, 1993.

  That is, in the up-converter shown in FIG. 1, the configuration of the class classification unit is the same, but instead of the prediction coefficient ROM 32, a prediction ROM in which a prediction value is stored is configured. The prediction value read by the class code is directly supplied to the conversion unit as the estimated interpolation pixel value.

  As a first method for obtaining such a predicted value, there is a learning method using a weighted average. More specifically, the above-described class classification is performed using the SD pixels around the interpolation target pixel, and the pixel values (using HD pixel values) of the interpolation target pixels integrated for each class are determined according to the number of interpolation target pixels. This is a method for obtaining a predicted value corresponding to each class by performing processing on various images when divided by the incremented frequency. As a second method for obtaining a predicted value, there is a learning method by normalization. More specifically, a block composed of a plurality of pixels including the interpolation target pixel is formed, and the value obtained by subtracting the reference value of the block from the pixel value of the interpolation target pixel is normalized by the dynamic range in the block. This is a method for obtaining a predicted value corresponding to each class by performing processing for various images on a value obtained by dividing the accumulated value of the normalized values by the accumulated frequency.

  In the first to fourth embodiments described above, the first class classification unit responds to the first class code (motion class code) formed according to the motion around the target pixel in the input image data. As described above, the second class classification unit adaptively selects the pixels used for class classification. In the first to fourth examples, as described above in the sixth example, The pixel used in the prediction calculation by the prediction calculation unit may be adaptively selected according to the first class code. Further, as described above in the seventh embodiment, both the pixel used for the class classification by the second class classification unit and the pixel used for the prediction calculation by the prediction calculation unit are adaptively set according to the first class code. You may make it select. In this way, image data with higher resolution can be generated.

  Further, in the above-described embodiment, ADRC is used as the second class classification unit that outputs the second class code based on the level distribution pattern of a plurality of pixels existing spatially and / or temporally around the target pixel. Although the case where a circuit is used has been described, the second class classification means of the present invention is not limited to this. For example, DCT (Discrete Cosine Transform) coding, DPCM (differential coding), vector quantization, subband coding Or a compression method such as wavelet conversion may be used.

  Further, in the above-described embodiment, as a class classification method of the first class classification unit that classifies the target pixel according to the movement around the target pixel in the input image data, a blotting method or an inter-frame difference value is used. A method of comparing with a predetermined threshold, a method of obtaining the direction of motion by detecting the minimum value of prediction errors among prediction values obtained corresponding to a plurality of motions, a simple blotting method, etc. Although the present invention is not limited to this, the present invention is not limited to this. For example, the motion class may be obtained by using a gradient method or a phase correlation method. Any type that can be classified into any of a predetermined number of classes based on the direction and magnitude of the movement is acceptable.

  In the above-described embodiments, the case where a ROM (read-only memory) is used as the prediction coefficient storage unit has been described. However, the present invention is not limited to this, and instead of this, a RAM (write-read memory), an SRAM, or the like May be used.

  Various modifications and application examples can be considered without departing from the gist of the present invention. Therefore, the gist of the present invention is not limited to the examples.

  The coefficient generation apparatus and the coefficient generation method of the present invention can be applied to an application for generating a coefficient for image signal conversion for converting, for example, a low resolution input image signal into a high resolution image signal.

It is a block diagram which shows the structure of the up converter by the Example of this invention. It is a block diagram which shows the structure of the learning circuit by an Example. It is a block diagram which shows the structure of the class classification | category part by 1st Example. It is a basic diagram which shows the example of a pattern classification | category at the time of forming the motion class in 1st Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 1st Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 1st Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 1st Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 1st Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 1st Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 1st Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 1st Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 1st Example. It is a block diagram which shows the structure of the class classification | category part by 2nd Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification selection part of 2nd Example. It is a block diagram which shows the structure of the class classification | category part by 3rd Example. It is a basic diagram which shows the example of a tap for class classification used when forming a motion class in 3rd Example. It is a basic diagram which shows the example of a pattern classification | category at the time of forming the motion class in 3rd Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 3rd Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 3rd Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 3rd Example. It is a basic diagram which shows the example of a prediction tap in 3rd Example. It is a block diagram which shows the structure of the class classification | category part by 4th Example. It is a basic diagram with which it uses for description of the movement class formation operation | movement in 4th Example. It is a block diagram which shows the structure of the up converter by 5th Example. It is a basic diagram which shows the positional relationship of SD pixel and HD pixel. It is a basic diagram which shows SD pixel used for a motion class classification | category in 5th Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 5th Example. It is a basic diagram which shows the tap pattern for class classification selected by the class classification pixel selection part of 5th Example. It is a basic diagram which shows the prediction tap pattern in 5th Example. It is a block diagram which shows the structure of the learning circuit by 5th Example. It is a block diagram which shows the structure of the up converter by 6th Example. It is a block diagram which shows the structure of the learning circuit by 6th Example. It is a basic diagram which shows the prediction tap pattern selected by the prediction calculation pixel selection part of 6th Example. It is a basic diagram which shows the prediction tap pattern selected by the prediction calculation pixel selection part of 6th Example. It is a basic diagram which shows the prediction tap pattern selected by the prediction calculation pixel selection part of 6th Example. It is a block diagram which shows the structure of the up converter by 7th Example. It is a block diagram which shows the structure of the learning circuit by 7th Example. It is a basic diagram which shows the example of spatial arrangement | positioning of SD pixel and HD pixel with which it uses for the conventional description. It is a block diagram which shows the structure of the conventional two-dimensional non-separable interpolation filter. It is a block diagram which shows the structure of the conventional horizontal / vertical separable interpolation filter. It is a block diagram which shows the structure of the up converter using a class classification adaptive process. It is a basic diagram which shows the example of arrangement | positioning of the tap for classification, and a prediction tap. It is a flow chart showing a prediction coefficient learning processing procedure. It is a block diagram which shows the structure of the learning circuit which calculates | requires a prediction coefficient.

Explanation of symbols

1, 5, 10, 30, 100, 140, 160... Upconverter, 11, 23, 31, 41, 60, 70, 90, 101, 141, 161... Class classification unit, 12, 84, 102, 142 , 162... Prediction calculation unit, 13, 32, 82, 99, 103, 143, 163... Prediction coefficient ROM, 20, 40, 120, 150, 170... Learning circuit, 24, 124, 151, 171. Coefficient calculation circuit, 50, 61, 71, 91... First class classification means, 52... Motion vector class detection section, 53, 64, 107... Class classification pixel selection means, 54A to 54H. , 55... Cluster top selection unit, 56, 68, 80, 108... ADRC class classification unit (second class classification means) 63... Motion detection unit, 72, 92. 110, prediction calculation pixel selection means, D1, input SD image data, D2, D32, CLASS10, CLASS12, class code, D3, prediction coefficient data, D4, D33, HD interpolation pixel data, D5 , D34 ... HD image data, CLASS0, CLASS2, CLASS4, CLASS6, CLASS8 '... Movement class code, CLASS1, CLASS3, CLASS5, CLASS7, CLASS9, CLASS11' ... ADRC class code.

Claims (5)

In a coefficient generation device that generates a coefficient for image signal conversion for converting a first input image signal into a second image signal different from the first input image signal,
A motion detector for detecting motion around the pixel of interest in the first learning image signal;
A classification pixel selection unit that selects a plurality of pixels in the first learning image signal based on the motion detected by the motion detection unit;
A class determination unit that determines a class of the pixel of interest based on a pattern of a plurality of pixels selected by the class classification pixel selection unit;
For each class, a plurality of pixels around the target pixel in the first learning image signal, and a second learning image corresponding to the first learning image signal and different from the first learning image signal A coefficient calculation unit that performs calculation using corresponding pixels of the image signal and calculates a coefficient for each class;
A coefficient generation apparatus comprising: a coefficient storage unit that stores the coefficient calculated by the coefficient calculation unit for each class. The class decision part
The coefficient generation apparatus according to claim 1, wherein the class is determined based on a motion from the motion detection unit. The first learning image signal is generated based on the second image signal,
The coefficient generation apparatus according to claim 1, wherein the first learning image signal is an image signal having a resolution lower than that of the second image signal. The first learning image signal is generated based on the second image signal,
The coefficient generation apparatus according to claim 1, wherein the first learning image signal is an image signal having a smaller number of pixels than the second image signal. In a coefficient generation method for generating a coefficient for image signal conversion for converting a first input image signal into a second image signal different from the first input image signal,
A motion detection step of detecting a motion around the target position in the first learning image signal;
A class classification pixel selection step of selecting a plurality of pixels in the first learning image signal based on the motion detected in the motion detection step;
A class determining step for determining a class of the target position based on a pattern of a plurality of pixels selected by the class-categorized pixel selection unit;
For each class, a plurality of pixels around the target position in the first learning image signal, and a second learning image corresponding to the first learning image signal and different from the first learning image signal A coefficient calculation step of performing calculation using corresponding pixels of the image signal and calculating a coefficient for each class;
And a coefficient storage step of storing the coefficient calculated in the coefficient calculation step for each class.

Images