JP2012248994A - Image information conversion apparatus, discrimination coefficient learning device, prediction coefficient learning device, processing method in them and program for allowing computer to perform processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve conversion accuracy of an image while an increase of a prediction coefficient is suppressed in an image information conversion apparatus for mutually converting images different in resolution.SOLUTION: A tap extraction section extracts a class tap and a prediction tap from a first image. A class classification section classifies the class tap into one of a plurality of classes. A discrimination coefficient selecting section selects a discrimination coefficient group corresponding to the classified class. A discrimination operation section discriminates a prediction tap type to which the prediction tap belongs on the basis of the selected discrimination coefficient group and pixel values of respective pixels in the prediction tap. A prediction coefficient selecting section selects a prediction coefficient group corresponding to combination of the classified class and the discriminated prediction tap type from the plurality of prediction coefficient groups. A prediction operation section performs a prediction operation in response to the selected prediction coefficient group and the pixel values of the respective pixels in the prediction tap.

Description

  The present technology relates to an image information conversion device, a discrimination coefficient learning device, a prediction coefficient learning device, a control method therefor, and a program that causes a computer to execute the method. Specifically, the present invention relates to an image information conversion apparatus that changes the resolution of an image, a control method thereof, and a program that causes a computer to execute the method. The present invention also relates to a discrimination coefficient learning device and a prediction coefficient learning device that learn a discrimination coefficient and a prediction coefficient to be used in resolution change, a control method in these, and a program that causes a computer to execute the method.

  Conventionally, an image information conversion apparatus classifies pixel groups in an image to be converted into a plurality of classes, and executes a class classification adaptive process for obtaining a pixel value to be converted by a different calculation for each class. Different images were converted to each other (for example, see Patent Document 1). Here, the class is each group into which the pixel group is classified based on the shape pattern of the three-dimensional distribution of each pixel value in the pixel group including the target pixel of interest in the conversion target image. Here, with respect to the shape pattern of the three-dimensional distribution, the time to which each of the two fields constituting one frame belongs is a time axis, the position of each pixel in the pixel group in each field is the horizontal axis, and the size of the pixel value Is assumed to be a three-dimensional coordinate system. In the three-dimensional distribution diagram in which each pixel is plotted in this coordinate system, the shape formed by each plot is the shape pattern of the three-dimensional distribution of each pixel value.

In this class classification adaptive processing, a predetermined calculation is performed using each pixel value in a predetermined number (for example, 13) of pixel groups including the target pixel in the image to be converted, so that the converted image Pixel values are calculated (predicted). Hereinafter, this calculation is referred to as a prediction calculation, and this pixel group is referred to as a “prediction tap”. The shape pattern of the three-dimensional distribution of the region including the target pixel is the target of class classification. In order to determine which class the shape pattern of the region including the target pixel belongs to, a predetermined pixel group including the target pixel is extracted as a class tap. Here, for example, when each pixel value is quantized with 8 bits, the shape pattern of the class tap is (2 ⁸ ) ¹³ patterns, which is a huge number. Therefore, in general, grouping is performed on these shape patterns as a reduction of the enormous number of groups. Grouping of three-dimensional distribution shape patterns is often performed by reducing the number of quantization bits assigned to the number of gradations of each pixel value by a compression method such as ADRC.

  When the class tap is classified into a class, the image information conversion apparatus predicts a pixel value in the image to be converted using a different prediction coefficient group for each class. Specifically, the image information conversion apparatus outputs a product sum of a prediction coefficient group corresponding to a class and each pixel value in the prediction tap as a predicted pixel value. As a result, the image to be converted is converted into an image having a resolution different from that of the image. These prediction coefficient groups are set in advance for each class before image conversion.

  Here, in order to set the prediction coefficient for each class in the image information conversion apparatus before image conversion, it is necessary to learn and obtain the prediction coefficient by a learning apparatus in advance. Specifically, the learning apparatus uses a certain resolution image as a teacher image, and generates a student image that is an image obtained by changing the resolution of the teacher image. The learning device calculates a prediction coefficient for each class so that an error between the predicted value predicted from the pixel value of the student image and the pixel value of the teacher image is minimized.

JP-A-7-79418

  However, in the above-described conventional technology, when an image information conversion apparatus converts an input image using a prediction coefficient, the image may be deteriorated. Specifically, the converted image may fail or may lack a high resolution feeling. This is because the characteristics of the input image have been diversified and the conversion accuracy of the image has been lowered because there is a characteristic difference between the input image and the image used in the environment in which the coefficient to be created in advance is created. Possible reasons for the diversification of input images include the development of editing technology and the diversification of formats, and the spread of digital broadcasting and Blu-ray Disc (Blu-ray Disc (registered trademark)). If the prediction coefficient for each shape pattern considering all the characteristics of various input images is set without reducing the number of prediction coefficients, the conversion accuracy is improved. However, in this configuration, since the number of shape patterns is enormous, a large number of prediction coefficients are required. For this reason, there is a problem that it is difficult to improve the conversion accuracy of the image.

  The present technology has been devised in view of such a situation, and in an image information conversion apparatus that mutually converts images having different resolutions, it is possible to improve the conversion accuracy of an image while suppressing an increase in a prediction coefficient. Objective.

  The present technology has been made to solve the above-described problems, and a first aspect of the present technology is a first pixel group including any pixel of interest in the first image as a pixel of interest. A tap extraction unit that extracts a class tap and a prediction tap that is a second pixel group including the pixel of interest from the first image, and a shape of a three-dimensional distribution of pixel values of each pixel in the class tap A class classification unit for classifying a pattern into one of a plurality of classes, and the prediction tap type as a coefficient group to be used in a discrimination calculation to determine which of the plurality of prediction tap types the prediction tap belongs to For each of the plurality of prediction tap types, the discrimination coefficient group corresponding to the classified class is selected from a plurality of discrimination coefficient groups set in association with the plurality of classes for each A discriminant calculation for discriminating the prediction tap type to which the prediction tap belongs from the result of executing the discrimination calculation based on the discrimination coefficient selection unit, the selected discrimination coefficient group, and the pixel value of each pixel in the prediction tap And a plurality of classes and a plurality of prediction tap types as coefficient groups to be used in a prediction calculation for predicting a pixel value of a pixel in a second image that is an image obtained by changing the resolution of the first image A prediction coefficient selection unit that selects the prediction coefficient group corresponding to the combination of the classified class and the determined prediction tap type from among a plurality of prediction coefficient groups set in association with each combination of An image information conversion apparatus including a prediction calculation unit that performs the prediction calculation based on the selected prediction coefficient group and a pixel value of each pixel in the prediction tap, and a control method thereof, Is a program for causing a computer to execute the method in rabbi. This brings about the effect that the prediction calculation is executed based on the prediction coefficient group corresponding to the combination of the classified class and the determined prediction tap type and each pixel value in the prediction tap.

  Also, in this first aspect, the class classification unit reduces the number of gradations of the pixel value of each pixel in the class tap and reduces the number of gradations to the shape pattern based on the pixel value. You may classify | categorize into either of several classes. Accordingly, there is an effect that the shape pattern is classified into one of a plurality of classes based on the pixel value obtained by reducing the number of gradations.

  In the first aspect, the discrimination calculation unit performs a product-sum operation on each discrimination coefficient in the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap as the discrimination calculation. A discrimination calculation execution unit, a comparison unit that compares and compares the result of the product-sum calculation in each of the plurality of prediction tap types with a predetermined threshold, and outputs a comparison result. And a prediction tap type determination unit that determines the prediction tap type to which the prediction tap belongs. This brings about the effect that the prediction tap type is determined based on the result of comparing the product-sum operation of each determination coefficient and each pixel value in the prediction tap with the threshold value.

  The first aspect further includes a recording unit that records an operation signal for adjusting the image quality of the second image, and the prediction tap type determination unit is configured to perform the image quality performed by the recorded operation signal. One of the plurality of prediction taps may be selected with priority given to the prediction tap type to which the prediction coefficient group suitable for the adjustment belongs. This brings about the effect that the prediction tap type to which the prediction coefficient group suitable for the second image to be corrected belongs is selected with priority.

  In the first aspect, the image processing apparatus further includes a viewing environment detection unit that detects a viewing environment parameter indicating the viewing environment of the second image, and the prediction tap type determination unit includes the viewing environment parameter indicated by the viewing environment parameter. One of the plurality of prediction tap types may be selected with priority given to the prediction tap type to which the prediction coefficient group suitable for the environment belongs. This brings about the effect that the prediction tap type to which the prediction coefficient group suitable for the viewing environment belongs is preferentially selected.

  In the first aspect, the viewing environment parameter may include illuminance of illumination in a room in which the display device that displays the second image is installed. This brings about the effect that the predicted tap type is discriminated in preference to the predicted tap type suitable for the illuminance in the room.

  In the first aspect, the viewing environment parameter may include a type of a display device that displays the second image. This brings about the effect that the predicted tap type is discriminated in preference to the predicted tap type suitable for the type of the display device.

  The first aspect may further include an average value calculation unit that calculates an average value of pixel values of each pixel in the prediction tap, and the prediction tap type determination unit includes the average value outside a predetermined range. In this case, the prediction tap type to which the prediction coefficient group suitable for the average value belongs may be preferentially selected and any one of the plurality of prediction tap types may be selected. This brings about the effect that the prediction tap type to which the prediction coefficient group suitable for the average value belongs is preferentially selected.

  In addition, a second aspect of the present technology provides a student image input unit that inputs a plurality of student images including a plurality of student images, each of a plurality of different images as student images, A tap extracting unit that extracts a class tap that is a first pixel group including the target pixel and a prediction tap that is a second pixel group including the target pixel from any of the images, with any pixel as the target pixel; A class classification unit that classifies a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into one of a plurality of classes, and the student image in the student image group as a discrimination image A target value generation unit that generates a first target value when a discrimination image is input and generates a second target value when the student image other than the discrimination image is input, and changes the resolution of the image Place to do The discriminant coefficient group to be used in the discriminant calculation for discriminating whether or not the image to be changed is similar to the learning image in FIG. 6 is the first and second target values and the pixel value of each pixel in the prediction tap. A discrimination coefficient learning device including a discrimination coefficient calculation unit that calculates for each class based on the above, a control method thereof, and a program for causing a computer to execute the method. As a result, the discriminant coefficient is calculated for each class in a plurality of student images.

  In the second aspect, the student image input unit includes a filter unit that generates the student image group by passing a teacher image, which is an image in which the resolution of the student image is changed, through a plurality of filters; A student image supply unit that inputs a plurality of generated student image groups a plurality of times may be provided. This brings about the effect that a student image group is generated by a plurality of filters.

  In the second aspect, the discrimination calculation is a product-sum calculation of each discrimination coefficient in the discrimination coefficient group and each pixel value in the prediction tap in the change target image, and the discrimination coefficient The calculation unit adds the result of the discrimination calculation in the student image other than the learning image and the second target value to the square sum of the difference between the result of the discrimination calculation in the learning image and the first target value. The discriminant coefficient group that minimizes the value obtained by adding the sum of squares of the difference between the two can be calculated for each of the plurality of classes. As a result, a group of discriminant coefficients that minimizes a value obtained by adding the square sum of the difference between the result of the discrimination calculation and the second target value to the sum of squares of the difference between the result of the discrimination calculation and the first target value is calculated. It brings about the effect of being.

  Further, the third aspect of the present technology includes a student image input unit that inputs each of a plurality of different images as a student image, and includes the pixel of interest with any pixel in the input student image as a pixel of interest. A tap extraction unit that extracts from the first image a class tap that is a first pixel group and a prediction tap that is a second pixel group including the pixel of interest; and a pixel value of each pixel in the class tap A class classification unit that classifies the shape pattern of the three-dimensional distribution into one of a plurality of classes, and one or more pixels in the teacher image that is an image in which the resolution of the input student image is changed is used as a teacher pixel A teacher pixel extraction unit to extract, and a prediction coefficient group to be used in a prediction calculation for predicting a pixel value of the teacher pixel based on the pixel value of each pixel in the prediction tap and the pixel value of the teacher pixel Prediction coefficient and a prediction coefficient calculator for calculating for each class learning apparatus, and a program for executing the control method and the method in a computer. This brings about the effect that the prediction coefficient is calculated for each class in a plurality of student images.

  In the third aspect, the student image input unit may generate and input the plurality of student images by passing the teacher image through a plurality of filters. This brings about the effect that a student image group is generated by a plurality of filters.

  According to the present technology, in an image information conversion apparatus that mutually converts images having different resolutions, an increase in prediction coefficient is suppressed, and an excellent effect of improving image conversion accuracy can be achieved.

It is a block diagram which shows the example of 1 structure of the image information conversion system in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the image information conversion apparatus in 1st Embodiment. It is a figure which shows an example of the positional relationship between the pixel of HD (High Definition) signal and the pixel of SD (Standard Definition) signal in 1st Embodiment. It is a figure which shows an example of the pixel group which comprises the prediction tap and class tap in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the discrimination coefficient selection part in 1st Embodiment. It is a figure which shows an example of the discrimination coefficient memorize | stored in the discrimination coefficient memory | storage part in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the discrimination | determination calculating part in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the prediction coefficient selection part in 1st Embodiment. It is a figure which shows an example of the prediction coefficient memorize | stored in the prediction coefficient memory | storage part in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the discrimination coefficient learning apparatus in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the filter part of the discrimination coefficient learning apparatus in 1st Embodiment. It is a figure which shows an example of the learning method of the discrimination coefficient in 1st Embodiment. It is a figure which shows the example of a setting of the target value in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the discrimination coefficient calculating part in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the normal equation production | generation part in 1st Embodiment. It is a figure which shows an example of the statistics of the matrix component in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the prediction coefficient learning apparatus in 1st Embodiment. It is a block diagram which shows the example of 1 structure of the filter part of the prediction coefficient learning apparatus in 1st Embodiment. It is a figure which shows an example of the learning method of the prediction coefficient in 1st Embodiment. It is a flowchart which shows an example of operation | movement of the image information converter in 1st Embodiment. It is a flowchart which shows an example of the class classification | category process in 1st Embodiment. It is a flowchart which shows an example of the prediction tap classification discrimination | determination process in 1st Embodiment. It is a flowchart which shows an example of operation | movement of the discrimination coefficient learning apparatus in 1st Embodiment. It is a flowchart which shows an example of operation | movement of the prediction coefficient learning apparatus in 1st Embodiment. It is a figure which shows an example of the learning method of the discrimination coefficient in a modification. It is a block diagram which shows one structural example of the image information conversion apparatus in 2nd Embodiment. It is a figure which shows an example of the candidate determination result row | line in 2nd Embodiment. It is a figure which shows the example of a setting of the shift amount in 2nd Embodiment. It is a block diagram which shows the example of 1 structure of the operation signal log | history information recording part in 2nd Embodiment. It is a block diagram which shows the example of 1 structure of the viewing-and-listening environment detection part in 2nd Embodiment. It is a block diagram which shows the example of 1 structure of the discrimination | determination calculating part in 2nd Embodiment. It is a flowchart which shows an example of the prediction tap classification discrimination | determination process in 2nd Embodiment. It is a block diagram which shows the example of 1 structure of the image information converter in 3rd Embodiment. It is a block diagram which shows the example of 1 structure of the average waveform calculating part in 3rd Embodiment.

Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order.
1. First embodiment (image information conversion process: an example in which a prediction coefficient group associated with a combination of a class and a prediction tap type is used)
2. Second Embodiment (Image Information Conversion Process: Example of Shifting Candidate Determination Result Sequence)
3. Third embodiment (image information conversion process: example of calculating an average waveform)

<1. First Embodiment>
[Configuration example of image information conversion system]
FIG. 1 is a block diagram illustrating a configuration example of an image information conversion system according to the first embodiment. This image information conversion system includes an image information conversion device 100, a discrimination coefficient learning device 200, and a prediction coefficient learning device 300.

  Each time an image signal is input, the image information conversion apparatus 100 converts the image signal into an image signal of an image having a resolution different from that of the image indicated by the image signal, and outputs the image signal. For example, the image information conversion apparatus 100 up-converts an image into a higher resolution image. Further, the image information conversion apparatus 100 uses a discrimination coefficient and a prediction coefficient, which will be described later, in the conversion of the image signal. The image information conversion apparatus 100 receives an image signal indicating each image constituting the moving image. The image signal is a signal having data for specifying the pixel value and coordinates of each pixel in the image. Hereinafter, an image signal indicating a low resolution image is referred to as a low resolution image signal, and an image signal indicating a high resolution image is referred to as a high resolution image signal. The low resolution image signal is, for example, an SD (Standard Definition) signal, and the high resolution image signal is, for example, an HD (High Definition) signal.

  The discrimination coefficient learning device 200 learns the discrimination coefficient based on the high resolution image signal and the discrimination image designation signal. The discrimination image designation signal will be described later. The discrimination coefficient learning device 200 outputs the learned discrimination coefficient to the image information conversion device 100 via the signal line 209.

  The prediction coefficient learning device 300 learns a prediction coefficient based on a high resolution image signal and a learning image designation signal. The learning image designation signal will be described later. The prediction coefficient learning apparatus 300 outputs the learned prediction coefficient to the image information conversion apparatus 100 via the signal line 309.

[Configuration example of image information conversion apparatus]
FIG. 2 is a block diagram illustrating a configuration example of the image information conversion apparatus 100. The image information conversion apparatus 100 includes a tap extraction unit 110, a class classification unit 120, a discrimination coefficient selection unit 130, a discrimination calculation unit 140, a prediction coefficient selection unit 150, and a prediction calculation unit 160.

  The tap extraction unit 110 extracts a class tap and a prediction tap from the low resolution image signal. The tap extraction unit 110 includes a prediction tap extraction unit 111 and a class tap extraction unit 112. A low resolution image signal is input to the prediction tap extraction unit 111 and the class tap extraction unit 112.

  The prediction tap extraction unit 111 extracts a prediction tap from the input image signal. The prediction tap is a predetermined number (for example, 13) of pixel groups extracted from the low resolution image in order to predict the pixel value of the high resolution image. The prediction tap is extracted for each target pixel with each pixel in the low resolution image as the target pixel. The pixel of interest is a pixel that serves as a reference in the prediction tap (for example, the center pixel). When the resolution of the low-resolution image is 720 × 480 pixels, 720 × 480 prediction taps with each pixel as the target pixel are extracted. Each time the prediction tap extraction unit 111 extracts a prediction tap, the prediction tap extraction unit 111 outputs the pixel value of each pixel in the extracted prediction tap to the discrimination calculation unit 140 and the prediction calculation unit 160 via the signal line 118.

  The class tap extraction unit 112 extracts a class tap from the low resolution image signal. The class tap is a predetermined number (for example, 9) of pixel groups including the target pixel in the low resolution image for class classification. The class tap is also extracted for each pixel in the image, like the prediction tap. Each time the class tap extraction unit 112 extracts a class tap, the class tap extraction unit 112 outputs the pixel value of each pixel in the extracted class tap to the class classification unit 120 via the signal line 119.

  The class classification unit 120 classifies the shape pattern of the three-dimensional distribution of pixel values of each pixel in the class tap into one of a plurality of classes. The class classification unit 120 includes a requantization unit 121 and a class code determination unit 122.

The requantization unit 121 requantizes each pixel so that the number of gradations of the pixel value of each pixel in the class tap decreases. Specifically, the pixel value of each pixel in the class tap is requantized using ADRC (Adaptive Dynamic Range Coding). In ADRC, the requantization unit 121 calculates the difference between the maximum value MAX and the minimum value MIN among the pixel values of each pixel in the class tap as the dynamic range DR. The requantization unit 121 divides a value obtained by subtracting the minimum value MIN from the pixel value of each pixel in the class tap by DR / 2 ^m (m is an integer of 1 or more). The requantization unit 121 outputs data obtained by arranging the division results for each pixel in a predetermined order based on the pixel position to the class code determination unit 122 as an ADRC code. For example, the division results for each pixel are arranged in ascending order of the total value of the x and y coordinates of the pixel. By setting the number of bits different from the number of bits representing the pixel value before conversion to m and executing the above-described ADRC, the number of gradations of the pixel value is changed. In this way, the process of changing the number of gradations for a pixel whose pixel value is represented (quantized) by a predetermined number of gradations is called requantization. The number of gradations of each pixel value in the class tap is reduced by performing re-quantization by setting m to a bit number (for example, 1) smaller than the number of bits (for example, 8) representing the pixel value before conversion. Is done. When class taps in which the same ADRC code is generated by reducing the number of gradations are classified into the same class, the class tap is classified into a plurality of classes indicated by each ADRC code. For example, when the number of gradations of each of the nine pixels in the class tap is reduced to 2 gradations, the class tap is classified into 2 ⁹ = 512 classes.

  The class code determination unit 122 determines a class code based on the ADRC code. The class code is information for identifying a class into which class taps are classified. The ADRC code and the class code are associated one to one. For example, the class code determination unit 122 stores a class code for each ADRC code, reads a class code corresponding to the input ADRC code, and determines the class code as a class tap class code. The class code determination unit 122 outputs the determined class code to the discrimination coefficient selection unit 130 and the prediction coefficient selection unit 150 via the signal line 129.

  The discrimination coefficient selection unit 130 selects a discrimination coefficient group from a plurality of discrimination coefficient groups based on the class code. One discrimination coefficient group includes the same number of discrimination coefficients as the number of pixels in the prediction tap (for example, 13). These discrimination coefficients are coefficients used in the discrimination calculation for discriminating the prediction tap type to which the extracted prediction tap should belong. Two or more prediction tap types are defined, and each prediction tap belongs to one of these prediction tap types.

  Here, the prediction tap type is a group in which a plurality of extracted prediction taps are classified according to a standard different from the class classification. Specifically, the prediction tap is classified into a plurality of prediction tap types based on the image type of the image from which the prediction tap is extracted when learning the prediction coefficient. For example, each prediction tap is classified into a plurality of prediction tap types according to the compression rate of the image from which the prediction tap is extracted at the time of learning.

  In the discrimination coefficient selection unit 130, a plurality of discrimination coefficient groups are set in advance in association with combinations of a plurality of classes and N (N is an integer of 2 or more) prediction tap types. For example, when the number of classes is 512, 512 × N discrimination coefficient groups are set. The discrimination coefficient selection unit 130 selects N discrimination coefficient groups corresponding to the class indicated by the class code from the 512 × N discrimination coefficient groups. The discrimination coefficient selection unit 130 outputs each selected discrimination coefficient group to the discrimination calculation unit 140 via the signal line 139.

  The discrimination computation unit 140 discriminates which of the N prediction tap types belongs to the prediction tap by executing a discrimination computation based on the prediction tap and the discrimination coefficient group. Details of the discrimination calculation will be described later. The discrimination calculation unit 140 generates a type code indicating the determined prediction tap type and outputs the type code to the prediction coefficient selection unit 150 via the signal line 149.

  The prediction coefficient selection unit 150 selects one of a plurality of prediction coefficient groups based on the class code and the type code. In the prediction coefficient selection unit 150, a plurality of prediction coefficient groups are set in advance in association with combinations of a plurality of classes and N prediction tap types. For example, when there are 512 classes, 512 × N prediction coefficient groups are set. When converting a low-resolution image signal into a high-resolution image signal, it is necessary to generate a plurality of (for example, four) high-resolution image signals from one low-resolution image signal. Therefore, each prediction coefficient group includes a set of a plurality of prediction coefficients as many as the number of high resolution image signals to be generated. Each set includes the same number of prediction coefficients as the number of pixels in the prediction tap (for example, 13). For example, when the number of pixels in the prediction tap is 13 and four high-resolution image signals are generated from one low-resolution signal, each prediction coefficient group includes four sets of 13 prediction coefficients. These prediction coefficients are coefficients to be used in the prediction calculation for predicting the pixel value of the pixel in the high resolution image. The prediction coefficient selection unit 150 outputs a prediction coefficient group corresponding to the combination of the prediction tap type indicated by the type code and the class indicated by the class code to the prediction calculation unit 160 via the signal line 159.

式１においてｗ_ｎ（ｎは１乃至１３の整数）は、予測係数群内の１つの高解像度画像信号に対応するセットにおける各予測係数である。ｘ_ｎは、予測タップ内の各画素の画素値である。予測演算部１６０は、予測演算の結果であるｙを、高解像度画像信号の画素値として出力する。１つの低解像度信号から４つの高解像度画像信号を生成する場合、予測演算部１６０は、１３個の予測係数からなるセットを４つ使用して４つの画素値を求める。 The prediction calculation unit 160 executes prediction calculation for predicting the pixel value of the high-resolution image based on the prediction tap and the prediction coefficient group. This prediction calculation is executed by, for example, the following formula 1 which is a linear linear combination formula.

In Equation 1, w _n (n is an integer of 1 to 13) is each prediction coefficient in the set corresponding to one high-resolution image signal in the prediction coefficient group. _xn is a pixel value of each pixel in the prediction tap. The prediction calculation unit 160 outputs y that is a result of the prediction calculation as a pixel value of the high-resolution image signal. When generating four high resolution image signals from one low resolution signal, the prediction calculation unit 160 obtains four pixel values using four sets of 13 prediction coefficients.

  Here, the relationship between the SD signal to be converted by the image information conversion apparatus 100 and the HD signal after conversion will be described. FIG. 3 is a diagram illustrating an example of a positional relationship between a pixel of the HD signal and a pixel of the SD signal according to the first embodiment. In FIG. 3, a circular mark is a pixel in the SD signal, and a diamond mark is a pixel in the HD signal. A shaded mark is a pixel in the first field, and a white mark is a pixel in the second field. Here, the first field and the second field are adjacent fields on the time axis, and an image of one frame is configured by these fields.

  FIG. 3A is a diagram illustrating an example of the positional relationship of each pixel in the vertical direction of the image. The horizontal axis of Fig.3 (a) is a time axis, and a vertical axis | shaft is a coordinate of the vertical direction of a screen. As illustrated in FIG. 3A, SD signal pixels are located between HD signal pixels adjacent in the vertical direction of each field. More specifically, when the distance between the pixels of the HD signal adjacent in the vertical direction is d, for example, at a position where the distance between the upper HD signal pixel and the SD signal is 1/4 × d. The pixel of the SD signal is located.

  FIG. 3B is a diagram illustrating an example of the positional relationship of each pixel in the horizontal direction of the image. The image in FIG. 3B is one frame configured by the first field and the second field. In FIG. 3B, the vertical axis is the vertical coordinate of the image, and the horizontal axis is the horizontal coordinate of the image. As illustrated in FIG. 3B, in the horizontal direction of the frame, an SD signal pixel is arranged in the middle between adjacent HD signal pixels.

  FIG. 4 is an example of a prediction tap and a class tap in the first embodiment. In FIG. 4, a circular mark is a pixel in the SD signal, and a diamond mark is a pixel in the HD signal. A shaded mark is a pixel in the first field, and a white mark is a pixel in the second field. The first field and the second field are adjacent fields on the time axis, and an image of one frame is constituted by these fields.

  Further, in FIG. 4, the crosses in the left first field indicate the position of the target pixel. The target pixel is a pixel serving as a reference in the class tap and the prediction tap. For example, the pixel at the center of the class tap and the prediction tap is set as the target pixel. The crosses in the second field on the right side indicate positions corresponding to the target pixel, that is, the same coordinates as the coordinates of the target image. Here, as described above, since the first field and the second field are fields constituting one frame, lines on which pixels are arranged are different in each field. For example, when pixels are arranged on odd lines in the first field, pixels are arranged on even lines in the second field. For this reason, the line including the x mark in the second field is not a line where the pixel is arranged, and actually, no pixel is arranged at the position of the x mark in the second field.

  FIG. 4A is a diagram illustrating an example of the extracted prediction tap. For example, 13 pixels p1 to p13 are extracted as prediction taps. The pixel p4 is a target pixel. Pixels p1 and p7 are pixels located above and below the pixel p4. The pixel p3 is a pixel located on the left side of the pixel p4, and the pixel p5 is a pixel located on the right side of the pixel p4. The pixel p2 is a pixel located on the left side of the pixel p3, and the pixel p6 is a pixel located on the right side of the pixel p5. Pixels p8 and p9 are pixels located above and below the position corresponding to the target pixel in the second field on the right side. The pixels p10 and p11 are pixels located on the left and right of the pixel p8, and the pixels p12 and p13 are pixels located on the left and right of the pixel p9.

  FIG. 4B is a diagram illustrating an example of extracted class taps. For example, nine pixels p1 to p9 are extracted as prediction taps. The pixels p1 to p9 in the class tap are pixels having the same coordinates as the pixels p1 to p9 in the prediction tap.

  FIG. 5 is a block diagram illustrating a configuration example of the discrimination coefficient selection unit 130 according to the first embodiment. The discrimination coefficient selection unit 130 includes a discrimination coefficient storage unit 131 and N discrimination coefficient reading units 133 that are associated with N prediction tap types.

  The discrimination coefficient storage unit 131 stores a plurality of discrimination coefficient groups in association with combinations of a plurality of classes and N prediction tap types. The discrimination coefficient storage unit 131 includes N discrimination coefficient memories 132 corresponding to the N discrimination coefficient reading units 133. The N discrimination coefficient memories 132 store discrimination coefficient groups for each class.

  The discrimination coefficient reading unit 133 reads a discrimination coefficient group corresponding to the class code. Each of the N discrimination coefficient reading units 133 receives the class code from the class classification unit 120. Each of the N discrimination coefficient reading units 133 reads out a discrimination coefficient group corresponding to the class indicated by the class code from the corresponding discrimination coefficient memory 132. As a result, N discrimination coefficient groups are read out. Each of the N discrimination coefficient reading units 133 outputs these discrimination coefficient groups to the discrimination calculation unit 140 via the signal line 139.

FIG. 6 is a diagram illustrating an example of a discrimination coefficient group stored in the discrimination coefficient storage unit 131. The N discriminant coefficient memories 132 are associated with the N prediction tap types on a one-to-one basis. The prediction tap type is classified according to the compression rate of the image from which the prediction tap is extracted when learning the prediction coefficient. For example, the discrimination coefficient memory # 1 is associated with the prediction tap type to which each prediction tap extracted from the image with the highest compression rate belongs, and each of the discrimination coefficient memories # 2 to #N is assigned in descending order of the compression rate. The prediction tap type is associated. In each discrimination coefficient memory, a discrimination coefficient group including discrimination coefficients v _{1 to} v ₁₃ is stored in association with 512 classes of class codes “000000000000” to “111111111”.

  FIG. 7 is a block diagram illustrating a configuration example of the discrimination calculation unit 140 according to the first embodiment. The discrimination coefficient calculation unit includes N discrimination calculators 142, N threshold processing units 143, candidate determination result string generation units 144, and prediction tap type determination units 146. The N discriminant calculators 142 are associated with the N discriminant coefficient reading units 133 on a one-to-one basis, and the N discriminant calculators 142 are associated with the N threshold processing units 143 on a one-to-one basis. ing.

Each of the N discrimination calculators 142 executes a discrimination calculation for determining whether or not a prediction tap belongs to the associated prediction tap type. Each of the N discrimination calculators 142 receives the pixel value of each pixel in the prediction tap extracted by the tap extraction unit 110 and the discrimination coefficient group from the corresponding discrimination coefficient reading unit 133. Each of the N discriminating calculators 142 uses these values to execute a discriminant calculation. This discrimination operation is executed by the following equation which is a linear linear combination equation, for example.

In the above equation, v _n (n is an integer of 1 to 13) is each discrimination coefficient in the discrimination coefficient group. _xn is a pixel value of each pixel in the prediction tap. The discrimination calculator 142 outputs Y, which is the result of the discrimination calculation, to the corresponding threshold value processing unit 143.

  Each of the N threshold value processing units 143 compares the result of the discrimination operation with a threshold value. Each of the N threshold value processing units 143 compares a predetermined same threshold value with the calculation result from the corresponding discrimination calculator 142. An example of setting the threshold will be described later. The comparison result indicates whether or not the prediction tap can belong to the prediction tap type corresponding to the threshold processing unit 143. If it can belong, the prediction tap type is a candidate for the prediction tap type to be finally determined as one. This comparison result is hereinafter referred to as a candidate determination result. For example, when the calculation result is equal to or greater than the threshold value, there is a high possibility that the prediction tap belongs to the corresponding prediction tap type, and the prediction tap type corresponds to a candidate. On the other hand, when the calculation result is less than the threshold value, it is unlikely that the prediction tap belongs to the corresponding prediction tap type, so the prediction tap type does not correspond to a candidate. Each of the N threshold processing units 143 generates a bit indicating a candidate determination result for the corresponding prediction tap type. For example, each of the N threshold processing units 143 generates a bit indicating “1” when the calculation result is equal to or greater than the threshold (that is, corresponds to a candidate), and the calculation result is less than the threshold (that is, , A bit indicating “0” is generated. Each of the N threshold processing units 143 outputs the generated 1 bit to the candidate determination result sequence generation unit 144.

  The candidate determination result sequence generation unit 144 generates a candidate determination result sequence from the N candidate determination results from the threshold processing unit 143. The candidate determination result sequence is a bit sequence in which N bits indicating N candidate determination results are arranged in a predetermined order. The candidate determination result sequence generation unit 144 arranges the candidate determination results based on the image type of the image from which the prediction coefficient group is extracted. For example, when each prediction tap is classified into a plurality of prediction tap types according to the compression rate of each image, the candidate determination result sequence generation unit 144 arranges the candidate determination results in descending order of the compression rate. The candidate determination result sequence generation unit 144 outputs the generated candidate determination result sequence to the prediction tap type determination unit 146.

  The prediction tap type determination unit 146 determines the prediction tap type to which the prediction tap type belongs based on the candidate determination result sequence. Specifically, the prediction tap type determination unit 146 selects one prediction tap type from the prediction tap type candidates indicated by the candidate determination result sequence. Priorities are set in advance for the N prediction tap types. When a plurality of prediction tap types become candidates, the prediction tap type determination unit 146 selects the prediction tap type with the highest priority. For example, the higher the compression rate of the image from which the prediction tap is extracted, the higher the priority of the prediction tap type related to that image. When candidate determination results are arranged in descending order of compression rate, higher priority is set in order from the top. The prediction tap type determination unit 146 outputs a type code indicating the selected prediction tap type to the prediction coefficient selection unit 150 via the signal line 149.

  FIG. 8 is a block diagram illustrating a configuration example of the prediction coefficient selection unit 150 according to the first embodiment. The prediction coefficient selection unit 150 includes a prediction coefficient storage unit 151, N prediction coefficient reading units 153, and a selector 154.

  The prediction coefficient storage unit 151 stores a plurality of prediction coefficient groups in association with combinations of a plurality of classes and N prediction tap types. The prediction coefficient storage unit 151 includes N prediction coefficient memories 152. N prediction tap types are assigned to the N prediction coefficient memories 152. Each of the N prediction coefficient memories 152 stores a prediction coefficient group corresponding to the assigned prediction tap type for each class.

  The prediction coefficient reading unit 153 reads a prediction coefficient group corresponding to the class indicated by the class code. The N prediction coefficient reading units 153 are associated with the N prediction coefficient memories 152. Each of the N prediction coefficient reading units 153 receives a class code from the class classification unit 120. Each of the N prediction coefficient reading units 153 reads a prediction coefficient group corresponding to the class indicated by the class code from the corresponding prediction coefficient memory 152. Each of the N prediction coefficient reading units 153 outputs the read prediction coefficient group to the selector 154.

  The selector 154 selects a prediction coefficient group corresponding to the prediction tap type indicated by the type code from the N prediction coefficient groups. Specifically, the selector 154 receives N prediction coefficient groups from the N prediction coefficient reading units 153 and receives a type code from the discrimination calculation unit 140. The selector 154 selects a prediction coefficient group corresponding to the prediction tap type indicated by the type code from the N prediction coefficient groups. The selector 154 outputs the selected prediction coefficient group to the prediction calculation unit 160 via the signal line 159.

  FIG. 9 is a diagram illustrating an example of a prediction coefficient group stored in the prediction coefficient storage unit 151. Each of the N prediction coefficient memories 152 stores prediction coefficient groups corresponding to N prediction tap types for the number of classes. For example, the prediction coefficient memory # 1 stores a prediction coefficient group corresponding to the prediction tap type to which each prediction tap in the highest compression image belongs, and the prediction coefficient memories # 2 to #N are arranged in descending order of the image compression rate. In addition, a prediction coefficient group corresponding to each of the N prediction tap types is stored. In each of the N prediction coefficient memories, a prediction coefficient group is stored in association with each of 512 classes of class codes “000000000000” to “111111111”. Each prediction coefficient group includes a set of prediction coefficients of the same number as the number of pixels in the prediction tap, as many as the number of pixel values to be predicted. For example, when the number of pixels in the prediction tap is 13, and each pixel value of four pixels at the upper left, lower left, upper right, and lower right positions is predicted, each prediction coefficient group includes 13 prediction coefficients. Contains four sets of

[Configuration example of discrimination coefficient learning device]
FIG. 10 is a block diagram illustrating a configuration example of the discrimination coefficient learning device 200 according to the first embodiment. The discrimination coefficient learning device 200 includes a filter unit 210, a target value supply unit 220, a tap extraction unit 230, a class classification unit 240, and a discrimination coefficient calculation unit 250.

  The filter unit 210 generates N different student images from a high-resolution image signal input as a teacher image. The teacher image is an image used for generating a student image. The student image is an image used for learning a discrimination coefficient and a prediction coefficient. A high resolution image signal input as a teacher image is input to the filter unit 210. The filter unit 210 generates N different student images by passing the teacher image through N filters that change the resolution. These student images are, for example, images having the same resolution and different compression rates. The filter unit 210 outputs a low-resolution image signal as a student image group including N student images to the tap extraction unit 230 N times via the signal line 219.

  The target value supply unit 220 supplies the target value Y according to the discrimination image designation signal. The discrimination image designation signal is a signal for designating one of N student images as a discrimination target student image. A student image to be determined is an image that should be determined to be an image similar to an input image in image conversion. The target value Y is a value that is set as a target value as a result of the discrimination calculation of Equation 2 above. A discrimination image designation signal is input N times to the target value supply unit 220, and a different student image is designated as a discrimination target in each discrimination image designation signal. The order of specifying the discrimination target in the N student images is arbitrary and is set by the user or the like. When the discrimination image designation signal is input, the target value supply unit 220 generates a target value Y to be given to each student image. Different target values Y are generated for the determination target student image and the other student images. Of the target values Y, the target value for the student image to be determined is Y1, and the other target values are Y2. For example, the target value supply unit 220 generates “1” as Y1 for the student image to be discriminated, and generates “0” as Y2 for the other student images. The target value supply unit 220 outputs the generated target value Y to the discrimination coefficient calculation unit 250 via the signal line 221.

  The configuration of the tap extraction unit 230 is the same as that of the tap extraction unit 110 in the image information conversion apparatus 100. The tap extraction unit 230 extracts a class tap and a prediction tap from the low resolution image signal. The tap extraction unit 230 outputs each pixel value in the extracted class tap to the class classification unit 240 via the signal line 238, and outputs each pixel value in the extracted prediction tap to the discrimination coefficient calculation unit 250 using the signal line 239. Output via.

  The configuration of the class classification unit 240 is the same as that of the class classification unit 120 in the image information conversion apparatus 100. The class classification unit 240 outputs a class code indicating the class into which the extracted class taps are classified to the discrimination coefficient calculation unit 250 via the signal line 249.

  The discrimination coefficient calculation unit 250 calculates a discrimination coefficient. The discrimination coefficient calculation unit 250 receives the target value Y from the target value supply unit 220, each pixel value in the prediction tap from the tap extraction unit 230, and the class code from the class classification unit 240. The discrimination coefficient calculation unit 250 calculates a discrimination coefficient group for each class indicated by the class code from the target value Y and each pixel value in the prediction tap. The discrimination coefficient calculation unit 250 outputs the calculated discrimination coefficient group as a discrimination coefficient group corresponding to the student image to be discriminated.

Here, the tap extraction unit 230 extracts the same number of prediction taps as the number of pixels in the student image in each of the N student images in the correct image group. For example, when the resolution of a student image is 720 × 480 pixels, 720 × 480 × N prediction taps are extracted from N student images. On the other hand, the class classification unit 240 obtains a class code indicating a class to which a class tap having the same pixel of interest as the prediction tap belongs in synchronization with the extraction of the prediction tap. The discrimination coefficient calculation unit 250 receives each pixel value in the prediction tap and the class to which the prediction tap belongs from the tap extraction unit 230 and the class classification unit 240. Each prediction tap having the same class code is hereinafter referred to as a “sample” of the class. For example, when 100 prediction taps corresponding to the class code “000000000000” are extracted from 720 × 480 × N prediction taps, the 100 prediction taps are samples of the class indicated by the class code. The The discrimination coefficient calculation unit 250 minimizes the sum of squares of the difference between each calculation result Y _k ′ obtained by performing the discrimination calculation of the above equation 2 and the target value Y (Y1 or Y2) for each pixel value in each sample. Are calculated for each class. k is an integer of 1 or more, and is a number assigned to each sample. For example, the discrimination coefficient calculation unit 250 generates a normal equation, which will be described later, and obtains a discrimination coefficient group by solving the normal equation.

A method for deriving a normal equation will be described. The difference ek in the _kth sample is obtained by the following equation.

The calculation result Y _k ′ of the above equation is obtained by the above equation 2. When the equation 2 a x _n pixel values in the k-th sample is replaced by x _{n_k} be substituted into the above equation 3, the following equation is obtained.

In the least square method, an optimum discrimination coefficient group is obtained by minimizing the sum of squares E expressed by the following equation.

In the above equation, K is the total number of samples belonging to the class to be statistics. The value of the discrimination coefficient that minimizes the square sum E is a value when the partial differentiation of the square sum E by the discrimination coefficient becomes zero. Expression 6 below is obtained by partial differentiation of the sum of squares E with each of the discrimination coefficients v _{1 to} v ₁₃ .

A discrimination coefficient group satisfying the above equation is an optimum discrimination coefficient group. From the above equation, the following equation is obtained.

Here, when the above equation 4 is partially differentiated by the discrimination coefficients v _{1 to} v ₁₃ , the following equation is obtained.

By substituting the above equation 8 into the above equation 7, the following equation is obtained.

Then, by substituting the above equation 4 to e _k of the above equation 9, the following equation is obtained.

By transforming the above equation, the following equation is obtained.

The above equation is expressed by the following equation, which is a normal equation, using a matrix.

  The above equation can be solved for each discrimination coefficient by using, for example, a sweep-out method (Gauss-Jordan elimination method). As a result, a discrimination coefficient that minimizes the difference between the result of the discrimination calculation and the target value is obtained.

  As described above, the target value Y1 is given to the student image to be discriminated, and the target value Y2 is given to the other student images. For this reason, when the image information conversion apparatus 100 performs the discrimination calculation on the input image using the discrimination coefficient group, the result of the discrimination calculation is Y1 when the image is similar to the student image to be discriminated. The value is close to Y2 and is close to Y2 when it is not similar to the student image to be determined. Therefore, by setting the threshold value to a value between these target values Y1 and Y2, and comparing this threshold value with the discrimination result, the image information conversion apparatus 100 allows the input image to be similar to the student image to be discriminated. It can be determined whether or not. When the target value “1” is given to the student image to be discriminated and the target value “0” is given to the other student images, the threshold is set to a value “0.5” between them. Is done. The image information conversion apparatus 100 can determine whether the input image is similar to the student image to be determined based on whether the determination result is equal to or greater than a threshold value.

  The filter unit 210 is an example of a student image input unit described in the claims. The target value supply unit 220 is an example of a target value generation unit described in the claims.

  FIG. 11 is a block diagram illustrating a configuration example of the filter unit 210 of the discrimination coefficient learning device according to the first embodiment. The filter unit 210 includes N two-dimensional filters 211, a student image storage unit 212, and a student image supply unit 213.

  The two-dimensional filter 211 generates a student image by changing the resolution of the teacher image. The same two teacher images are input to the N two-dimensional filters 211. The N two-dimensional filters 211 change the resolution of the teacher image and perform different image processing to generate different N student images. For example, the N two-dimensional filters 211 generate N student images with different compression rates. N student images are stored in the student image storage unit 212.

  The student image storage unit 212 stores a student image group including N student images. The student image supply unit 213 reads a student image group from the student image storage unit 212 and supplies it to the tap extraction unit 230. The student image supply unit 213 executes the supply of the student image group N times.

  The two-dimensional filter 211 is an example of a filter described in the claims.

  FIG. 12 is a diagram illustrating an example of a discrimination coefficient learning method according to the first embodiment. As illustrated in FIG. 12, a student image of N rows and N columns is generated in all learning. The student image in each row is a student image in a student image group generated in each of N times of learning. In each row, the student images are arranged in order of compression rate. The student images in each column are student images generated from the same two-dimensional filter in each of N times of learning. The shaded student image is a student image to be determined. For example, in the first learning, the student image # 1-1 generated from the two-dimensional filter # 1 is designated as a discrimination target. In this case, among the student images # 1-1 to # N-1 generated for the first time, the target value Y1 is set to the student image # 1-1, and the target value is set to the student images # 1-2 to 1-N. Y2 is set. In the second learning, student image # 2-2 is designated as a discrimination target. In this case, among the student images # 2-1 to # N-2 generated for the second time, the target value Y1 is set to the student image # 2-2, and the student image 2-1 and the student images # 2-3 to The target value Y2 is set to 2-N. In this way, learning is performed N times, and the discrimination coefficient is learned for each of the N student images.

  FIG. 13 is a diagram illustrating a setting example of the target value in the first embodiment. When the student image # 1 is designated as the discrimination target, the target value Y1 of “1” is set for the student image # 1, and the target value Y2 of “0” is set for the other student images. . When the student image # 2 is designated as the discrimination target, the target value Y1 of “1” is set for the student image # 2, and the target value Y2 of “0” is set for the other student images. . In this way, different target values are given to the student image to be determined and the other student images.

  FIG. 14 is a block diagram illustrating a configuration example of the discrimination coefficient calculation unit 250. The discrimination coefficient calculation unit 250 includes a normal equation generation unit 260, a discrimination coefficient determination unit 270, and a discrimination coefficient storage unit 280. The discrimination coefficient determination unit 270 includes a statistical processing unit 271, a matrix component total value storage unit 272, and a discrimination coefficient calculator 273.

  The normal equation generation unit 260 generates a matrix component of the normal equation in one sample based on each pixel value in the prediction tap and the target value. The normal equation generation unit 260 outputs the generated matrix components to the statistical processing unit 271.

  The statistical processing unit 271 performs statistical processing on the matrix components of each sample belonging to the class for each class. The statistical processing unit 271 receives the matrix component from the normal equation generation unit 260 and the class code from the class classification unit 240. The statistical processing unit 271 adds all matrix components having the same class code. Hereinafter, the added matrix component is referred to as a matrix component total value. For example, when receiving the class code and the matrix component, the statistical processing unit 271 reads the matrix component total value corresponding to the class code from the matrix component total value storage unit, and adds the received matrix component to the read value. The statistical processing unit 271 stores the added value in the matrix component total value storage unit 272 as a new matrix component total value in the class code. When the statistical processing is completed for N student images, the statistical processing unit 271 issues a statistical end notification for notifying the end of the statistics and outputs the statistical end notification to the discrimination coefficient calculator 273.

  The discrimination coefficient calculator 273 calculates a discrimination coefficient group for each class. When receiving the target value Y and the statistics end notification, the discrimination coefficient calculator 273 reads the matrix component total value for each class from the matrix component total value storage unit 272. The discriminant coefficient calculator 273 calculates a discriminant coefficient group for each class by solving the normal equation exemplified in Equation 12 above. The discrimination coefficient calculator 273 stores each calculated discrimination coefficient group in the discrimination coefficient storage unit 280 as a discrimination coefficient group corresponding to the student image to be discriminated indicated by the target value Y.

  The discrimination coefficient storage unit 280 stores a plurality of discrimination coefficient groups in association with combinations of a plurality of classes and N student images. Each of the N student images is associated with the N prediction tap types on a one-to-one basis. The discrimination coefficient group stored in the discrimination coefficient storage unit 280 is output to the image information conversion apparatus 100 via the signal line 209.

FIG. 15 is a block diagram illustrating a configuration example of the normal equation generation unit 260. The normal equation generation unit 260 includes multiplier groups 261 to 273 and a multiplier group 274. Each multiplier group includes 13 multipliers. Multipliers 261-1 through 261-13 in the multiplier group 261, respectively, and the pixel values _{x 1,} the _x 1 _{x 1} to _x 1 _{x 13} is the result of multiplying the _{x 1} to _{x 13,} the normal equation Is output as a matrix component of the first row vector on the left side of. Similarly, the multiplier groups 262 to 273 output matrix components from the second row vector on the left side to the 13th row vector on the left side. The multipliers 274-1 to 271-13 in the multiplier group 274 respectively convert x ₁ Y to x ₁₃ Y, which are the results of multiplying the target value Y and x _{1 to} x ₁₃ by the right side of the normal equation. Output as a matrix component of a single column vector.

  FIG. 16 is a diagram illustrating an example of matrix component statistics in the matrix component total value storage unit 272 according to the first embodiment. As shown in FIG. 16, for each class code, the matrix components of each sample are added to obtain the total value of the administrative components. Specifically, the total value of the matrix components for the left side first row vector to the left side 13th row vector and the right side first column vector of Equation 12 is obtained.

[Configuration example of prediction coefficient learning device]
FIG. 17 is a block diagram illustrating a configuration example of the prediction coefficient learning apparatus 300 according to the first embodiment. The prediction coefficient learning device 300 includes a filter unit 310, a teacher pixel extraction unit 320, a tap extraction unit 330, and a class classification unit 340.

  The filter unit 310 generates N different student images from the high-resolution image signal input as the teacher image. A high resolution image signal is input to the filter unit 310 as a teacher image. The filter unit 310 generates N different student images by passing the teacher image through N filters. These student images are images having different compression rates, for example. The filter unit 310 selects a student image designated by the learning image designation signal from the N student images, and outputs the selected student image to the tap extraction unit 330 via the signal line 319.

  The teacher pixel extraction unit 320 extracts a learning target pixel as a teacher pixel from a high-resolution image signal input as a teacher image. Each pixel corresponding to a plurality of high resolution image signals to be generated by the image information conversion apparatus 100 is extracted from one low resolution image signal. For example, for each pixel in the student image, four teacher pixels are extracted from the teacher image. The teacher pixel extraction unit 320 outputs the extracted pixel value of each teacher pixel as a target value in the prediction calculation to the prediction coefficient calculation unit 350 via the signal line 321.

  The configuration of the tap extraction unit 330 is the same as that of the tap extraction unit 110 in the image information conversion apparatus 100. The tap extraction unit 330 extracts class taps and prediction taps from the low resolution image signal. The tap extraction unit 330 outputs each pixel value in the extracted class tap to the class classification unit 340 via the signal line 338, and outputs each pixel value in the extracted prediction tap to the prediction coefficient calculation unit 350 with the signal line 339. Output via.

  The configuration of the class classification unit 340 is the same as that of the class classification unit 120 in the image information conversion apparatus 100. The class classification unit 340 outputs a class code indicating the class into which the extracted class taps are classified to the prediction coefficient calculation unit 350 via the signal line 349.

The prediction coefficient calculation unit 350 calculates a prediction coefficient. The prediction coefficient calculation unit 350 includes a pixel value of each teacher pixel from the teacher pixel extraction unit 320, each pixel value in the prediction tap from the tap extraction unit 330, a class code from the class classification unit 340, and a learning image. The specified signal is input. The prediction coefficient calculation unit 350 calculates a prediction coefficient group for each class indicated by the class code from the pixel value of the teacher pixel that is the target value and each pixel value in the prediction tap. Each prediction coefficient in the prediction coefficient group is calculated using, for example, the following equation.

In the above equation, k is a number assigned to each sample. _{xn_k} is each pixel value in the kth sample. y _k is a pixel value of any of the teacher pixels corresponding to the k-th sample. K is the total number of samples belonging to one class. The prediction coefficients w _{1 to} w ₁₃ are each prediction coefficient in one set in the prediction coefficient group. Equation 13 is derived by the same method as Equation 12. The discrimination coefficient calculation unit 350 outputs each calculated prediction coefficient group as a prediction coefficient group corresponding to a student image designated by the learning image designation signal.

  The filter unit 310 is an example of a student image input unit described in the claims.

  FIG. 18 is a block diagram illustrating a configuration example of the filter unit 310 of the prediction coefficient learning device according to the first embodiment. The filter unit 310 includes N two-dimensional filters 311 and a selector 312.

  The two-dimensional filter 311 is a filter similar to the two-dimensional filter 211 in the discrimination coefficient learning device 200. The selector 312 selects a student image designated by the learning image designation signal from N student images generated by the N two-dimensional filters 311. The selector 312 outputs a low resolution image signal as the selected student image to the tap extraction unit 330 via the signal line 319.

  The two-dimensional filter 311 is an example of a filter described in the claims.

  FIG. 19 is a diagram illustrating an example of a prediction coefficient learning method according to the first embodiment. In each of the N learnings, a student image designated as a learning target is generated by any one of the N two-dimensional filters. For example, in the first learning, student image # 1 is designated as a learning target. In the second learning, student image # 2 is designated. In this way, learning is performed N times, and the prediction coefficient is learned for each of the N student images.

[Operation example of image information converter]
FIG. 20 is a flowchart illustrating an example of the operation of the image information conversion apparatus 100 according to the first embodiment. The image information conversion apparatus 100 starts an image information conversion process for converting the resolution of an image when a low-resolution image signal is input.

  The image information conversion apparatus 100 uses any pixel of the low-resolution image as a target pixel, and extracts a class tap and a prediction tap centered on the target pixel (step S905).

  The image information conversion apparatus 100 executes a class classification process for classifying the class tap into any of a plurality of classes (step S910). The image information conversion apparatus 100 executes a prediction tap type determination process for determining the prediction tap type to which the prediction tap belongs (step S920).

  Then, the image information conversion apparatus 100 selects a prediction coefficient group corresponding to the combination of the classified class and the determined prediction tap type (step S930). The image information conversion apparatus 100 outputs the result of the prediction calculation using the selected prediction coefficient group as the pixel value of the high resolution image (step S935).

  The image information conversion apparatus 100 determines whether or not the calculation of all the pixels of the high resolution image has been completed (step S940). If the calculation for all the pixels has not been completed (step S940: No), the image information conversion apparatus 100 returns to step S905. If the calculation for all the pixels has been completed (step S940: Yes), the image information conversion apparatus 100 ends the image information conversion process.

  FIG. 21 is a flowchart illustrating an example of class classification processing according to the first embodiment. The class classification unit 120 re-quantizes each pixel in the class tap to generate an ADRC code (step S911). The class classification unit 120 determines a class code based on the ADRC code (step S912). After step S912, the class classification unit 120 ends the class classification process.

  FIG. 22 is a flowchart illustrating an example of the prediction tap type determination process according to the first embodiment. The discrimination coefficient selection unit 130 reads N discrimination coefficient groups corresponding to the class code from the N discrimination coefficient memories 132 (step S921). The discrimination calculation unit 140 performs N discrimination calculations using the discrimination coefficient group to generate a candidate determination result sequence (step S922). The discrimination calculation unit 140 discriminates the prediction tap type to which the prediction tap belongs from the candidate determination result sequence (step S924). After step S924, the discrimination calculation unit 140 ends the prediction tap type discrimination process.

[Operation example of discrimination coefficient learning device]
FIG. 23 is a flowchart illustrating an example of the operation of the discrimination coefficient learning device 200 according to the first embodiment. The discrimination coefficient learning device 200 starts a discrimination coefficient learning process for learning a discrimination coefficient when a high-resolution image signal and a discrimination image designation signal are input.

  The discrimination coefficient learning device 200 supplies N student images generated from the N two-dimensional filters to the tap extraction unit 230 (step S950). The discrimination coefficient learning device 200 sets the target value of the student image to be discriminated as 1 and the other target value as 0, and gives the target value to N student images (step S955).

  The discrimination coefficient learning device 200 extracts a class tap and a prediction tap using any pixel as a target pixel (step S960). The discrimination coefficient learning device 200 executes a class classification process (step S965). The discriminant coefficient learning apparatus 200 generates each matrix component of the normal equation for each class, and executes the statistical process (step S970). The discrimination coefficient learning device 200 determines whether or not the statistics of the matrix components of the normal equation have been completed for all the pixels of the N student images (step S975). If the statistics are not completed (step S975: No), the discrimination coefficient learning device 200 returns to step S960.

  If the statistics are completed (step S975: Yes), the discrimination coefficient learning device 200 calculates a discrimination coefficient group from the normal equation for each class (step S980). The discrimination coefficient learning device 200 associates the calculated discrimination coefficient group with the prediction tap type related to the designated student image (step S985).

  The discrimination coefficient learning device 200 determines whether learning of all the two-dimensional filters has been completed (step S990). If learning of any two-dimensional filter has not ended (step S990: No), the discrimination coefficient learning device 200 returns to step S950. If learning has ended (step S990: Yes), the discrimination coefficient learning device 200 ends the discrimination coefficient learning process.

[Operation example of prediction coefficient learning device]
FIG. 24 is a flowchart illustrating an example of the operation of the prediction coefficient learning device 300 according to the first embodiment. The prediction coefficient learning device 300 starts a prediction coefficient learning process for learning a prediction coefficient when a high-resolution image signal and a learning image designation signal are input. The prediction coefficient learning process is the same as the discrimination coefficient learning process illustrated in FIG. 23 except that steps S951, S956, and S976 are executed instead of steps S950, S955, and S975.

  The prediction coefficient learning device 300 generates a designated student image (step S951). The prediction coefficient learning device 300 extracts teacher pixels from the teacher image (step S956). After generating a normal equation for each class (step S970), if the statistics are not completed (step S976: No), the discrimination coefficient learning device 200 returns to step S956. If the statistics are finished (step S976: Yes), the discrimination coefficient learning device 200 calculates a discrimination coefficient group from the normal equation for each class (step S980).

  As described above, according to the first embodiment, the image information conversion apparatus 100 classifies the class taps into classes by reducing pixel values, and determines the prediction tap type to which the prediction tap belongs by a determination operation. Then, the image information conversion apparatus 100 selects a prediction coefficient group corresponding to the combination of the class classified from the plurality of prediction coefficient groups and the determined prediction tap type, and executes the prediction calculation. Since the image is converted using more prediction coefficient groups than when only class classification is performed, the conversion accuracy of the image is improved. Moreover, since the prediction coefficient group is set as many as the number of combinations of classes and prediction tap types, an increase in the prediction coefficient group is suppressed.

  In addition, the image information conversion apparatus 100 performs a product-sum operation on a plurality of discrimination coefficients and the pixel value of each pixel in the prediction tap, and the prediction to which the prediction tap belongs based on a comparison result between the calculation result and a predetermined threshold value. Determine the tap type. Since the calculation is simple, the process for determining the prediction tap type is fast.

  Further, the discrimination coefficient learning device 200 generates a student image group, gives a first target value Y1 to a student image to be discriminated, and gives a second target value Y2 to other student images. On the basis of the target value and the pixel value of each pixel in the prediction tap, a discrimination coefficient group is calculated for each class. Thereby, a discrimination coefficient group for discriminating whether or not the image is similar to the student image to be discriminated is learned for each N class of student images.

  The discrimination coefficient learning device 200 generates a student image group by passing the teacher image through a plurality of filters. For this reason, a plurality of student images having different characteristics can be easily generated by using a plurality of filters having different characteristics.

  Further, the discrimination coefficient learning device 200 gives the target value Y1 to the student image to be discriminated and the target value Y2 to the other student images in the calculation of the discrimination coefficient using the least square method. Therefore, the image information conversion apparatus 100 can determine whether the input image is similar to the student image to be determined by determining which of Y1 and T2 is the result of the determination operation. it can.

  Further, the prediction coefficient learning device 300, for each of a plurality of student images, predicts the prediction coefficient for each class based on the pixel value of each pixel in the prediction tap in the prediction tap in the student image and the pixel value of the teacher pixel. Compute a group. Thereby, a prediction coefficient group is learned for each class for N student images.

  The prediction coefficient learning apparatus 300 generates a plurality of student images by passing the teacher image through a plurality of filters. For this reason, a plurality of student images having different characteristics can be easily generated by using a plurality of filters having different characteristics.

  Note that the image information conversion apparatus 100 upconverts the low resolution image signal to the high resolution image signal, but can also downconvert the high resolution image signal to the low resolution image signal. In the case of down-conversion, one prediction tap coefficient group includes only one set composed of the same number of prediction coefficients as the number of pixels in the prediction tap.

  Further, although the number of pixels in the class tap is different from the number of pixels in the prediction tap, the number of pixels in the class tap may be the same as the number of pixels in the prediction tap.

  Further, the pixel group of the class tap and the prediction tap is not limited to the pixel group illustrated in FIG. For example, a square pixel group composed of 3 × 3 pixels centered on the target pixel may be used as a class tap or a prediction tap.

  Further, as illustrated in FIG. 4, the low-resolution image is an image in an interlaced moving image, but may be an image in a non-interlaced moving image.

  The class classification unit 120 performs class classification by reducing the number of gradations. However, the class classification can be performed using a method other than the reduction of the number of gradations. For example, the class classification unit 120 can also classify a class based on a result obtained by matching a shape pattern of a three-dimensional distribution defined in advance for each class with a shape pattern of a class tap. Further, the class classification unit 120 reduces the number of gradations by ADRC, but the number of gradations may be reduced by a process other than ADRC. For example, the class classification unit 120 may reduce the number of gradations by comparing each pixel value in the class tap with a predetermined value (for example, 128) without calculating the dynamic range.

  In addition, in the classification of the prediction tap type, each prediction tap is classified into a plurality of prediction tap types according to the compression rate of the image from which the prediction tap is extracted at the time of learning. The prediction tap may be classified. For example, by learning from student images generated by a plurality of two-dimensional filters having different effects such as blurring and contour enhancement, the prediction taps may be classified based on the degree of blurring and contour enhancement applied to the image. Good.

  Further, the discrimination calculation unit 140 of the image information conversion apparatus 100 performs the discrimination calculation by the linear linear combination formula exemplified in the above formula 1, but the discrimination calculation is not limited to the above formula 1. For example, the calculation may be a quadratic expression including the product of the discrimination coefficient and the square of the pixel value. In this case, the above-mentioned 12 normal equations are also changed.

  Moreover, although the prediction calculation part 160 of the image information conversion apparatus 100 is performing the prediction calculation by the linear linear combination formula illustrated in the said Formula 2, the prediction calculation is not limited to the said Formula 2. For example, the calculation may be performed by a quadratic expression including the product of the prediction coefficient and the square of the pixel value. In this case, the normal equation (13) is also changed.

  The discrimination coefficient storage unit 131 includes N discrimination coefficient memories 132, but the configuration of the discrimination coefficient storage unit 131 is not limited to this configuration. For example, the discrimination coefficient storage unit 131 may include a single memory having N areas.

  The prediction coefficient storage unit 151 includes N prediction coefficient memories 152, but the configuration of the prediction coefficient storage unit 151 is not limited to this configuration. For example, the prediction coefficient storage unit 151 may include a single memory having N areas.

  The discrimination coefficient learning device 200 generates N student images using N filters. However, if N student images can be supplied, the discrimination coefficient learning apparatus 200 is limited to a configuration including N filters. Not. For example, the discrimination coefficient learning device 200 may be configured to receive N student images generated in advance without including N filters.

  Further, the discrimination coefficient learning apparatus 200 learns the discrimination coefficient using one teacher image, but two or more teacher images can also be used. In this case, the student image group generated from a plurality of teacher images may be supplied N times to learn the discrimination coefficient, or learning that supplies the student image group generated from one teacher image N times is performed a plurality of times. May be. Similarly, the prediction coefficient learning apparatus 300 may use two or more teacher images.

<Modification>
[Operation example of discrimination coefficient learning device]
FIG. 25 is a diagram illustrating an example of a discrimination coefficient learning method according to the modification. The discrimination coefficient learning apparatus 200 according to the modification is different from the discrimination coefficient learning apparatus 200 according to the first embodiment in that only one two-dimensional filter 211 is provided. As shown in FIG. 25, N teacher images # 1 to #N are supplied to the discrimination coefficient learning device 200. The teacher image # 1 is a natural image, and the teacher image #N is a CG (Computer Graphics) image. Teacher images # 2 to N-1 are images arranged in the order close to natural images. The student images # 1 to #N are generated by passing the teacher images # 1 to #N through the two-dimensional filter 211 and changing the resolution. The student image supply unit 213 supplies the student image groups of student images # 1 to N to the tap extraction unit 230 a plurality of times. In each learning, one of the N student images is designated as a determination target student image. A similar learning method is used in learning of the prediction coefficient.

  Thus, according to the modification, the discrimination coefficient learning device 200 learns the discrimination coefficient in each student image close to a natural image or a CG image. For this reason, the image information conversion apparatus 100 can determine whether the image input using the determination coefficient is close to a natural image or a CG image.

<2. Second Embodiment>
[Configuration example of image information conversion apparatus]
FIG. 26 is a block diagram illustrating a configuration example of the image information conversion apparatus 100 according to the second embodiment. The image information conversion apparatus 100 according to the second embodiment differs from the image information conversion apparatus 100 according to the first embodiment in that a determination calculation unit 141 is provided instead of the determination calculation unit 140. Further, the present embodiment is different from the image information conversion apparatus 100 of the first embodiment in that it further includes an operation signal history information recording unit 170, a viewing environment detection unit 180, and an image quality adjustment unit 190.

  The image quality adjustment unit 190 adjusts the image quality of the high-quality image generated by the prediction calculation unit 160 according to the operation signal. The operation signal is a signal instructing to change the value of the image quality parameter that affects the image quality of the high resolution image. The image quality parameter is, for example, brightness (that is, luminance), gamma, contrast, and the like. The image quality adjustment unit 190 outputs a high-quality image signal indicating the adjusted high-quality image to a display device or the like.

  The operation signal history information recording unit 170 records the value of the image quality parameter adjusted by the operation signal output to the image quality adjustment unit 190. The operation signal history information recording unit 170 records the adjusted image quality parameter value history, and estimates the content of adjustment performed on the high-resolution image from the history. The operation signal history information recording unit 170 outputs the shift amount A by which the candidate determination result sequence is shifted based on the estimated image quality adjustment to the discrimination calculation unit 141 via the signal line 179.

  Here, the shift amount A is an amount by which the candidate determination result sequence is bit-shifted in a direction in which the prediction tap type to which the prediction coefficient group suitable for the estimated adjustment belongs has priority over the prediction tap type to which the inappropriate prediction coefficient group belongs. is there. Whether or not each prediction coefficient group is suitable for the estimated image quality adjustment is determined based on whether or not an appropriate high-resolution image is obtained after the adjustment when the prediction coefficient group is selected. For example, when an adjustment to increase the brightness is estimated, it is better to use a prediction coefficient group learned from an image having a compression rate lower than the actual compression rate of the input low-resolution image. A high-resolution image with higher image quality is generated by a prediction calculation using a prediction coefficient group learned at a compression rate lower than the actual compression rate. In this way, an image in which a high resolution feeling is emphasized is easier to see under high luminance and the image quality looks better. If the brightness is adjusted based on this tendency, the candidate determination result sequence is shifted in a direction in which the prediction tap type to which the prediction coefficient group learned from the low-compressed student image belongs is prioritized. Conversely, when the brightness is adjusted to low luminance, the candidate determination result sequence is shifted in a direction in which the prediction tap type corresponding to the highly compressed student image is prioritized.

  The viewing environment detection unit 180 detects a viewing environment parameter indicating a viewing environment for viewing a high-resolution image. The viewing environment parameter is, for example, the illuminance in the room where the display device that displays the high-resolution image is installed, or the type of the display device. Specifically, the type of the display device is a type classified based on a physical screen size of the display device or a display method such as liquid crystal or plasma display. The illuminance in the room is acquired by a sensor signal from an optical sensor. The viewing environment detection unit outputs a shift amount B by which the candidate determination result sequence is shifted based on the viewing environment parameter to the determination calculation unit 141 via the signal line 189.

  Here, the shift amount B is an amount by which the candidate determination result sequence is bit-shifted in a direction in which the prediction tap type to which the prediction coefficient group suitable for the viewing environment indicated by the viewing environment parameter belongs is prioritized. Whether or not each prediction coefficient group is suitable for a certain viewing environment is determined depending on whether or not a high-resolution image suitable for viewing in the viewing environment is generated when that prediction coefficient group is selected. For example, in a bright room, a high-resolution image that is easier to see can be obtained by using a prediction coefficient group that has been learned with an image having a compression rate lower than the actual compression rate of the input low-resolution image. Based on this tendency, when the illuminance is high, the candidate determination result sequence is shifted in a direction in which the prediction tap type to which the prediction coefficient group learned from the low-compressed student image belongs is prioritized. Conversely, when the illuminance is low, the candidate determination result sequence is shifted in a direction in which the predicted tap type corresponding to the highly compressed student image is prioritized.

  The discrimination calculation unit 141 bit-shifts the generated candidate determination result sequence by the shift amounts A and B. The shift operation is executed by, for example, a logical shift in which 0 is inserted into an empty bit. And the discrimination | determination calculating part 141 discriminate | determines a prediction tap classification based on the shifted candidate determination result sequence | sequence, and outputs a classification code.

  The operation signal history information recording unit 170 is an example of a recording unit described in the claims.

FIG. 27 is a diagram illustrating a configuration example of a candidate determination result sequence. In the candidate determination result sequence, the candidate determination results are arranged in descending order of compression rate. More specifically, the candidate determination result b ₁ of the prediction taps type according to the highest image compression ratio at the top is arranged, thereafter, the candidate determination result highly compressed sequence b ₂ to b _N are arranged. The direction in which this candidate determination result sequence is shifted is set by the sign of the shift amount. For example, when shifting in a direction with a high compression rate,-is set for the sign of the shift amount, and when shifting in a direction with a low compression rate, + is set for the sign of the shift amount. Zeros are inserted in bits vacated by the shift.

  FIG. 28 is a diagram illustrating a setting example of the shift amount A. The shift amount A is set so that a prediction coefficient group suitable for image quality adjustment is used. For example, let us consider a case where the contrast value is adjusted in 100 steps from 1 to 100 as the image quality parameter, and the contrast increases as the value increases. When image quality adjustment with high contrast is performed, the image quality of the image after adjustment looks better when the prediction coefficient group learned from the low-compression student image is used. On the other hand, when the contrast is weak, the image quality looks better when the prediction coefficient group learned from the highly compressed student image is used. Based on this tendency, when the contrast statistic (for example, the average value) is 1 to 30, a shift amount −1 for shifting 1 bit toward the prediction tap type related to the highly compressed image is set. When the average contrast value is 31 to 70, it is not necessary to bit-shift the candidate determination result sequence, and a shift amount of 0 is set. When the average contrast value is 71 to 100, a shift amount +1 for shifting by 1 bit toward the prediction tap type related to the low-compression image is set. Similarly to the shift amount A, the shift amount B is set in advance for each viewing environment parameter statistic.

  FIG. 29 is a block diagram illustrating a configuration example of the operation signal history information recording unit 170 according to the second embodiment. The operation signal history information recording unit 170 includes an operation signal analysis unit 171, a statistical processing unit 172, and a shift amount output unit 173.

  The operation signal analyzer 171 analyzes the detected operation signal. In the analysis of the operation signal, the adjusted image quality parameter value and the like are analyzed. The operation signal analysis unit 171 outputs the analysis result to the statistical processing unit 172.

  The statistical processing unit 172 performs statistical processing on the analysis result of the operation signal. The statistical processing unit 172 calculates a statistic such as an average value or a median value of image quality parameter values within a predetermined period, for example. The statistical processing unit 172 outputs the result of the statistical processing to the shift amount output unit 173.

  The shift amount output unit 173 determines the shift amount A based on the statistical result. The shift amount A is preset for each statistic. The shift amount output unit 173 outputs the determined shift amount to the discrimination calculation unit 141 via the signal line 179.

  FIG. 30 is a block diagram illustrating a configuration example of the viewing environment detection unit 180 according to the second embodiment. The viewing environment detection unit 180 includes a sensor signal analysis unit 181, a statistical processing unit 182, and a shift amount output unit 183.

  The sensor signal analysis unit 181 analyzes the sensor signal. The sensor signal is, for example, a signal indicating the illuminance in the room detected by the optical sensor in the room where the display device for viewing a high-resolution image is installed. The sensor signal analysis unit 181 outputs the analysis result to the statistical processing unit 182.

  The statistical processing unit 182 performs statistical processing on the analysis result of the sensor signal. For example, the statistical processing unit 182 calculates a statistic such as an average value or median value of illuminance within a predetermined period. The statistical processing unit 182 outputs the result of the statistical processing to the shift amount output unit 183.

  The shift amount output unit 183 determines the shift amount B based on the statistical result. The shift amount B is preset for each statistic. The shift amount output unit 183 outputs the determined shift amount to the discrimination calculation unit 141 via the signal line 189.

  FIG. 31 is a block diagram illustrating a configuration example of the discrimination calculation unit 141 according to the second embodiment. The configuration of the discrimination calculation unit 141 is different from the discrimination calculation unit 140 of the first embodiment in that it further includes a shift unit 145. In the discrimination calculation unit 141, the candidate determination result sequence generation unit 144 outputs the candidate determination result sequence to the shift unit 145.

  The shift unit 145 performs a shift operation on the candidate determination result sequence. The shift unit 145 receives the candidate determination result sequence from the candidate determination result sequence generation unit 144 and receives the shift amounts A and B from the operation signal history information recording unit 170 and the viewing environment detection unit 180. The shift unit 145 performs a shift operation on the candidate determination result sequence based on the shift amounts A and B. The shift unit 145 outputs the candidate determination result sequence after the shift calculation to the prediction tap type determination unit 146.

[Operation example of image information converter]
FIG. 32 is a flowchart illustrating an example of a prediction tap type determination process according to the second embodiment. The prediction tap type discrimination process in the second embodiment is different from the first embodiment in that step S923 is further executed.

  After generating the candidate determination result sequence (step S922), the discrimination calculation unit 141 shifts the candidate determination result sequence based on the shift amount (step S923). The discrimination calculation unit 141 determines the prediction tap type from the shifted candidate determination result sequence (step S924).

  As described above, according to the second embodiment, the discrimination calculation unit 141 bit-shifts the candidate determination result sequence in a direction in which the prediction tap type to which the prediction coefficient group suitable for image quality adjustment belongs is prioritized. For this reason, high conversion accuracy can be maintained even when image adjustment is performed by an operation signal.

  Further, the discrimination calculation unit 141 bit-shifts the candidate determination result sequence in a direction in which the prediction tap type to which the prediction coefficient group suitable for the viewing environment belongs is prioritized. For this reason, high conversion accuracy can be maintained regardless of the viewing environment.

  The operation signal history information recording unit 170 can further acquire information for specifying the operated user from the operation signal. For example, in a remote control device that transmits an operation signal, user authentication is performed by inputting a code that identifies the user during operation, and the operation signal history information recording unit 170 acquires the code from the operation signal. Identify users. In this case, statistical processing is performed for each user.

  When there are a plurality of remote control devices that can transmit an operation signal, the operation signal history information recording unit 170 may analyze which device the operation signal is from. In this case, statistical processing is performed for each device.

  The viewing environment detection unit 180 can also detect a parameter other than illuminance as long as the parameter indicates the viewing environment. For example, the viewing environment detection unit 180 may detect the physical screen size of a display device that displays a high-resolution image. In a prediction calculation for predicting a high-resolution image suitable for viewing on a display device having a large screen size, a prediction coefficient group learned from a low-compression student image is suitable. The shift amount is determined based on this tendency.

  In addition, in the classification of the prediction tap type, each prediction tap is classified into a plurality of prediction tap types based on the compression rate of the image from which the prediction tap is extracted at the time of learning. Taps may be classified. For example, the prediction tap type may be classified according to whether the source image at the time of learning is closer to a natural image or a CG image. When the prediction tap type is classified based on whether it is close to a natural image or a CG image, the stronger the contrast, the more suitable the prediction coefficient group learned from an image close to the CG image. The shift amount is determined based on this tendency.

<Third Embodiment>
[Configuration example of image information conversion apparatus]
FIG. 33 is a block diagram illustrating a configuration example of the image information conversion apparatus 100 according to the third embodiment. The image information conversion apparatus 100 according to the third embodiment is different from the image information conversion apparatus 100 according to the first embodiment in that an average waveform calculation unit 185 is further provided. The prediction tap extraction unit 111 in the second modification further outputs each pixel value in the prediction tap to the average waveform calculation unit 185.

  The average waveform calculation unit 185 calculates an average value of pixel values of each pixel in the prediction tap as an average waveform. The average waveform calculation unit 185 outputs the shift amount C for shifting the candidate determination result sequence to the determination calculation unit 141 via the signal line 189 when the calculated average waveform is outside the range of the average waveform at the time of learning. .

  The shift amount C is an amount by which the candidate determination result sequence is bit-shifted in a direction in which the prediction tap type to which the prediction coefficient group suitable for the average waveform belongs is prioritized. Whether each prediction coefficient group is suitable for the average waveform depends on the image quality of the high-quality image converted from the low-resolution image when selecting that prediction coefficient group than when selecting the other prediction coefficient group. Judgment is based on whether or not the image is visible. For example, when a low-resolution image that is brighter than the student image at the time of learning is input using the pixel value as a luminance value, a prediction coefficient group that is learned with an image having a compression rate lower than the actual compression rate of the input low-resolution image A relatively appropriate high-resolution image is generated by using.

  FIG. 34 is a block diagram illustrating a configuration example of the average waveform calculator 185 according to the third embodiment. The average waveform calculator 185 includes an average waveform calculator 186 and a shift amount output unit 187.

  The average waveform calculator 186 calculates an average waveform from each pixel value in the prediction tap. The average waveform calculator 186 outputs the calculated average waveform to the shift amount output unit 187.

  The shift amount output unit 187 determines the shift amount C based on the average waveform. The shift amount output unit 187 outputs the determined shift amount C to the discrimination calculation unit 141. The shift amount C is preset for each value of the average waveform. The discrimination calculation unit 141 shifts the candidate determination result sequence based on the shift amount C.

  As described above, according to the third embodiment, the image information conversion apparatus 100 shifts the candidate determination result sequence in a direction in which the prediction tap type to which the prediction coefficient group suitable for the average waveform belongs is prioritized. For this reason, even when a low-resolution image outside the range of the average waveform at the time of learning is input, an appropriate prediction tap coefficient group is selected and the image quality is unlikely to deteriorate.

  The above-described embodiment shows an example for embodying the present technology, and the matters in the embodiment and the invention-specific matters in the claims have a corresponding relationship. Similarly, the invention specific matter in the claims and the matter in the embodiment of the present technology having the same name as this have a corresponding relationship. However, the present technology is not limited to the embodiment, and can be embodied by making various modifications to the embodiment without departing from the gist thereof.

  Further, the processing procedure described in the above embodiment may be regarded as a method having a series of these procedures, and a program for causing a computer to execute these series of procedures or a recording medium storing the program. You may catch it. As this recording medium, for example, a CD (Compact Disc), an MD (MiniDisc), a DVD (Digital Versatile Disk), a memory card, a Blu-ray Disc (registered trademark), or the like can be used.

In addition, this technique can also take the following structures.
(1) Using any pixel in the first image as a target pixel, a class tap that is a first pixel group including the target pixel and a prediction tap that is a second pixel group including the target pixel are A tap extraction unit for extracting from one image;
A class classification unit for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into one of a plurality of classes;
A plurality of discriminant coefficient groups set in association with the plurality of classes for each prediction tap type as a coefficient group to be used in a discriminant calculation for discriminating to which of the plurality of predictive tap types the predictive tap belongs A discrimination coefficient selection unit that selects the discrimination coefficient group corresponding to the classified class in each of the plurality of prediction tap types from
A discriminating operation unit that discriminates the prediction tap type to which the prediction tap belongs from a result of executing the discriminant operation based on the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap;
Each combination of the plurality of classes and the plurality of prediction tap types as a coefficient group to be used in a prediction calculation for predicting a pixel value of a pixel in the second image which is an image in which the resolution of the first image is changed A prediction coefficient selection unit that selects the prediction coefficient group corresponding to a combination of the classified class and the determined prediction tap type from among a plurality of prediction coefficient groups set in association with
An image information conversion apparatus comprising: a prediction calculation unit that executes the prediction calculation based on the selected prediction coefficient group and a pixel value of each pixel in the prediction tap.
(2) The class classification unit subtracts the number of gradations of the pixel value of each pixel in the class tap and converts the shape pattern of the plurality of classes based on the pixel value obtained by reducing the number of gradations. The image information conversion apparatus according to (1), wherein the image information conversion apparatus is classified into one of the above.
(3) The discrimination calculation unit
A discrimination calculation execution unit that executes a product-sum calculation of each discrimination coefficient in the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap as the discrimination calculation;
A comparison unit that outputs, as a comparison result, each of the results of comparing and comparing the result of the product-sum operation in each of the plurality of prediction tap types with a predetermined threshold;
The image information conversion device according to (1) or (2), further including: a prediction tap type determination unit that determines the prediction tap type to which the prediction tap belongs based on the comparison result.
(4) a recording unit that records an operation signal for adjusting the image quality of the second image;
The prediction tap type determination unit selects one of the plurality of prediction taps in preference to the prediction tap type to which the prediction coefficient group suitable for the image quality adjustment performed by the recorded operation signal belongs. The image information conversion apparatus according to (3).
(5) a viewing environment detection unit that detects a viewing environment parameter indicating the viewing environment of the second image;
The prediction tap type determination unit selects one of the plurality of prediction tap types in preference to the prediction tap type to which the prediction coefficient group suitable for the viewing environment indicated by the viewing environment parameter belongs ( 3) The image information conversion apparatus described.
(6) The image information conversion device according to (5), wherein the viewing environment parameter includes illuminance of illumination in a room in which the display device that displays the second image is installed.
(7) The image information conversion device according to (5) or (6), wherein the viewing environment parameter includes a type of a display device that displays the second image.
(8) further comprising an average value calculation unit for calculating an average value of pixel values of each pixel in the prediction tap;
The predicted tap type determination unit
When the average value is outside a predetermined range, the prediction tap type to which the prediction coefficient group suitable for the average value belongs is prioritized and any one of the plurality of prediction tap types is selected (3) The image information conversion apparatus described.
(9) Using any pixel in the first image as a target pixel, a class tap that is a first pixel group including the target pixel and a prediction tap that is a second pixel group including the target pixel are A tap extraction step for extracting from one image;
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A plurality of discriminant coefficient groups set in association with the plurality of classes for each prediction tap type as a coefficient group to be used in a discriminant calculation for discriminating to which of the plurality of predictive tap types the predictive tap belongs A discrimination coefficient selection step of selecting the discrimination coefficient group corresponding to the classified class in each of the plurality of prediction tap types from
A discriminant calculation step of discriminating the prediction tap type to which the prediction tap belongs from a result of executing the discrimination calculation based on the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap;
Each combination of the plurality of classes and the plurality of prediction tap types as a coefficient group to be used in a prediction calculation for predicting a pixel value of a pixel in the second image which is an image in which the resolution of the first image is changed A prediction coefficient selection step of selecting the prediction coefficient group corresponding to the combination of the classified class and the determined prediction tap type from among a plurality of prediction coefficient groups set in association with
A control method for an image information conversion apparatus, comprising: a prediction calculation step that executes the prediction calculation based on the selected prediction coefficient group and a pixel value of each pixel in the prediction tap.
(10) Using any pixel in the first image as a target pixel, a class tap that is a first pixel group including the target pixel and a prediction tap that is a second pixel group including the target pixel are A tap extraction step for extracting from one image;
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A plurality of discriminant coefficient groups set in association with the plurality of classes for each prediction tap type as a coefficient group to be used in a discriminant calculation for discriminating to which of the plurality of predictive tap types the predictive tap belongs A discrimination coefficient selection step of selecting the discrimination coefficient group corresponding to the classified class in each of the plurality of prediction tap types from
A discriminant calculation step of discriminating the prediction tap type to which the prediction tap belongs from a result of executing the discrimination calculation based on the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap;
Each combination of the plurality of classes and the plurality of prediction tap types as a coefficient group to be used in a prediction calculation for predicting a pixel value of a pixel in the second image which is an image in which the resolution of the first image is changed A prediction coefficient selection step of selecting the prediction coefficient group corresponding to the combination of the classified class and the determined prediction tap type from among a plurality of prediction coefficient groups set in association with
A program for causing a computer to execute a prediction calculation step of executing the prediction calculation based on the selected prediction coefficient group and a pixel value of each pixel in the prediction tap.
(11) A student image input unit that inputs a plurality of student image groups including a plurality of student images, each of a plurality of different images as student images,
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from the image, using any pixel in the input student image as the target pixel. A tap extraction unit to extract;
A class classification unit for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into one of a plurality of classes;
A first target value is generated when the discrimination image is input using any one of the student images in the student image group as a discrimination image, and the second target value is input when the student image other than the discrimination image is input. A target value generation unit for generating a target value of
When the resolution of an image is changed, a discrimination coefficient group to be used in a discrimination calculation for discriminating whether or not the image to be changed is similar to the learning image is set in the first and second target values and the prediction tap. A discrimination coefficient learning device comprising: a discrimination coefficient calculation unit that calculates each class based on a pixel value of each pixel.
(12) The student image input unit includes a filter unit that generates the student image group by passing a teacher image, which is an image in which the resolution of the student image is changed, through a plurality of filters;
The discrimination coefficient learning device according to (11), further comprising a student image supply unit that inputs the generated plurality of student image groups a plurality of times.
(13) The discrimination operation is a product-sum operation of each discrimination coefficient in the discrimination coefficient group and each pixel value in the prediction tap in the change target image,
The discriminant coefficient calculating unit adds the result of the discriminant calculation in the student image other than the learning image and the second sum of squares of the difference between the result of the discriminant calculation in the learning image and the first target value. The discriminant coefficient learning device according to (11) or (12), wherein the discriminant coefficient group that minimizes a value obtained by adding a sum of squares of differences from the target value is calculated for each of the plurality of classes.
(14) A student image input step of inputting a plurality of student image groups each including a plurality of student images, each of a plurality of different images as a student image,
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from the image, using any pixel in the input student image as the target pixel. Tap extraction step to extract;
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A first target value is generated when the discrimination image is input using any one of the student images in the student image group as a discrimination image, and the second target value is input when the student image other than the discrimination image is input. A target value generation step for generating a target value of
When the resolution of an image is changed, a discrimination coefficient group to be used in a discrimination calculation for discriminating whether or not the image to be changed is similar to the learning image is set in the first and second target values and the prediction tap. A discriminant coefficient learning device control method comprising: a discriminant coefficient calculation step for calculating each class based on a pixel value of each pixel.
(15) A student image input step for inputting a plurality of student image groups each including a plurality of student images, each of a plurality of different images as student images,
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from the image, using any pixel in the input student image as the target pixel. Tap extraction step to extract;
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A first target value is generated when the discrimination image is input using any one of the student images in the student image group as a discrimination image, and the second target value is input when the student image other than the discrimination image is input. A target value generation step for generating a target value of
When the resolution of an image is changed, a discrimination coefficient group to be used in a discrimination calculation for discriminating whether or not the image to be changed is similar to the learning image is set in the first and second target values and the prediction tap. And a discrimination coefficient calculation step for calculating each class based on the pixel value of each pixel.
(16) a student image input unit that inputs each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from any pixel in the input student image as the target pixel. A tap extraction unit that extracts from the image of
A class classification unit for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into one of a plurality of classes;
A teacher pixel extraction unit that extracts one or more pixels in the teacher image, which is an image obtained by changing the resolution of the input student image, as a teacher pixel;
A prediction coefficient calculation unit that calculates a prediction coefficient group to be used in a prediction calculation for predicting the pixel value of the teacher pixel for each class based on the pixel value of each pixel in the prediction tap and the pixel value of the teacher pixel. A prediction coefficient learning apparatus comprising:
(17) The prediction coefficient learning device according to (16), wherein the student image input unit generates and inputs the plurality of student images by passing the teacher image through a plurality of filters.
(18) a student image input step of inputting each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from any pixel in the input student image as the target pixel. Tap extraction step to extract from the image of
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A teacher pixel extraction step of extracting one or more pixels in the teacher image, which is an image obtained by changing the resolution of the input student image, as a teacher pixel;
A prediction coefficient calculation step of calculating a prediction coefficient group to be used in a prediction calculation for predicting the pixel value of the teacher pixel for each class based on the pixel value of each pixel in the prediction tap and the pixel value of the teacher pixel. A control method for a prediction coefficient learning apparatus comprising:
(19) A student image input step of inputting each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from any pixel in the input student image as the target pixel. Tap extraction step to extract from the image of
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A teacher pixel extraction step of extracting one or more pixels in the teacher image, which is an image obtained by changing the resolution of the input student image, as a teacher pixel;
A prediction coefficient calculation step of calculating a prediction coefficient group to be used in a prediction calculation for predicting the pixel value of the teacher pixel for each class based on the pixel value of each pixel in the prediction tap and the pixel value of the teacher pixel. A program that causes a computer to execute.

DESCRIPTION OF SYMBOLS 100 Image information converter 110,230,330 Tap extraction part 111 Prediction tap extraction part 112 Class tap extraction part 120,240,340 Class classification part 121 Requantization part 122 Class code determination part 130 Discrimination coefficient selection part 131,280 Discrimination Coefficient storage unit 132 Discrimination coefficient memory 133 Discrimination coefficient reading unit 140, 141 Discrimination calculation unit 142 Discrimination calculator 143 Threshold processing unit 144 Candidate determination result sequence generation unit 145 Shift unit 146 Prediction tap type determination unit 150 Prediction coefficient selection unit 151 Prediction coefficient Storage unit 152 Prediction coefficient memory 153 Prediction coefficient reading unit 154, 312 Selector 160 Prediction calculation unit 170 Operation signal history information recording unit 171 Operation signal analysis unit 172, 182, 271 Statistical processing unit 173, 183, 187 Shift amount output unit 180 Viewing Environment detection unit 181 Sensor signal analysis unit 185 Average waveform calculation unit 186 Average waveform calculation unit 190 Image quality adjustment unit 200 Discriminant coefficient learning device 210, 310 Filter unit 211, 311 Two-dimensional filter 212 Student image storage unit 213 Student image supply unit 220 Target Value supply unit 250 Discrimination coefficient calculation unit 260 Normal equation generation unit 270 Discrimination coefficient determination unit 272 Matrix component total value storage unit 273 Discrimination coefficient calculator 300 Prediction coefficient learning device 320 Teacher pixel extraction unit 350 Prediction coefficient calculation unit

Claims (19)

Using any pixel in the first image as a target pixel, the first image includes a class tap that is a first pixel group including the target pixel and a prediction tap that is a second pixel group including the target pixel. A tap extractor for extracting from
A class classification unit for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into one of a plurality of classes;
A plurality of discriminant coefficient groups set in association with the plurality of classes for each prediction tap type as a coefficient group to be used in a discriminant calculation for discriminating to which of the plurality of predictive tap types the predictive tap belongs A discrimination coefficient selection unit that selects the discrimination coefficient group corresponding to the classified class in each of the plurality of prediction tap types from
A discriminating operation unit that discriminates the prediction tap type to which the prediction tap belongs from a result of executing the discriminant operation based on the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap;
Each combination of the plurality of classes and the plurality of prediction tap types as a coefficient group to be used in a prediction calculation for predicting a pixel value of a pixel in the second image which is an image in which the resolution of the first image is changed A prediction coefficient selection unit that selects the prediction coefficient group corresponding to a combination of the classified class and the determined prediction tap type from among a plurality of prediction coefficient groups set in association with
An image information conversion apparatus comprising: a prediction calculation unit that executes the prediction calculation based on the selected prediction coefficient group and a pixel value of each pixel in the prediction tap.   The class classification unit subtracts the number of gradations of the pixel value of each pixel in the class tap and assigns the shape pattern to one of the plurality of classes based on the pixel value obtained by reducing the number of gradations. The image information conversion apparatus according to claim 1, wherein classification is performed. The discrimination calculation unit
A discrimination calculation execution unit that executes a product-sum calculation of each discrimination coefficient in the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap as the discrimination calculation;
A comparison unit that outputs, as a comparison result, each of the results of comparing and comparing the result of the product-sum operation in each of the plurality of prediction tap types with a predetermined threshold;
The image information conversion apparatus according to claim 1, further comprising: a prediction tap type determination unit that determines the prediction tap type to which the prediction tap belongs based on the comparison result. A recording unit that records an operation signal for adjusting an image quality of the second image;
The prediction tap type determination unit selects one of the plurality of prediction taps in preference to the prediction tap type to which the prediction coefficient group suitable for the image quality adjustment performed by the recorded operation signal belongs. The image information conversion apparatus according to claim 3. A viewing environment detection unit for detecting a viewing environment parameter indicating a viewing environment of the second image;
The prediction tap type determination unit selects one of the plurality of prediction tap types by giving priority to the prediction tap type to which the prediction coefficient group suitable for the viewing environment indicated by the viewing environment parameter belongs. 3. The image information conversion apparatus according to 3. The image information conversion apparatus according to claim 5, wherein the viewing environment parameter includes illuminance of illumination in a room in which a display device that displays the second image is installed. The image information conversion device according to claim 5, wherein the viewing environment parameter includes a type of a display device that displays the second image. Further comprising an average value calculation unit for calculating an average value of pixel values of each pixel in the prediction tap;
The predicted tap type determination unit
4. When the average value is outside a predetermined range, the prediction tap type to which the prediction coefficient group suitable for the average value belongs is preferentially selected and one of the plurality of prediction tap types is selected. Image information conversion device. Using any pixel in the first image as a target pixel, the first image includes a class tap that is a first pixel group including the target pixel and a prediction tap that is a second pixel group including the target pixel. Tap extraction step to extract from,
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A plurality of discriminant coefficient groups set in association with the plurality of classes for each prediction tap type as a coefficient group to be used in a discriminant calculation for discriminating to which of the plurality of predictive tap types the predictive tap belongs A discrimination coefficient selection step of selecting the discrimination coefficient group corresponding to the classified class in each of the plurality of prediction tap types from
A discriminant calculation step of discriminating the prediction tap type to which the prediction tap belongs from a result of executing the discrimination calculation based on the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap;
Each combination of the plurality of classes and the plurality of prediction tap types as a coefficient group to be used in a prediction calculation for predicting a pixel value of a pixel in the second image which is an image in which the resolution of the first image is changed A prediction coefficient selection step of selecting the prediction coefficient group corresponding to the combination of the classified class and the determined prediction tap type from among a plurality of prediction coefficient groups set in association with
A control method for an image information conversion apparatus, comprising: a prediction calculation step that executes the prediction calculation based on the selected prediction coefficient group and a pixel value of each pixel in the prediction tap. Using any pixel in the first image as a target pixel, the first image includes a class tap that is a first pixel group including the target pixel and a prediction tap that is a second pixel group including the target pixel. Tap extraction step to extract from,
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A plurality of discriminant coefficient groups set in association with the plurality of classes for each prediction tap type as a coefficient group to be used in a discriminant calculation for discriminating to which of the plurality of predictive tap types the predictive tap belongs A discrimination coefficient selection step of selecting the discrimination coefficient group corresponding to the classified class in each of the plurality of prediction tap types from
A discriminant calculation step of discriminating the prediction tap type to which the prediction tap belongs from a result of executing the discrimination calculation based on the selected discrimination coefficient group and a pixel value of each pixel in the prediction tap;
Each combination of the plurality of classes and the plurality of prediction tap types as a coefficient group to be used in a prediction calculation for predicting a pixel value of a pixel in the second image which is an image in which the resolution of the first image is changed A prediction coefficient selection step of selecting the prediction coefficient group corresponding to the combination of the classified class and the determined prediction tap type from among a plurality of prediction coefficient groups set in association with
A program for causing a computer to execute a prediction calculation step of executing the prediction calculation based on the selected prediction coefficient group and a pixel value of each pixel in the prediction tap. A student image input unit for inputting a plurality of student image groups including a plurality of student images, each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from the image, using any pixel in the input student image as the target pixel. A tap extraction unit to extract;
A class classification unit for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into one of a plurality of classes;
A first target value is generated when the discrimination image is input using any one of the student images in the student image group as a discrimination image, and the second target value is input when the student image other than the discrimination image is input. A target value generation unit for generating a target value of
When the resolution of an image is changed, a discrimination coefficient group to be used in a discrimination calculation for discriminating whether or not the image to be changed is similar to the learning image is set in the first and second target values and the prediction tap. A discrimination coefficient learning device comprising: a discrimination coefficient calculation unit that calculates each class based on a pixel value of each pixel. The student image input unit includes a filter unit that generates the student image group by passing a teacher image, which is an image in which the resolution of the student image is changed, through a plurality of filters;
The discrimination coefficient learning device according to claim 11, further comprising a student image supply unit that inputs the generated plurality of student image groups a plurality of times. The discrimination operation is a product-sum operation of each discrimination coefficient in the discrimination coefficient group and each pixel value in the prediction tap in the change target image,
The discriminant coefficient calculating unit adds the result of the discriminant calculation in the student image other than the learning image to the square sum of the difference between the result of the discriminant calculation in the learning image and the first target value. The discriminant coefficient learning device according to claim 11, wherein the discriminant coefficient group that minimizes a value obtained by adding a sum of squares of differences from the target value is calculated for each of the plurality of classes. A student image input step of inputting a plurality of student image groups including a plurality of student images, each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from the image, using any pixel in the input student image as the target pixel. Tap extraction step to extract;
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A first target value is generated when the discrimination image is input using any one of the student images in the student image group as a discrimination image, and the second target value is input when the student image other than the discrimination image is input. A target value generation step for generating a target value of
When the resolution of an image is changed, a discrimination coefficient group to be used in a discrimination calculation for discriminating whether or not the image to be changed is similar to the learning image is set in the first and second target values and the prediction tap. A discriminant coefficient learning device control method comprising: a discriminant coefficient calculation step for calculating each class based on a pixel value of each pixel. A student image input step of inputting a plurality of student image groups including a plurality of student images, each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from the image, using any pixel in the input student image as the target pixel. Tap extraction step to extract;
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A first target value is generated when the discrimination image is input using any one of the student images in the student image group as a discrimination image, and the second target value is input when the student image other than the discrimination image is input. A target value generation step for generating a target value of
When the resolution of an image is changed, a discrimination coefficient group to be used in a discrimination calculation for discriminating whether or not the image to be changed is similar to the learning image is set in the first and second target values and the prediction tap. And a discrimination coefficient calculation step for calculating each class based on the pixel value of each pixel. A student image input unit for inputting each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from any pixel in the input student image as the target pixel. A tap extraction unit that extracts from the image of
A class classification unit for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into one of a plurality of classes;
A teacher pixel extraction unit that extracts one or more pixels in the teacher image, which is an image obtained by changing the resolution of the input student image, as a teacher pixel;
A prediction coefficient calculation unit that calculates a prediction coefficient group to be used in a prediction calculation for predicting the pixel value of the teacher pixel for each class based on the pixel value of each pixel in the prediction tap and the pixel value of the teacher pixel. A prediction coefficient learning apparatus comprising: The prediction coefficient learning device according to claim 16, wherein the student image input unit generates and inputs the plurality of student images by passing the teacher image through a plurality of filters. A student image input step for inputting each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from any pixel in the input student image as the target pixel. Tap extraction step to extract from the image of
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A teacher pixel extraction step of extracting one or more pixels in the teacher image, which is an image obtained by changing the resolution of the input student image, as a teacher pixel;
A prediction coefficient calculation step of calculating a prediction coefficient group to be used in a prediction calculation for predicting the pixel value of the teacher pixel for each class based on the pixel value of each pixel in the prediction tap and the pixel value of the teacher pixel. A control method for a prediction coefficient learning apparatus comprising: A student image input step for inputting each of a plurality of different images as a student image;
A class tap which is a first pixel group including the target pixel and a prediction tap which is a second pixel group including the target pixel are selected from any pixel in the input student image as the target pixel. Tap extraction step to extract from the image of
A class classification step for classifying a shape pattern of a three-dimensional distribution of pixel values of each pixel in the class tap into any of a plurality of classes;
A teacher pixel extraction step of extracting one or more pixels in the teacher image, which is an image obtained by changing the resolution of the input student image, as a teacher pixel;
A prediction coefficient calculation step of calculating a prediction coefficient group to be used in a prediction calculation for predicting the pixel value of the teacher pixel for each class based on the pixel value of each pixel in the prediction tap and the pixel value of the teacher pixel. A program that causes a computer to execute.

Images