KR20060136335A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

 The image processing apparatus includes a down converter and an image converter for converting the first image data into second image data of higher quality. The image converter is a prediction tap extraction unit that extracts, as prediction taps, a plurality of pixels used to predict a subject pixel of second image data from first image data, and classifies the class of the target pixels from first image data. A class tap extracting unit for extracting a plurality of pixels used to classify as a class tap, a class classifying unit for classifying a class of the target pixel based on the class tap, and coefficients corresponding to the plurality of classes predetermined by learning; A coefficient output unit for outputting a coefficient corresponding to the class of the target pixel, and a calculator for determining the target pixel by performing a prediction operation using the coefficient corresponding to the class of the target pixel and the prediction tap.

Image Processing, High Definition, HD, SD, D1, Class, Upconvert, Downconvert, Upconverter, Downconverter

Description

 Image processing apparatus, image processing method, and program {IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND PROGRAM}

1 is a block diagram showing the configuration of a broadcast system according to an embodiment of the present invention.

2 is a diagram showing main parts of a broadcasting station.

3 is a block diagram showing the configuration of a digital broadcast receiver.

4 is a flowchart illustrating an operation performed by a digital broadcast receiving apparatus.

5 is a block diagram showing an example of the configuration of an image converter.

6 is a flowchart for describing processing by the image converter 48.

7 is a block diagram illustrating a configuration example of a learning device that learns tap coefficients.

8 is a flowchart showing a process of learning tap coefficients.

9 is a block diagram showing another configuration example of an image converter.

10 is a flowchart for explaining processing performed by an image converter.

11 is a block diagram illustrating another configuration example of a learning device that performs learning of coefficient seed data.

12 is a flowchart illustrating a learning process for performing learning of coefficient seed data.

Figure 13 illustrates another approach to the learning of coefficient seed data.

14 is a block diagram illustrating another configuration example of a learning device for learning coefficient seed data.

15 is a block diagram showing a configuration example of a computer according to an embodiment of the present invention.

Cross Reference of Related Application

The present invention includes an object related to Japanese Patent Application No. JP-2005-1859987, filed with the Japan Patent Office on June 27, 2005, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD The present invention relates to an image processing apparatus, an image processing method, and a program, and more particularly, to an image processing apparatus, an image processing method, and a program for allowing a user to enjoy high quality image data.

In the conventional terrestrial analog broadcasting, an SD (Standard Definition) image having an aspect ratio of 4: 3 corresponding to a composite signal conforming to the NTSC (National Television System Committee) method was broadcast.

In recent years, terrestrial digital broadcasting has been started in addition to existing terrestrial analog broadcasting. In terrestrial digital broadcasting, since an SD image having an aspect ratio of 16: 9 and a high definition (HD) image are broadcast in the form of a component signal, a so-called "D1 image". High quality images can be provided.

Since broadcast equipment such as SD images (hereinafter, referred to as "D1 images") with an aspect ratio of 16: 9 and cameras for capturing HD images are expensive, it takes time for such expensive broadcast equipment to be widely distributed to broadcasting stations.

Therefore, some broadcasting stations convert the composite signal image to be broadcast by terrestrial analog broadcasting into a component signal image, and then up-convert by increasing the number of pixels by interpolating the component signal image. By this operation, only the format of the resultant pixel is the same as the D1 image or HD image broadcasted by terrestrial digital broadcasting.

By converting a composite signal image into a component signal image and up-converting the component signal image, an image having the same format as a D1 image or an HD image is referred to as a " quasi-D1 image " or " quasi-HD image " quasi-HD image) ".

Since the quasi-D1 image and the quasi-HD image are images generated by converting a composite signal image into a component signal image, such quasi-image includes noise called so-called "cross-color" or "dot crawl." Occurs. In addition, since the quasi-D1 image and the quasi-HD image are images obtained by up-converting an image obtained by converting a composite signal image into a component signal image, so-called cross color or dot crawl becomes more noticeable by up-conversion. In particular, in the quasi-HD image, since the number of pixels interpolated by the up-conversion is larger than that of the quasi-D1 image, so-called cross color or dot crawl becomes more noticeable.

In consideration of the characteristics peculiar to the composite signal, a method of converting an image into a high quality image is proposed, for example, in Japanese Unexamined Patent Publication No. 10-056622.

Even if the quasi-HD image is broadcast by terrestrial digital broadcasting, since only the format of the quasi-HD image is converted in the same way as the HD image, it is difficult for a user (viewer) to enjoy a high-definition image with such a quasi-HD image.

Therefore, there is a demand for a user to enjoy high quality images.

According to an embodiment of the present invention, there is provided an image processing apparatus for processing first image data and outputting second image data of higher quality than the first image data.

The image processing apparatus uses down conversion means for down converting the input image data by reducing the number of pixels of the input image data, and using the input image data down converted by the down conversion means as the first image data. And image conversion means for converting the first image data into the second image data.

The image conversion means comprises: prediction tap extracting means for extracting, as a prediction tap, a plurality of pixels used to predict a subject pixel of the second image data from the first image data, the first Class tap extracting means for extracting a plurality of pixels used to classify the target pixel into one of a plurality of classes from image data as a class tap, and class classification for classifying the class of the target pixel based on the class tap Means, coefficient output means for outputting a coefficient corresponding to the class of the target pixel from among coefficients corresponding to a plurality of classes predetermined by learning, and the coefficient and the prediction tap corresponding to the class of the target pixel. And calculating means for determining the target pixel by performing a prediction operation.

The above-described image storing apparatus may further include determining means for determining the type of the input image data. If the determining means determines that the type of the input image data is up-converted image data obtained by increasing the number of pixels of the image data having different types, the down-converting means decreases the number of pixels of the input image data. The input image data is down converted.

The input image data may be broadcast image data.

The image processing apparatus described above includes selecting means for selecting one coefficient set from among a plurality of coefficient sets associated with a plurality of corresponding upconversion techniques used for the upconversion, based on channel information about a channel of the broadcast image data. It may further include. The coefficient output means may output a coefficient corresponding to the class of the target pixel among the coefficient sets selected by the selection means.

According to another embodiment of the present invention, there is provided an image processing method for processing first image data and outputting second image data of higher quality than the first image data.

The image processing method may include: down-converting the input image data by reducing the number of pixels of input image data, and using the down-converted image data as the first image data, using the first image data as the first image data. Converting into two image data.

The converting of the first image data into the second image data may include extracting, as a prediction tap, a plurality of pixels used to predict a target pixel of the second image data from the first image data. Extracting, as class taps, a plurality of pixels used to classify the target pixel into one of a plurality of classes from image data, classifying the class of the target pixel based on the class tap, predetermined by learning; Outputting a coefficient corresponding to the class of the target pixel among coefficients corresponding to a plurality of classes, and determining the target pixel by performing a prediction operation using the coefficient corresponding to the class of the target pixel and the prediction tap. It includes a step.

According to another embodiment of the present invention, a program is provided for causing a computer to execute image processing for processing first image data and outputting second image data of higher quality than the first image data. The program may further include down-converting the input image data by reducing the number of pixels of input image data, and using the down-converted image data as the first image data, using the first image data in the second image. Converting to data. The converting of the first image data into the second image data may include extracting, as a prediction tap, a plurality of pixels used to predict a target pixel of the second image data from the first image data. Extracting, as class taps, a plurality of pixels used to classify the target pixel into one of a plurality of classes from image data, classifying the class of the target pixel based on the class tap, predetermined by learning; Outputting a coefficient corresponding to the class of the target pixel among coefficients corresponding to a plurality of classes, and determining the target pixel by performing a prediction operation using the coefficient corresponding to the class of the target pixel and the prediction tap. It includes a step.

According to another embodiment of the present invention, there is provided an image processing apparatus that processes first image data and outputs second image data of higher quality than the first image data. The image processing apparatus uses down conversion means for down converting the input image data by reducing the number of pixels of the input image data, and using the input image data down converted by the down conversion means as the first image data. And image conversion means for converting the first image data into second image data.

The image converting means includes predictive tap extracting means for extracting, as a predictive tap, a plurality of pixels used to predict a target pixel of the second image data from the first image data, and a plurality of the target pixels from the first image data. Class tap extracting means for extracting a plurality of pixels used to classify as one of the classes of as class taps, class classifying means for classifying the class of the target pixel based on the class taps, predetermined parameters, and seed of coefficients Generating means for generating coefficients corresponding to each of the plurality of classes from the seed data used for?), And coefficients for outputting coefficients corresponding to the class of the target pixel among the coefficients corresponding to the plurality of classes generated by the generating means Output means and coefficients and images corresponding to the class of the target pixel By using the prediction tap for performing predictive computation includes computation means for determining the target pixel.

The above-described image storing apparatus may further include determining means for determining the type of the input image data. If the determining means determines that the type of the input image data is up-converted image data obtained by increasing the number of pixels of the image data having different types, the down-converting means decreases the number of pixels of the input image data. And down-convert the input image data.

The input image data may be broadcast image data.

The generating means may generate coefficients corresponding to each of the plurality of classes by using channel information about a channel of the broadcast image data as the predetermined parameter.

According to another embodiment of the present invention, there is provided an image processing method for processing first image data and outputting second image data of higher quality than the first image data. The image processing method may include: down-converting the input image data by reducing the number of pixels of input image data; and using the down-converted input image data as first image data, using the first image data as a second image. Converting the image data. The converting the first image data into the second image data may include extracting, as prediction taps, a plurality of pixels used to predict a target pixel of second image data from the first image data. Extracting, as class taps, a plurality of pixels used to classify the target pixel into one of a plurality of classes from image data, classifying the class of the target pixel based on the class tap, predetermined parameters and coefficients Generating coefficients corresponding to each of a plurality of classes from seed data used as seeds and predetermined by learning, and outputting coefficients corresponding to the class of the target pixel among the generated coefficients corresponding to the plurality of classes And a coefficient corresponding to the class of the target pixel and the prediction tap. Determining the target pixel by performing a prediction operation.

According to another embodiment of the present invention, a program is provided for causing a computer to execute image processing for processing first image data and outputting second image data of higher quality than the first image data. The program may further include down-converting the input image data by reducing the number of pixels of the input image data, and using the down-converted input image data as the first image data, using the first image data as the second image data. Converting. The converting of the first image data into the second image data may include extracting, as a prediction tap, a plurality of pixels used to predict a target pixel of second image data from the first image data. Extracting, as a class tap, a plurality of pixels used to classify the target pixel into one of a plurality of classes, class classifying means for classifying the class of the target pixel based on the class tap, predetermined parameters; Generating coefficients corresponding to each of a plurality of classes from seed data used as seed of coefficients and predetermined by learning, and outputting coefficients corresponding to the class of the target pixel among the generated coefficients corresponding to the plurality of classes And coefficients corresponding to the class of the target pixel and the Determining the target pixel by performing a prediction operation using a prediction tap.

According to an embodiment of the present invention, a user can enjoy high quality image data.

Prior to describing the embodiments of the present invention, the corresponding relationship between the features of the claims and the embodiments of the present invention will be described below. This description is intended to confirm that the embodiments supporting the claimed subject matter are described in the detailed description of the invention. Thus, although a component is not described in the following embodiments as relating to any feature of the invention, it does not necessarily mean that the component is not related to any feature of the claims. On the contrary, if a component is described herein as being related to a certain feature of the claims, it does not necessarily mean that the component is not related to any other feature of the claims.

An image processing apparatus (for example, the digital broadcast receiving apparatus 2 of FIG. 3) according to an embodiment of the present invention processes first image data and outputs second image data of higher quality than the first image data. do. The image processing apparatus includes a down converting means (e.g., the down converter 45 of FIG. 3) for reducing the number of pixels of the input image data and down converting the input image data, and the input image data down converted by the down converting means. Image conversion means (for example, the image converter 48 in Fig. 3) for converting the first image data into the second image data, using the first image data. Prediction tap extracting means (e.g., tap extracting unit 161 of FIG. 5) for extracting, as a predictive tap, a plurality of pixels used to predict a target pixel of the second image data from the first image data; Class tap extracting means for extracting, as a class tap, a plurality of pixels used to classify the target pixel into one of a plurality of classes from one image data (for example, in FIG. Tap extraction unit 162], class classification means (e.g., class classification unit 163 of FIG. 5) for classifying the class of the target pixel based on the class taps;

Coefficient output means (e.g., coefficient memory 164 of FIG. 5) which outputs coefficients corresponding to the class of the target pixel among coefficients corresponding to a plurality of classes determined in advance by learning, and the target pixel Calculation means (eg, the prediction unit 165 of FIG. 5) for determining the target pixel by performing a prediction operation using the coefficient corresponding to the class and the prediction tap.

The image processing apparatus may further include judging means (for example, the judging unit 44 of FIG. 3) for judging the type of the input image data. If the determining means determines that the type of the input image data is up-converted image data obtained by increasing the number of pixels of the image data having different types, the down-converting means decreases the number of pixels of the input image data. Down conversion of the input image data.

Selecting means for selecting one coefficient set from among a plurality of coefficient sets associated with a plurality of corresponding upconversion techniques used for the upconversion based on channel information about a channel of the broadcast image data [Example For example, the coefficient selector 160 of FIG. 5 may be further included. The coefficient output means outputs a coefficient corresponding to the class of the target pixel among the coefficient sets selected by the selection means.

According to an embodiment of the present invention, an image processing method or program for processing first image data and outputting second image data having a higher quality than the first image data may reduce the number of pixels of the input image data so that the input is performed. Down-converting image data (for example, step S6 in FIG. 4), and converting the first image data into the second image data using the down-converted image data as the first image data. A step (for example, step S7 of FIG. 4) is included. The converting of the first image data into the second image data may include extracting, as a prediction tap, a plurality of pixels used to predict a target pixel of the second image data from the first image data (for example, For example, step S1111 of FIG. 6), extracting, as a class tap, a plurality of pixels used to classify the target pixel into one of a plurality of classes from the first image data (for example, step S1111 of FIG. 6). Classifying the class of the target pixel based on the class tap (for example, step S1112 of FIG. 6), and corresponding to the class of the target pixel among coefficients corresponding to a plurality of classes predetermined by learning. Outputting a coefficient (e.g., step S1113 in FIG. 6), and performing a prediction operation using the coefficient corresponding to the class of the target pixel and the prediction tap. By performing the step of determining the target pixel and a (for example, step S1114 in Fig. 6).

An image processing apparatus (for example, the digital broadcast receiving apparatus 2 of FIG. 3) according to another embodiment of the present invention processes the first image data to generate second image data of higher quality than the first image data. Output The image processing apparatus down-converts the down-converting means (e.g., the down converter 45 in FIG. 3) for down-converting the input image data by reducing the number of pixels of the input image data. Image conversion means (for example, the image converter 48 of FIG. 3) for converting the first image data into the second image data by using the received input image data as the first image data. The image converting means includes predictive tap extracting means (for example, a tap extracting unit of FIG. 9) for extracting, as a predictive tap, a plurality of pixels used to predict a target pixel of the second image data from the first image data. (161)] class tap extracting means (for example, a tap extracting unit of FIG. 9) for extracting, as class taps, a plurality of pixels used to classify the target pixel into one of a plurality of classes from the first image data; 162), class classification means (e.g., the class classification unit 163 of FIG. 9) for classifying the class of the target pixel based on the class tap, and used as a seed of a predetermined parameter and coefficient. Generating means (e.g., coefficient generator 166 of FIG. 9) for generating coefficients corresponding to each of a plurality of classes from the seed data to be generated, and the plurality of classes generated by the generating means. Coefficient output means (for example, the coefficient memory 164 of FIG. 9) for outputting a coefficient corresponding to the class of the target pixel among the corresponding coefficients, and a coefficient corresponding to the class of the target pixel and the prediction tap Calculation means (for example, the prediction unit 165 of FIG. 9) for determining the target pixel by performing a prediction operation using the "

The image processing apparatus may further include judging means (for example, the judging unit 44 of FIG. 3) for judging the type of the input image data. If the determining means determines that the type of the input image data is up-converted image data obtained by increasing the number of pixels of the image data having different types, the down-converting means decreases the number of pixels of the input image data. Down conversion of the input image data.

According to another embodiment of the present invention, an image processing method or program for processing first image data and outputting second image data of higher quality than the first image data may reduce the number of pixels of the input image data to thereby input the image data. Down-converting image data (for example, step S6 in FIG. 4), and using the down-converted input image data as the first image data, converting the first image data to the second image data. A step (for example, step S7 of FIG. 4) is included. The converting of the first image data into the second image data may include extracting, as a prediction tap, a plurality of pixels used to predict a target pixel of the second image data from the first image data (for example, For example, step S1215 of FIG. 10), extracting, as a class tap, a plurality of pixels used to classify the target pixel into one of a plurality of classes from the first image data (for example, step S1215 of FIG. 10). Classifying the class of the target pixel based on the class tap (for example, step S1216 of FIG. 10), a plurality of classes from predetermined parameters and seed data used as seeds of coefficients and predetermined by learning. Generating coefficients corresponding to each one (for example, step S1213 of FIG. 10), wherein the generation coefficients among the generated coefficients corresponding to the plurality of classes are generated. Outputting a coefficient corresponding to the class of the pixel (for example, step S1217 of FIG. 10), and performing a prediction operation using the coefficient corresponding to the class of the target pixel and the prediction tap to obtain the target pixel. Determining (eg, step S1218 of FIG. 10).

EMBODIMENT OF THE INVENTION Hereinafter, with reference to an accompanying drawing, embodiment of this invention is described in detail.

1 is a block diagram illustrating a broadcast system according to an embodiment of the present invention. Here, the system is a logical collection of a plurality of devices, and the devices do not necessarily need to be in the same housing.

The broadcasting station 1 broadcasts an image or sound of a certain program by terrestrial analog broadcasting or terrestrial digital broadcasting. Other broadcasting stations also broadcast images or audio by terrestrial analog broadcasting or terrestrial digital broadcasting. Below, only the image of the program broadcast by the broadcasting station 1 is demonstrated, and the description regarding the audio | voice of the same program is abbreviate | omitted.

The digital broadcast receiver 2 receives an image as a program transmitted from the broadcasting station 1 to terrestrial digital broadcasting, and displays the received image on a monitor (not shown). The analog broadcast receiver 3 receives an image as a program transmitted from the broadcasting station 1 to terrestrial analog broadcast, and displays the received image on a monitor (not shown).

FIG. 2 is a block diagram showing the main part of the broadcasting station 1 shown in FIG.

The broadcasting station 1 includes an SD camera 11 which picks up an SD image having an aspect ratio of 4: 3. The broadcasting station 1 converts (encodes) the SD image associated with the component signal picked up by the SD camera 11 into a composite signal of the NTSC system by the NTSC encoder 21, and then converts the NTSC composite signal into a terrestrial wave. Broadcast by analog broadcast.

When the broadcasting station 1 includes the SD camera 12 which picks up a D1 image, and the HD camera 13 which picks up an HD image, as shown by the dotted line in FIG. 2, it captures by the SD camera 12 The D1 image of the component signal thus obtained and the HD image of the component signal picked up by the HD camera 13 are broadcasted on the channel of the frequency band allocated to the broadcasting station 1 by terrestrial digital broadcasting.

When the broadcasting station 1 does not include the SD camera 12, the NTSC decoder 22 converts (decodes) the NTSC composite signal output from the NTSC encoder 21 into a component signal, thereby converting the up converter 23 into a component. Supplies). The up converter 23 then up-converts the component signal so that the component signal image is in the same format as the D1 image output from the SD camera 12, and broadcasts the resulting quasi-D1 image by terrestrial digital broadcasting.

When the broadcasting station 1 does not include the HD camera 13, the NTSC decoder 22 converts (decodes) the NTSC composite signal output from the NTSC encoder 21 into a component signal, thereby converting the up converter 24 into a component. To feed. The up converter 24 then up-converts the component signal so that the component signal image is in the same format as the HD image output from the HD camera 13, and broadcasts the resulting quasi-HD image in terrestrial digital broadcasting.

FIG. 3 is a block diagram showing the configuration of the digital broadcast receiver 2 shown in FIG. The tuner 41 receives a signal of terrestrial digital broadcasting, extracts a signal component corresponding to a channel selected by a user through a remote commander (not shown) from the received signal, and then supplies the extracted signal component to the demultiplexer 42. do. The demultiplexer 42 separates the necessary packet from the signal component output from the tuner 41 and supplies the packet to the decoder 43. The decoder 43 then decodes the MPEG (Moving Picture Expert Group) encoded image data contained in the packet according to the MPEG method, and supplies the resulting image data to the determination unit 44.

Although not described with reference to FIG. 2, the broadcasting station 1 performs MPEG encoding on image data, and broadcasts the encoded image data by terrestrial digital broadcasting.

The determination unit 44 determines the type of image data supplied from the decoder 43, and supplies the image data to the down converter 45, the image converter 46, or the image converter 47 based on the determination result. do.

More specifically, the judging unit 44 converts the type of the image data supplied from the decoder 43 into a component signal of the NTSC system into a component signal, and up-converts the component signal by up-converting the component signal by increasing the number of pixels. When it determines with quasi-HD image data which is one kind of the image data which was obtained, the quasi-HD image data which is this image data is supplied to the down converter 45. FIG.

The determination unit 44 converts the type of image data from the decoder 43 into a component signal of the NTSC system into a component signal, and increases the number of pixels of the up-converted image data obtained by up-converting the component signal. When it is determined that the other type is quasi-D1 image data or when it is determined that it is D1 image data, the quasi-D1 image data or D1 image data which is the image data is supplied to the image converter 47.

When determining the kind of the image data from the decoder 43 as the HD image data, the determination unit 44 supplies the image converter 46 with the HD image data which is the image data.

The determination as to whether the image data supplied from the decoder 43 is HD image data or quasi-HD image data, or whether the image data is D1 image data or quasi-D1 image data is a format of image data, i.e., 1 to form image data. It may be made based on the number of pixels of the frame.

The determination as to whether the image data supplied from the decoder 43 is HD image data or quasi-HD image data can be made based on the activity of the image data. More specifically, HD image data is high resolution, but quasi HD image data is not so high resolution since it is only image data obtained by converting a composite signal of NTSC type into a component signal. Therefore, when the image data from the decoder 43 shows the activity of a certain threshold value or more, the image data can be determined as HD image data. Conversely, when the image data from the decoder 43 shows activity below a certain threshold, the image data can be determined as quasi-HD image data.

Both HD image data and quasi-HD image data have an aspect ratio of 16: 9. In practice, however, quasi-HD image data is image data obtained by converting an NTSC composite signal having an aspect ratio of 4: 3 into a component signal and up-converting the component signal to an aspect ratio of 16: 9. Therefore, black is added to the left and right edges of 4: 3 image data in the resulting semi-HD call data. Therefore, the determination as to whether the image data from the decoder 43 is HD image data or quasi-HD image data can be made by determining whether black image data is observed at the left and right ends of the image data.

In the example of FIG. 3, since the image data supplied from the decoder 43 is supplied to the image converter 47 regardless of whether it is D1 image data or quasi-D1 image data, the image data is quasi-D1 image data or D1. The determination as to whether it is image data is unnecessary. However, if this determination is made, it can be made similarly to determining whether it is HD image data or quasi-HD image data.

The down converter 45 reduces the pixel number of the quasi-HD image data supplied from the determination unit 44 to the number of pixels of the component signal image output from the NTSC decoder 22 shown in FIG. The down converter 45 also down-converts the quasi-HD image data by using a low pass filter (LPF), and then supplies the resulting SD image data to the image converter 48.

The image converter 46 converts the HD image data supplied from the determination unit 44 into higher quality HD image data, and then supplies the resulting HD image data to the image output unit 49.

 The image converter 47 converts the quasi-D1 image data or D1 image data supplied from the determination unit 44 into higher quality HD image data, and then supplies the resulting HD image data to the image output unit 49. do.

 The image converter 48 converts the SD image data supplied from the down converter 45 into higher quality HD image data, and then supplies the resulting HD image data to the image output unit 49.

 The image output unit 49 supplies HD image data supplied from any of the image converters 46 to 48 to a monitor (not shown), and displays the HD image data on the monitor.

With reference to the flowchart of FIG. 4, the operation performed by the digital broadcast receiver 2 shown in FIG. 3 will be described below.

In the digital broadcast receiver 2, in step S1, the tuner 41 receives a signal of terrestrial digital broadcast, extracts (selects) a signal component corresponding to the channel selected by the user through the remote commander from the signal, The extracted signal component is supplied to the demultiplexer 42. The demultiplexer 42 then separates the necessary packets from the signal components supplied from the tuner 41 and supplies the packets to the decoder 43. After that, in step S2, the decoder 43 decodes the MPEG coded image data contained in the packet according to the MPEG method, and supplies the resulting image data to the determination unit 44.

In step S3, the determination unit 44 determines the type (format) of the image data supplied from the decoder 43, and based on the determination result, the image data is converted into the down converter 45 and the image converter 46. ) Or to the image converter 47.

When it is determined in step S3 that the image data is HD image data, the determination unit 44 supplies HD image data, which is image data, to the image converter 46.

Then, in step S4, the image converter 46 sets the HD image data supplied from the determination unit 44 as the first image data, and sets the HD image data that is higher in quality than the first image data to the second image data. Set to. Then, the image converter 46 converts the first image data into the second image data, and supplies the second image data to the image output unit 49. The image output unit 49 then supplies HD image data to the monitor.

If it is determined in step S3 that the image data from the decoder 43 is D1 image data or quasi-D1 image data, the determination unit 44 supplies the image data to the image converter 47.

Subsequently, in step S5, the image converter 47 sets the D1 image data or quasi-D1 image data supplied from the determination unit 44 as the first image data, and removes the HD image data of higher quality than the first image data. 2 Set to image data. The image converter 47 then converts the first image data into second image data, and supplies the resulting high quality HD image data to the image output unit 49. The image output unit 49 then supplies HD image data to the monitor.

If it is determined in step S3 that the image data from the decoder 43 is quasi-HD image data, the determination unit 44 supplies the quasi-HD image data to the down converter 45.

Thereafter, in step S6, the down converter 45 selects the quasi-HD image data so that the number of pixels of the quasi-HD image data is reduced to be equal to the number of pixels of the component signal image output from the NTSC decoder 22 shown in FIG. The down conversion is performed, and the resultant SD image data is supplied to the image converter 48.

Then, in step S7, the image converter 48 sets the SD image data supplied from the down converter 45 as the first image data, and further sets the HD image data of higher quality than the first image data as the second image data. Set it. Then, the image converter 48 converts the first image data into the second image data, and supplies the resulting high quality HD image data to the image output unit 49. The image output unit 49 then supplies HD image data to the monitor.

The image converter 48 broadcasts channel information about a channel currently received from the tuner 41, that is, quasi-HD image data up-converting the SD image data supplied from the down converter 45 to the image converter 48. Receive information about the channel being played. The image converter 48 converts SD image data supplied from the down converter 45 into HD image data based on channel information about a channel for broadcasting quasi-HD image data obtained by up-converting the SD image data.

The image conversion processing performed by the image converter 48 of FIG. 3 is as follows.

The image converter 48 performs image conversion for converting the first image data into second image data of higher quality than the first image data.

 As described above, the image converter 48 sets SD image data supplied from the down converter 45 as first image data, and sets HD image data of higher quality than the first image data as second image data. Next, image conversion processing for converting the first image data into the second image data is performed. According to this image conversion process, various processing operations can be realized depending on how the first and second image data are defined.

More specifically, when the second image data is set to high resolution image data and the first image data is set to low resolution image data obtained by lowering the resolution of the second image data, the image conversion process improves the resolution. May be a treatment. If the second image data is set to high S / N (Siginal / Noise) image data, and the first image data is low S / N image data obtained by lowering the S / N of the second image data, image conversion processing May be a noise removal process. If the second image data is set as HD image data and the first image data is SD image data obtained by lowering the number of pixels and the resolution of the second image data, the image conversion process may be a process of converting the SD image into an HD image. have.

5 is a block diagram showing an example of the functional configuration of an image converter 48 that performs the above-described image conversion processing.

The picture converter 48 includes a coefficient selector 160, tap extraction units 161, 162, class classification unit 163, coefficient memory 164, and prediction unit 165.

The image converter 48 receives SD image data output from the down converter 45 shown in FIG. 3 as first image data. The image converter 48 also further tunes the channel information on the channel of the SD image data output from the down converter 45, that is, the channel information on the channel broadcasting the quasi-HD image data before being down-converted into the SD image data. (41) (FIG. 3).

SD image data, which is the first image data, is supplied to the tap extracting units 161, 162, and channel information about a channel of the SD image data is supplied to the coefficient selector 160.

The coefficient selector 160 selects one set of tap coefficients from among a plurality of tap coefficient sets stored in the coefficient memory 164 to be described later based on the channel information about the SD image data. By "memory bank swtching" the set of selected tap coefficients is validated. That is, the coefficient memory 164 reads out and outputs tap coefficients corresponding to the information supplied from the class classification unit 163 among the plurality of tap coefficient sets, and the coefficient selector 160 reads the tap coefficients by the coefficient memory 164. Select the set of tap coefficients containing the coefficients. The set of tap coefficients read by the coefficient memory 164 is referred to as an " effective set of tap coefficients. &Quot;

The tap extracting unit 161 sets one by one pixels constituting the HD image data as second image data to be converted from the first image data as target pixels, and further sets several pixels constituting the first image data, It extracts as a prediction tap used to predict the pixel value of the target pixel. In this case, since the HD image data as the second image data has not yet been determined, it is only assumed in a virtual manner.

More specifically, the tap extracting unit 161 may include a plurality of pixels (for example, pixels closest to the target pixel and pixels adjacent to the spatially adjacent pixel) that are spatially or temporally close to the target pixel. Extract).

The tap extracting unit 162 extracts several pixels constituting the first image data as a class tap used to classify the class of the target pixel, that is, as a class tap used to classify the target pixel into one class. do.

For simplicity, the prediction tap and the class tap are assumed to be composed of the same tap structure, that is, the same pixel. However, prediction taps and class taps may have different tap structures.

The prediction tap extracted by the tap extraction unit 161 is supplied to the prediction unit 165, and the class tap extracted by the tap extraction unit 162 is supplied to the class classification unit 163.

The class classification unit 163 classifies the class of the target pixel based on the class tap supplied from the tap extraction unit 162, and supplies the class code corresponding to the determined class to the coefficient memory 164.

As a method of classifying, ADRC (Adaptive Dynamic Range Coding) can be adopted, for example.

In this method, the pixel value of the pixels constituting the class tap is subjected to ADRC processing, and the class of the target pixel is determined according to the resulting ADRC code.

In K-bit ADRC, the maximum value MAX and the minimum value MIN of the pixel values of the pixels constituting the class tap are detected, and then DR = MAX-MIN, and the local dynamic range of the set of pixels constituting the class tap. Set to). Based on this dynamic range DR, the pixel value of the pixels constituting the class tap is requantized into K bits. More specifically, the splits subtracting the minimum value MIN from the pixel value of each pixel constituting the class tap, and the result by DR / 2 ^K (quantizes). Then, a bit string of K bit pixel values constituting the class taps, arranged in a predetermined order, is output as an ADRC code. When the class tap is 1-bit ADRC processed, the minimum value MIN is subtracted from the pixel value of each pixel constituting the class tap, and the result is divided by 1/2 of the difference between the maximum value MAX and the minimum value MIN Truncation), the pixel value of each pixel can be binarized (1 bit). Subsequently, the bit strings arranged in a predetermined predetermined order are output as ADRC codes.

The class classification unit 163 can output the pattern of the level distribution of the pixel values of the pixels constituting the class tap as a class code. However, in this case, if the class tap is composed of N pixels, and K bits are assigned to pixel values of each pixel, the number of cases of class codes output from the class classification unit 163 is (2 ^N ) ^K. ) Is an immense number that is exponentially proportional to the number of bits K allocated to the pixel value of the pixel.

Therefore, the class classification unit 163 preferably performs class classification after compressing the information amount of the class tap by the above-described ADRC processing or vector quantization.

The coefficient memory 164 stores a set of tap coefficients of each class predetermined by the learning to be described later, and is stored at an address corresponding to the class code supplied from the class classification unit 163 among the set of stored tap coefficients. The tap coefficient corresponding to the existing tap coefficient, that is, the class represented by the class code supplied from the class classification unit 163 is output.

The quasi-HD image data is image data before conversion to SD image data, and is used as first image data that is an object subjected to image conversion processing by the image converter 48. HD image data ". As described with reference to Fig. 2, the semi-HD image data is obtained by converting the composite signal image data into component signal image data in the broadcasting station 1, and up-converting the component signal image data.

In this case, various upconversion techniques can be employed. For example, a pixel having the same pixel value as that of the pixel of the image data to be up-converted can be interpolated into component signal image data. Alternatively, a pixel having a weighted average value of pixel values of a plurality of pixels of the image data to be up-converted can be interpolated as component signal image data.

The upconversion technique employed may vary depending on the broadcast station.

Therefore, in the image converter 48, regardless of which upconversion technique is employed to obtain quasi-HD image data, the SD image data down-converted from the quasi-HD image data can be converted into HD image data in an appropriate manner. The coefficient memory 164 stores a plurality of tap coefficient sets associated with various upconversion techniques.

As mentioned above, the upconversion technique depends on the broadcast station, i.e. on the channel. Therefore, the coefficient selector 160 selects one set of tap coefficient sets from among the plurality of tap coefficient sets stored in the coefficient memory 164 based on the channel information about the channel of the SD image data as the first image data. . That is, the coefficient selector 160 performs a tap coefficient suitable for the image conversion processing to be performed on the SD image data down-converted by the up-conversion technique employed by the broadcasting station associated with the channel indicated by the channel information. Select a set and validate the set of selected tap coefficients.

The coefficient memory 164 then selects from the set of valid tap coefficients the tap coefficients stored at an address corresponding to the class code supplied from the class classification unit 163, that is, the tap coefficients corresponding to the class of the target pixel. Output

Therefore, the tap coefficients output from the coefficient memory 164 are image converted to be performed on the SD image data down-converted the quasi-HD image data acquired by the up-conversion technique employed by the broadcasting station corresponding to the channel indicated by the channel information. Tap coefficient appropriate for processing.

The tap coefficient is a coefficient multiplied by the input data in the so-called "tap" in the digital filter.

The prediction unit 165 receives the prediction taps from the tap extraction unit 161 and the tap coefficients from the coefficient memory 164, and uses the prediction taps and the tap coefficients to predict the true value of the target pixel. Perform a predetermined prediction operation to determine. As a result, the prediction unit 165 determines (determines) the pixel value (prediction value) of the target pixel, that is, the pixel value of the pixel constituting the second image data.

With reference to the flowchart of FIG. 6, the image conversion process performed by the image converter 48 shown in FIG. 5 is demonstrated below.

In step S1110, the coefficient selector 160 corresponds to the channel indicated by the channel information among the plurality of tap coefficient sets stored in the coefficient memory 164 based on the channel information received from the tuner 41 (FIG. 3). A set of tap coefficients suitable for the image conversion processing to be performed on the down-converted SD image data obtained by the up-conversion technique employed by the broadcasting station is selected to validate the selected tap coefficients.

On the other hand, the tap extracting unit 161 selects a pixel constituting the HD image data as the second image data corresponding to the SD image data as the first image data supplied from the down converter 45 (FIG. 3). Select sequentially as Then, in step S1111, the tap extraction units 161 and 162 extract the prediction tap and the class tap for the target pixel selected by the tap extraction unit 161, respectively. The tap weight tool unit 161 supplies the prediction tap to the prediction unit 165 and the class tap to the class classification unit 163.

In step S1112, as the class tap for the target pixel is received from the tap extracting unit 162, the class classification unit 163 classifies the class of the target pixel based on the received class tap. In addition, the class classification unit 163 outputs a class code indicating the determined class of the target pixel to the coefficient memory 164.

Then, in step S1113, the coefficient memory 164, from the set of valid tap coefficients selected by the coefficient selector 160, corresponds to the tap coefficient corresponding to the class code supplied from the class classification unit 163, that is, the target pixel. Read the tap coefficient corresponding to the class. The coefficient memory 164 also outputs the tap coefficients to the prediction unit 165.

Next, in step S1114, the prediction unit 165 performs a predetermined prediction operation by using the prediction taps output from the tap extraction unit 161 and the tap coefficients output from the coefficient memory 164 to perform the prediction of the target pixel. The pixel value is determined, and the pixel value of the determined target pixel is supplied to the image output unit 49 (FIG. 3).

In step S1115, the tap extraction unit 161 then determines whether there is a non-selected pixel among the second image data. If it is determined in step S1115 that there are no pixels selected, the tap extracting unit 161 sets the non-selected pixels as the target pixels and returns to step S110. Step S1110 and subsequent steps are then repeated.

If it is determined in step S1115 that there are no pixels selected in the second image data, the process ends.

The prediction operation performed by the prediction unit 165 and the learning of tap coefficients stored in the coefficient memory 164 will be described in detail.

Now, as an image conversion process, a low quality image in which the high quality image data is set as the second image data and the high quality image data is filtered using a low pass filter (LPF) to reduce the image quality (resolution) of the high quality image data. Assume that the data is set as the first image data, and then a prediction tap is extracted from the low quality image data, and the pixel value of the high quality image pixel is determined according to a predetermined prediction operation using the prediction tap and the tap coefficient. .

As a predetermined prediction operation, a linear prediction operation may be performed. Therefore, the pixel value y of the high definition pixel can be determined by the following linear equation:

However, in formula (1), x _n represents a pixel of the nth low quality image data (hereinafter referred to as a "low quality pixel") constituting a prediction tap of a corresponding high definition pixel, and w _n represents the nth low quality The nth tap coefficient multiplied by the pixel is shown. In equation (1), the prediction tap is composed of N low-resolution pixels x ₁ , x ₂ ,..., And x _N.

The pixel value y of the high-definition pixel can be obtained not by the linear equation shown in equation (1) but by a higher-order equation.

Now, if the true value of the pixel value of the high-definition pixel of the k-th sample is represented by y _k , and the predicted value of the true value y _k obtained by equation (1) is represented by y _k ＇, the prediction error e _k is expressed by the following equation (2). You can get it.

Since the predicted value y _k 의 of formula (2) is determined by formula (1), equation (3) can be obtained by substituting y _{k 의} of formula (2) with formula (1).

However, in formula (3), x _{n and k} represent the nth low quality pixels constituting the prediction taps for the high quality pixels of the kth sample.

The tap coefficient w _n which causes the prediction error e _k in the formula (3) or (2) to be zero is the optimum tap coefficient for predicting the high quality pixel. In general, however, it is difficult to obtain such a tap coefficient w _n for all high quality pixels.

Thus, for example, using the least square method as the norm indicating that the tap coefficient w _n is optimal, the optimum tap coefficient w _n minimizes the total E of the squared errors represented by equation (4). It can be obtained by

However, in Equation (4), K is a set of high quality pixels y _k and low quality pixels x _{1, k} , x _{2, k} , ..., and x _{N , k} constituting a prediction tap for the high quality pixel y _k . The number of samples (number of samples for learning) is shown.

As shown in Formula (5), the minimum value of the total E of the square error of Formula (4) can be calculated | required by w _n which makes the result obtained by partial-dividing the total E by the tap coefficient w _n to become 0.

When partial expression of equation (3) with the tap coefficient w _n is obtained, equation (6) can be obtained.

Equation (7) can be obtained from equations (5) and (6).

By substituting equation (3) into e _k of equation (7), equation (7) can be expressed by the normal equation shown in equation (8).

The normal equation of Equation (8) can be solved for the tap coefficient w _n using, for example, a sweeping out method (Gauss-Jordan's erase method).

The normal equation of equation (8) can be solved for each class, and the optimum tap coefficient (tap coefficient that minimizes the total E of the squared errors) w _n can be obtained for each class.

The image converter 48 shown in Fig. 5 converts the SD image data as the first image data into the HD image data as the second image data by performing an operation according to equation (1) using tap coefficients of each class. can do.

FIG. 7 is a block diagram showing an example of the configuration of a learning apparatus that learns by solving the normal equation of equation (8) for each class to find the tap coefficient w _n .

Learning image data used for learning the tap coefficient w _n is input to the learning apparatus. As the learning image data, for example, HD image data obtained by the HD camera 13 (FIG. 2) or HD image data of higher quality can be used.

In the learning apparatus, the learning image data is supplied to the teacher data generator 171 and the student data generator 173.

The teacher data generator 171 generates teacher data corresponding to the second image data from the learning image data, and supplies the teacher data to the teacher data storage unit 172. That is, the teacher data generator 171 supplies HD image data, which is image data for training, to the teacher data storage unit 172 as it is as teacher data.

The teacher data storage unit 172 stores HD image data supplied from the teacher data generator 171 as teacher data.

The student data generator 173 generates student data corresponding to the first image data from the learning image data, and supplies the student data to the student data storage unit 174.

More specifically, the student data generator 173 generates teacher data (HD image data) from the learning image data, similarly to the teacher data generator 171, and then down-converts the HD image data to convert the SD camera ( 11) SD image data of a component signal of image quality equivalent to the SD image data output from (Fig. 2) is generated. Thereafter, the student data generator 173 converts the SD image data of the component signal into the SD image data of the composite signal as in the broadcasting station 1 (FIG. 2), and converts the SD image data of the composite signal into the component signal. Up-converting by converting the image into SD image data again to generate quasi-HD image data. Thereafter, the student data generator 173 down-converts quasi-HD image data in the same manner as the down converter 45 (FIG. 3), and converts the resulting SD image data into the student data storage unit 174 as student data. Supply.

The student data storage unit 174 stores student data supplied from the student data generator 173.

The tap extracting unit 175 sets the pixels constituting the HD image data as the teacher data stored in the teacher data storage unit 172 as target pixels one by one. The tab extraction unit 175 then extracts predetermined predetermined pixels constituting SD image data as student data stored in the student data storage unit 174 with respect to the target pixel, and the tab shown in FIG. 5. A prediction tap having the same tap structure as the prediction tap acquired by the extraction unit 161 (a target that is the same as the positional relationship to the target pixel of the pixel constituting the prediction tap obtained by the tap extraction unit 161 shown in FIG. 5). A prediction tap consisting of pixels of student data in a positional relationship to the pixels). The tap extraction unit 175 then supplies the determined predictive taps to the adder 190.

The tap extracting unit 176 extracts a predetermined pixel constituting SD image data as student data stored in the student data storage unit 174 with respect to the target pixel, and extracts it to the tap extracting unit 162 shown in FIG. Determine the class tab of the same tab structure as the tab structure of the class tab obtained. The tap extraction unit 176 supplies the determined class tap to the class classification unit 177.

The class classification unit 177 performs class classification like the class classification unit 163 shown in FIG. 5 based on the class tap output from the tap extraction unit 176, and adds a class code corresponding to the determined class. Output to 190.

The adder 190 reads the target pixel from the teacher data storage unit 172, and for each of the class codes supplied from the class classification unit 177, predicts the target pixel supplied from the tap extraction unit 175 with the target pixel. The pixels of the student data to be composed are added together.

That is, the adder 190 is the pixel value y _k of the target pixel of the teacher data stored in the teacher data storage unit 172, and the pixel value of the pixel of the student data constituting the predictive tap output from the tap extraction unit 175. x _{n , k,} and a class code indicating the class of the target pixel output from the class classification unit 177 are supplied.

The adder 190 then uses the prediction taps (student data) x _{n , k} for each class corresponding to the class code supplied from the class classification unit 177 in the matrix on the left side of the equation (8). The same operation as multiplication (x _{n , k} x _{n ' , k} ) and sum (Σ) of student data is performed.

The adder 190 also uses the prediction taps (student data) x _{n , k} and the teacher data y _k for each class corresponding to the class code supplied from the class classification unit 177, to the right side of equation (8). the student data in the vector x _n, and performs operations such as multiplication (x _{n, k} y _k) and the summation (Σ) of the teacher data y _k and a _k.

That is, the adder 190 are determined for the subject pixel from the previous training data, the formula (8), the left side of the component of the matrix of (Σx _{n, k} x _{n ', k)} and the components of the right side of the vector (Σx _{n, k} y _k ) is stored in the built-in memory (not shown), and the teacher for the target pixel of the current teacher data for the components (Σx _{n , k} x _{n ' , k} ) and the components (Σx _{n , k} y _k ). Add the corresponding components x _{n, k + 1} x _{n ' , k + 1} or x _{n , k + 1} y _{k +1} , calculated using data y _{k +1} and student data x _{n , k + 1} [Addition represented by the sum of formula (8)].

The adder 190 performs the above-described addition to all the pixels of the teacher data stored in the teacher data storage unit 172, establishes a regular equation shown in equation (8) for each class, and then normalizes the normal. The equation is fed to the tap coefficient calculator 191.

The tap coefficient calculator 191 solves a regular equation for each class supplied from the adder 190 to obtain an optimum tap coefficient w _n .

With reference to the flowchart of FIG. 8, the learning process performed by the learning apparatus shown in FIG. 7 is demonstrated below.

In step S1121, the teacher data generator 171 and the student data generator 173 generate teacher data and student data from the learning image data, respectively.

More specifically, the teacher data generator 171 generates image data of image quality equivalent to HD image data obtained by the HD camera 13 (FIG. 2), and outputs the generated image data as teacher data. . In addition, similar to the teacher data generator 171, the student data generator 171 generates teacher data (HD image data), and then down-converts the HD image data to output from the SD camera 11 (FIG. 2). SD image data of a component signal having the same image quality as that of the SD image data is generated. The student data generator 173 then converts the SD image data of the component signal into the SD image data of the composite signal and converts the SD image data of the composite signal to the SD of the component signal, as in the broadcasting station 1 (FIG. 2). Up-conversion by converting back into image data produces quasi-HD image data. Thereafter, the student data generator 173 down-converts quasi-HD image data in the same manner as the down converter 45 (FIG. 3), and outputs the resulting SD image data to the student data storage unit 174 as student data. do.

The teacher data output from the teacher data generator 171 is supplied to and stored in the teacher data storage unit 172. The student data output from the student data generator 173 is supplied to and stored in the student data storage unit 174.

Then, in step S1122, the tap extraction unit 175 selects an unselected pixel among the teacher data stored in the teacher data storage unit 172. The tap extraction unit 175 further determines a prediction tap from the student data stored in the student data storage unit 174 with respect to the target pixel, and supplies the prediction tap to the adder 190. The tap extraction unit 176 determines the class tap from the student data stored in the student data storage unit 174 with respect to the target pixel, and supplies the class tap to the class classification unit 177.

Then, in step S1123, the class classification unit 177 performs class classification of the target pixel based on the class tap supplied from the tap extraction unit 176, and outputs a class code associated with the determined class to the adder 190. FIG. .

In step S1124, the adder 190 reads the target pixel from the teacher data storage unit 172, and classifies the target pixel and the pixels of the student data constituting the prediction taps supplied from the tap extraction unit 175. The addition is performed according to equation (8) consisting of regular equations established for the class associated with the class code supplied from unit 177.

The tap extracting unit 175 determines in step S1125 whether there are any unselected pixels among the teacher data stored in such teacher data storage unit 172. In step S1125, if not selected teacher data is found, the tap extraction unit 175 sets the unselected pixel as the target pixel and returns to step S1122. Step S1122 and subsequent steps are then repeated.

In step S1125, if it is determined that there are no pixels selected in the teacher data, the adder 190 adds the matrix of the left side and the vector of the right side in the equation (8) established for each class to the tap coefficient calculator 191. Supply.

Then, in step S1126, the tap coefficient calculator 191 solves the regular equation of equation (8) for each class supplied from the adder 190, obtains the set of tap coefficients w _{n of} each class, and then ends the processing. do.

Since the number of training image data is insufficient, there may be some classes that cannot determine a sufficient number of regular equations required to obtain tap coefficients. For such a class, the tap coefficient calculator 191 outputs a default (default) tap coefficient.

In the learning device, the student data generator 173 employs a plurality of upconversion techniques to obtain quasi-HD image data generated in the process of generating SD image data as student data, and the plurality of upconversion techniques associated with the plurality of upconversion techniques. Get student data. Using the plurality of student data, a plurality of sets of tap coefficients corresponding to the plurality of upconversion techniques are obtained.

In the coefficient memory 164 of the image converter 48 shown in FIG. 5, a set of a plurality of tap coefficients w _m for the individual classes obtained as described above are stored.

The image converters 46 and 47 shown in FIG. 3 are configured similarly to the image converter 48 shown in FIG. However, the image converters 46 and 47 are not provided with the coefficient selector 160 indicated by dotted lines in FIG. 5, and the set of tap coefficients stored in the coefficient memory 164 of the image converters 46 and 47 is an image. It is different from that of the converter 48.

More specifically, the image converter 46 sets the HD image data to the first image data and the higher quality HD image data to the second image data, and then converts the first image data to the second image data. . Accordingly, the set of tap coefficients obtained by performing the learning process shown in FIG. 8 using student data as first image data and second image data as teacher data is the coefficient memory 164 of the image converter 46. Is remembered.

The image converter 47 sets the D1 image data or the quasi-D1 image data to the first image data and the HD image data to the second image data, and then converts the first image data to the second image data. Therefore, using the first image data as the student data and the second image data as the teacher data, the set of tap coefficients obtained by performing the learning process shown in Fig. 8 is the coefficient memory 164 of the image converter 47. Is remembered.

In the learning process shown in Fig. 8, the student data generator 173 uses SD image data obtained by down-converting quasi-HD image data as student data. Alternatively, the SD image data of the component signal having the same image quality as the SD image data output from the SD camera 11 (FIG. 2) can be generated by down-converting the HD image data as the teacher data, and the SD of the component signal. The image data can be converted into SD image data of the composite signal, and then the SD image data of the component signal having the same image quality as that of the image data output from the NTSC decoder 22 shown in FIG. Can be reconverted to Thereafter, the resulting SD image data can be used as student data.

9 is a block diagram showing another example of the functional configuration of the image converter 48 shown in FIG. In FIG. 9, the same reference numerals are given to components corresponding to those of FIG. 5, and description thereof is omitted.

The image converter 48 of FIG. 9 is provided with the tap extraction units 161 and 162, the class classification unit 163, the coefficient memory 164, and the prediction unit 165, so that the image converter of FIG. Similar to (48). However, in the image converter 48 of FIG. 9, instead of the coefficient selector 160 provided in the image converter 48 shown in FIG. 5, the coefficient generator 166 and the coefficient seed memory 167 are provided. , And parameter memory 168 are installed.

The coefficient generator 166 generates a set of tap coefficients for each class based on the coefficient seed data stored in the coefficient seed memory 167 and the parameters stored in the parameter memory 168, and then generates the generated tap coefficients. Is stored by overwriting the previous tap coefficient stored in the coefficient memory 164.

The coefficient seed memory 167 stores coefficient seed data for each class obtained by learning coefficient seed data described later. The coefficient seed data is used as so-called "seeds" which generate tap coefficients.

The parameter memory 168 overwrites and stores the channel information supplied from the tuner 41 (FIG. 3) as a parameter used for generating tap coefficients, overwriting the previous parameter stored in the parameter memory 168.

With reference to the flowchart of FIG. 10, the image conversion process performed by the image converter 48 shown in FIG. 9 is demonstrated below.

The tap extracting unit 161 sequentially sets the pixels constituting the HD image data as the second image data corresponding to the SD image data as the first image data supplied from the down converter 45 (FIG. 3) as the target pixels. Then, in step S1211, the parameter memory 168 determines whether new channel information different from the channel information currently stored in the parameter memory 168 has been supplied from the tuner 41 (Fig. 3) as a parameter. If it is determined in step S1211 that new channel information has been supplied, the process proceeds to step S1212, and the parameter memory 168 overwrites and stores the new channel information with the previous channel information.

If it is determined in step S1211 that new channel information has not been supplied, the process skips step S1212 and proceeds to step S1213.

Therefore, in the parameter memory 168, a channel for broadcasting quasi-HD image data before down-converting the SD image data as the first image data supplied from the down converter 45 (FIG. 3) to the image converter 48 is provided. Related channel information is stored as a parameter.

In step S1213, the coefficient generator 166 reads the coefficient seed data set for each class from the coefficient seed memory 167, and also reads the corresponding parameter from the parameter memory 168, and writes to the coefficient seed data and parameters. Based on this, a set of tap coefficients for each class is obtained. Then, in step S1214, the coefficient generator 166 supplies the coefficient memory 164 with the set of tap coefficients for each class by overwriting the previous set of tap coefficients.

In step S1215, the tap extracting units 161, 162 extract the predictive tap and the class tap for the target pixel, respectively, from the SD picture data as the first picture data supplied from the down converter 45 (FIG. 3). The tap extraction unit 161 supplies the prediction tap to the prediction unit 165 and the class tap to the class classification unit 163.

In step S1216, the class classification unit 163 classifies the class of the target pixel based on the class tap as the class tap for the target pixel is received from the tap extraction unit 162, and also represents the determined class. The code is output to the coefficient memory 164.

In step S1217, the coefficient memory 164 reads and outputs the tap coefficient stored in the address corresponding to the class code supplied from the class classification unit 163, that is, the tap coefficient associated with the class of the target pixel. In addition, the coefficient memory 164 also outputs the tap coefficient to the prediction unit 165.

Then, in step S1218, the prediction unit 165 performs a predetermined prediction operation by using the prediction taps output from the tap extraction unit 161 and the tap coefficients obtained from the coefficient memory 164, so as to determine the target pixel. Find the pixel value.

The tap extracting unit 161 then determines whether there is a pixel not selected among the second image data in step S1219. In step S1219, when an unselected pixel is found among the second image data, the tap extracting unit 161 sets the unselected pixel as the target pixel and returns to step S1211. Then, step S1211 and subsequent steps are repeated.

If it is determined in step S1219 that there are no pixels selected in the second image data, the process ends.

In the flowchart of FIG. 10, steps S1213 and S1214 may be executed only in the parameter memory 168 when the previous parameter is overwritten by a new parameter, and in other cases, steps S1213 and S1214 may be omitted.

The following describes the prediction operation performed by the prediction unit 165, the generation of tap coefficients performed by the coefficient generator 166, and the learning of coefficient seed data stored in the coefficient seed memory 167.

As in the image converter 48 shown in FIG. 5, the image converter 48 shown in FIG. 9 sets the SD image data output from the down converter 45 (FIG. 3) as the first image data, and sets the HD image data. Is set as the second image data, and then, using the tap coefficients for each class, the first image data is converted into the second image data according to, for example, a linear prediction operation represented by equation (1).

However, in the image converter 48 shown in Fig. 9, the coefficient generator 166 generates the tap coefficient w _n from the coefficient seed data stored in the coefficient seed memory 167 and the parameters stored in the parameter memory 168. do. Now assume that coefficient generator 166 uses the coefficient seed data and parameters to generate the tap coefficient w _n , for example, according to equation (9).

In the formula (9), β _{m and n} represent m-th coefficient seed data used for obtaining the _n- th tap coefficient w _n , and z represents a parameter. In formula (9), the tap coefficient w _n is obtained using M coefficient seed data β _{1, n} , β _{2, n} ,..., And β _{M, n} .

The equation for obtaining the tap coefficient w _n from the coefficient seed data _{βm, n} and the parameter z is not limited to the expression (9).

The value z ^{m -1} determined by the parameter z in equation (9) is defined in equation (10) by introducing a new variable t _m .

Formula (11) can be obtained by substituting Formula (10) into Formula (9).

According to equation (11), the tap coefficient w _n can be obtained by a linear equation of the coefficient seed data _{βm, n} and the variable t _m .

Now, if the true value of the pixel value of the high-definition pixel of the k-th sample is represented by y _k , and the predicted value of the true value y _k obtained by the formula (1) is represented by y _{k ＇} , the prediction error e _k is expressed by the above-described equation (2). Can be determined by the same formula (12).

(12) predicted value y _{k 'is} determined by the formula (1), y _k in equation _(12), Substituting in equation (1), it is possible to obtain a formula (3), the same formula (13) and of .

However, in formula (13), x _{n and k} represent the nth low quality pixels constituting the prediction taps for the high quality pixels of the kth sample.

Formula (14) can be obtained by substituting Formula (11) into w _n of Formula (13).

The coefficient seed data β _{m and n} for which the prediction error e _k in Equation (14) is 0 is the optimal coefficient seed data for predicting the corresponding high quality pixel. In general, however, it is difficult to obtain such coefficient seed data β _{m and n} for all high definition pixels.

So, for example, if the least square method is used as the norm indicating that the coefficient seed data β _{m, n} is optimal, the optimal coefficient seed data β _{m, n} is the sum of the squared errors represented by the equation (15). It can be found by minimizing E.

However, in Equation (15), K is a set of high quality pixels y _k and low quality pixels x _{1, k} , x _{2, k} , ..., and x _{N , k} constituting a prediction tap for the high quality pixel y _k . The number of samples (number of samples for learning) is shown.

The minimum value of the total E of the squared errors of the formula (15) is determined by β _{m, n} which makes the result obtained by partial derivative of the total E with the coefficient seed data β _{m, n} as shown in the formula (16).

Formula (17) can be obtained by substituting Formula (13) into Formula (16).

X _{i , p, j, q} and Y _{i , p} are defined by equations (18) and (19), respectively.

In this case, equation (17) can be expressed by the regular equation of equation (20) using X _{i , p, j, q} and Y _{i , p} .

The normal equation represented by equation (20) can be solved for the coefficient seed data β _{m and n} using, for example, a sweeping method.

In the image converter 48 shown in Fig. 9, a plurality of high-definition pixels y ₁ , y ₂ ,..., And y _k of an image having the same image quality as the HD image data as the second image data are used as teacher data, Using low-quality pixels x _{1, k} , x _{2, k} , ..., x _{N , k} of the image having the same image quality as the SD image data down-converted from the semi-HD image data as the first image data, as student data, The coefficient seed data β _{m, n} obtained by performing learning to solve equation (20) is stored in the coefficient seed memory 167. The coefficient generator 166 generates the tap coefficient w _n from the coefficient seed data β _{m, n} and the parameter z stored in the parameter memory 168 in accordance with Expression (9). Prediction unit 165, by using _the pixel x _n of the first image data forming the prediction tap for the tap coefficient w _n and the target pixel, calculates the equation (1), the target pixels constituting the second image data, The prediction value of the pixel value is obtained.

FIG. 11 is a block diagram showing another configuration example of a learning device that learns by solving a normal equation expressed by Expression (20) to obtain coefficient seed data β _{m and n} . In FIG. 11, since the same reference numerals are given to the same components as those of FIG. 7, the description thereof will be omitted.

Learning image data used for learning the coefficient seed data β _{m and n} is supplied to the learning device, more specifically to the teacher data generator 171 and the student data generator 173.

As described with reference to FIG. 7, the teacher data generator 171 generates teacher data having the same image quality as the HD image data as the second image data from the image data for learning, and converts the generated teacher data into the teacher data storage unit 172. To feed.

The teacher data storage unit 172 stores HD image data as teacher data supplied from the teacher data generator 171.

As described with reference to FIG. 7, the student data generator 173 generates student data having the same image quality as the SD image data as the first image data from the learning image data, and generates the generated student data in the student data storage unit 174. To feed.

However, in the learning device shown in FIG. 11, the student data generator 173 generates student data corresponding to the parameter z supplied from the parameter generator 180.

That is, the student data generator 173 supplies not only the learning image data but also some values which can be determined by the channel information as the parameter z stored in the parameter memory 168 of FIG. 9 supplied from the parameter generator 180. Receive. More specifically, for example, z = 0, 1, 2, ..., and Z are supplied from the parameter generator 180 to the student data generator 173.

As described with reference to FIG. 7, the student data generator 173, like the teacher data generator 171, generates teacher data (HD image data) from the image data for learning, and then down-converts the HD image data. SD image data of the component signal having the same image quality as the SD image data output from the SD camera 11 (Fig. 2) is generated. The student data generator 173 then converts the SD image data of the component signal into the SD image data of the composite signal and converts the SD image data of the composite signal into the component signal, as in the broadcasting station 1 (FIG. 2). Upconversion by reconversion to SD image data produces quasi-HD image data. Thereafter, the student data generator 173 down-converts quasi-HD image data in the same manner as the down converter 45 (FIG. 3), and converts the resulting SD image data into the student data storage unit 174 as student data. Supply.

The student data generator 173 employs an upconversion technique associated with the parameter z supplied from the parameter generator 180 as an upconversion technique for obtaining quasi-HD image data generated in the process of generating SD image data as student data. do.

That is, the student data generator 173 performs up-conversion by using the same up-conversion technique as the up-conversion technique used to obtain quasi-HD image data at a broadcasting station associated with the channel indicated by the channel information as the parameter z. Generate data.

Then, the student data generator 173 down-converts the quasi-HD image data and supplies it to the student data storage unit 174 as student data.

Therefore, the student data generator 173 can obtain, as the student data, SD image data of the same image quality as the SD image data obtained by downconverting the semi-HD image data broadcasted in the channel indicated by the channel information as the parameter z.

In this case, in the student data generator 173, student data of the type (Z + 1) corresponding to the parameters z = 0, 1, 2), ..., and Z associated with different upconversion techniques are generated.

The student data storage unit 174 stores (Z + 1) type SD image data as student data supplied from the student data generator 173.

The tap extracting unit 175 sets the pixel constituting the HD image data as the teacher data stored in the teacher data storage unit 172 as the target pixel, and serves as the student data stored in the student data storage unit 174. Predetermined pixels constituting the SD image data are extracted, and the predicted tap having the same tap structure as the tap structure of the predicted tap acquired by the tap extracting unit 161 shown in FIG. 9 is determined. The tap extraction unit 175 supplies the predictive tap to the adder 178.

The tap extracting unit 176 extracts predetermined pixels constituting the SD image data as the student data stored in the student data storage unit 174, and selects the class taps obtained by the tap extracting unit 162 shown in FIG. Determines the class tab of the same tab structure as the tab structure. The tap extraction unit 176 supplies the class tap to the class classification unit 177.

The tap extracting units 175, 176, upon receiving the parameter z generated by the parameter generator 180, use the student data associated with the parameter z supplied from the parameter generator 180 to select the prediction tap and the class tap. Create each.

The class classification unit 177 performs class classification like the class classification unit 163 shown in FIG. 9 based on the class tap output from the tap extraction unit 176, and adds a class code corresponding to the determined class. Output to (178).

The adder 178 then reads the target pixel from the teacher data storage unit 172 and writes the target pixel, the student data constituting the predictive tap supplied from the tap extraction unit 175, and the parameter z associated with the student data. Is performed for each class code supplied from the class classification unit 177.

That is, the teacher data y _k stored in the teacher data storage unit 172, the predicted taps x _{i , k} (x _{j , k} ) output from the tap extracting unit 175, and the class classification unit 177 are outputted. In addition to the class code, the parameter z generated when the student data used for the prediction tap is generated is also supplied from the parameter generator 180 to the adder 178.

The adder 178 then uses the prediction taps (student data) x _{i , k} (x _{j , k} ) and the parameter z for each class associated with the class code supplied from the class classification unit 177, In the matrix on the left side of equation (20) _, the multiplication (x _{i , k} t _p x _{j , k} t _{q) of the} student data and the parameter z to find the components X _{i , p, j, q} defined in equation (18) ) And summation (Σ). In equation (18), t _p is calculated from the parameter z according to equation (10). The same applies to t _q in equation (18).

The adder 178 also uses the prediction tab (student data) x _{i , k} , teacher data y _k , and parameter z for each class associated with the class code supplied from the class classification unit 177, to express the equation ( In the vector on the right side of 20) _, the multiplication of the student data x _{i , k} , the teacher data y _k , and the parameter z (x _{i , k} t _p y _k ) to find the components Y _{i , p} defined in equation (19) ) And summation (Σ). In equation (19), t _p is calculated from the parameter z according to equation (10).

That is, the adder 178 stores the components X _{i , p, j, q} of the matrix on the left side and components Y _{i , p} of the vector on the right side in the equation (20) obtained for the previous target pixel. And corresponding components x _{i , k} t _p calculated using teacher data y _k , student data x _{i , k} (x _{j , k} ), and parameter z for the current target pixel. x _{j , k} t _q Or add x _{i , k} t _p y _k to each of the components X _{i , p, j, q of the matrix,} or components Y _{i , p} of the vector [addition represented by the sum of the formulas (18) or (10) )

The adder 178 performs the above-described addition on all the pixels of the teacher data stored in the teacher data storage unit 172 with respect to all the parameters z (0, 1, ..., and Z), so that each class , A normal equation shown in equation (20) is established, and the normal equation is supplied to the coefficient seed calculator 179.

The coefficient seed calculator 179 solves a normal equation for each class supplied from the adder 178 to obtain coefficient seed data β _{m and n} for each class.

The parameter generator 180 generates some values that can be determined as the parameter z supplied to the parameter memory 168 shown in FIG. 9, for example z = 0, 1, 2, ..., Z, The generated parameter z is supplied to the student data generator 173. The parameter generator 180 also supplies the generated parameter z to the tap extraction units 175 and 176 and the adder 178.

With reference to the flowchart of FIG. 12, the learning process performed by the learning apparatus shown in FIG. 11 is demonstrated.

In step S1221, the teacher data generator 171 generates teacher data having the same image quality as the HD image data as the second image data from the learning image data.

The student data generator 173 performs a process such as up-conversion according to Z + 1 values (0, 1, ..., and Z) of the parameter z output from the parameter generator 180, thereby learning image data. (Z + 1) type of student data having the same image quality as that of the SD image data as the first image data.

The teacher data output from the teacher data generator 171 is supplied to and stored in the teacher data storage unit 172, and the student data of type (Z + 1) output from the student data generator 173 is a student data storage unit ( 174 is stored.

Then, in step S1222, the parameter generator 180 sets the parameter z to an initial value, for example, 0, and supplies the parameter z to the tap extracting units 175 and 176 and the adder 178.

In step S1223, the tap extracting unit 175 sets a pixel not selected from the teacher data stored in the teacher data storage unit 172 as the target pixel. The tap extracting unit 175 then extracts pixels from the student data corresponding to the parameter z output from the parameter generator 180 and stored in the student data storage unit 174 to generate predictive taps, and generate the prediction The tab is fed to adder 178. In addition, the tap extracting unit 176 extracts pixels from student data corresponding to the parameter z stored in the student data storage unit 174 and output from the parameter generator 180 to generate a class tap, and generates the generated class tap. Supply to class classification unit 177.

Then, in step S1224, the class classification unit 177 classifies the class of the target pixel based on the class tap, and outputs a class code associated with the determined class to the adder 178.

In step S1225, the adder 178 reads the target pixel from the teacher data storage unit 172, and the target pixel, the prediction tap supplied from the tap extraction unit 175, and the parameter z output from the parameter generator 180. Using, compute the components x _{i , k} t _p x _{j , k} t _q and the components x _{i , k} t _p y _k of the vector on the right side of equation (20). The adder 178 classifies the class components 177 into matrix components x _{i , k} t _p x _{j , k} t _q and vector components x _{i , k} t _p y _k calculated from the target pixel, the prediction tap, and the parameter z. Add to the previous components associated with the class code output from.

In step S1226, the parameter generator 180 determines whether the parameter z that it outputs is equal to the maximum value Z. If it is determined in step S1226 that the parameter z is not equal to the maximum value Z (less than the maximum value Z), the process proceeds to step S1227. In step S1227, the parameter generator 180 adds 1 to the parameter z, and as a result outputs the parameter to the tap extracting units 175 and 176, and the adder 178 as a new parameter z, and the process returns to step S1223. do.

If it is determined in step S1226 that the parameter z is equal to the maximum value Z, the process proceeds to step S1228 to determine whether there are any unselected pixels among the teacher data stored in the teacher data storage unit 172. In step S1228, if an unselected pixel is found, the tap extraction unit 175 sets the unselected pixel as the target pixel and returns to step S1222. Then, step S1222 and subsequent steps are repeated.

In step S1228, when it is determined that there are pixels which are not selected among the teacher data, the adder 178 supplies the coefficient seed calculator 179 with the matrix on the left side and the vector on the right side in the equation (20) acquired for each class. Supply.

Then, in step S1229, the coefficient seed calculator 179 solves the normal equation of equation (20) for each class, obtains coefficient seed data β _m, n for the corresponding class, and ends the processing.

Because of the insufficient number of training image data, there may be several classes that make it difficult to determine a sufficient number of normal equations needed to obtain coefficient seed data. For such a class, the coefficient seed calculator 179 outputs default coefficient seed data.

In the learning apparatus shown in Fig. 11, the linear prediction equation of equation (1) is used by using HD image data as teacher data and using SD image data corresponding to parameter z, which is generated from the HD image data, as student data. We learn to directly obtain the coefficient seed data β _{m, n} that minimizes the total sum of squared errors of the predicted values y of the teacher data predicted at. Alternatively, the coefficient seed data βm, n can be learned as shown in FIG.

In the modification shown in Fig. 13, the prediction value y of the teacher data predicted by the linear prediction equation of equation (1) by using SD image data corresponding to the parameter z as student data generated from the HD image data as the teacher data. Tap coefficient w _n , which minimizes the sum of squared errors of _, is used for each of the values of parameter z (z = 0, 1,..., And Z) using tap coefficient w _n and student data x _n . It is saved every time. Then, using the obtained tap coefficient wn as the teacher data and the parameter z as the student data, as the teacher data predicted from the coefficient seed data β _{m, n} , and the variable tm associated with the parameter z by equation (11), The learning which calculates | requires the coefficient seed data (beta) _{m, n} which minimizes the sum total of the square error of the prediction value of the tap coefficient w _n is performed.

That is, as in the learning apparatus shown in FIG. 7, the tap coefficient w _n that minimizes the total E of the squared error of the predicted value y of the teacher data predicted by the linear prediction equation of the equation (1), i. _n can be obtained for each value of the parameter z (z = 0, 1, ..., and Z) for each class by solving the normal equation of Formula (8).

According to equation (11), the tap coefficient can be obtained from the coefficient seed data β _{m, n} and the variable t _m corresponding to the parameter z. _"Assuming that, as expressed by equation (21), determined that the tap coefficient w _n by the optimum tap coefficients w _n and equation _(11), the tap coefficient determined by equation (11) w _n error of The coefficient seed data β _{m, n} for which e _{n is} 0 is optimal coefficient seed data for obtaining an optimal tap coefficient w _n . However, it is difficult to obtain such coefficient seed data β _{m and n} for all tap coefficients w _n .

By substituting Equation (11) into Equation (21), Equation (21) can be transformed into Equation (22).

For example, if the least square method is adopted as the norm indicating that the coefficient seed data β _{m, n} is optimal, the optimal coefficient seed data β _{m, n} is the total E of the squared errors represented by the equation (23). It can be found by minimizing.

The minimum value of the sum of the square error E of Expression (23), to _n, the total E the coefficient seed data β _m, the coefficient seed data to zero the result obtained by partial derivatives with _n β _m, as shown in equation (24) Is determined by

Equation (25) can be obtained by substituting equation (22) into equation (24).

X _{i , j} and Y _i are defined as shown in equations (26) and (27), respectively.

In this case, equation (25) can be represented by the regular equation of equation (28) using X _{i , j} and Y _i .

For example, the regular equation of equation (28) can be solved for the coefficient seed data β _{m and n} by using the sweeping method.

FIG. 14 shows another example of the configuration of a learning apparatus that learns to solve coefficient seed data β _{m and n} by solving the normal equation of equation (28). In FIG. 14, the same reference numerals are given to the components corresponding to the components of FIG. 7 or 11, and description thereof will be omitted.

The adder 190 is supplied with a class code for the target pixel output from the class classification unit 177 and a parameter z output from the parameter generator 180. Then, the adder 190 reads the target pixel from the teacher data storage unit 172, and, for the student data constituting the target pixel and the predictive tap supplied from the tap extraction unit 175, the class classification unit 177 Is added to each of the class codes supplied from the C) and each of the values of the parameter z output from the parameter generator 180.

That is, the adder 190 is a teacher data y _k stored in the teacher data storage unit 172, a predicted tap x _{n , k} output from the tap extracting unit 175 _, and a class output from the class classification unit 177. The code and parameter z output from the parameter generator 180 are supplied to correspond to student data and used to determine the prediction taps x _{n , k} .

Adder 190 then for each class corresponding to the class code supplied from class classification unit 177 and for each of the values of parameter z output from parameter generator 180, prediction tap (student data) x _{n Using, k} , arithmetic operations such as multiplication (x _{n , k} x _{n ' , k} ) and summation (Σ) of the student data in the matrix on the left side of equation (8) are performed.

The adder 190 also includes a prediction tap (student data) x for each class corresponding to the class code supplied from the class classification unit 177 and for each of the values of the parameter z output from the parameter generator 180. _{n, k} and the teacher data y using _k, (8) the student data in the right side of the vector x _n of _{a, k,} and the teacher data y _k of the multiplication (x _{n, k} y _k) and the summation (Σ), such as Perform the operation.

That is, the adder 190 is the left side of the component of the matrix in equation 8 is calculated for the target pixel from the previous training data of (Σx _{n, k} x _{n ', k)} and the components of the right side of the vector (Σx _{n, k} y _k ) is stored in a built-in memory (not shown), and then calculated using teacher data y _{k +1} and student data x _{n , k + 1} for the target pixel of the current teacher data, Corresponding components x _{n , k + 1} x _{n ' , k + 1} or x _{n , k + 1} y _{k +1} are components (Σx _{n , k} x _{n' , k} ) or components of the vector (Σx _{n, k} y _k ), respectively (addition expressed by the summation of Equation (8)).

The adder 190 performs the above-described addition to all the pixels of the teacher data stored in the teacher data storage unit 172, and, for each of the classes and the values of the parameter z, respectively, the regular equation shown in equation (8) is obtained. And the regular equation is supplied to the tap coefficient calculator 191.

The tap coefficient calculator 191 solves the normal equation supplied from the adder 190 for each of the classes and the values of the parameter z, obtains an optimal tap coefficient wn for each of the classes, and supplies it to the adder 192.

The adder 192 adds, for each class, to the variable t _m corresponding to the parameter z and the optimal tap coefficient w _n .

That is, the adder 192 uses the component t _i (tj) determined from the parameter z in the formula (10), and the components X _{i and j} defined in the formula (26) in the matrix on the left side of the formula (28). For each class, the same operation as the multiplication (t _i t _j ) and sum (Σ) of the variable t _{i (} t _j ) corresponding to the parameter z is performed.

In this case, the components X _{i , j} are determined only by the parameter z, regardless of the class. Thus, the components X _{i , j} are sufficient to be calculated once for all classes without calculating the components X _{i , j} of each of the classes.

The adder 192 also uses the variable t _i obtained by the formula (10) from the parameter z and the component defined in equation (27) in the vector on the right side of the equation (28), using the optimal tap coefficient w _n for each class to find the Y _i, and performs operations such as multiplication with the variable t _{i (n} tiw) of the optimum tap coefficients w _n and the summation (Σ) corresponding to the parameter z.

The adder 192 obtains the components X _{i , j} represented by the formula (26) and the component Y _i represented by the formula (27), thereby establishing a regular equation of the formula (28) for each class, The normal equation is supplied to the coefficient seed calculator 193.

The coefficient seed calculator 193 solves the normal equation of equation (28) for each class supplied from the adder 192 to obtain coefficient seed data β _{m and n} for each corresponding class.

In the coefficient seed memory 167 of the image converter 48 shown in FIG. 9, coefficient seed data β _{m and n} for each of the classes obtained as described above are stored.

As described above, in the digital broadcast receiver 2 shown in FIG. 3, when the determination unit 44 determines the type of the image data and determines that the image data is quasi-HD image data, the down converter By 45, the quasi-HD image data is down-converted by reducing the number of pixels of the quasi-HD image data. Then in the image converter 48,

Image conversion processing by operation using tap coefficients for each class determined by learning, or tap coefficients for each class generated from coefficient seed data and predetermined parameters determined by learning (hereinafter, "class classification adaptive processing"). To convert the down-converted SD image data into high-definition HD image data.

Therefore, the user of the digital broadcast receiver 2 can enjoy high quality image data.

As described with reference to Fig. 2, the quasi-HD image data converts only composite signal image data of the NTSC system into component signal image data, and captures the number of pixels of the image data of the component signal with the HD camera 13. Image data obtained by increasing the number of pixels of one HD image and performing up-conversion. Therefore, if the number of pixels of the image data of the component signal converted from the composite signal image data of the NTSC system is one fifth of the number of pixels of the HD image data captured by the HD camera 13, five pixels of the semi-HD image data. The amount of information about is only an amount of information about one pixel of the image data of the component signal converted from the composite signal image data beam of the NTSC system.

Therefore, roughly speaking, when class classification adaptive processing is performed using quasi-HD image data as the first image data, image data of a component signal converted from composite signal image data of the NTSC system is used as the first image data. The number of pixels five times as many as in the case of performing the class classification adaptation process should be used to perform the calculation and class classification of equation (1). Otherwise, it becomes difficult to obtain comparable performance in the case where the class classification adaptation process is performed using the image data of the component signal converted from the composite signal image data of the NTSC system as the first image data.

If a large number of pixels are used for the calculation or class classification in equation (1), the throughput increases by that amount.

Therefore, as described above, the down converter 45 reduces the number of pixels of the quasi-HD image data to down-convert the quasi-HD image data to the SD image data, and then the image converter 48 removes the SD image data. Class classification adaptation processing is performed using one image data. As a result, high quality image data can be obtained with a small processing amount.

In addition, as shown in FIG. 5, from among a plurality of tap coefficient sets corresponding to a plurality of upconversion techniques, one set of tap coefficient sets is selected based on channel information about a channel of image data, and the set of selected tap coefficients is selected. Perform class classification adaptive processing using. Alternatively, as shown in Fig. 9, a set of tap coefficients is generated using channel information as a parameter z, and then class classification adaptive processing is performed using the generated set of tap coefficients. In this way, class classification adaptation processing can be performed using tap coefficients suitable for the up-conversion technique employed by the broadcasting station associated with the channel broadcasting the image data. As a result, high quality image data can be obtained.

The quasi-D1 image data is image data obtained by converting the composite signal image data of the NTSC system into component signal image data in the same way as the quasi-HD image data, increasing the number of pixels of the image data of the component signal, and up-converting them. However, since the number of pixels of the quasi-D1 arc image data interpolated by up-conversion is smaller than the pixel portion of the quasi-HD image data, the digital broadcast receiver 2 does not distinguish between the D1 image data and the quasi-D1 image data. Processing is performed by the converter 47. Alternatively, the quasi-D1 image data may be processed similarly to the quasi-HD image data separately from the D1 image data. More specifically, quasi-D1 image data can be down-converted by reducing its number of pixels to the number of pixels of image data obtained by converting the composite signal image data of the NTSC system into the image data of the component signal, similarly to the quasi-HD image data. Then, class classification adaptation processing is performed on the down-converted image data.

In the block including the determination unit 44, the down converter 45, the image converters 46, 47, and 48, and the image output unit 49 shown in FIG. Processing can also be performed on the image data reproduced from the recording medium.

The above-described series of processing operations performed by the determination unit 44, the down converter 45, and the image converter 48 can be executed by hardware or software. When using software, the corresponding software program is installed in a general purpose computer, for example.

Fig. 15 shows a block diagram of a computer on which a program for executing a series of processing operations described above is installed.

The program is recorded in advance in the hard disk 105 or the ROM 103 as a recording medium built in the computer.

In contrast, the program includes a removable recording medium 111 such as a flexible disk, a compact disc read only memory (CD-ROM), a magneto optical (MO) disk, a digital versatile disc (DVD), a magnetic disk, a semiconductor memory, and the like. In addition, it may be stored (recorded) temporarily or permanently. Such a removable recording medium 111 may be provided as so-called "package software".

As described above, the program can be installed in the computer from the removable recording medium 111. Alternatively, the program can be transferred to a computer from a download site, wirelessly via a satellite for digital satellite broadcasting, or wired via a network such as a local area network (LAN) or the Internet. Thereafter, the computer can receive the program transmitted by the Kongsin unit 108 and install the program on the built-in hard disk 105.

The computer has a CPU (Central Processing Unit) 102 built in. The input / output interface 110 is connected to the CPU 102 via the bus 101. The CPU 102 executes a program stored in the ROM 103 when the user receives a command by operating the input unit 107 including a keyboard, a mouse, or a microphone through the input / output interface 110. Unlike this, the CPU 102 may be a program stored in the hard disk 105, a program transmitted from a satellite or a network, received by the communication unit 108, and then installed in the hard disk 105, or a drive 109. The program installed on the hard disk 105 after being read from the removable recording medium 111 attached to the CD) is loaded into the RAM 104, and the loaded program is executed. The CPU 102 can execute the processing according to the flowchart described above or the processing performed on the components shown in the block diagram. If necessary, the CPU 102 outputs the processing result from the output unit 106 including the liquid crystal display (LCD) or the speaker, or transmits the data from the communication unit 108 or the input / output interface 110. Through the hard disk 105 can be recorded.

In this specification, the steps of constructing a program for causing a computer to execute various processes are not necessarily processed in time series in the order shown in the flowchart. Alternatively, the steps may be executed in parallel or separately (eg, parallelism or object processing).

The program may be executed by one computer or distributedly processed by a plurality of computers.

Of course, various modifications, combinations, subcombinations, and changes may be made by those skilled in the art according to design requirements and other factors within the scope of the appended claims or their equivalents.

According to the present invention, a user can enjoy high quality image data.

Claims (14)

An image processing apparatus which processes first image data and outputs second image data of higher quality than the first image data. Down-converting means for downconverting the input image data by reducing the number of pixels of the input image data; And Image conversion means for converting the first image data into the second image data using the input image data down-converted by the down converting means as the first image data. Including; The image conversion means, Prediction tap extracting means for extracting, from the first image data, a plurality of pixels used to predict a subject pixel of the second image data as a prediction tap; Class tap extracting means for extracting, as a class tap, a plurality of pixels used to classify the target pixel into one of a plurality of classes from the first image data; Class classification means for classifying a class of the target pixel based on the class tap; Coefficient output means for outputting a coefficient corresponding to the class of the target pixel from among coefficients corresponding to a plurality of classes, predetermined by learning; And And calculation means for determining the target pixel by performing a prediction operation using the coefficients corresponding to the class of the target pixel and the prediction tap. In claim 1, Determining means for determining the type of the input image data, If the determining means determines that the type of the input image data is up-converted image data obtained by increasing the number of pixels of the image data having different types, the down-converting means decreases the number of pixels of the input image data. And down converting the input image data. In claim 2, And the input image data is broadcast image data. In claim 3, Selecting means for selecting one coefficient set from among a plurality of coefficient sets associated with a plurality of corresponding up-conversion techniques used for the up-conversion, based on channel information about a channel of the broadcast image data, And the coefficient output means outputs a coefficient corresponding to the class of the target pixel among the coefficient sets selected by the selection means. An image processing method for processing first image data and outputting second image data of higher quality than the first image data, Down converting the input image data by reducing the number of pixels of the input image data; And Converting the first image data to the second image data using the down-converted image data as the first image data Including; Converting the first image data into the second image data, Extracting, from the first image data, a plurality of pixels used to predict a target pixel of the second image data as a prediction tap, Extracting, from the first image data, a plurality of pixels used to classify the target pixel into one of a plurality of classes as class taps, Classifying the class of the target pixel based on the class tap; Outputting a coefficient corresponding to the class of the target pixel from among coefficients corresponding to a plurality of classes predetermined by learning, and And determining the target pixel by performing a prediction operation using coefficients corresponding to the class of the target pixel and the prediction tap. A program for causing a computer to execute image processing for processing first image data and outputting second image data of higher quality than the first image data, Down converting the input image data by reducing the number of pixels of the input image data; And Converting the first image data to the second image data using the down-converted image data as the first image data Including; Converting the first image data into the second image data, Extracting, from the first image data, a plurality of pixels used to predict a target pixel of the second image data as a prediction tap, Extracting, from the first image data, a plurality of pixels used to classify the target pixel into one of a plurality of classes as class taps, Classifying the class of the target pixel based on the class tap; Outputting a coefficient corresponding to the class of the target pixel from among coefficients corresponding to a plurality of classes predetermined by learning, and And determining the target pixel by performing a prediction operation using coefficients corresponding to the class of the target pixel and the prediction tap. An image processing apparatus which processes first image data and outputs second image data of higher quality than the first image data. Down converting means for downconverting the input image data by reducing the number of pixels of the input image data; And Image conversion means for converting the first image data into the second image data using the input image data down-converted by the down converting means as the first image data. Including; The image conversion means, Prediction tap extracting means for extracting, as predictive taps, a plurality of pixels used to predict a target pixel of the second image data from the first image data; Class tap extracting means for extracting, as class taps, a plurality of pixels used to classify the target pixel into one of a plurality of classes from the first image data; Class classification means for classifying a class of the target pixel based on the class tap; Generating means for generating coefficients corresponding to each of the plurality of classes from predetermined parameters and seed data used as seeds of the coefficients; Coefficient output means for outputting a coefficient corresponding to the class of the target pixel among the coefficients corresponding to the plurality of classes generated by the generating means, and And computing means for determining the target pixel by performing a prediction operation using coefficients corresponding to the class of the target pixel and the prediction tap. In claim 7, Determining means for determining the type of the input image data, If the determining means determines that the type of the input image data is up-converted image data obtained by increasing the number of pixels of the image data having different types, the down-converting means decreases the number of pixels of the input image data. And down converting the input image data. In claim 8, And the input image data is broadcast image data. In claim 9, And said generating means generates coefficients corresponding to each of said plurality of classes using channel information about a channel of said broadcast image data as said predetermined parameter. As an image processing method for processing first image data and outputting second image data of higher quality than the first image data. Down converting the input image data by reducing the number of pixels of the input image data; And Converting the first image data to the second image data using the down-converted input image data as the first image data Including; Converting the first image data into the second image data, Extracting, from the first image data, a plurality of pixels used to predict a target pixel of the second image data as a prediction tap, Extracting, from the first image data, a plurality of pixels used to classify the target pixel into one of a plurality of classes as class taps, Classifying the class of the target pixel based on the class tap; Generating coefficients corresponding to each of the plurality of classes from predetermined parameters and seed data used as seeds of coefficients and predetermined by learning, Outputting a coefficient corresponding to the class of the target pixel among the generated coefficients corresponding to the plurality of classes, and And determining the target pixel by performing a prediction operation using coefficients corresponding to the class of the target pixel and the prediction tap. A program for causing a computer to execute image processing for processing first image data and outputting second image data of higher quality than the first image data, Down converting the input image data by reducing the number of pixels of the input image data; And Converting the first image data to the second image data using the down-converted input image data as the first image data Including; Converting the first image data into the second image data, Extracting, from the first image data, a plurality of pixels used to predict a target pixel of the second image data as a prediction tap, Extracting, from the first image data, a plurality of pixels used to classify the target pixel into one of a plurality of classes as class taps, Classifying the class of the target pixel based on the class tap; Generating coefficients corresponding to each of the plurality of classes from predetermined parameters and seed data used as seeds of coefficients and predetermined by learning, Outputting a coefficient corresponding to the class of the target pixel among the generated coefficients corresponding to the plurality of classes, and And determining the target pixel by performing a prediction operation using coefficients corresponding to the class of the target pixel and the prediction tap. An image processing apparatus which processes first image data and outputs second image data of higher quality than the first image data. A down-conveter configured to down convert the input image data by reducing the number of pixels of the input image data; And An image converter configured to convert the first image data into the second image data using the input image data down-converted by the down converter as the first image data. Including; The image converter, A prediction tap extracting unit, configured to extract, from the first image data, a plurality of pixels used to predict a target pixel of the second image data as a prediction tap; A class tap extracting unit configured to extract, as class taps, a plurality of pixels used to classify the target pixel into one of a plurality of classes from the first image data; A class classification unit configured to classify the class of the target pixel based on the class tap, A coefficient output unit configured to output a coefficient corresponding to the class of the target pixel among coefficients corresponding to a plurality of classes, predetermined by learning, and And a calculator configured to determine the target pixel by performing a prediction operation using the coefficients corresponding to the class of the target pixel and the prediction tap. An image processing apparatus which processes first image data and outputs second image data of higher quality than the first image data. A down converter configured to down convert the input image data by reducing the number of pixels of the input image data; And An image converter for converting the first image data into the second image data using the input image data down-converted by the down converter as the first image data Including; The image converter, A prediction tap extracting unit, configured to extract, from the first image data, a plurality of pixels used to predict a target pixel of the second image data as a prediction tap; A class tap extracting unit configured to extract, as class taps, a plurality of pixels used to classify the target pixel into one of a plurality of classes from the first image data; A class classification unit configured to classify the class of the target pixel based on the class tap, A generator configured to generate coefficients corresponding to each of the plurality of classes from predetermined parameters and seed data used as seeds for the coefficients, A coefficient output unit configured to output a coefficient corresponding to the class of the target pixel among the coefficients corresponding to the plurality of classes generated by the generator, and And a calculator configured to determine the target pixel by performing a prediction operation using the coefficients corresponding to the class of the target pixel and the prediction tap.

Images