JP2005236633A - Signal processing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily realize many functions with single hardware. <P>SOLUTION: Receiving units 211 and 221 each receive a command sequence consisting of a plurality of commands to be transmitted from a control device 202, and supply it to R-ICs 212, 222, respectively. The R-IC 212 changes the internal structure in response to at least one command in the command sequence from the receiving unit 211, processes a first signal to be input thereinto, and outputs a second signal. The R-IC 222 changes the internal structure in response to at least one command in the command sequence from the receiving unit 221, processes a second signal to be output from the R-IC 212, and outputs a third signal. This apparatus is applicable to, for example, an IC (Integrated Circuit) chip and the like. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a signal processing apparatus and a signal processing method, and in particular, by switching the internal structure of hardware according to a plurality of commands, it is possible to easily realize many functions with a single hardware. The present invention relates to a signal processing apparatus and a signal processing method that can be performed.

  In recent years, various functions have been made into software as computers become faster and cheaper. That is, not only for general-purpose computers, but also for cellular phones and other PDAs (Personal Digital Assistant), AV (Audio Visual) devices such as television receivers, and home appliances such as electronic rice cookers, various processes are performed by CPU ( A processor such as a central processing unit (DSP) or a digital signal processor (DSP) is executed by executing software (program).

  Along with the softwareization of such functions, the software has become complicated and enormous, increasing the burden on the computer (CPU, DSP, etc.). Thus, an apparatus having a novel processor that reduces the processing load on the computer system has been proposed (see, for example, Patent Document 1).

JP 2002-358294 A.

  As described above, as long as various processes are performed by causing a computer to execute software, it is expected that development of more and more complicated and enormous software will be required in the future. And development of such software requires a lot of labor and time.

  In particular, when high-speed processing is required, for example, parallel processing by a plurality of processors is required. In the production of software for parallel processing, it is necessary to perform programming with a huge number of steps in consideration of the operation timing of multiple processors, and the burden on the software producer (engineer) is considerable. Become.

  On the other hand, if only a certain process is performed, there is a method of developing, for example, an IC (Integrated Circuit) (LSI (Large Scale Integration)) as dedicated hardware for performing the process.

  However, a dedicated hardware for performing a certain process can perform only that process, which is versatile.

  The present invention has been made in view of such a situation, and makes it possible to easily realize many functions with a single hardware without performing programming with a large number of steps. Is.

  In the signal processing device of the present invention, the first signal processing means switches the internal structure in accordance with at least one command in the command sequence composed of a plurality of commands, performs signal processing on the first signal, The second signal is output, and the second signal processing means switches the internal structure according to at least one command in the command sequence, performs signal processing on the second signal, and outputs the third signal. It is characterized by outputting.

  In the signal processing method of the present invention, the first signal processing means switches the internal structure in accordance with at least one command in a command sequence composed of a plurality of commands, and performs signal processing on the first signal. The first signal processing step for outputting the second signal and the second signal processing means switch the internal structure in accordance with at least one command in the command sequence and perform signal processing on the second signal. And a second signal processing step of outputting a third signal.

  Another signal processing apparatus according to the present invention switches an internal structure in accordance with at least one command in a command string made up of a plurality of commands, performs signal processing on the first signal, and outputs a second signal. Wherein the second signal is signal-processed by the second signal processing means for switching the internal structure in accordance with at least one command in the command sequence. .

  According to still another signal processing device of the present invention, the first signal processing means for switching the internal structure in accordance with at least one command in the command sequence including a plurality of commands processes the first signal. Second signal processing means for signal processing a second signal to be output, wherein the second signal processing means switches the internal structure according to at least one command in the command sequence, and Signal processing is performed, and a third signal is output.

  In the signal processing device and the signal processing method of the present invention, the first signal processing means switches the internal structure in response to at least one command in a command sequence made up of a plurality of commands, and outputs the first signal. Signal processing is performed to output a second signal. Then, the second signal processing means switches the internal structure in accordance with at least one command in the command sequence, performs signal processing on the second signal, and outputs a third signal.

  In another signal processing apparatus of the present invention, the first signal processing means switches the internal structure in accordance with at least one command in a command string made up of a plurality of commands, and performs signal processing on the first signal. Then, the second signal is output. The second signal is signal-processed by second signal processing means for switching the internal structure in accordance with at least one command in the command sequence.

  In still another signal processing device of the present invention, the first signal processing means for switching the internal structure in response to at least one command in a command sequence made up of a plurality of commands performs signal processing on the first signal. In the second signal processing means for signal processing the second signal to be output, the internal structure is switched in response to at least one command in the command sequence, and the second signal is signal-processed, A third signal is output.

  According to the present invention, it is possible to easily realize many functions with a single hardware.

  First, the correspondence between the constituent elements (invention specific items) described in the claims and the specific examples in the embodiments of the invention will be exemplified. In addition, this description is for confirming that the specific example which supports the invention described in the claim is described in the embodiment of the invention. Therefore, even if there are specific examples that are described in the embodiment of the invention but are not described here as corresponding to the configuration requirements, the specific examples are not included in the configuration. It does not mean that it does not correspond to a requirement. On the contrary, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the embodiments of the invention are described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, a divisional application will be made. Nor does it deny the existence of an invention added by amendment.

The signal processing device according to claim 1 is:
In a signal processing apparatus including first and second signal processing means (for example, R-ICs 212, 222, and 232 in FIG. 14),
The first signal processing means switches the internal structure in accordance with at least one command of a command string made up of a plurality of commands, performs signal processing on the first signal, and outputs a second signal. ,
The second signal processing means switches an internal structure in accordance with at least one command in the command sequence, performs signal processing on the second signal, and outputs a third signal. And

The signal processing device according to claim 3 is:
It further comprises command sequence generation means (for example, the control device 202 in FIG. 14) that generates the command sequence in response to an external signal.

The signal processing device according to claim 6 is:
The first signal processing means includes
Class tap selection means (for example, tap selection unit 253 in FIG. 27) for selecting a class tap used for classifying the signal of interest into any one of the plurality of classes from the first signal; ,
Class classification means for classifying the signal of interest based on the class tap (for example, the class classification unit 254 in FIG. 27);
A prediction tap selection unit (for example, a tap selection unit 252 in FIG. 27) that selects, from the first signal, a prediction tap used for calculation with the tap coefficient when obtaining the signal of interest;
Tap coefficient output means (for example, coefficient output unit 255 in FIG. 27) for outputting the tap coefficient of the class of the signal of interest;
Calculation means for obtaining the signal of interest by performing a calculation using the tap coefficient of the class of the signal of interest and the prediction tap selected for the signal of interest (for example, the prediction calculation unit 256 in FIG. 27). It is characterized by having and.

The signal processing device according to claim 9 is:
The second signal processing means includes
Class tap selection means (for example, tap selection unit 253 in FIG. 27) for selecting a class tap used for classifying the signal of interest into any one of the plurality of classes from the second signal; ,
Class classification means for classifying the signal of interest based on the class tap (for example, the class classification unit 254 in FIG. 27);
A prediction tap selection unit (for example, a tap selection unit 252 in FIG. 27) that selects, from the second signal, a prediction tap used for calculation with the tap coefficient when obtaining the signal of interest;
Tap coefficient output means (for example, coefficient output unit 255 in FIG. 27) for outputting the tap coefficient of the class of the signal of interest;
Calculation means for obtaining the signal of interest by performing a calculation using the tap coefficient of the class of the signal of interest and the prediction tap selected for the signal of interest (for example, the prediction calculation unit 256 in FIG. 27). It is characterized by having and.

A signal processing device according to claim 12 is provided.
Image signal output means (for example, a tuner unit 311 in FIG. 32) for receiving a broadcast signal and outputting the first image signal obtained from the broadcast signal is further provided.

A signal processing device according to claim 13 is provided.
A motion vector detection means (for example, a signal processing chip 334 in FIG. 35) that performs a motion vector detection process according to one command in the command sequence;
The first or second signal processing means performs signal processing using a motion vector detected by the motion vector detection processing.

The signal processing device according to claim 14 comprises:
The image processing apparatus further includes image signal storage means (for example, a memory unit 333 in FIG. 35) that stores the third image signal in response to one command in the command sequence.

The signal processing method according to claim 15 comprises:
In a signal processing method of a signal processing apparatus comprising first and second signal processing means,
The first signal processing means switches the internal structure in accordance with at least one command in a command string made up of a plurality of commands, performs signal processing on the first signal, and outputs a second signal. A first signal processing step (eg, steps S215 and S216 _{1 in} FIG. 17);
The second signal processing means switches an internal structure in accordance with at least one command in the command sequence, performs signal processing on the second signal, and outputs a third signal. And a signal processing step (for example, steps S215 and S216 _{2 in} FIG. 17).

The signal processing device according to claim 16 comprises:
First signal processing means (for example, a first signal processing means for switching the internal structure in accordance with at least one command in a command sequence of a plurality of commands, signal-processing the first signal, and outputting the second signal) R-IC 212) of FIG.
The second signal is signal-processed by second signal processing means (for example, R-IC 222 in FIG. 14) that switches an internal structure in response to at least one command in the command sequence. And

The signal processing device according to claim 17 comprises:
A first signal processing means (for example, the R-IC 212 in FIG. 14) that switches the internal structure in response to at least one command in a command sequence composed of a plurality of commands performs signal processing on the first signal. Second signal processing means (for example, R-IC 222 in FIG. 14) for processing the second signal to be output;
The second signal processing means switches an internal structure in accordance with at least one command in the command sequence, performs signal processing on the second signal, and outputs a third signal. And

  Hereinafter, embodiments of various apparatuses (systems) to which the present invention is applied will be described. Before that, class classification adaptive processing used for signal processing performed by the various apparatuses will be described. The class classification adaptive processing is an example of processing used for signal processing performed by various devices, and the signal processing performed by various devices may not use the class classification adaptive processing.

  Here, the class classification adaptation process will be described by taking an image conversion process for converting the first image data (image signal) to the second image data (image signal) as an example.

  The image conversion processing for converting the first image data into the second image data is various signal processing depending on the definition of the first and second image data.

  That is, for example, if the first image data is image data with low spatial resolution and the second image data is image data with high spatial resolution, the image conversion process creates a spatial resolution that improves the spatial resolution ( (Improved) processing.

  Further, for example, if the first image data is low S / N (Siginal / Noise) image data and the second image data is high S / N image data, the image conversion process may reduce noise. It can be referred to as noise removal processing to be removed.

  Further, for example, if the first image data is image data having a predetermined number of pixels (size) and the second image data is image data in which the number of pixels of the first image data is increased or decreased, The image conversion process can be referred to as a resizing process for resizing (enlarging or reducing) an image.

  In addition, for example, if the first image data is image data with low time resolution and the second image data is image data with high time resolution, the image conversion process creates time resolution that improves the time resolution ( (Improved) processing.

  Further, for example, the first image data is set as decoded image data obtained by decoding image data encoded in units of blocks such as MPEG (Moving Picture Experts Group) encoding, and the second image data. Is image data before encoding, it can be said that the image conversion process is a distortion removal process for removing various distortions such as block distortion caused by MPEG encoding and decoding.

  In the spatial resolution creation process, when converting the first image data that is image data with low spatial resolution into the second image data that is image data with high spatial resolution, the second image data is converted into the second image data. The image data can have the same number of pixels as the one image data, or the image data can have more pixels than the first image data. When the second image data is image data having a larger number of pixels than the first image data, the spatial resolution creation process is a process for improving the spatial resolution and resizing to increase the image size (number of pixels). It is also a process.

  As described above, according to the image conversion process, various signal processes can be realized depending on how the first and second image data are defined.

  In the class classification adaptive processing as the image conversion processing as described above, the target pixel of interest (pixel value) in the second image data is classified into one of a plurality of classes. The pixel of interest (pixel value) is obtained by calculation using the tap coefficient of the class obtained by the above and the pixel (pixel value) of the first image data selected for the pixel of interest.

  That is, FIG. 1 shows a configuration example of an image conversion apparatus 1 that performs image conversion processing by class classification adaptation processing.

  In the image conversion apparatus 1, the image data supplied thereto is supplied to the tap selection units 12 and 13 as the first image data.

  The pixel-of-interest selection unit 11 sequentially sets pixels constituting the second image data as the pixel of interest, and supplies information representing the pixel of interest to a necessary block.

  The tap selection unit 12 selects some of the pixels (the pixel values) constituting the first image data used to predict the target pixel (the pixel values thereof) as prediction taps.

  Specifically, the tap selection unit 12 selects, as prediction taps, a plurality of pixels of the first image data that are spatially or temporally close to the temporal and spatial positions of the target pixel.

  The tap selection unit 13 selects some of the pixels constituting the first image data used for classifying the target pixel into any of several classes as class taps. That is, the tap selection unit 13 selects a class tap in the same manner as the tap selection unit 12 selects a prediction tap.

  Note that the prediction tap and the class tap may have the same tap structure or may have different tap structures.

  The prediction tap obtained by the tap selection unit 12 is supplied to the prediction calculation unit 16, and the class tap obtained by the tap selection unit 13 is supplied to the class classification unit 14.

  The class classification unit 14 classifies the target pixel based on the class tap from the tap selection unit 13 and supplies a class code corresponding to the resulting class to the coefficient output unit 15.

  Here, as a method of classifying, for example, ADRC (Adaptive Dynamic Range Coding) or the like can be employed.

  In the method using ADRC, the pixels constituting the class tap are subjected to ADRC processing, and the class of the pixel of interest is determined according to the ADRC code obtained as a result.

In the K-bit ADRC, for example, the maximum value MAX and the minimum value MIN of the pixels constituting the class tap are detected, and DR = MAX-MIN is set as the local dynamic range of the set, and this dynamic range Based on DR, the pixel value of each pixel constituting the class tap is requantized to K bits. That is, the pixel value of each pixel forming the class taps, the minimum value MIN is subtracted, and the subtracted value is divided by DR / 2 ^K (requantization). A bit string obtained by arranging the pixel values of the K-bit pixels constituting the class tap in a predetermined order is output as an ADRC code. Therefore, for example, when the class tap is subjected to 1-bit ADRC processing, the pixel value of each pixel constituting the class tap is divided by the average value of the maximum value MAX and the minimum value MIN (rounded down). Thereby, the pixel value of each pixel is set to 1 bit (binarized). Then, a bit string in which the 1-bit pixel values are arranged in a predetermined order is output as an ADRC code.

Note that, for example, the level distribution pattern of the pixel values of the pixels constituting the class tap may be output to the class classification unit 14 as it is as a class code. However, in this case, if the class tap is composed of pixel values of N pixels and K bits are assigned to the pixel values of each pixel, the number of class codes output by the class classification unit 14 Is (2 ^N ) ^K , which is a huge number that is exponentially proportional to the number K of bits of the pixel value of the pixel.

  Accordingly, the class classification unit 14 preferably performs class classification by compressing the information amount of the class tap by the above-described ADRC processing or vector quantization.

  The coefficient output unit 15 stores tap coefficients for each class obtained by learning described later, and is further stored in an address corresponding to the class code supplied from the class classification unit 14 among the stored tap coefficients. The tap coefficient (the tap coefficient of the class represented by the class code supplied from the class classification unit 14) is output. This tap coefficient is supplied to the prediction calculation unit 16.

  Here, the tap coefficient corresponds to a coefficient that is multiplied with input data in a so-called tap in the digital filter.

  The prediction calculation unit 16 acquires the prediction tap output from the tap selection unit 12 and the tap coefficient output from the coefficient output unit 15, and uses the prediction tap and the tap coefficient to predict the true value of the target pixel. Predetermined calculation for obtaining is performed. Thereby, the prediction calculation unit 16 obtains and outputs the pixel value (predicted value) of the pixel of interest, that is, the pixel value of the pixels constituting the second image data.

  Next, image conversion processing by the image conversion apparatus 1 in FIG. 1 will be described with reference to the flowchart in FIG.

  In step S11, the pixel-of-interest selecting unit 11 selects one of the pixels constituting the second image data for the first image data input to the image conversion device 1 that has not yet been set as the pixel of interest. The pixel is selected as the target pixel, and the process proceeds to step S12. That is, for example, the pixel-of-interest selection unit 11 selects, as pixels of interest, pixels that have not yet been selected as pixels of interest in the raster scan order among the pixels constituting the second image data.

  In step S12, the tap selection units 12 and 13 respectively select the prediction tap and the class tap for the pixel of interest from the first image data supplied thereto. The prediction tap is supplied from the tap selection unit 12 to the prediction calculation unit 16, and the class tap is supplied from the tap selection unit 13 to the class classification unit 14.

  The class classification unit 14 receives the class tap for the target pixel from the tap selection unit 13, and classifies the target pixel based on the class tap in step S13. Further, the class classification unit 14 outputs a class code representing the class of the target pixel obtained as a result of the class classification to the coefficient output unit 15, and proceeds to step S14.

  In step S14, the coefficient output unit 15 acquires and outputs the tap coefficient stored at the address corresponding to the class code supplied from the class classification unit 14. Further, in step S14, the prediction calculation unit 16 acquires the tap coefficient output by the coefficient output unit 15, and the process proceeds to step S15.

  In step S <b> 15, the prediction calculation unit 16 performs a predetermined prediction calculation using the prediction tap output from the tap selection unit 12 and the tap coefficient acquired from the coefficient output unit 15. Thereby, the prediction calculation part 16 calculates | requires and outputs the pixel value of an attention pixel, and progresses to step S16.

  In step S <b> 16, the target pixel selection unit 11 determines whether there is second image data that has not yet been set as the target pixel. If it is determined in step S16 that there is still second image data that is not the target pixel, the process returns to step S11, and the same processing is repeated thereafter.

  If it is determined in step S16 that there is no second image data that has not yet been set as the target pixel, the process ends.

  Next, the prediction calculation in the prediction calculation unit 16 of FIG. 1 and the learning of the tap coefficient stored in the coefficient output unit 15 will be described.

  Now, for example, the high-quality image data (high-quality image data) is used as the second image data, and the high-quality image data is filtered by an LPF (Low Pass Filter) to reduce the image quality (resolution). Using the low-quality image data (low-quality image data) as the first image data, a prediction tap is selected from the low-quality image data, and using the prediction tap and the tap coefficient, pixels (high-quality image data) Consider that the pixel value of (image quality pixel) is obtained (predicted) by a predetermined prediction calculation.

  For example, when a linear primary prediction calculation is adopted as the predetermined prediction calculation, the pixel value y of the high-quality pixel is obtained by the following linear linear expression.

・・・（１）

... (1)

In Equation (1), x _n represents a pixel value of an nth low-quality image data pixel (hereinafter referred to as a low-quality pixel as appropriate) that constitutes a prediction tap for the high-quality pixel y, and w _n represents the nth tap coefficient multiplied by the nth low image quality pixel (its pixel value). In equation (1), the prediction tap is assumed to be composed of N low image quality pixels x ₁ , x ₂ ,..., X _N.

  Here, the pixel value y of the high-quality pixel can be obtained not by the linear primary expression shown in Expression (1) but by a higher-order expression of the second or higher order.

Now, when the true value of the pixel value of the high-quality pixel of the k-th sample is expressed as y _k and the predicted value of the true value y _k obtained by the equation (1) is expressed as y _k ′, the prediction error e _k Is expressed by the following equation.

・・・（２）

... (2)

Now, since the predicted value y _k ′ of Equation (2) is obtained according to Equation (1), the following equation is obtained by replacing y _k ′ of Equation (2) according to Equation (1).

・・・（３）

... (3)

In Equation (3), x _{n, k} represents the nth low-quality pixel that constitutes the prediction tap for the high-quality pixel of the k-th sample.

Tap coefficient w _n for the prediction error e _k 0 of the formula (3) (or Equation (2)) is, is the optimal for predicting the high-quality pixel, for all the high-quality pixel, such In general, it is difficult to obtain a large tap coefficient w _n .

Therefore, as the standard for indicating that the tap coefficient w _n is optimal, for example, when adopting the method of least squares, optimal tap coefficient w _n, the sum E of square errors expressed by the following formula It can be obtained by minimizing.

・・・（４）

... (4)

However, in Equation (4), K is the high image quality pixel y _k and the low image quality pixels x _{1, k} , x _{2, k} ,..., X _N constituting the prediction tap for the high image quality pixel y _{k. , k} represents the number of samples (the number of learning samples).

The minimum value of the sum E of square errors of Equation (4) (minimum value), as shown in Equation (5), given that by partially differentiating the sum E with the tap coefficient w _n by w _n to 0.

・・・（５）

... (5)

Therefore, when partial differentiation of above equation (3) with the tap coefficient w _n, the following equation is obtained.

・・・（６）

... (6)

  From the equations (5) and (6), the following equation is obtained.

・・・（７）

... (7)

By substituting equation (3) into e _k in equation (7), equation (7) can be expressed by the normal equation shown in equation (8).

・・・（８）

... (8)

Normal equation of Equation (8), for example, by using a like sweeping-out method (Gauss-Jordan elimination method) can be solved for the tap coefficient w _n.

The normal equation of Equation (8), by solving for each class, the optimal tap coefficient (here, the tap coefficient that minimizes the sum E of square errors) to w _n, can be found for each class .

Next, FIG. 3 shows an example of the configuration of a learning apparatus 21 performs learning for determining the tap coefficient w _n by solving the normal equations in equation (8).

Learning image storage unit 31 stores the learning image data used for learning of the tap coefficient w _n. Here, as the learning image data, for example, high-resolution image data with high resolution can be used.

  The teacher data generation unit 32 reads learning image data from the learning image storage unit 31. Further, the teacher data generation unit 32 generates, from the learning image data, teacher data for the tap coefficient learning (true value), that is, teacher data that becomes a pixel value of a mapping destination as a prediction calculation according to the equation (1). And supplied to the teacher data storage unit 33. Here, for example, the teacher data generation unit 32 supplies high-quality image data as learning image data to the teacher data storage unit 33 as teacher data as it is.

  The teacher data storage unit 33 stores high-quality image data as teacher data supplied from the teacher data generation unit 32.

  The student data generation unit 34 reads learning image data from the learning image storage unit 31. Further, the student data generation unit 34 generates, from the learning image data, a student who learns the tap coefficient, that is, student data that is a pixel value to be converted by mapping as a prediction calculation according to Expression (1). The data is supplied to the storage unit 174. Here, the student data generation unit 34 generates low-quality image data by filtering the high-quality image data as the learning image data, for example, to reduce the resolution, and this low-quality image data is converted into the low-quality image data. , And supplied to the student data storage unit 35 as student data.

  The student data storage unit 35 stores the student data supplied from the student data generation unit 34.

  The learning unit 36 sequentially sets pixels constituting the high-quality image data as the teacher data stored in the teacher data storage unit 33 as the target pixel, and the student data stored in the student data storage unit 35 for the target pixel. Among the low image quality pixels constituting the low image quality image data, a low image quality pixel having the same tap structure as that selected by the tap selection unit 12 in FIG. 1 is selected as a prediction tap. Further, the learning unit 36 uses each pixel constituting the teacher data and the prediction tap selected when the pixel is the target pixel, and establishes a normal equation of Expression (8) for each class. By solving, the tap coefficient for each class is obtained.

  That is, FIG. 4 shows a configuration example of the learning unit 36 of FIG.

  The target pixel selection unit 41 sequentially selects pixels constituting the teacher data stored in the teacher data storage unit 33 as the target pixel, and supplies information representing the target pixel to a necessary block.

  The tap selection unit 42 selects the same pixel that is selected by the tap selection unit 12 in FIG. 1 from the low-quality pixels constituting the low-quality image data as the student data stored in the student data storage unit 35 for the target pixel. Thus, a prediction tap having the same tap structure as that obtained by the tap selection unit 12 is obtained and supplied to the adding unit 45.

  The tap selection unit 43 is the same pixel as the pixel of interest selected by the tap selection unit 13 in FIG. 1 from the low image quality pixels constituting the low image quality image data as the student data stored in the student data storage unit 35. Thus, a class tap having the same tap structure as that obtained by the tap selection unit 13 is obtained and supplied to the class classification unit 44.

  The class classification unit 44 performs the same class classification as the class classification unit 14 of FIG. 1 based on the class tap output from the tap selection unit 43, and adds the class code corresponding to the resulting class to the addition unit 45. Output.

  The adding unit 45 reads the teacher data (pixels) that are the target pixel from the teacher data storage unit 33, and configures the target pixel and the prediction tap for the target pixel supplied from the tap selection unit 42. Addition for data (pixels) is performed for each class code supplied from the class classification unit 44.

That is, the addition unit 45 is supplied with the teacher data y _k stored in the teacher data storage unit 33, the prediction tap x _{n, k} output from the tap selection unit 42, and the class code output from the class classification unit 44. .

Then, the adding unit 45 uses the prediction tap (student data) x _{n, k} for each class corresponding to the class code supplied from the class classifying unit 44 _, and uses the prediction data (student data) x _{n, k} for the student data in the matrix on the left side of equation (8) (X _{n, k} x _{n ′, k} ) and computation corresponding to summation (Σ).

Furthermore, the adding unit 45 also uses the prediction tap (student data) x _{n, k} and the teacher data y _k for each class corresponding to the class code supplied from the class classification unit 44, and the equation (8) Multiplication (x _{n, k} y _k ) of student data x _{n, k} and teacher data y _k in the vector on the right side and calculation corresponding to summation (Σ) are performed.

That is, the adding unit 45 performs the left-side matrix component (Σx _{n, k} x _{n ′, k} ) and the right-side vector component ( Σx _{n, k} y _k ) is stored in its built-in memory (not shown), and the matrix component (Σx _{n, k} x _{n ′, k} ) or vector component (Σx _{n, k} y _k) ), The corresponding component x _{n, k + 1} x _n calculated using the teacher data y _{k + 1} and the student data x _{n, k + 1} for the teacher data newly set as the pixel of interest. _{', k + 1} or x _{n, k + 1} y _{k + 1} is added (addition represented by the summation of equation (8) is performed).

  Then, the addition unit 45 performs the above-described addition using all the teacher data stored in the teacher data storage unit 33 (FIG. 3) as the target pixel, so that the normality shown in Expression (8) is obtained for each class. When the equation is established, the normal equation is supplied to the tap coefficient calculation unit 46.

Tap coefficient calculating section 46 solves the normal equations for each class supplied from the adder 45, for each class, and outputs the determined optimal tap coefficient w _n.

The coefficient output unit 15 in the image conversion apparatus 1 in FIG. 1, the tap coefficient w _n for each class determined as described above is stored.

  Here, as described above, various tap coefficients may be used depending on how image data to be used as student data corresponding to the first image data and image data to be used as teacher data corresponding to the second image data are selected. What performs the image conversion process of this can be obtained.

  That is, as described above, the high-quality image data is used as teacher data corresponding to the second image data, and the low-quality image data obtained by degrading the spatial resolution of the high-quality image data is used as the first image data. As a tap coefficient, the first image which is low-quality image data (SD (Standard Definition) image) is shown as the first tap coefficient from the top of FIG. It is possible to obtain what performs image conversion processing as spatial resolution creation processing for converting data into second image data that is high-quality image data (HD (High Definition) image data) with improved spatial resolution. .

  In this case, the first image data (student data) may have the same or less number of pixels as the second image data (teacher data).

  Also, for example, the high-quality image data is used as teacher data, and the tap coefficient is learned by using, as the student data, image data on which noise is superimposed on the high-quality image data as the teacher data. As shown second from the top in FIG. 5, the first image data, which is low S / N image data, is high S / N image data obtained by removing (reducing) noise contained therein. What performs an image conversion process as a noise removal process converted into 2nd image data can be obtained.

  Further, for example, by performing tap coefficient learning by using a certain piece of image data as teacher data and learning the tap coefficient using image data obtained by thinning out the number of pixels of the image data as the teacher data as the tap data, FIG. As shown in the third from the top, enlargement processing (resizing processing) for converting first image data that is a part of image data into second image data that is enlarged image data obtained by enlarging the first image data ) For performing the image conversion process.

  Note that the tap coefficient for performing the enlargement processing is a tap using high-quality image data as teacher data and low-quality image data obtained by degrading the spatial resolution of the high-quality image data by thinning out the number of pixels as student data. It can also be obtained by learning coefficients.

  In addition, for example, by using the high frame rate image data as the teacher data and learning the tap coefficient using the image data obtained by thinning out the frames of the high frame rate image data as the teacher data as the student data, taps are performed. As a coefficient, as shown in the fourth (first bottom) from the top in FIG. 5, as a time resolution creation process for converting the first image data having a predetermined frame rate into the second image data having a high frame rate. What performs an image conversion process can be obtained.

  Next, processing (learning processing) of the learning device 21 in FIG. 3 will be described with reference to the flowchart in FIG.

  First, in step S21, the teacher data generation unit 32 and the student data generation unit 34 generate teacher data and student data from the learning image data stored in the learning image storage unit 31, and the teacher data storage unit 33 and the student data generation unit 34 are supplied and stored.

  Note that what kind of student data and teacher data are generated in the teacher data generation unit 32 and the student data generation unit 34, respectively, is a tap coefficient for which of the types of image conversion processing as described above. It depends on what you learn.

  Thereafter, the process proceeds to step S22, and in the learning unit 36 (FIG. 4), the target pixel selection unit 41 selects, as the target pixel, the teacher data stored in the teacher data storage unit 33 that has not yet been set as the target pixel. Then, the process proceeds to step S23. In step S23, the tap selection unit 42 selects a pixel as student data to be a prediction tap from the student data stored in the student data storage unit 35 for the target pixel, and supplies the selected pixel to the adding unit 45. Again, the unit 43 selects student data as class taps from the student data stored in the student data storage unit 35 for the pixel of interest, and supplies the selected class data to the class classification unit 44.

  In step S24, the class classification unit 44 classifies the target pixel based on the class tap for the target pixel, and outputs the class code corresponding to the class obtained as a result to the addition unit 45. The process proceeds to step S25.

  In step S <b> 25, the adding unit 45 reads the target pixel from the teacher data storage unit 33, and obtains the target pixel and student data constituting the prediction tap selected for the target pixel supplied from the tap selection unit 42. Addition of the target expression (8) is performed for each class code supplied from the class classification unit 44, and the process proceeds to step S26.

  In step S <b> 26, the pixel-of-interest selection unit 41 determines whether teacher data that is not yet a pixel of interest is stored in the teacher data storage unit 33. If it is determined in step S26 that teacher data that is not a pixel of interest is still stored in the teacher data storage unit 33, the process returns to step S22, and the same processing is repeated thereafter.

  If it is determined in step S26 that the teacher data that is not the pixel of interest is not stored in the teacher data storage unit 33, the adding unit 45 obtains the class obtained by the processes in steps S22 to S26 so far. The matrix on the left side and the vector on the right side in each equation (8) are supplied to the tap coefficient calculation unit 46, and the process proceeds to step S27.

In step S27, the tap coefficient calculation unit 46 solves each class by solving a normal equation for each class composed of a matrix on the left side and a vector on the right side in the equation (8) for each class supplied from the addition unit 45. each and determines and outputs the tap coefficient w _n, the process ends.

  It should be noted that due to the number of learning image data being insufficient, etc., there may occur a class in which the number of normal equations necessary for obtaining the tap coefficient cannot be obtained. The tap coefficient calculation unit 46 outputs a default tap coefficient, for example.

  Next, FIG. 7 shows a configuration example of an image conversion device 51 that is another image conversion device that performs image conversion processing by class classification adaptation processing.

  In the figure, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the image conversion apparatus 51 is configured in the same manner as the image conversion apparatus 1 in FIG. 1 except that a coefficient output unit 55 is provided instead of the coefficient output unit 15.

  In addition to the class (class code) supplied from the class classification unit 14, the coefficient output unit 55 is supplied with, for example, a parameter z input from the outside in response to a user operation. The coefficient output unit 55 generates a tap coefficient for each class corresponding to the parameter z as will be described later, and predicts the tap coefficient of the class from the class classification unit 14 out of the tap coefficients for each class. To the unit 16.

  FIG. 8 shows a configuration example of the coefficient output unit 55 of FIG.

  The coefficient generator 61 generates a tap coefficient for each class based on the coefficient seed data stored in the coefficient seed memory 62 and the parameter z stored in the parameter memory 63, and supplies the tap coefficient to the coefficient memory 64. Remember to overwrite.

  The coefficient seed memory 62 stores coefficient seed data for each class obtained by learning of coefficient seed data described later. Here, the coefficient seed data is so-called seed data that generates tap coefficients.

  The parameter memory 63 stores the parameter z input from the outside in accordance with a user operation or the like in an overwritten form.

  The coefficient memory 64 stores the tap coefficient for each class supplied from the coefficient generation unit 61 (the tap coefficient for each class corresponding to the parameter z). And the coefficient memory 64 reads the tap coefficient of the class supplied from the class classification | category part 14 (FIG. 7), and outputs it to the prediction calculating part 16 (FIG. 7).

  In the image conversion apparatus 51 of FIG. 7, when the parameter z is input from the outside to the coefficient output unit 55, the parameter z is overwritten in the parameter memory 63 of the coefficient output unit 55 (FIG. 8). Remembered.

  When the parameter z is stored in the parameter memory 63 (when the storage content of the parameter memory 63 is updated), the coefficient generation unit 61 reads out the coefficient seed data for each class from the coefficient seed memory 62 and from the parameter memory 63. The parameter z is read, and the tap coefficient for each class is obtained based on the coefficient seed data and the parameter z. Then, the coefficient generation unit 61 supplies the tap coefficient for each class to the coefficient memory 64 and stores it in the form of overwriting.

  The image conversion device 51 stores tap coefficients, and generates and outputs tap coefficients corresponding to the parameter z in a coefficient output unit 55 provided in place of the coefficient output unit 15 that outputs the tap coefficients. Except for this, processing similar to the processing according to the flowchart of FIG. 2 performed by the image conversion apparatus 1 of FIG. 1 is performed.

  Next, prediction calculation in the prediction calculation unit 16 in FIG. 7, generation of tap coefficients in the coefficient generation unit 61 in FIG. 8, and learning of coefficient seed data stored in the coefficient seed memory 62 will be described.

  As in the embodiment of FIG. 1, low-quality image data in which high-quality image data (high-quality image data) is used as second image data and the spatial resolution of the high-quality image data is reduced. (Low-quality image data) is the first image data, a prediction tap is selected from the low-quality image data, and the pixel value of the high-quality pixel that is a pixel of the high-quality image data is selected using the prediction tap and the tap coefficient. For example, consider obtaining (predicting) by linear primary prediction calculation of equation (1).

  Here, the pixel value y of the high-quality pixel can be obtained not by the linear primary expression shown in Expression (1) but by a higher-order expression of the second or higher order.

In the embodiment of FIG. 8, the coefficient generation unit 61 generates the tap coefficient w _n from the coefficient seed data stored in the coefficient seed memory 62 and the parameter z stored in the parameter memory 63. generating the tap coefficient w _n in the coefficient generating unit 61, for example, to be performed by the following equation using the coefficient seed data and the parameter z.

・・・（９）

... (9)

However, in the equation (9), beta _{m, n} denotes the m-th coefficient seed data used for determining the n-th tap coefficient w _n. In the equation (9), the tap coefficient w _n is obtained using M coefficient seed data β _{1, n} , β _{2, n} ,..., Β _{M, n} .

Here, the formula for obtaining the tap coefficient w _n from the coefficient seed data β _{m, n} and the parameter z is not limited to the formula (9).

Now, a value z ^m−1 determined by the parameter z in equation (9) is defined by the following equation by introducing a new variable t _m .

・・・（１０）

... (10)

  By substituting equation (10) into equation (9), the following equation is obtained.

・・・（１１）

(11)

According to equation (11), the tap coefficient w _n will be asked by the linear first-order equation of the coefficient seed data beta _{m, n} and the variable t _m.

Now, when the true value of the pixel value of the high-quality pixel of the k-th sample is expressed as y _k and the predicted value of the true value y _k obtained by the equation (1) is expressed as y _k ′, the prediction error e _k is expressed by the following equation.

・・・（１２）

(12)

Now, since the predicted value y _k ′ of Expression (12) is obtained according to Expression (1), the following expression is obtained by replacing y _k ′ of Expression (12) according to Expression (1).

・・・（１３）

... (13)

In Equation (13), x _{n, k} represents the n-th low-quality pixel that constitutes the prediction tap for the high-quality pixel of the k-th sample.

To w _n of formula (13), by substituting equation (11), the following equation is obtained.

・・・（１４）

(14)

Prediction error e _k coefficient seed data beta _{m, n} to 0 in Equation (14), is the optimal for predicting the high-quality pixel, for all the high-quality pixel, such coefficient seed data It is generally difficult to obtain β _{m, n} .

Therefore, as a standard indicating that the coefficient seed data β _{m, n} is optimal, for example, when the least square method is adopted, the optimal coefficient seed data β _{m, n} is expressed by the following equation. It can be obtained by minimizing the sum E of square errors.

・・・（１５）

... (15)

However, in Expression (15), K is a high-quality pixel y _k and low-quality pixels x _{1, k} , x _{2, k} ,..., X _N that constitute a prediction tap for the high-quality pixel y _{k. , k} represents the number of samples (the number of learning samples).

The minimum value (minimum value) of the sum E of squared errors in Equation (15) is β _{m, where} 0 is a partial differentiation of the sum E with coefficient seed data β _{m, n} as shown in Equation (16) _. given by _n .

・・・（１６）

... (16)

  By substituting equation (13) into equation (16), the following equation is obtained.

・・・（１７）

... (17)

Now, X _{i, p, j, q} and Y _{i, p} are defined as shown in equations (18) and (19).

・・・（１８）

... (18)

・・・（１９）

... (19)

In this case, Expression (17) can be expressed by a normal equation shown in Expression (20) using X _{i, p, j, q} and Y _{i, p} .

・・・（２０）

... (20)

The normal equation of Expression (20) can be solved for the coefficient seed data β _{m, n} by using, for example, a sweeping method (Gauss-Jordan elimination method) or the like.

In the image conversion device 51 of FIG. 7, a large number of high-quality pixels y ₁ , y ₂ ,..., Y _K are used as teacher data for learning, and prediction taps for the respective high-quality pixels y _k are used. low quality pixels x _{1, k} that _{constitute, x 2, k, ···,} x N, as student data serving as a student learning _k, the vertical and solves learning normal equation of formula (20) for each class The coefficient seed data β _{m, n} for each class obtained by performing is stored in the coefficient seed memory 62 of the coefficient output unit 55 (FIG. 8), and the coefficient generation unit 61 stores the coefficient seed data β _{m, n.} and _n, from the stored parameter z in the parameter memory 63, according to equation (9), the tap coefficient w _n for each class is generated. Then, the prediction calculation unit 16 uses the tap coefficient w _n and the low image quality pixel (the pixel of the first image data) x _n constituting the prediction tap for the pixel of interest as the high image quality pixel, and the equation (1) ) Is calculated, the pixel value of the target pixel as a high-quality pixel (predicted value close to it) is obtained.

Next, FIG. 9 shows a configuration example of a learning device 71 that performs learning for obtaining coefficient seed data β _{m, n} for each class by solving the normal equation of Expression (20) for each class. .

  In the figure, portions corresponding to those in the learning device 21 of FIG. 3 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the learning device 71 is provided with a student data generation unit 74 and a learning unit 76 instead of the student data generation unit 34 and the learning unit 36, respectively, and a parameter generation unit 81 is newly provided. 3 is configured in the same manner as the learning device 21 in FIG.

  The student data generation unit 74 generates student data from the learning image data, and supplies the student data to the student data storage unit 35 to store the same as in the student data generation unit 34 of FIG.

  However, in addition to the learning image data, the student data generation unit 74 is supplied with some values in the range that can be taken by the parameter z supplied to the parameter memory 63 of FIG. It has become. That is, assuming that the value that can be taken by the parameter z is a real number in the range of 0 to Z, the student data generation unit 74 has, for example, z = 0, 1, 2,. It is supplied from the section 81.

  The student data generation unit 74 filters low-quality image data as student data by filtering high-quality image data as learning image data using, for example, an LPF having a cutoff frequency corresponding to the parameter z supplied thereto. Is generated.

  Accordingly, the student data generation unit 74 generates Z + 1 types of low-quality image data as student data having different spatial resolutions for the high-quality image data as the learning image data.

  Note that, here, for example, as the value of the parameter z increases, the high-quality image data is filtered by using an LPF with a high cutoff frequency, and low-quality image data as student data is generated. Therefore, here, the lower the image quality data corresponding to the larger parameter z, the higher the spatial resolution.

  In the present embodiment, in order to simplify the explanation, the student data generation unit 74 reduces the spatial resolution of the high-quality image data in both the horizontal and vertical directions by an amount corresponding to the parameter z. Assume that low-quality image data is generated.

  The learning unit 76 uses the teacher data stored in the teacher data storage unit 33, the student data stored in the student data storage unit 35, and the parameter z supplied from the parameter generation unit 81, so that the coefficient seed data for each class. Is output.

  The parameter generation unit 81 generates, for example, z = 0, 1, 2,..., Z as described above as several values in the range that the parameter z can take, and learns with the student data generation unit 74. To the unit 76.

  Next, FIG. 10 shows a configuration example of the learning unit 76 of FIG. In the figure, portions corresponding to those in the learning unit 36 in FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  As with the tap selection unit 42 in FIG. 4, the tap selection unit 92 performs the tap in FIG. 7 from the low quality pixels that constitute the low quality image data as the student data stored in the student data storage unit 35 for the target pixel. A prediction tap having the same tap structure as that selected by the selection unit 12 is selected and supplied to the addition unit 95.

  As with the tap selection unit 43 in FIG. 4, the tap selection unit 93 also taps the pixel of interest from the low-quality pixels constituting the low-quality image data as the student data stored in the student data storage unit 35 with respect to the target pixel. A class tap having the same tap structure as the selection unit 13 selects is selected and supplied to the class classification unit 44.

  However, in FIG. 10, the parameter z generated by the parameter generation unit 81 in FIG. 9 is supplied to the tap selection units 42 and 43, and the tap selection units 42 and 43 are supplied from the parameter generation unit 81. Prediction taps and class taps are generated from student data generated corresponding to the parameter z to be generated (here, low-quality image data as student data generated using the LPF of the cutoff frequency corresponding to the parameter z). Select each one.

  The adding unit 95 reads out the target pixel from the teacher data storage unit 33 in FIG. 9, student data constituting the target pixel, the prediction tap configured for the target pixel supplied from the tap selection unit 42, and the student The addition for the parameter z when the data is generated is performed for each class supplied from the class classification unit 44.

In other words, the addition unit 95 includes the teacher data y _k as the target pixel stored in the teacher data storage unit 33 and the prediction tap x _{i, k} (x _{j, k} ) for the target pixel output from the tap selection unit 42. The class of the pixel of interest output from the class classification unit 44 is supplied, and the parameter z when the student data constituting the prediction tap for the pixel of interest is generated is supplied from the parameter generation unit 81.

Then, the adding unit 95 uses the prediction tap (student data) x _{i, k} (x _{j, k} ) and the parameter z for each class supplied from the class classification unit 44 _, and the matrix on the left side of Expression (20). Of the student data and the parameter z for obtaining the component X _{i, p, j, q} defined by the equation (18) (x _{i, k} t _p x _{j, k} t _q ) and summation (Σ ) Is performed. Incidentally, t _p of formula (18), according to equation (10), is calculated from the parameter z. The same applies to _{tq in} equation (18).

Further, the adding unit 95 also uses the prediction tap (student data) x _{i, k} , the teacher data y _k , and the parameter z for each class corresponding to the class code supplied from the class classification unit 44, and uses the formula z Multiplication (x _{i, k} t _p ) of student data x _{i, k} , teacher data y _k , and parameter z for obtaining the component Y _{i, p} defined by equation (19) in the vector on the right side of (20) y _k ) and a calculation corresponding to summation (Σ). Incidentally, t _p of formula (19), according to equation (10), is calculated from the parameter z.

That is, the adding unit 95 performs the left-side matrix component X _{i, p, j, q} and the right-side vector component Y _{i, p} in Expression (20) previously obtained for the teacher data set as the target pixel. Is stored in its built-in memory (not shown), and the teacher data newly set as the pixel of interest for the matrix component X _{i, p, j, q} or the vector component Y _{i, p} , The corresponding component x _{i, k} t _p x _{j, k} t _q or x _i, calculated using the teacher data y _k , student data x _{i, k} (x _{j, k} ), and parameter z _k t _p y _k is added (addition represented by summation in the component X _{i, p, j, q in the} equation (18) or the component Y _{i, p} in the equation (19) is performed).

  Then, the addition unit 95 performs the above-described addition for all the parameter data z of 0, 1,..., Z, using all the teacher data stored in the teacher data storage unit 33 as the target pixel. Thus, when the normal equation shown in the equation (20) is established for each class, the normal equation is supplied to the coefficient seed calculating unit 96.

The coefficient seed calculator 96 obtains and outputs coefficient seed data β _{m, n} for each class by solving the normal equation for each class supplied from the adder 95.

  Next, processing (learning processing) of the learning device 71 in FIG. 9 will be described with reference to the flowchart in FIG.

  First, in step S31, the teacher data generation unit 32 and the student data generation unit 74 generate and output teacher data and student data from the learning image data stored in the learning image storage unit 31, respectively. . That is, the teacher data generation unit 32 outputs the learning image data as it is, for example, as teacher data. In addition, the student data generation unit 74 is supplied with a parameter z of Z + 1 values generated by the parameter generation unit 81. For example, the student data generation unit 74 filters the learning image data with an LPF having a cutoff frequency corresponding to the parameter z of Z + 1 values (0, 1,..., Z) from the parameter generation unit 81. Thus, the Z + 1 frame student data is generated and output for the teacher data (learning image data) of each frame.

  The teacher data output from the teacher data generation unit 32 is supplied to and stored in the teacher data storage unit 33, and the student data output from the student data generation unit 74 is supplied to and stored in the student data storage unit 35.

  Thereafter, the process proceeds to step S32, and the parameter generation unit 81 sets the parameter z as an initial value, for example, 0, and supplies the parameter z to the tap selection units 42 and 43 and the addition unit 95 of the learning unit 76 (FIG. 10). Then, the process proceeds to step S33. In step S33, the pixel-of-interest selection unit 41 proceeds to step S34 using the teacher data stored in the teacher data storage unit 33 as the pixel of interest that has not yet been the pixel of interest.

  In step S34, the tap selection unit 42 stores the student data corresponding to the parameter z output from the parameter generation unit 81 and stored in the student data storage unit 35 for the target pixel (for learning corresponding to the teacher data serving as the target pixel). A prediction tap is selected from the student data generated by filtering the image data with the LPF having the cutoff frequency corresponding to the parameter z, and supplied to the adding unit 95. Furthermore, in step S34, the tap selection unit 43 selects a class tap from the student data corresponding to the parameter z output from the parameter generation unit 81 and stored in the student data storage unit 35 for the target pixel, and the class classification unit. 44.

  In step S35, the class classification unit 44 classifies the pixel of interest based on the class tap for the pixel of interest, and outputs the class of the pixel of interest obtained as a result to the adding unit 95. Proceed to S36.

In step S <b> 35, the adding unit 95 reads the target pixel from the teacher data storage unit 33, uses the target pixel, the prediction tap supplied from the tap selection unit 42, and the parameter z output from the parameter generation unit 81. In step 20), the left side matrix component x _{i, K} t _{p xj, K} t _q and the right side vector component x _{i, K} t _p y _K are calculated. Further, the addition unit 95 applies the target pixel, the prediction tap, and the parameter z to the matrix component and the vector component that have already been obtained, corresponding to the class of the target pixel from the class classification unit 44. The matrix components x _{i, K} t _p x _{j, K} t _q obtained from the above are added to the vector components x _{i, K} t _p y _K , and the process proceeds to step S37.

  In step S37, the parameter generation unit 81 determines whether or not the parameter z output by itself is equal to Z which is the maximum value that can be taken. In step S36, when it is determined that the parameter z output from the parameter generation unit 81 is not equal to the maximum value Z (less than the maximum value Z), the process proceeds to step S38, and the parameter generation unit 81 sets the parameter z to 1 is added, and the added value is output as a new parameter z to the tap selection units 42 and 43 and the addition unit 95 of the learning unit 76 (FIG. 10). And it returns to step S34 and the same process is repeated hereafter.

  If it is determined in step S37 that the parameter z is equal to the maximum value Z, the process proceeds to step S39, where the pixel-of-interest selection unit 41 stores teacher data that is not yet a pixel of interest in the teacher-data storage unit 33. Determine whether or not. If it is determined in step S38 that teacher data that is not a pixel of interest is still stored in the teacher data storage unit 33, the process returns to step S32, and the same processing is repeated thereafter.

  If it is determined in step S39 that the teacher data that is not the pixel of interest is not stored in the teacher data storage unit 33, the adding unit 95 calculates the formula (20) for each class obtained by the processing so far. The matrix on the left side and the vector on the right side in) are supplied to the coefficient seed calculating unit 96, and the process proceeds to step S40.

In step S40, the coefficient seed calculation unit 96 solves each class by solving a normal equation for each class constituted by a matrix on the left side and a vector on the right side in the equation (20) for each class supplied from the addition unit 95. Each time, coefficient seed data β _{m, n} is obtained and output, and the process is terminated.

  In addition, due to the number of learning image data being insufficient, etc., there may occur a class in which the number of normal equations necessary for obtaining coefficient seed data cannot be obtained. The coefficient seed calculation unit 96 outputs default coefficient seed data, for example.

By the way, in the learning device 71 of FIG. 9, the high-quality image data as the learning image data is used as the teacher data, and the low-quality image data in which the spatial resolution of the high-quality image data is degraded in accordance with the parameter z. Is the coefficient seed data β _{m, n} that minimizes the sum of the square error of the predicted value y of the teacher data predicted from the tap coefficient w _n and the student data x _{n by} the linear linear expression of Expression (1). However, the learning of the coefficient seed data β _{m, n} can be performed as follows, for example.

That is, the high-resolution image data as learning image data is used as teacher data, and the high-resolution image data is filtered by an LPF having a cutoff frequency corresponding to the parameter z, thereby reducing the horizontal resolution and the vertical resolution. First, the square error of the predicted value y of the teacher data predicted by the linear primary prediction expression of Expression (1) using the tap image w _n and the student data x _n as the student data as the low-quality image data thus made. the tap coefficient w _n for the sum to a minimum (in this case, z = 0, 1, · · ·, Z) value of the parameter z seek each. Then, the tap coefficient w _n which is determined for each value of the parameter z as well as the teaching data, a parameter z as the learner data, a parameter z is the coefficient seed data beta _{m, n} and student data by equation (11) obtaining coefficient seed data beta _{m, n} that minimizes the sum of square errors of the prediction value of the tap coefficient w _n as a teacher data predicted from corresponding variable t _m.

Here, the tap coefficient w _n that minimizes the sum E of the squared errors of the predicted values y of the teacher data predicted by the linear primary prediction expression of Expression (1) is the case in the learning device 21 of FIG. Similarly to the above, by solving and solving the normal equation of the equation (8), each class can be obtained for each value of the parameter z (z = 0, 1,..., Z).

Incidentally, the tap coefficient is obtained from the coefficient seed data β _{m, n} and the variable t _m corresponding to the parameter z, as shown in the equation (11). Now, assuming that the tap coefficient obtained by this equation (11) is represented as w _n ′, the optimum tap coefficient w _n represented by the following equation (21) and equation (11) are obtained. coefficient seed data beta _{m, n} of the error e _n of the tap coefficient w _n 'and 0, but the optimum coefficient seed data for determining the optimal tap coefficient w _n, for all of the tap coefficients w _n, It is generally difficult to obtain such coefficient seed data β _{m, n} .

・・・（２１）

(21)

  Equation (21) can be transformed into the following equation by Equation (11).

・・・（２２）

(22)

Therefore, as a standard indicating that the coefficient seed data β _{m, n} is optimal, for example, if the least square method is adopted, the optimal coefficient seed data β _{m, n} is _expressed by the following equation. This can be obtained by minimizing the sum E of squared errors.

・・・（２３）

(23)

The minimum value (minimum value) of the sum E of square errors in equation (23) is β _{m, where} 0 is a partial differentiation of the sum E with coefficient seed data β _{m, n} , as shown in equation (24) _. given by _n .

・・・（２４）

... (24)

  By substituting equation (22) into equation (24), the following equation is obtained.

・・・（２５）

... (25)

Now, X _{i, j,} and Y _i are defined as shown in equations (26) and (27).

・・・（２６）

... (26)

・・・（２７）

... (27)

In this case, Expression (25) can be expressed by a normal equation shown in Expression (28) using X _{i, j} and Y _i .

・・・（２８）

... (28)

The normal equation of Expression (28) can also be solved for the coefficient seed data β _{m, n} by using, for example, a sweeping method (Gauss-Jordan elimination method).

Next, FIG. 12 shows a configuration example of the learning device 101 that performs learning for obtaining the coefficient seed data β _{m, n} by building and solving the normal equation of Expression (28).

  In the figure, portions corresponding to those in the learning device 21 of FIG. 3 or the learning device 71 of FIG. 9 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the learning device 101 is configured in the same manner as the learning device 71 in FIG. 9 except that a learning unit 106 is provided instead of the learning unit 76.

  FIG. 13 shows a configuration example of the learning unit 106 of FIG. In the figure, portions corresponding to those in the learning unit 36 of FIG. 4 or the learning unit 76 of FIG. 10 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  The adding unit 115 is supplied with the class of the pixel of interest output from the class classification unit 44 and the parameter z output from the parameter generation unit 81. Then, the adding unit 115 reads out the target pixel from the teacher data storage unit 33, and adds the target pixel and the student data constituting the prediction tap for the target pixel supplied from the tap selection unit 42 as targets. Is performed for each class supplied from the class classification unit 44 and for each value of the parameter z output from the parameter generation unit 81.

That is, the adding unit 115 includes the teacher data y _k stored in the teacher data storage unit 33 (FIG. 12), the prediction tap x _{n, k} output from the tap selection unit 42, the class output from the class classification unit 44, And the parameter z when the student data which comprises the prediction tap _{xn, k} which the parameter production | generation part 81 (FIG. 12) outputs is produced | generated is supplied.

Then, the adding unit 115 uses the prediction tap (student data) x _{n, k} for each class supplied from the class classification unit 44 and for each value of the parameter z output from the parameter generation unit 81, 8) Multiply (x _{n, k} x _{n ′, k} ) between student data in the left-side matrix and an operation corresponding to summation (Σ).

Furthermore, the addition unit 115 also uses the prediction tap (student data) x _{n, k} and the teacher data for each class supplied from the class classification unit 44 and for each value of the parameter z output from the parameter generation unit 81. with y _k, performed student data x _n in the vector of the right side of equation _{(8), k} and multiplication of teacher data _{y k (x n, k y} k) and, a calculation corresponding to summation (sigma).

That is, the adding unit 115 lastly calculates the left-side matrix component (Σx _{n, k} x _{n ′, k} ) and the right-side vector component ( Σx _{n, k} y _k ) is stored in its built-in memory (not shown), and the matrix component (Σx _{n, k} x _{n ′, k} ) or vector component (Σx _{n, k} y _k) ), The corresponding component x _{n, k + 1} x _n calculated using the teacher data y _{k + 1} and the student data x _{n, k + 1} for the teacher data newly set as the pixel of interest. _{', k + 1} or x _{n, k + 1} y _{k + 1} is added (addition represented by the summation of equation (8) is performed).

  Then, the adding unit 115 performs the above-described addition using all the teacher data stored in the teacher data storage unit 33 as the target pixel, thereby obtaining the equation (8) for each value of the parameter z for each class. When the normal equation shown in (2) is established, the normal equation is supplied to the tap coefficient calculation unit 46.

  Therefore, the adding unit 115 forms the normal equation of the equation (8) for each class, similarly to the adding unit 45 of FIG. However, the addition unit 115 is further different from the addition unit 45 of FIG. 4 in that a normal equation of Expression (8) is established for each value of the parameter z.

The tap coefficient calculation unit 46 solves a normal equation for each value of the parameter z for each class supplied from the adding unit 115, thereby obtaining an optimum tap coefficient w _n for each value of the parameter z for each class. Obtained and supplied to the add-in unit 121.

Adder 121 adds as a parameter generating unit 81 (variable t _m corresponding to) the parameter z supplied from (12), directed to the optimum tap coefficients w _n supplied from the tap coefficient calculating section 46 For each class.

That is, the adding unit 121 uses the variable t _i (t _j ) obtained by the equation (10) from the parameter z supplied from the parameter generating unit 81 (FIG. 12), and in the matrix on the left side of the equation (28), Multiplication (t _i t _j ) between variables t _i (t _j ) corresponding to the parameter z for _{obtaining the} component X _{i, j} defined by the equation (26), and an operation corresponding to summation (Σ) , For each class.

Here, since the component X _{i, j} is determined only by the parameter z and has no relation to the class _, the calculation of the component X _{i, j} does not actually need to be performed for each class, and is performed once. Just do it.

Further, the adding unit 121 uses the variable t _i obtained from the parameter z supplied from the parameter generating unit 81 by the equation (10) and the optimum tap coefficient w _n supplied from the tap coefficient calculating unit 46, Multiplication (t _i w _n ) of the variable t _i corresponding to the parameter z for _{obtaining the} component Y _i defined in the expression (27) and the optimum tap coefficient w _n in the vector on the right side of the expression (28); An operation corresponding to summation (Σ) is performed for each class.

The adding unit 121 obtains the component X _{i, j} represented by the equation (26) and the component Y _i represented by the equation (27) for each class, thereby obtaining the equation (28) for each class. The normal equation is supplied to the coefficient seed calculation unit 122.

The coefficient seed calculating unit 122 calculates and outputs coefficient seed data β _{m, n} for each class by solving the normal equation of the equation (28) for each class supplied from the adding unit 121.

The coefficient seed memory 62 in the coefficient output unit 55 of FIG. 8 may store the coefficient seed data β _{m, n} for each class obtained as described above.

  In the learning of the coefficient seed data, the student data corresponding to the first image data and the teacher data corresponding to the second image data are the same as in the tap coefficient learning described with reference to FIG. Depending on how the data is selected, the coefficient seed data can be obtained by performing various image conversion processes.

  That is, in the above-described case, the learning image data is directly used as teacher data corresponding to the second image data, and the low-quality image data in which the spatial resolution of the learning image data is deteriorated is the first image data. Since the coefficient seed data is learned as the student data corresponding to the image data, the first image data is converted into the second image data with improved spatial resolution as the coefficient seed data. What can perform image conversion processing as spatial resolution creation processing can be obtained.

  In this case, the image conversion apparatus 51 in FIG. 7 can improve the horizontal resolution and vertical resolution of the image data to a resolution corresponding to the parameter z.

  Further, for example, the high-quality image data is used as the teacher data, and the image data obtained by superimposing the noise corresponding to the parameter z on the high-quality image data as the teacher data is used as the student data. By performing learning, the coefficient seed data is obtained by performing image conversion processing as noise removal processing for converting the first image data into second image data from which the noise included therein is removed (reduced). Can be obtained. In this case, the image conversion apparatus 51 of FIG. 7 can obtain S / N image data corresponding to the parameter z.

  Further, for example, certain image data is used as teacher data, and image data obtained by thinning out the number of pixels of the image data as the teacher data in accordance with the parameter z is used as student data, or image data of a predetermined size is used. The coefficient seed data is obtained by performing the learning of the coefficient seed data by using the image data obtained by thinning out the pixels of the image data as the student data at the thinning rate corresponding to the parameter z as the teacher data. It is possible to obtain one that performs image conversion processing as resizing processing for converting one image data into second image data whose size is enlarged or reduced. In this case, the image conversion apparatus 51 in FIG. 7 can obtain image data enlarged or reduced to a size corresponding to the parameter z.

Incidentally, in the above case, defines the tap coefficient w _n, in as shown in Equation _{(9), β 1, n} z 0 + β 2, n z 1 + ··· + β M, n z M-1 and, by the equation (9), the spatial resolution in the horizontal and vertical directions, both have been to obtain the tap coefficient w _n for improving corresponding to the parameter z, the tap coefficient w _n is horizontal It is also possible to obtain a resolution and a vertical resolution that are independently improved corresponding to the independent parameters z _x and z _y .

That is, the tap coefficients w _n, instead of the equation (9), for example, cubic polynomial ^{_{β 1, n z x 0 z}} y 0 + β 2, n z x 1 z y 0 + β 3, n z x 2 z y ⁰ + β _{4, n} z _x ³ z _y ⁰ + β _{5, n} z _x ⁰ z _y ¹ + β _{6, n} z _x ⁰ z _y ² + β _{7, n} z _x ⁰ z _y ³ + β _{8, n} z _x ¹ z _y with defined ^{_{1 + β 9, n z x}} 2 z y 1 + β 10, n z x 1 z y 2, the variable t _m defined in formula (10), in place of the equation (10), for example, t ₁ = Z _x ⁰ z _y ⁰ , t ₂ = z _x ¹ z _y ⁰ , t ₃ = z _x ² z _y ⁰ , t ₄ = z _x ³ z _y ⁰ , t ₅ = z _x ⁰ z _y ¹ , t ₆ = Z _x ⁰ z _y ² , t ₇ = z _x ⁰ z _y ³ , t ₈ = z _x ¹ z _y ¹ , t ₉ = z _x ² z _y ¹ , t ₁₀ = z _x ¹ z _y ² . Again, the tap coefficient w _n is finally can be represented by the formula (11), therefore, and the learning device 71 in FIG. 9, in the learning apparatus 101 of FIG. 12, the parameter z _x and z _y Correspondingly, the horizontal resolution and the vertical resolution are obtained by performing learning using the image data obtained by degrading the horizontal resolution and the vertical resolution of the teacher data as student data, and obtaining the coefficient seed data β _{m, n} . in response to independent parameters z _x and z _y, it can be determined tap coefficients w _n to improve independently.

In addition, for example, in addition to the parameters z _x and z _y corresponding to the horizontal resolution and the vertical resolution, respectively, by introducing a parameter z _t corresponding to the resolution in the time direction, the horizontal resolution, the vertical resolution, and the time resolution are independent parameters z _x, z _y, corresponding to the z _t, it is possible to determine the tap coefficient w _n to improve independently.

As for the resizing process, as in the spatial resolution creation processing, the horizontal and vertical directions, other tap coefficients w _n to resize magnification of both corresponding to the parameter z (or reduction ratio), the horizontal and vertical It is possible to obtain a tap coefficient w _n whose size is independently resized with an enlargement factor corresponding to the parameters z _x and z _y , respectively.

Furthermore, and the learning device 71 in FIG. 9, in the learning apparatus 101 of FIG. 12, with deteriorating the horizontal resolution and vertical resolution of the teacher data corresponding to the parameter z _x, the noise to the teacher data corresponding to the parameter z _y the additional image data by performing learning by using as the student data, by obtaining the coefficient seed data beta _{m, n,} improves the horizontal resolution and vertical resolution corresponding to the parameter z _x, corresponding to the parameter z _y it can be obtained tap coefficients w _n to perform noise removal and.

  Next, a first embodiment of the present invention will be described.

  FIG. 14 shows an example of the configuration of an AV system as the first embodiment of the present invention.

  In FIG. 14, the AV system is composed of a main body 200 and a remote controller (remote commander) 201.

  The main body 200 includes a control device 202, a DVD (Digital Versatile Disc) recorder unit 203, an HDD (Hard Disk Drive) recorder unit 204, and a television receiver unit 205.

  The remote controller 201 includes an operation unit 201A and a transmission unit 201B.

  The operation unit 201 </ b> A is operated by the user when inputting various commands to the main body 200. The transmission unit 201B emits an operation signal corresponding to the operation of the operation unit 201A, for example, by infrared rays. In addition, the transmission unit 201B can also transmit the operation signal by radio according to, for example, the Bluetooth (trademark) standard.

  The control device 202 includes a transmission / reception unit 202A and a control unit 202B.

  The transmission / reception unit 202A receives (receives) an operation signal transmitted from the remote controller 201 by infrared rays and supplies the operation signal to the control unit 202B. The transmission / reception unit 202A transmits (broadcasts) a command string, which is a sequence of one or more commands supplied from the control unit 202B, wirelessly or by wire. Here, it is assumed that the transmission / reception unit 202A transmits the command string wirelessly.

  The control unit 202B generates a command sequence including one or more commands for the DVD recorder unit 203, the HDD recorder unit 204, or the television receiver unit 205 in response to the operation signal supplied from the transmission / reception unit 202A, and transmits / receives To the unit 202A.

  The DVD recorder unit 203 includes a receiving unit 211, an R-IC (Reconfigurable-IC) 212, a recording unit 213, a DVD 214, and a playback unit 215, and includes image data of a television broadcast program received by a tuner (not shown) The image data supplied from the outside, such as image data input via an external terminal (not shown), is subjected to signal processing as necessary and supplied to the HDD recorder unit 204 at the subsequent stage or recorded on the DVD 214. Also, the DVD recorder unit 203 reproduces image data from the DVD 214, performs signal processing as necessary, and supplies it to the HDD recorder unit 204 at the subsequent stage.

  That is, in the DVD recorder unit 203, the reception unit 211 receives a command sequence transmitted wirelessly from the transmission / reception unit 202 </ b> A of the control device 202 and supplies it to the R-IC 212.

  The R-IC 212 is a one-chip IC whose internal structure is reconfigurable (reconfigurable), and in accordance with at least one command in the command sequence supplied from the receiving unit 211. The structure is switched, and image data supplied from the outside or image data supplied from the playback unit 215 is subjected to signal processing (data processing) as necessary, and supplied to the HDD recorder unit 204 or the recording unit 213.

  Note that the entire receiving unit 211 and the R-IC 212 can be configured as a one-chip IC. The same applies to the entire receiving unit 221 and the R-IC 222 described later, and the entire receiving unit 231 and the R-IC 232.

  The recording unit 213 performs necessary processing such as MPEG encoding on the image data supplied from the R-IC 212 to convert the image data into recording data that conforms to the DVD standard, and stores it on the DVD 214. Record. The DVD 214 can be easily attached to and detached from the DVD recorder unit 203.

  The playback unit 215 reads the recorded data from the DVD 214, decodes it into image data (plays back the image data), and supplies it to the R-IC 212.

  The HDD recorder unit 204 includes a receiving unit 221, an R-IC 222, a recording unit 223, an HD 224, and a playback unit 225, and performs signal processing on image data supplied from the previous DVD recorder unit 203 as necessary. And supplied to the television receiver 205 at the subsequent stage or recorded on the HD 224. Also, the HDD recorder unit 204 reproduces image data from the HD 224, performs signal processing as necessary, and supplies it to the television receiver unit 205 at the subsequent stage.

  That is, in the HDD recorder unit 204, the reception unit 221 receives a command string transmitted from the transmission / reception unit 202 </ b> A of the control device 202 wirelessly and supplies it to the R-IC 222.

  Similar to the R-IC 212, the R-IC 222 is a one-chip IC whose internal structure is reconfigurable, and the internal structure is changed according to at least one command in the command sequence supplied from the receiving unit 221. The image data supplied from the DVD recorder unit 203 at the previous stage or the image data supplied from the playback unit 225 is subjected to signal processing as necessary, and supplied to the television receiver unit 205 or the recording unit 223.

  The recording unit 223 performs necessary processing on the image data supplied from the R-IC 222 and records it in the HD 224.

  The playback unit 225 reads (plays back) the recording data from the HD 224 and supplies it to the R-IC 222.

  The television receiver unit 205 includes a receiving unit 231, an R-IC 232, and a display 233. The television receiver unit 205 performs signal processing on the image data supplied from the preceding HD recorder unit 204 and supplies the image data to the display 233. To display.

  That is, in the television receiver unit 205, the receiving unit 231 receives a command sequence transmitted wirelessly from the transmission / reception unit 202 </ b> A of the control device 202 and supplies it to the R-IC 232.

  Similar to the R-IC 212, the R-IC 232 is a one-chip IC whose internal structure is reconfigurable, and the internal structure is changed according to at least one command in the command sequence supplied from the receiving unit 231. The image data supplied from the HD recorder unit 204 at the previous stage is subjected to signal processing as necessary and supplied to the display 233.

  The display 233 displays the image data supplied from the R-IC 232.

  In FIG. 14, the DVD recorder unit 203, the HDD recorder unit 204, and the television receiver unit 205 are assumed to be housed in one casing as the main body 200, but the DVD recorder unit 203, HDD recorder The unit 204 and the television receiver unit 205 may be independent devices housed in a single casing. In this case, the DVD recorder unit 203, the HDD recorder unit 204, and the television receiver unit 205 can exchange necessary data (signals) in a wired or wireless manner.

  In FIG. 14, the control device 202 generates a command sequence corresponding to the operation signal from the remote controller 201 and transmits it to the DVD recorder unit 203, HDD recorder unit 204, and television receiver unit 205 wirelessly. However, for example, the remote controller 201 generates a command sequence corresponding to the operation signal and transmits it to the DVD recorder unit 203, the HDD recorder unit 204, and the television receiver unit 205 wirelessly. Is also possible.

  Next, the operation of the main body 200 of the AV system shown in FIG. 14 will be described with reference to FIGS.

  When the user operates the operation unit 201A of the remote controller 201, the transmission unit 201B transmits an operation signal corresponding to the operation.

  And in the control apparatus 202, the transmission / reception part 202A receives the operation signal from the transmission part 201B of the remote control 201, and supplies it to the control part 202B.

  The control unit 202B receives the operation signal from the transmission / reception unit 202A, and determines in step S201 whether or not the operation signal instructs recording of the image data from the outside onto the DVD 214.

  If it is determined in step S201 that the operation signal is for instructing recording of image data from the outside onto the DVD 214, the process proceeds to step S202, and the control unit 202B displays a command sequence corresponding to the operation signal. Generate.

  Here, FIG. 16 shows an example of a command string generated by the control unit 202B in step S202 of FIG.

  The command sequence in FIG. 16 is composed of one noise removal processing command and two Null commands.

  The first (first) noise removal processing command in the command sequence of FIG. 16 requests the DVD recorder unit 203 (R-IC 212) to perform noise removal processing to remove (reduce) noise as signal processing. To do. The second Null command from the head requests the HDD recorder unit 204 (the R-IC 222) to do nothing as signal processing. Further, the third Null command from the head requests the television receiver unit 205 (R-IC 232) to do nothing as signal processing.

  Note that the Null command is not necessary, and in this case, the command sequence in FIG. 16 is composed of one noise removal processing command for the DVD recorder unit 203.

  Returning to FIG. 15, when the control unit 202B generates a command sequence in step S202, the control unit 202B supplies the command sequence to the transmission / reception unit 202A, and the process proceeds to step S203. In step S203, the transmission / reception unit 202A transmits (broadcasts) the command string from the control unit 202B, and the process proceeds to step S204.

  In step S204, the reception unit 211 of the DVD recorder unit 203, the reception unit 221 of the HDD recorder unit 204, and the reception unit 231 of the television receiver unit 205 receive the command sequence transmitted by the transmission / reception unit 202A in step S203. Further, in step S204, the receiving units 211, 221, and 231 supply the command strings received from the transmitting / receiving unit 202A to the R-ICs 212, 222, and 232, respectively, and the process proceeds to step S205.

  In step S205, each of the R-ICs 212, 222, and 232 switches its internal structure in accordance with, for example, a command for itself among the command sequences supplied from the reception units 211, 211, and 231 and proceeds to step S206. .

  That is, in this case, the R-IC 212 of the DVD recorder unit 203 switches its internal structure to perform noise removal processing in accordance with the head noise removal processing command in the command sequence of FIG. Also, the R-IC 221 of the HDD recorder unit 204 and the R-IC 231 of the television receiver unit 205 do not perform signal processing in response to the second and third Null commands in the command sequence of FIG. , Switching its internal structure.

  In step S206, each of the R-ICs 212, 222, and 232 performs signal processing to be performed by the internal structure switched in step S205, and ends the processing.

  That is, in this case, the R-IC 212 performs noise removal processing on the image data input from the outside of the DVD recorder unit 203 and supplies it to the recording unit 213. The recording unit 213 performs MPEG encoding on the image data from the R-IC 212 and records it on the DVD 214. That is, the DVD 214 records image data obtained by the R-IC 212 performing noise removal processing on image data input from the outside.

  Accordingly, in this case, only the R-IC 212 that performs signal processing among the R-ICs 212, 222, and 232 functions as a device that performs noise removal processing.

  On the other hand, if the control unit 202B determines in step S201 that the operation signal does not command recording of external image data onto the DVD 214, the control unit 202B proceeds to step S211 in FIG. It is determined whether or not it is an instruction to reproduce the recorded image data.

  If it is determined in step S211 that the operation signal is an instruction to reproduce the image data recorded on the DVD 214, the process proceeds to step S212, and the control unit 202B generates a command sequence corresponding to the operation signal. .

  Here, FIG. 18 shows an example of a command string generated by the control unit 202B in step S212 of FIG.

  The command sequence in FIG. 18 includes one distortion removal processing command, one temporal resolution creation processing command, and one spatial resolution creation processing command.

  The leading distortion removal processing command in the command sequence of FIG. 18 requests the R-IC 212 of the DVD recorder unit 203 to perform distortion removal processing for removing block distortion or the like caused by MPEG encoding / decoding as signal processing. To do. The second time resolution creation processing command from the head requests the R-IC 222 of the HDD recorder unit 204 to perform time resolution creation processing for improving the time resolution of image data as signal processing. Furthermore, the third spatial resolution creation processing command from the head requests the R-IC 232 of the television receiver unit 205 to perform spatial resolution creation processing for improving the spatial resolution of image data as signal processing.

  Returning to FIG. 17, when the control unit 202B generates a command string in step S212, the command string is supplied to the transmission / reception unit 202A, and the process proceeds to step S213. In step S213, the transmission / reception unit 202A transmits (broadcasts) the command string from the control unit 202B, and the process proceeds to step S214.

  In step S214, the reception unit 211 of the DVD recorder unit 203, the reception unit 221 of the HDD recorder unit 204, and the reception unit 231 of the television receiver unit 205 receive the command sequence transmitted by the transmission / reception unit 202A in step S213. Furthermore, in step S214, each of the reception units 211, 211, and 231 supplies the command strings received from the transmission / reception unit 202A to the R-ICs 212, 222, and 232, respectively, and the process proceeds to step S215.

  In step S215, each of the R-ICs 212, 222, and 232 switches the internal structure in accordance with, for example, a command for the command sequence supplied from the reception units 211, 211, and 231 and proceeds to step S216. .

  That is, in this case, the R-IC 212 of the DVD recorder unit 203 switches its internal structure so as to perform distortion removal processing in accordance with the first distortion removal processing command in the command sequence of FIG. Further, the R-IC 221 of the HDD recorder unit 204 switches its internal structure so as to perform the time resolution creation processing in accordance with the second time resolution creation processing command from the top of the command sequence of FIG. Further, the R-IC 231 of the television receiver unit 205 switches its internal structure so as to perform spatial resolution creation processing in accordance with the third spatial resolution creation processing command from the top of the command sequence of FIG. .

  In step S216, each of the R-ICs 212, 222, and 232 performs signal processing that should be performed by the internal structure switched in step S215, and ends the processing.

In other words, in this case, in the DVD recorder unit 203, the playback unit 215 reads out image data from the DVD 214, performs MPEG decoding, and supplies it to the R-IC 212. R-IC 212, in step S216 _1, image data supplied from the reproducing unit 215, i.e., subjected to the image data block distortion or the like occurs, the distortion removal processing by the MPEG coding / decoding, HDD recorder 204 To the output.

In the HDD recorder unit 204, image data from the R-IC 212 of the DVD recorder unit 203 is supplied to the R-IC 222. R-IC 222, in step S216 _2, the image data from the R-IC 212, subjected to temporal resolution creation processing, and outputs to the television receiver 205.

In the television receiver unit 205, image data from the R-IC 222 of the HDD recorder unit 204 is supplied to the R-IC 232. R-IC 232, in step S216 _3, the image data from the R-IC 222, subjected to spatial resolution creation processing, and supplies the display 233.

  Therefore, in this case, the R-IC 212 is a device (IC) that performs distortion removal processing, the R-IC 222 is a device that performs time resolution creation processing, and the R-IC 232 is a device that performs spatial resolution creation processing. Each will work.

  As described above, the distortion removal processing is performed by the R-IC 212 for the image data reproduced from the DVD 214, and the time resolution creation processing is performed by the R-IC 222 for the processing result. By performing the spatial resolution creation process with the R-IC 232 for the result, block distortion and the like are removed (reduced) on the display 233, and an image with high temporal resolution and spatial resolution is displayed.

  Here, for example, assuming that image data of a movie is recorded on the DVD 214, so-called 2-3 pull-down is performed when converting a movie into image data of the television system. The image data is degraded in time resolution due to the 2-3 pulldown.

  Therefore, the R-IC 222 can recover the time resolution of the image data degraded by 2-3 pulldown (reducing the degree of degradation of the time resolution) by performing the time resolution creation process as described above.

  On the other hand, if the control unit 202B determines in step S211 that the operation signal does not instruct the reproduction of the image data recorded on the DVD 214, the control unit 202B proceeds to step S221 in FIG. It is determined whether or not data is to be recorded on the HD 224.

  If it is determined in step S221 that the operation signal is a command to record external image data to the HD 224, the process proceeds to step S222, and the control unit 202B displays a command sequence corresponding to the operation signal. Generate.

  Here, FIG. 20 shows an example of a command string generated by the control unit 202B in step S222 of FIG.

  The command sequence in FIG. 20 includes one noise removal processing command, one spatial resolution creation processing command, and one Null command.

  The head distortion removal processing command in the command sequence in FIG. 20 requests the R-IC 212 of the DVD recorder unit 203 to perform noise removal processing as signal processing. The second spatial resolution creation processing command from the head requests the R-IC 222 of the HDD recorder unit 204 to perform spatial resolution creation processing as signal processing. Further, the third Null command from the head requests that the R-IC 232 of the television receiver unit 205 do nothing as signal processing.

  Returning to FIG. 19, when the control unit 202B generates a command sequence in step S222, the control unit 202B supplies the command sequence to the transmission / reception unit 202A, and proceeds to step S223. In step S223, the transmission / reception unit 202A transmits (broadcasts) the command string from the control unit 202B, and the process proceeds to step S224.

  In step S224, the reception unit 211 of the DVD recorder unit 203, the reception unit 221 of the HDD recorder unit 204, and the reception unit 231 of the television receiver unit 205 receive the command sequence transmitted by the transmission / reception unit 202A in step S223. Further, in step S224, each of the reception units 211, 211, and 231 supplies the command strings received from the transmission / reception unit 202A to the R-ICs 212, 222, and 232, respectively, and the process proceeds to step S225.

  In step S225, each of the R-ICs 212, 222, 232 switches its internal structure in accordance with, for example, a command for itself in the command sequence supplied from the receiving units 211, 211, 231 and proceeds to step S226. .

  That is, in this case, the R-IC 212 of the DVD recorder unit 203 switches its internal structure so as to perform noise removal processing according to the head noise removal processing command in the command sequence of FIG. Further, the R-IC 221 of the HDD recorder unit 204 switches its internal structure so as to perform the spatial resolution creation processing in accordance with the second spatial resolution creation processing command from the top of the command sequence of FIG. Furthermore, the R-IC 231 of the television receiver unit 205 switches its internal structure so as not to perform signal processing in response to the third Null command from the top in the command sequence of FIG.

  In step S226, each of the R-ICs 212, 222, and 232 performs signal processing to be performed by the internal structure switched in step S225, and ends the processing.

  That is, in this case, in the DVD recorder unit 203, the R-IC 212 performs noise removal processing on the image data input from the outside of the DVD recorder unit 203 and outputs it to the HDD recorder 204.

  In the HDD recorder unit 204, image data from the R-IC 212 of the DVD recorder unit 203 is supplied to the R-IC 222. The R-IC 222 performs spatial resolution creation processing on the image data from the R-IC 212 and supplies the image data to the recording unit 223. The recording unit 223 performs MPEG encoding on the image data from the R-IC 222 and records it in the HD 224. That is, the HD 224 records image data obtained by the R-IC 212 performing noise removal processing on the image data input from the outside, and then the R-IC 222 performing spatial resolution creation processing thereafter. The

  Therefore, in this case, the R-IC 212 functions as a device that performs distortion removal processing, and the R-IC 222 functions as a device that performs spatial resolution creation processing.

  On the other hand, if the control unit 202B determines in step S221 that the operation signal does not command recording of image data from the outside to the HD 224, the control unit 202B proceeds to step S231 in FIG. It is determined whether or not it is an instruction to reproduce the recorded image data.

  If it is determined in step S231 that the operation signal is an instruction to reproduce the image data recorded in the HD 224, the process proceeds to step S232, and the control unit 202B generates a command sequence corresponding to the operation signal. .

  Here, FIG. 22 shows an example of a command string generated by the control unit 202B in step S232 of FIG.

  The command sequence in FIG. 22 is composed of one Null command, one distortion removal processing command, and one spatial resolution creation processing command.

  The leading Null command in the command sequence in FIG. 22 requests the R-IC 212 of the DVD recorder unit 203 to do nothing as signal processing. Further, the second distortion removal processing command from the head requests the R-IC 222 of the HDD recorder unit 204 to perform distortion removal processing as signal processing. Further, the third spatial resolution creation processing command from the top requests the R-IC 232 of the television receiver unit 205 to perform spatial resolution creation processing as signal processing.

  Returning to FIG. 21, when the control unit 202B generates a command string in step S232, the control unit 202B supplies the command string to the transmission / reception unit 202A, and the process proceeds to step S233. In step S233, the transmission / reception unit 202A transmits (broadcasts) the command string from the control unit 202B, and the process proceeds to step S234.

  In step S234, the reception unit 211 of the DVD recorder unit 203, the reception unit 221 of the HDD recorder unit 204, and the reception unit 231 of the television receiver unit 205 receive the command sequence transmitted by the transmission / reception unit 202A in step S233. Furthermore, in step S234, each of the reception units 211, 211, and 231 supplies the command strings received from the transmission / reception unit 202A to the R-ICs 212, 222, and 232, respectively, and the process proceeds to step S235.

  In step S235, each of the R-ICs 212, 222, and 232 switches its internal structure in accordance with, for example, a command for itself among the command sequences supplied from the reception units 211, 211, and 231 and proceeds to step S236. .

  That is, in this case, the R-IC 212 of the DVD recorder unit 203 switches its internal structure so as not to perform signal processing in response to the first Null command in the command sequence of FIG. Further, the R-IC 221 of the HDD recorder unit 204 switches its internal structure so as to perform the distortion removal processing in accordance with the second distortion removal processing command from the top in the command sequence of FIG. Furthermore, the R-IC 231 of the television receiver unit 205 switches its internal structure so as to perform spatial resolution creation processing in accordance with the third spatial resolution creation processing command from the top of the command sequence of FIG. .

  In step S236, each of the R-ICs 212, 222, and 232 performs signal processing to be performed by the internal structure switched in step S235, and ends the processing.

  That is, in this case, in the HDD recorder unit 204, the playback unit 225 reads out image data from the HD 224, performs MPEG decoding, and supplies the image data to the R-IC 222. The R-IC 222 performs distortion removal processing on the image data supplied from the reproduction unit 225, that is, image data in which block distortion or the like is generated by MPEG encoding / decoding, and outputs the processed image data to the television receiver unit 205. To do.

  In the television receiver unit 205, image data from the R-IC 222 of the HDD recorder unit 204 is supplied to the R-IC 232. The R-IC 232 performs spatial resolution creation processing on the image data from the R-IC 222 and supplies the image data to the display 233.

  Therefore, in this case, the R-IC 222 functions as a device that performs distortion removal processing, and the R-IC 232 functions as a device that performs spatial resolution creation processing.

  As described above, the distortion removal processing is performed by the R-IC 222 for the image data reproduced from the HD 224, and the spatial resolution creation processing is performed by the R-IC 232 for the processing result. In this case, the block distortion and the like are removed (reduced), and an image with a high spatial resolution is displayed.

  On the other hand, when the control unit 202B determines in step S231 that the operation signal does not instruct reproduction of the image data recorded in the HDD 224, the control unit 202B proceeds to step S241 in FIG. It is determined whether or not it is a command for enlarging (zooming) data.

  If it is determined in step S241 that the operation signal is an instruction to enlarge the image data from the outside, the process proceeds to step S242, and the control unit 202B generates a command string corresponding to the operation signal.

  Here, FIG. 24 shows an example of a command string generated by the control unit 202B in step S242 of FIG.

  The command sequence in FIG. 24 is composed of three spatial resolution creation processing commands.

  24, the first, second and third spatial resolution creation processing commands are the R-IC 212 of the DVD recorder unit 203, the R-IC 222 of the HDD recorder unit 204, and the R of the television receiver unit 205. -The IC 232 requests spatial resolution creation processing as signal processing.

  Returning to FIG. 23, when the control unit 202B generates a command sequence in step S242, the control unit 202B supplies the command sequence to the transmission / reception unit 202A, and proceeds to step S243. In step S243, the transmission / reception unit 202A transmits (broadcasts) the command string from the control unit 202B, and the process proceeds to step S244.

  In step S244, the reception unit 211 of the DVD recorder unit 203, the reception unit 221 of the HDD recorder unit 204, and the reception unit 231 of the television receiver unit 205 receive the command sequence transmitted by the transmission / reception unit 202A in step S243. Further, in step S244, each of the reception units 211, 211, and 231 supplies the command strings received from the transmission / reception unit 202A to the R-ICs 212, 222, and 232, respectively, and the process proceeds to step S245.

  In step S245, each of the R-ICs 212, 222, and 232 switches its internal structure in accordance with, for example, a command for itself in the command sequence supplied from the receiving units 211, 211, and 231 and proceeds to step S246. .

  That is, in this case, the R-IC 212 of the DVD recorder unit 203 switches its internal structure so as to perform spatial resolution creation processing in accordance with the top spatial resolution creation processing command in the command sequence of FIG. Similarly, the R-IC 221 of the HDD recorder unit 204 and the R-IC 231 of the television receiver unit 205 also correspond to the second and third spatial resolution creation processing commands from the top in the command sequence of FIG. Each internal structure is switched to perform the spatial resolution creation process.

  In step S246, each of the R-ICs 212, 222, and 232 performs signal processing that should be performed by the internal structure switched in step S245, and ends the processing.

  That is, in this case, in the DVD recorder unit 203, the R-IC 212 performs spatial resolution creation processing on the image data input from the outside of the DVD recorder unit 203 and outputs the image data to the HDD recorder 204.

  In the HDD recorder 204, image data from the R-IC 212 of the DVD recorder unit 203 is supplied to the R-IC 222. The R-IC 222 performs spatial resolution creation processing on the image data from the R-IC 212 and outputs the processed image data to the television receiver unit 205.

  In the television receiver unit 205, image data from the R-IC 222 of the HDD recorder unit 204 is supplied to the R-IC 232. The R-IC 232 performs spatial resolution creation processing on the image data from the R-IC 222 and supplies the image data to the display 233.

  Therefore, in this case, each of the R-ICs 212, 222, and 232 functions as a device that performs a spatial resolution creation process.

  Now, the image conversion process as the spatial resolution creation process performed in each of the R-ICs 212, 222, and 232 converts the first image data into the second image data having a larger number of pixels than the first image data. In other words, if the spatial resolution creation processing performed in each of the R-ICs 212, 222, and 232 is also resizing processing, the R-IC 212 is also resizing processing for image data from the outside. Spatial resolution creation processing is performed. The R-IC 222 performs the spatial resolution creation process, which is also the resizing process, for the result of the spatial resolution creation process. Further, the R-IC 232 targets the result of the spatial resolution creation process. The spatial resolution creation process, which is also the resizing process, is performed.

  As a result, an image obtained by enlarging image data from the outside is displayed on the display 233.

  On the other hand, when the control unit 202B determines in step S241 that the operation signal does not instruct the enlargement display of the image data from the outside, the control unit 202B proceeds to step S251 in FIG. It is determined whether or not it is a command to display in slow motion.

  If it is determined in step S251 that the operation signal is for instructing to display image data from the outside in slow motion, the process proceeds to step S252, and the control unit 202B displays a command sequence corresponding to the operation signal. Generate.

  Here, FIG. 26 shows an example of a command string generated by the control unit 202B in step S252 of FIG.

  The command sequence in FIG. 26 is composed of three time resolution creation processing commands.

  26, the first, second and third temporal resolution creation processing commands are the R-IC 212 of the DVD recorder unit 203, the R-IC 222 of the HDD recorder unit 204, and the R of the television receiver unit 205. The IC 232 requests time resolution creation processing as signal processing.

  Returning to FIG. 25, when the control unit 202B generates a command sequence in step S252, the command sequence is supplied to the transmission / reception unit 202A, and the process proceeds to step S253. In step S253, the transmission / reception unit 202A transmits (broadcasts) the command string from the control unit 202B, and the process proceeds to step S254.

  In step S254, the reception unit 211 of the DVD recorder unit 203, the reception unit 221 of the HDD recorder unit 204, and the reception unit 231 of the television receiver unit 205 receive the command sequence transmitted by the transmission / reception unit 202A in step S253. Further, in step S254, each of the reception units 211, 221, and 231 supplies the command strings received from the transmission / reception unit 202A to the R-ICs 212, 222, and 232, respectively, and the process proceeds to step S255.

  In step S255, each of the R-ICs 212, 222, and 232 switches its internal structure in accordance with, for example, a command for itself in the command sequence supplied from the receiving units 211, 221, and 231 and the process proceeds to step S256. .

  That is, in this case, the R-IC 212 of the DVD recorder unit 203 switches its internal structure so as to perform the time resolution creation process in accordance with the head time resolution creation process command in the command sequence of FIG. Similarly, the R-IC 221 of the HDD recorder unit 204 and the R-IC 231 of the television receiver unit 205 also correspond to the second and third temporal resolution creation processing commands from the top in the command sequence of FIG. Each internal structure is switched to perform the time resolution creation process.

  In step S256, each of the R-ICs 212, 222, and 232 performs signal processing to be performed by the internal structure switched in step S255, and ends the processing.

  That is, in this case, in the DVD recorder unit 203, the R-IC 212 performs time resolution creation processing on the image data input from the outside of the DVD recorder unit 203 and outputs it to the HDD recorder 204.

  In the HDD recorder 204, image data from the R-IC 212 of the DVD recorder unit 203 is supplied to the R-IC 222. The R-IC 222 performs time resolution creation processing on the image data from the R-IC 212 and outputs the image data to the television receiver unit 205.

  In the television receiver unit 205, image data from the R-IC 222 of the HDD recorder unit 204 is supplied to the R-IC 232. The R-IC 232 performs time resolution creation processing on the image data from the R-IC 222 and supplies it to the display 233.

  Therefore, in this case, each of the R-ICs 212, 222, and 232 functions as a device that performs time resolution creation processing.

  Now, the image conversion process as the time resolution creation process performed by each of the R-ICs 212, 222, and 232 is the second image having the number of frames (or fields) larger than that of the first image data. Assuming that the image data is also converted, the R-IC 212 performs time resolution creation processing on image data from the outside, and obtains image data with an increased number of frames. Then, the R-IC 222 performs the time resolution creation process on the image data obtained by the R-IC 212 with the increased number of frames, and obtains image data with the number of frames further increased. Thereafter, the R-IC 232 also performs time resolution creation processing on the image data with the increased number of frames obtained by the R-IC 222 to obtain image data with an increased number of frames.

  As described above, the image data with the increased number of frames is supplied to the display 233 and displayed at the same frame (field) rate as when the image data from the outside is displayed. Will be displayed.

  Note that the processing of each step described in the flowcharts of FIGS. 15, 17, 19, 21, 23, and 25 is processing performed by the R-ICs 212, 222, and 232, which are hardware. . However, the processing in steps S201 through S203 in FIG. 15, the processing in steps S211 through S213 in FIG. 17, the processing in steps S221 through S223 in FIG. 19, the processing in steps S231 through S233 in FIG. 21, and the processing in steps S241 through S243 in FIG. The processing and the processing of steps S251 to S253 in FIG. 25 can also be performed by causing a computer such as a microcomputer to execute a program.

  Next, FIG. 27 shows a configuration example of the R-IC 212 of FIG. The other R-ICs 222 and 232 are configured similarly.

  The R-IC 212 switches its internal structure in accordance with commands constituting the command sequence supplied from the receiving unit 211 (FIG. 14), and performs various signal processing using the above-described class classification adaptive processing.

  That is, the R-IC 212 includes a target pixel selection unit 251, tap selection units 252 and 253, a class classification unit 254, a coefficient output unit 255, and a prediction calculation unit 256. These pixel-of-interest selection unit 251, tap selection units 252, 253, class classification unit 254, coefficient output unit 255, and prediction calculation unit 256 are the pixel-of-interest selection unit 11, tap selection units 12, 13, class classification unit of FIG. 14, the coefficient output unit 15, and the prediction calculation unit 16.

  Therefore, the R-IC 212 converts the first image data supplied thereto into the second image data and outputs the second image data.

  The coefficient output unit 255 is supplied with commands for the R-IC 212 (DVD recorder unit 203) in the command sequence received by the receiving unit 211 (FIG. 14).

  FIG. 28 shows a configuration example of the coefficient output unit 255 of FIG.

In FIG. 28, the coefficient output unit 255 includes coefficient memories 261 ₁ , 261 ₂ , 261 ₃ , and 261 ₄ , and a selection unit 262.

In the coefficient memories 261 ₁ , 261 ₂ , 261 ₃ , and 261 ₄ , tap coefficients for noise removal processing, tap coefficients for distortion removal processing, tap coefficients for time resolution creation processing, which are obtained by learning performed in advance, Tap coefficients for spatial resolution creation processing are stored.

The coefficient memories 261 _{1 to} 261 ₄ are supplied with classes (class codes) output from the class classification unit 254 in FIG. Each of the coefficient memories 261 _{1 to} 261 ₄ reads out the tap coefficient of the class from the class classification unit 254 and outputs it to the selection unit 262.

As described above, the selection unit 262 is supplied with the tap coefficient read from each of the coefficient memories 261 _{1 to} 261 ₄ , and R of the command sequence received by the reception unit 211 (FIG. 14). -A command for IC 212 is supplied. The selection unit 262 selects any one of the output ends of the coefficient memories 261 _{1 to} 261 ₄ according to the command supplied from the reception unit 211, and the selected output end is used as the prediction calculation unit 256 (FIG. 27). ), The internal structure of the R-IC 212 is switched.

Here, when the selection unit 262 selects, for example, the output end of the coefficient memory 261 ₁ among the coefficient memories 261 _{1 to} 261 ₄ , and connects the selected output end to the input end of the prediction calculation unit 256 , the prediction computation unit 256, it is possible to tap coefficients for noise removal processing which is read out from the coefficient memory 261 ₁ is supplied, as a result, R-IC 212 will serve as a device for performing a noise removal process.

Similarly, the selection unit 262 selects the output ends of the coefficient memories 261 ₂ , 261 ₃ , and 261 ₄ among the coefficient memories 261 _{1 to} 261 ₄ , and uses the selected output ends as the input ends of the prediction calculation unit 256. , The prediction calculation unit 256 has a distortion removal tap coefficient read from the coefficient memory 261 ₂ , 261 ₃ , 261 ₄ , a temporal resolution creation process tap coefficient, and a spatial resolution creation process tap coefficient. Are supplied respectively. As a result, when the selection unit 262 selects the output terminals of the coefficient memories 261 ₂ , 261 ₃ , and 261 ₄ among the coefficient memories 261 _{1 to} 261 ₄ , the R-IC 212 performs a distortion removal process. It functions as a device for performing time resolution creation processing and a device for performing spatial resolution creation processing.

  Next, the processing of the R-IC 212 of FIG. 27 will be described with reference to the flowchart of FIG.

  When a command for the R-IC 212 is supplied from the reception unit 211 in FIG. 14 to the R-IC 212, the R-IC 212 switches the internal structure in accordance with the command in step S261.

That is, the command from the receiving unit 211 is supplied to the coefficient output unit 255. In the coefficient output unit 255 (FIG. 28), the selection unit 262 performs a signal processing tap corresponding to the command from the reception unit 211 in the coefficient memories 261 _{1 to} 261 ₄ in accordance with the command from the reception unit 211. The output end of the coefficient stored is selected, and the selected output end is connected to the input end of the prediction calculation unit 256 (FIG. 27), thereby switching the internal structure of the R-IC 212.

  Thereafter, the process proceeds to step S262, and thereafter, in steps S262 to S267, the same processing as in steps S11 to S16 of FIG. 2 is performed.

  That is, in step S262, the pixel-of-interest selecting unit 251 selects one of the pixels constituting the second image data corresponding to the first image data input to the R-IC 212 that has not yet been set as the pixel of interest. Then, the pixel of interest is selected and the process proceeds to step S263.

  In step S263, the tap selection units 252 and 253 respectively select the prediction tap and the class tap for the target pixel from the first image data supplied thereto. The prediction tap is supplied from the tap selection unit 252 to the prediction calculation unit 256, and the class tap is supplied from the tap selection unit 253 to the class classification unit 254.

  The class classification unit 254 receives the class tap for the target pixel from the tap selection unit 253, and classifies the target pixel based on the class tap in step S264. Further, the class classification unit 254 outputs the class of the target pixel obtained as a result of the class classification to the coefficient output unit 255, and the process proceeds to step S265.

In step S265, the coefficient output unit 255 outputs the tap coefficient of the class supplied from the class classification unit 254. That is, the coefficient output unit 255 (FIG. 28) selects the tap coefficient of the class supplied from the class classification unit 254 from the coefficient memory 261 _{1 to} 261 ₄ from which the selection unit 262 selects the output end. Read and output to the prediction calculation unit 256. In step S265, the prediction calculation unit 256 acquires the tap coefficient output by the coefficient output unit 255, and the process proceeds to step S266.

  In step S266, the prediction calculation unit 256 performs the prediction calculation of Expression (1) using the prediction tap output from the tap selection unit 252 and the tap coefficient acquired from the coefficient output unit 255. Thereby, the prediction calculation unit 256 obtains and outputs the pixel value of the target pixel, and proceeds to step S267.

  In step S267, the pixel-of-interest selecting unit 251 determines whether there is second image data that has not yet been set as the pixel of interest. If it is determined in step S267 that there is still second image data that is not the pixel of interest, the process returns to step S262, and the same processing is repeated thereafter.

  If it is determined in step S267 that there is no second image data that has not yet been set as the target pixel, the process ends.

  Next, FIG. 30 shows another configuration example of the R-IC 212 of FIG. In the figure, portions corresponding to those in FIG. 27 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the R-IC 212 in FIG. 30 is configured in the same manner as in FIG. 27 except that a coefficient output unit 275 is provided instead of the coefficient output unit 255.

  Here, the other R-ICs 222 and 232 can also be configured similarly to the R-IC 212 shown in FIG.

  Similarly to the case of FIG. 27, the R-IC 212 of FIG. 30 switches its internal structure in accordance with a command supplied from the reception unit 211 (FIG. 14), and various signals using the above-described class classification adaptive processing. Process. That is, the R-IC 212 in FIG. 30 also converts the first image data supplied thereto into the second image data and outputs it.

  A command from the receiving unit 211 (FIG. 14) is supplied to the coefficient output unit 275. Further, the parameter z is supplied to the coefficient output unit 275 from the outside of the R-IC 212.

  Here, the parameter z can be given, for example, when the user operates the operation unit 201A of the remote controller 201 (FIG. 14).

  That is, the user operates the remote controller 201 to instruct the main body 200 to record or play back image data on the DVD 214 or HD 224, enlarge an image on the display 233, or display a slow motion as described above. can do. In response to the command, the R-IC 212 performs noise removal processing, distortion removal processing, temporal resolution creation processing, or spatial resolution creation processing on the first image data input thereto. Second image data obtained as a result of the processing is output.

  In the R-IC 212 having the configuration shown in FIG. 27, since the fixed tap coefficient is stored in the coefficient output unit 255, for example, the degree of noise removal by the noise removal process (the degree of S / N improved by the noise removal process) ), The degree of distortion removal by the distortion removal process, and the degree to which the high-frequency component increases by the time resolution creation process and the spatial resolution creation process are also fixed, but the preference of these degrees generally differs for each user.

  Furthermore, as described above, the slow motion display of the image is performed by the temporal resolution creation process, and the enlarged display of the image is performed by the spatial resolution creation process, but the display rate (frame rate) when performing the slow motion display Or, there are cases where the user wants to specify an enlargement ratio when performing enlarged display.

  Therefore, in the embodiment of FIG. 30, the user operates the remote controller 201 (FIG. 14) to instruct recording or playback of image data on the DVD 214 or HD 224, enlargement display of images on the display 233, slow motion display, and the like. Separately, the degree of noise removal by the noise removal process, the degree of distortion removal by the distortion removal process, the extent to which high frequency components increase by the time resolution creation process and the spatial resolution creation process, and when performing slow motion display A display rate and an enlargement rate for enlarging display can be specified.

  This designation can be performed, for example, by the user operating the operation unit 201A of the remote controller 201. That is, when the user operates the operation unit 201A of the remote controller 201, the degree of noise removal by the noise removal process, the degree of distortion removal by the distortion removal process, the time resolution creation process, and the spatial resolution creation process increase high frequency components. When a value such as a display rate when performing slow motion display or an enlargement rate when performing enlarged display is designated, the transmission unit 201B transmits a parameter z corresponding to the designated value. The parameter z is received by the receiving units 211, 221, and 231 via the control device 202, and supplied to the R-ICs 212, 222, and 232, respectively.

  In the R-IC 212 in FIG. 30, the parameter z from the reception unit 211 is supplied to the coefficient output unit 275.

  FIG. 31 shows a configuration example of the coefficient output unit 275 of FIG.

  The coefficient output unit 275 includes a coefficient generation unit 281, a coefficient type output unit 282, a parameter memory 283, and a coefficient memory 284. The coefficient generation unit 281, coefficient type output unit 282, parameter memory 283, and coefficient memory 284 are stored in the coefficient generation unit 61, coefficient type memory 62, parameter memory 63, and coefficient memory 64 that constitute the coefficient output unit 55 of FIG. 8. Each corresponds.

  In the coefficient output unit 275 of FIG. 31, the parameter z from the reception unit 211 (FIG. 14) is supplied to and stored in the parameter memory 283. Further, commands for the R-IC 212 in the command sequence received by the receiving unit 211 are supplied to the coefficient seed output unit 282. The class output from the class classification unit 254 (FIG. 30) is supplied to the coefficient memory 284.

The coefficient seed output unit 282 includes coefficient seed memories 291 ₁ , 291 ₂ , 291 ₃ , and 291 ₄ , and a selection unit 292.

The coefficient seed memories 291 ₁ , 291 ₂ , 291 ₃ , and 291 ₄ store coefficient seed data for noise removal processing, coefficient seed data for distortion removal processing, and time resolution creation processing that are obtained by learning performed in advance. Coefficient seed data and coefficient resolution data for spatial resolution creation processing are stored.

Then, the coefficient seed memories 291 _{1 to} 291 ₄ read out the coefficient seed data for each class stored therein and output them to the selection unit 292.

As described above, the selection unit 292 is supplied with the coefficient seed data read from each of the coefficient seed memories 291 _{1 to} 291 ₄ , and among the command strings received by the reception unit 211 (FIG. 14). , A command for the R-IC 212 is supplied. The selection unit 292 selects one of the output ends of the coefficient seed memories 291 _{1 to} 291 ₄ in accordance with the command supplied from the reception unit 211, and the selected output end is input to the coefficient generation unit 281. By connecting to the end, the internal structure of the R-IC 212 is switched.

Here, connection selection unit 292, of the coefficient seed memory 291 ₁ to 291 _4, for example, to select the output terminal of the coefficient seed memory 291 _1, and the selected output terminal, the input terminal of the coefficient generator 281 If you, the coefficient generation unit 281, coefficient seed data for noise removal processing which is read out from the coefficient seed memory 291 ₁ is supplied.

Similarly, the selection unit 292 selects the output end of the coefficient type memories 291 ₂ , 291 ₃ , 291 _{4 from} the coefficient type memories 291 _{1 to} 291 ₄ , and the selected output end is used as the coefficient generation unit 281. When connected to the input terminal, the coefficient generation unit 281 has the coefficient seed data for distortion removal processing, the coefficient seed data for temporal resolution creation processing, and the spatial resolution creation read out from the coefficient seed memories 291 ₂ , 291 ₃ and 291 _4. Coefficient seed data for processing is supplied.

  The coefficient generation unit 281 calculates a tap corresponding to the parameter z by calculating Expression (9) based on the coefficient seed data supplied from the coefficient seed output unit 282 and the parameter z stored in the parameter memory 283. A coefficient is generated for each class, supplied to the coefficient memory 284, and stored in an overwritten form.

  When the class is supplied from the class classification unit 254 (FIG. 30), the coefficient memory 284 taps the class from the class classification unit 254 among the tap coefficients corresponding to the parameter z stored therein. The coefficient is read and output to the prediction calculation unit 256 (FIG. 30).

  The R-IC 212 of FIG. 30 having the coefficient output unit 275 as described above stores a fixed tap coefficient, and a coefficient output provided in place of the coefficient output unit 255 of FIG. 27 that outputs the tap coefficient. The same process as in FIG. 27 is performed except that the unit 275 generates and outputs a tap coefficient corresponding to the parameter z.

Thus, in the coefficient output unit 275 (FIG. 31), the selection unit 292, of the coefficient seed memory 291 ₁ to 291 _4, for example, to select the output terminal of the coefficient seed memory 291 _1, thereby, the coefficient generation unit 281 When the coefficient seed data for noise removal processing read from the coefficient seed memory 291 ₁ is supplied to the coefficient generation unit 281, the coefficient seed data and the parameter z stored in the parameter memory 283 are used. A tap coefficient for noise removal processing corresponding to the parameter z is generated and stored in the coefficient memory 284. Accordingly, in this case, tap coefficients for noise removal processing are supplied from the coefficient output unit 275 to the prediction calculation unit 256, and as a result, the R-IC 212 functions as a device that performs noise removal processing. Become.

Similarly, the selection unit 292 selects the output end of the coefficient type memory 291 ₂ , 291 ₃ , or 291 _{4 from} the coefficient type memories 291 _{1 to} 291 ₄ , thereby causing the coefficient generation unit 281 to have the coefficient type stored in the coefficient type memory 281. When the coefficient seed data for distortion removal processing, the coefficient seed data for temporal resolution creation processing, or the coefficient seed data for spatial resolution creation processing read from the memory 291 ₂ , 291 ₃ , or 291 ₄ is supplied, In the coefficient generation unit 281, from the coefficient seed data and the parameter z stored in the parameter memory 283, the tap coefficient for distortion removal processing, the tap coefficient for temporal resolution creation processing, or the spatial resolution corresponding to the parameter z Tap coefficients for creation processing are generated and stored in the coefficient memory 284. Therefore, when the selection unit 292 selects the output end of the coefficient type memory 291 ₂ , 291 ₃ , or 291 ₄ among the coefficient type memories 291 _{1 to} 291 ₄ , the coefficient calculation unit 256 outputs the prediction calculation unit 256. Is supplied with a tap coefficient for distortion removal processing, a tap coefficient for temporal resolution creation processing, or a tap coefficient for spatial resolution creation processing. As a result, the R-IC 212 is a device that performs noise removal processing. It functions as a device for performing time resolution creation processing and a device for performing spatial resolution creation processing.

  Further, in the R-IC 212 of FIG. 30, since the tap coefficient corresponding to the parameter z is stored in the coefficient memory 284 of the coefficient output unit 275 (FIG. 31), the user operates the remote controller 201 (FIG. 14). Accordingly, the degree of noise removal by the noise removal process, the degree of distortion removal by the distortion removal process, and the degree of increase of the high frequency components by the time resolution creation process and the spatial resolution creation process can be set as desired. It is also possible to specify a display rate when performing slow motion display and an enlargement rate when performing enlarged display.

  As described above, in each of the R-ICs 212, 222, and 232, the internal structure is switched in accordance with at least one command in the command sequence, the image data input thereto is subjected to signal processing, and the signal processing is performed. Since the resulting image data is output, a plurality of functions can be easily realized with a single hardware.

  Further, since the R-IC 222 performs signal processing on the signal processing result of the R-IC 212 and the R-IC 232 performs signal processing on the signal processing result of the R-IC 222, the R-IC 212, 222, 232 as a whole is more This function can be easily realized.

  The AV system in FIG. 14 is provided with three R-ICs 212, 222, and 232, but the AV system may be configured with one, two, or four or more R-ICs. Is possible.

  Next, a second embodiment of the present invention will be described.

  FIG. 32 shows an example of the configuration of a television receiver 301 as a second embodiment of the present invention.

  An antenna 302 is connected to the television receiver 301. The antenna 302 receives a transmission signal of a television broadcast program as a broadcast wave (radio wave) transmitted from a broadcast station (not shown), and supplies it to the television receiver 301. The television receiver 301 receives a transmission signal from the antenna 301 and selects a television broadcast program (signal) of a predetermined channel included in the transmission signal in accordance with an operation signal from the remote controller (remote commander) 303. The station displays an image as the television broadcast program and outputs sound.

  That is, the television receiver 301 has a tuner unit 311, and a transmission signal from the antenna 302 is supplied to the tuner unit 311. The tuner unit 311 receives a transmission signal from the antenna 302, and according to control of the system controller 318, an image (data) and sound (data) as a television broadcast program of a predetermined channel from the transmission signal received from the antenna 302. Acquired by selecting a channel.

  Further, the tuner unit 311 supplies the sound of the selected channel to the amplifier circuit 312 and also outputs the R (Red) signal, the G (Green) signal, and the B (Blue) signal of the image to the signal processing units 314 and 315. , 316 respectively.

  The amplifier circuit 312 amplifies the sound from the tuner unit 311 and supplies the amplified sound to the speaker 313 for output.

  In accordance with control from the system controller 318, the signal processing units 314 to 316 respectively process the R signal, G signal, and B signal of the image from the tuner unit 311, and supply them to the display 317 to display the corresponding image. Display.

  The system controller 318 controls the tuner unit 311 and the signal processing units 314 to 316 by supplying control signals to the tuner unit 311 and the signal processing units 314 to 316 in accordance with the operation signal supplied from the remote control receiving unit 319. .

  The remote control receiving unit 319 receives, for example, infrared or other wireless signals as operation signals transmitted from the remote control 303 when the user operates the remote control 303, and supplies them to the system controller 318.

  In the television receiver 301 configured as described above, the tuner unit 311 receives a transmission signal from the antenna 302 and, according to the control of the system controller 318, the television of a predetermined channel from the transmission signal received from the antenna 302. Select the image and sound as a John broadcast program. The tuner unit 311 supplies the selected channel audio to the amplifier circuit 312 and also supplies the image R signal, G signal, and B signal (image data) to the signal processing units 314 to 316, respectively. .

  In the amplifier circuit 312, the sound from the tuner unit 311 is amplified and supplied to the speaker 313 for output.

  On the other hand, the signal processing units 314 to 316 respectively perform signal processing on the R signal, G signal, and B signal (image data) of the image from the tuner unit 311 in accordance with control from the system controller 318. In the signal processing units 314 to 316, image data obtained as a result of each signal processing is supplied to the display 317, and a corresponding image is displayed on the display 317.

  Note that the broadcast received by the television receiver 301 is not particularly limited. That is, the television receiver 301 can receive, for example, satellite broadcast, terrestrial broadcast, analog broadcast, digital broadcast, and other arbitrary broadcasts.

  Further, in the following description, the explanation about the sound is omitted unless particularly necessary.

  Next, control of the tuner unit 311 and the signal processing units 314 to 316 by the system controller 318 in FIG. 32 will be described.

  Here, in the signal processing units 314 to 316, the same processing is performed except that the signals processed in each are R signal, G signal, and B signal. Therefore, the signal processing units 314 to 316 are described below. Of these, only the signal processing unit 314 will be described.

  In the following description, it is assumed that the image output from the tuner unit 311 is a progressive image (non-interlace image). However, the image output from the tuner unit 311 may be an interlaced image. In this case, the “frame” in the description of the second embodiment below can be a “field”.

  The system controller 318 performs control to switch the operation mode of the television receiver 301 to the multi-screen mode or the normal screen mode in accordance with an operation signal supplied from the remote control 303 via the remote control receiving unit 319. That is, the system controller 318 switches the operation mode of the television receiver 301 to the multi-screen mode or the normal screen mode in accordance with the operation of the remote controller 303 by the user, and performs the processing of the operation mode. And controls the signal processing units 314 to 316.

  The tuner unit 311 and the signal processing units 314 to 316 perform multi-screen mode processing or normal screen mode processing under the control of the system controller 318.

  Here, image data supplied (input) from the tuner unit 311 to the signal processing unit 314 is referred to as input image data, and image data obtained as a result of signal processing in the signal processing unit 314 is referred to as output image data. The multi-screen mode is a mode in which, for example, the same frame of each of the input image data of a plurality of channels obtains output image data displayed in a so-called multi-screen and displays it on the display 317. The normal screen mode is 1 In this mode, output image data corresponding to the input image data of the channel is obtained and displayed on the display 317.

  In the multi-screen mode, since images of a plurality of channels are displayed on a multi-screen, the user can easily select a channel that broadcasts a desired program by viewing the multi-screen image. It becomes possible.

  In the multi-screen mode, a cursor (for example, a frame) for designating an image of an arbitrary channel among the images of a plurality of channels displayed on the multi-screen is displayed, and the cursor is used to operate the remote controller 303 by the user. It can be moved according to. In this case, the sound of the channel of the image designated by the cursor can be output from the speaker 313.

  FIG. 33 is a diagram for explaining control of the tuner unit 311 by the system controller 318 of FIG. In FIG. 33, the vertical axis represents the vertical direction of the image, and the horizontal axis represents time (elapsed time).

  When the operation mode is the normal screen mode, the system controller 318 controls the tuner unit 311 so as to select a channel corresponding to the operation of the remote controller 303 by the user.

  Therefore, for example, assuming that the user has selected channel CH1 by operating the remote controller 303, the tuner unit 311 continues to select channel CH1 from the transmission signal from the antenna 302. Thus, as shown in the upper side of FIG. 33, the image (data) of the channel CH1 is supplied to the signal processing unit 314 in the frame period.

  On the other hand, when the operation mode is the multi-screen mode, the system controller 318 controls the tuner unit 311 so as to sequentially change the channel to be selected, for example, in the frame period.

Therefore, if the number of channels to be changed is, for example, four channels CH1, CH2, CH3, and CH4, the tuner unit 311 has the first frame as shown in the lower side of FIG. 33, for example. The channel CH1 is selected at the timing, the channel CH2 is selected at the timing of the next frame, and the channels CH3 and CH4 are sequentially selected in the same manner. After that, the tuner unit 311 selects the channel CH1 again, and subsequently selects the channels CH1 to CH4 sequentially in the same manner. As a result, assuming that the frame period is represented by T ₁ , the image of the channel CH1 is supplied from the tuner unit 311 to the signal processing unit 314 at a period four times the frame period T ₁ . That is, the tuner 311, at 4 times the period of a frame period T _1, the image of the channel CH1 dropped frames of every three frames has occurred is output. The same applies to the images of the other channels CH2 to CH4.

Here, the number of channels to be changed is not limited to four channels. Assuming that the number of channels to be changed is represented by N, in the multi-screen mode, a cycle in which an image of one channel that is to be changed is supplied from the tuner 311 to the signal processor 314 is a frame. It becomes N times the period T _1.

  In the multi-screen mode, for example, all the channels that can be received by the television receiver 301 can be set as channels to be selected, or a plurality of channels selected by the user can be changed. It can also be the target channel. Further, in the multi-screen mode, when the number of channels to be selected for channel selection, that is, the number of channels of an image displayed in one frame is larger than a predetermined threshold, the image may be scrolled and displayed. Is possible.

  Here, in the following, in order to simplify the description, it is assumed that the channel to be changed in tuning of the tuner unit 311 in the multi-screen mode is fixed to the above-described channels CH1 to CH4.

  Next, FIG. 34 shows a case where an image of one channel CH1 obtained by the tuner unit 311 in the normal screen mode is simply displayed and an image of four channels CH1 to CH4 obtained by the tuner unit 311 in the multi-screen mode. The multi-screen having the same number of divisions as the number of channels, that is, the case of simply displaying on the 4-screen multi-screen is shown.

  In the multi-screen mode, one frame is equally divided into, for example, 4 × 2 small screens, and images of 4 channels CH1 to CH4 are respectively displayed. That is, here, it is assumed that images of channels CH1 to CH4 are displayed on the upper left, upper right, lower left, and lower right small screens of the four divided multi-screens.

In normal screen mode, the tuner unit 311, as described with reference to FIG. 33, the image of one channel CH1 is obtained in the frame cycle T _1. Therefore, if you simply display one image channel CH1 obtained by the tuner section 311, as shown in FIG. 34 upper, a frame period T _1, the image of the channel CH1 is displayed.

On the other hand, in the multi-screen mode, as described with reference to FIG. 33, the tuner unit 311 obtains images of the four channels CH1 to CH4 with a period 4T ₁ that is four times the frame period T ₁ . Therefore, when the images of the four channels CH1 to CH4 obtained by the tuner unit 311 are simply displayed on a 4-screen multi-screen, the multi-screen is displayed at the timing of the first frame as shown in the lower side of FIG. The first frame of the image of the channel CH1 obtained by the tuner unit 311 is displayed on the upper left small screen, and at the timing of the second frame, the first frame obtained on the upper right small screen of the multi screen is obtained by the tuner unit 311. The second frame of the image of channel CH2 is displayed. Furthermore, at the timing of the third frame, the third frame of the image of the channel CH3 obtained by the tuner unit 311 is displayed on the lower left small screen of the multi screen, and at the timing of the fourth frame, the third frame of the multi screen is displayed. The fourth frame of the image of the channel CH4 obtained by the tuner 311 is displayed on the lower right small screen. After that, at the timing of the fifth frame, the fifth frame of the image of the channel CH1 obtained by the tuner unit 311 is displayed on the small screen at the upper left of the multi-screen. The images of the channels CH1 to CH4 obtained in the cycle 4T ₁ are displayed on the corresponding small screens of the multi screen.

In the multi-screen mode, after a channel that is the tuner unit 311 is selected, a small screen that displays an image of the channel is frozen until the next channel is selected, for example. Then, in the tuner unit 311, each of the four channels CH1 to CH4 is selected in a time-division manner with a period of 4T ₁ , so the images of the four channels CH1 to CH4 displayed on the multi-screen are rough and difficult to see. It will be a thing.

  The television receiver 301 in FIG. 32 can display such a multi-screen. However, as described later, the signal processing unit 314 generates an output image that displays a smooth multi-screen. And can be displayed.

  That is, FIG. 35 shows a configuration example of the signal processing unit 314 of FIG.

  The signal processing unit 314 includes a command sequence generation unit 330, signal processing chips 331 and 332, a memory unit 333, and a signal processing chip 334.

  Here, the command sequence generation unit 330, the signal processing chips 331 and 332, the memory unit 333, and the signal processing chip 334 are each configured by, for example, a one-chip IC. Note that the entire signal processing unit 314 can be configured by a single-chip IC, or two or more of the command sequence generation unit 330, the signal processing chips 331 and 332, the memory unit 333, and the signal processing chip 334. Can be a one-chip IC.

  The command string generation unit 330 is supplied with a control signal from the system controller 318 (FIG. 32). The command sequence generation unit 330 generates a command sequence including a plurality of commands in response to a control signal from the system controller 318, and transmits the command sequence to the signal processing chips 331, 332, the memory unit 333, and the signal processing chip 334, for example, wirelessly. To do.

  The signal processing chip 331 is supplied with input image data as an image output from the tuner unit 311 of FIG. Further, a motion vector as a signal processing result of the signal processing chip 334 is supplied to the signal processing chip 331.

  The signal processing chip 331 receives the command sequence transmitted from the command sequence generation unit 330, and is reconfigurable in accordance with the commands constituting the command sequence, similar to the R-IC 212 of the first embodiment. The internal structure is switched to. Furthermore, the signal processing chip 331 performs signal processing on the input image data from the tuner unit 311 as first image data in the class classification adaptive processing, and the resulting image data (second image data in the class classification adaptive processing). Is output to the signal processing chip 332.

  In the signal processing chip 331, the input image data from the tuner unit 311 is signal-processed using the motion vector supplied from the signal processing chip 334.

  In addition, the signal processing chip 331 switches the internal structure thereof, and as signal processing, for example, spatial resolution creation processing that improves the spatial resolution of the image, or resizing processing that resizes the image to generate a reduced image (hereinafter, referred to as “processing”). (Also referred to as reduced image generation processing).

  As described above, the signal processing chip 332 is supplied not only with image data as a signal processing result of the signal processing chip 331 but also with a motion vector as a signal processing result of the signal processing chip 334.

  The signal processing chip 332 receives the command sequence transmitted from the command sequence generation unit 330, and is reconfigurable in accordance with the commands constituting the command sequence, similar to the R-IC 212 of the first embodiment. The internal structure is switched to. Further, the signal processing chip 332 performs signal processing on the image data output from the signal processing chip 331 as first image data in the class classification adaptive processing, and the resulting image data (second image in the class classification adaptive processing). Data) is output to the memory unit 333.

  Note that also in the signal processing chip 332, image data from the signal processing chip 331 is signal-processed using the motion vector supplied from the signal processing chip 334.

  Further, the signal processing chip 332 performs, for example, noise removal processing for removing image noise or time resolution creation processing for improving the time resolution of the image as the signal processing by switching the internal structure.

  The memory unit 333 receives the command sequence transmitted from the command sequence generation unit 330, and stores the image data output from the signal processing chip 332 in a built-in memory (not shown) according to the commands constituting the command sequence. ) And the image data stored in the memory is read out and supplied as output image data to the display 317 (FIG. 32).

  Similarly to the signal processing chip 331, the signal processing chip 334 is supplied with input image data as an image output from the tuner unit 311 in FIG.

  The signal processing chip 334 receives the command sequence transmitted from the command sequence generation unit 330, and is reconfigurable in accordance with the commands constituting the command sequence, similar to the R-IC 212 of the first embodiment. The internal structure is switched to. Further, the signal processing chip 334 detects the motion vector by performing signal processing on the input image data from the tuner unit 311, and supplies the motion vector to the signal processing chips 331 and 332.

  Note that the signal processing chip 334 switches the internal structure, and as signal processing, for example, motion vector processing for detecting a motion vector for a normal screen described later, or motion for detecting a motion vector for a multi screen described later. Perform vector processing.

  Next, the command sequence generated by the command sequence generation unit 330 in FIG. 35 will be described with reference to FIG.

  That is, FIG. 36 illustrates an example of a command sequence generated by the command sequence generation unit 330.

  The system controller 318 (FIG. 32) supplies a control signal instructing to perform the normal screen mode process or the multi-screen mode process to the command string generation unit 330.

  When the control signal supplied from the system controller 318 instructs to perform normal screen mode processing, the command sequence generation unit 330 displays a command sequence shown in FIG. 36 (hereinafter referred to as normal mode command sequence as appropriate). Generated). When the control signal supplied from the system controller 318 instructs to perform the multi-screen mode processing, the command sequence generation unit 330 uses the command sequence shown in the lower part of FIG. Mode command sequence).

  The normal screen mode command sequence shown in FIG. 36 includes a normal screen motion vector detection command, a spatial resolution creation command, a noise removal command, and a normal screen memory control command sequentially arranged from the top. In addition, the multi-screen mode command sequence shown in the lower part of FIG. 36 is configured by sequentially arranging a multi-screen motion vector detection command, a reduced image generation command, a time resolution creation command, and a multi-screen memory control command from the top. ing.

  The normal screen motion vector detection command is a command for requesting execution of a motion vector detection process for detecting a motion vector for a normal screen from an image. The spatial resolution creation command or the noise removal command is a class classification described above. This command requests execution of spatial resolution creation processing or noise removal processing by adaptive processing.

  The normal screen memory control command is a command for requesting execution of read / write processing to / from the memory of the one-channel image data for displaying the one-channel image data on the entire screen of the display 317.

  The multi-screen motion vector detection command is a command for requesting execution of a motion vector detection process for detecting a multi-screen motion vector from an image, and the reduced image generation command or the temporal resolution creation command is described above. This is a command for requesting execution of reduced image generation processing or time resolution creation processing as resizing processing by class classification adaptation processing.

  The multi-screen memory control command is a command for requesting execution of read / write processing of the multi-channel image data to the memory for displaying the multi-channel image data on the display 317.

  Here, the command sequence in FIG. 36 is simply configured by sequentially arranging one or more commands, but each command configuring the command sequence includes an IC that is to execute processing corresponding to the command, That is, in the embodiment of FIG. 35, it is possible to add IC information for specifying any one of the signal processing chips 331 and 332, the memory unit 333, or the signal processing chip 334. In this case, each of the signal processing chips 331 and 332, the memory unit 333, or the signal processing chip 334 corresponds to a command to which IC information representing itself is added in the command sequence received from the command sequence generation unit 330. Execute the process.

  In the second embodiment, as shown in FIG. 36, the command sequence is simply configured by sequentially arranging one or more commands. In this case, each of the signal processing chips 331, 332, the memory unit 333, or the signal processing chip 334 executes, for example, a process corresponding to a command that can be executed in the command sequence received from the command sequence generation unit 330. To do.

  That is, when the normal screen mode command sequence in FIG. 36 is transmitted by the command sequence generation unit 330 (FIG. 35), the signal processing chips 331, 332, the memory unit 333, or the signal processing chip 334 respectively The screen mode command sequence is received, and the process corresponding to the command executable by itself in the normal screen mode command sequence is executed.

  In this case, the signal processing chip 334 detects the motion vector for the normal screen mode using the internal structure as signal processing in accordance with, for example, the normal screen motion vector detection command in the normal screen mode command sequence. Switch to perform motion vector detection processing.

  The signal processing chip 334 detects the motion vector for the normal screen mode from the image data of the channel CH1, for example, as the image data of one channel supplied from the tuner unit 311 in the normal screen mode, and the signal processing chip. 331 and 332 are supplied.

  The signal processing chip 331 switches the internal structure of the normal screen mode command sequence in FIG. 36 to perform spatial resolution creation processing as signal processing, for example, according to the spatial resolution creation command.

  Then, the signal processing chip 331 uses the motion vector supplied from the signal processing chip 334 to perform spatial resolution creation processing on the image data of the channel CH1 supplied from the tuner unit 311 and obtains the resulting image data Is output to the signal processing chip 332.

  Here, FIG. 37 shows image data input and output in the signal processing chip 331 that performs the spatial resolution creation processing.

In FIG. 37, the i-th frame (image thereof) is represented by t _i .

The command sequence generation unit 330 (FIG. 35) transmits the normal screen mode command sequence in FIG. 36 when the normal screen mode is selected as the operation mode. In the normal screen mode, the tuner unit 311 (FIG. 32) selects, for example, the channel CH1 as a predetermined channel, so that the signal processing chip 331 has a channel as shown in the upper side of FIG. frame t _i of the input image data of CH1 is sequentially inputted at a frame period T _1.

The signal processing chip 331 performs the spatial resolution creation process on the frame t _i of the input image data of the channel CH1 input thereto, and the spatial resolution is generated by the spatial resolution creation process as shown in the lower side of FIG. Frames t _i of the improved image data of channel CH1 are sequentially output at a frame period T ₁ .

The frame t _i of the image of the channel CH 1 whose spatial resolution is improved is sequentially supplied to the signal processing chip 332.

  The signal processing chip 332 switches the internal structure of the normal screen mode command sequence in FIG. 36 to perform noise removal processing as signal processing in accordance with, for example, a noise removal command.

  Then, the signal processing chip 332 uses the motion vector supplied from the signal processing chip 334 to perform noise removal processing on the image data of the channel CH1 supplied from the signal processing chip 331, and the resulting image data Is output to the memory unit 333.

  Here, FIG. 38 shows image data input and output in the signal processing chip 332 that performs noise removal processing.

In FIG. 38, the i-th frame (image thereof) is represented by t _i .

For the signal processing chip 331 332, as shown in FIG. 38 upper, frame t _i of the image data of the channel CH1 with improved spatial resolution is successively inputted at a frame period T _1.

The signal processing chip 332 performs noise removal processing on the frame t _i of the image data of the channel CH1 input thereto, and the S / N is improved by the noise removal processing as shown in the lower side of FIG. and sequentially outputs the frames t _i of the image data of the channel CH1 in a frame period T _1.

The frame t _i of the image of the channel CH1 with improved S / N is sequentially supplied to the memory unit 333.

  The memory unit 333 follows each frame of image data output from the signal processing chip 332 as it is in the built-in memory according to, for example, a normal screen memory control command in the normal screen mode command sequence in FIG. The image data frame temporarily stored by writing is read out as a frame of output image data and supplied to the display 317 (FIG. 32).

  Accordingly, when the normal screen mode command sequence in FIG. 36 is transmitted by the command sequence generation unit 330 (FIG. 35), the CH1 image data received by the tuner unit 311 (FIG. 32) is displayed on the display 317. An image with improved spatial resolution and S / N is displayed.

  On the other hand, when the multi-screen mode command sequence in the lower part of FIG. 36 is transmitted by the command sequence generation unit 330 (FIG. 35), the signal processing chips 331 and 332, the memory unit 333, or the signal processing chip 334 respectively A screen mode command sequence is received, and processing corresponding to a command that can be executed by itself in the multi-screen mode command sequence is executed.

  In this case, the signal processing chip 334 detects a motion vector for the multi-screen mode using the internal structure as signal processing in response to, for example, a multi-screen motion vector detection command in the multi-screen mode command sequence. Switch to perform motion vector detection processing.

  The signal processing chip 334 detects a motion vector for the multi-screen mode from, for example, image data of the channels CH1 to CH4 as the image data of a plurality of channels supplied in time division from the tuner unit 311 in the multi-screen mode. To the signal processing chips 331 and 332.

  The signal processing chip 331 performs reduced image generation processing (resizing to reduce the size) using the internal structure as signal processing in accordance with, for example, a reduced image generation command in the multi-screen mode command sequence in the lower part of FIG. Switch to perform resizing.

  The signal processing chip 331 uses the motion vector supplied from the signal processing chip 334 to perform reduced image generation processing on the image data of the channels CH1 to CH4 supplied in a time division manner from the tuner unit 311. The resulting reduced image data of CH1 to CH4 is output to the signal processing chip 332.

  Here, FIG. 39 shows image data input and output in the signal processing chip 331 that performs reduced image generation processing.

In FIG. 39, the i-th frame (image thereof) is represented by t _i .

The command sequence generation unit 330 (FIG. 35) transmits the multi-screen mode command sequence in the lower part of FIG. 36 when the multi-screen mode is selected as the operation mode. In the multi-screen mode, the tuner unit 311 (FIG. 32) selects, for example, channels CH1 to CH4 as a plurality of channels in a time-sharing manner. As shown, frames t _i of image data of channels CH1 to CH4 are input in a time division manner.

That is, when the operation mode is the multi-screen mode, as described above, the tuner unit 311 converts the image data of the four channels CH1 to CH4 in which frames are dropped every three frames in the cycle 4T ₁ into the signal processing unit 314 ( FIG. 35). Thus, supplied to the signal processing chip 331 of the signal processing section 314, 4 to the image data of the channels CH1 to CH4, respectively, at a period 4T _1, it dropped frames every three frames occurs.

  Here, a frame in which frame dropping occurs, that is, a frame not received by the tuner unit 311 is hereinafter referred to as a missing frame as appropriate. In addition, a frame received by the tuner unit 311 with respect to a missing frame is hereinafter referred to as a non-missing frame as appropriate.

The signal processing chip 331 performs reduced image generation processing on the frames t _i of the image data of the channels CH1 to CH4 supplied thereto, and is obtained by the reduced image generation processing as shown in the lower side of FIG. Further, reduced image data obtained by reducing the image data of channels CH1 to CH4 is output.

The period of the reduced image data signal processing chip 331 is output is a frame period T _1, the period in which the reduced image data is output for one channel certain of the channels CH1 through CH4 are 4T _1. That is, there is a missing frame in the reduced image data of each channel of the channels CH1 to CH4. That is, when attention is paid to the reduced image data of a certain channel of channels CH1 to CH4, there are three missing frames between a certain non-missing frame and the next non-missing frame.

  The reduced image data of the channels CH1 to CH4 is supplied from the signal processing chip 331 to the signal processing chip 332.

  The signal processing chip 332 switches the internal structure of the normal screen mode command sequence in the lower part of FIG. 36 to perform time resolution creation processing as signal processing, for example, in accordance with a time resolution creation command.

  Then, the signal processing chip 332 uses the motion vector supplied from the signal processing chip 334 to perform time resolution creation processing on the reduced image data of the channels CH1 to CH4 supplied from the signal processing chip 331, and as a result The obtained image data is output to the memory unit 333.

  Here, FIG. 40 shows image data input and output in the signal processing chip 332 that performs the time resolution creation processing.

In FIG. 40, the i-th frame of channel CH # k is represented as CH # k (t _i ).

In the multi-screen mode, the tuner unit 311 tunes the channels CH1 to CH4 in a time division manner, so that the signal processing chips 331 to 332 receive the channels CH1 to CH4 as shown at the top of FIG. each frame of the reduced image data are sequentially input at a period 4T _1.

That is, in FIG. 40, at the timing of the frame t _1, frame CH1 is the first frame of the channel CH1 (t ₁₎ is, at the timing of the frame t _2, a second frame of the channel CH2 frame CH2 (t ₂ ) is the timing of frame t ₃ and frame CH ₃ (t ₃ ), which is the third frame of channel CH 3, and frame CH ₄ (t ₄ ), which is the fourth frame of channel CH ₄ , at the timing of frame t ₄ However, at the timing of frame t ₅ , frame CH 1 (t ₅ ), which is the fifth frame of channel CH 1, is input to signal processing chip 332, and in the same manner, each frame of channels CH 1 to CH 4 has a period of 4T. ₁ is entered sequentially.

  The signal processing chip 332 generates reduced image data of each of the channels CH1 to CH4 having no missing frame by performing a time resolution creation process on the reduced image data of each of the channels CH1 to CH4 having the missing frame.

That is, in the signal processing chip 332, as shown second from the top in FIG. 40, a frame (image) CH1 (t ₂ ), CH1 (t ₃ ), CH1 (t ₄ ) that is a missing frame of the channel CH1. , CH1 (t ₆ ),... Is generated, and the frame CH2 (t ₁ ), CH2 (t ₃ ), CH2 (t ₄ ), reduced image data of the channel CH2 in which CH2 (t ₅ ),... Exists is generated. Further, in the signal processing chip 332, a channel CH3 in which frames CH3 (t ₁ ), CH3 (t ₂ ), CH3 (t ₄ ), CH3 (t ₅ ),. Furthermore, there are frames CH4 (t ₁ ), CH4 (t ₂ ), CH4 (t ₃ ), CH4 (t ₅ ),... That are missing frames of the channel CH4. Reduced image data of channel CH4 is generated.

In this way, the signal processing chip 332 obtains and outputs reduced image data in which a frame exists at the frame period T ₁ for each of the four channels CH1 to CH4.

The frame t _i of the reduced image data of the lost frame is not the channel CH1 through CH4 are sequentially supplied to the memory unit 333.

  The memory unit 333 stores the reduced image data of the channels CH1 to CH4 output from the signal processing chip 332 in accordance with, for example, a multi-screen memory control command in the multi-screen mode command sequence in the lower part of FIG. The same frame of the reduced image data of the channels CH1 to CH4 is synthesized by writing to and reading from, thereby generating a frame of output image data for displaying the same frame of each of the reduced image data of the channels CH1 to CH4 on a multi-screen Then, it is supplied to the display 317.

  That is, as shown at the bottom of FIG. 40, the memory unit 333 converts the reduced image data of the same frame of the channels CH1 to CH4 from the signal processing chip 332 into the upper left of the multi-screen, according to the multi-screen memory control command. By writing into the storage area of the memory corresponding to the upper right, lower left, and lower right small screens, a frame of output image data for displaying the same frame of each of the reduced image data of channels CH1 to CH4 on a multi-screen is generated (stored).

  Further, the memory unit 333 reads out the frame of the output image data stored in the memory in accordance with the multi-screen memory control command and supplies it to the display 317.

  Accordingly, when the multi-screen mode command sequence in the lower part of FIG. 36 is transmitted by the command sequence generation unit 330 (FIG. 35), the display 317 smoothly displays the motion of the images of the channels CH1 to CH4 having no missing frame. A multi-screen is displayed.

  Next, the processing of the signal processing unit 314 in FIG. 35 will be described with reference to the flowchart in FIG.

  When a control signal is transmitted from the system controller 318 (FIG. 32) to the signal processing unit 314 (or 316), in the signal processing unit 314, in step S301, the command sequence generation unit 330 outputs the control signal. Then, the process proceeds to step S302.

  In step S302, the command sequence generation unit 330 generates a command sequence including one or more commands corresponding to the control signal received from the system controller 318, transmits the command sequence wirelessly, and proceeds to step S303.

  In step S <b> 303, each block constituting the signal processing unit 314, that is, the signal processing chips 331 and 332, the memory unit 333, and the signal processing chip 334 receives the command sequence transmitted from the command sequence generation unit 330. Furthermore, in step S303, each of the signal processing chips 331, 332, and 334 recognizes a command that can be executed from the command in the command sequence received from the command sequence generation unit 330, and performs processing corresponding to the command ( Signal processing). That is, each of the signal processing chips 331, 332, and 334 switches to its own internal structure in accordance with a command that can be executed by itself, and performs signal processing corresponding to the command.

  Then, the process proceeds from step S303 to S304, and the signal processing chip 334 executes the signal processing determined in step S303 on the image data supplied from the tuner unit 311 (FIG. 32), and the signal processing result is obtained as a signal. The processing chips 331 and 332 are supplied, and the process proceeds to step S305.

  In step S305, the signal processing chip 331 executes the signal processing determined in step S303 on the image data supplied from the tuner unit 311 (FIG. 32), and the image data as the signal processing result is subjected to signal processing. The chip 332 is supplied, and the process proceeds to step S306.

  In step S306, the signal processing chip 332 executes the signal processing determined in step S303 on the image data supplied from the signal processing chip 331, and supplies the image data as the signal processing result to the memory unit 333. Then, the process proceeds to step S307.

  Here, the signal processing chips 331 and 332 perform signal processing using the signal processing result of the signal processing chip 334.

  In step S307, the memory unit 333 stores the image data supplied from the signal processing chip 332 in accordance with a command that can be executed by the command in the command sequence received from the command sequence generation unit 330 in step S303. And the written image data is read out and output to the display 317 (FIG. 32), and the process is terminated.

  Note that the processes in steps S301 to S307 in FIG. 41 are processes performed by the signal processing chips 331 and 332, the memory unit 333, the signal processing chip 334, and the like as hardware. However, the processing of steps S301 to S303 can also be performed by causing a computer such as a microcomputer to execute a program.

  Next, FIG. 42 is a block diagram illustrating a configuration example of the signal processing chip 334 of FIG.

  Image (input image) data output from the tuner unit 311 (FIG. 32) is supplied to the frame memory 341 and the motion detection circuit 344.

  The receiving unit 340 receives the command sequence transmitted from the command sequence generating unit 330. Further, the reception unit 340 recognizes a command that can be executed by the signal processing chip 334 from the command sequence received from the command sequence generation unit 330, and supplies the command to the selector 343 and the selection unit 345.

  The frame memory 341 temporarily stores the frame of the input image data supplied thereto, thereby delaying it by one frame and supplying it to the delay circuit 342 and the selector 343. Therefore, when the input image data of the (n + 4) th frame is supplied to the motion detection circuit 344, the input image data of the (n + 3) th frame before that frame is supplied from the frame memory 341 to the delay circuit 342 and the selector 343. .

  The delay circuit 342 delays the frame of the input image data supplied from the frame memory 341 by 3 frames and supplies the delayed image to the selector 343. Accordingly, the frame of the input image data supplied from the tuner unit 311 to the signal processing chip 334 is delayed by a total of four frames in the frame memory 341 and the delay circuit 342 and supplied to the selector 343, so that the motion detection circuit 344 When the input image of the (n + 4) th frame is supplied, the input image data of the nth frame before the 4th frame is supplied from the delay circuit 342 to the selector 343.

  As described above, the selector 343 is supplied with the input image of the (n + 3) th frame from the frame memory 341 and the input image of the nth frame from the delay circuit 342. In response to the command supplied from the receiving unit 340, the selector 343 uses either the output of the frame memory 341 or the output of the delay circuit 342 as an input to the motion detection circuit 344. The internal structure of 334 is switched.

  Accordingly, one of the n + 3th frame input image data output from the frame memory 341 or the nth frame input image data output from the delay circuit 342 is supplied to the motion detection circuit 344 via the selector 343. Is done.

  The motion detection circuit 344 detects and selects the motion vector of each pixel of the frame of the input image data supplied via the selector 343 by referring to the input image data supplied from the tuner unit 311 (FIG. 32). Supplied to the unit 345.

  In response to the command supplied from the reception unit 340, the selection unit 345 directly or without reducing the size of the motion vector supplied from the motion detection circuit 344 to the signal processing chips 331 and 332 (FIG. 35).

  Next, processing of the signal processing chip 334 of FIG. 42 will be described with reference to the flowchart of FIG.

  In the signal processing chip 334, when a command sequence is transmitted from the command sequence generation unit 330 (FIG. 35), the reception unit 340 receives the command sequence in step S311. Further, the reception unit 340 recognizes a command that can be executed by the signal processing chip 334 from the command sequence received from the command sequence generation unit 330, supplies the command to the selector 343 and the selection unit 345, and starts from step S311. The process proceeds to S312.

  Here, in the second embodiment, as the commands that can be executed by the signal processing chip 334, for example, as described above, the normal screen motion vector detection command in the normal screen mode command sequence in FIG. There are multi-screen motion vector detection commands in the multi-screen mode command sequence at the bottom of FIG.

  Therefore, when receiving the normal screen mode command sequence in FIG. 36, the receiving unit 340 supplies the normal screen motion vector detection command in the sequence to the selector 343 and the selection unit 345. When receiving the multi-screen mode command sequence in the lower part of FIG. 36, the receiving unit 340 supplies the multi-screen motion vector detection command in the multi-screen mode command sequence to the selector 343 and the selecting unit 345.

  In step S <b> 312, the selector 343 causes either the output of the frame memory 341 or the output of the delay circuit 342 to be input to the motion detection circuit 344 in accordance with the command supplied from the reception unit 340. The internal structure of the signal processing chip 334 is switched. Further, in step S312, the selection unit 345 outputs the motion vector supplied from the motion detection circuit 344 as it is or reduces the size thereof by half according to the command supplied from the reception unit 340. As such, it sets itself (switches its internal structure) and proceeds to step S313.

  That is, when the command supplied from the reception unit 340 to the selector 343 and the selection unit 345 is a normal screen motion vector detection command, the selector 343 uses the output of the frame memory 341 as an input to the motion detection circuit 344. Thus, the internal structure of the signal processing chip 334 is switched to perform processing for detecting a motion vector for the normal screen mode as signal processing. Furthermore, the selection unit 345 sets itself so as to output the motion vector supplied from the motion detection circuit 344 as it is.

  On the other hand, when the command supplied from the reception unit 340 to the selector 343 and the selection unit 345 is a multi-screen motion vector detection command, the selector 343 uses the output of the delay circuit 342 as an input to the motion detection circuit 344. Thus, the internal structure of the signal processing chip 334 is switched so as to perform processing for detecting a motion vector for the multi-screen mode as signal processing. Furthermore, the selection unit 345 sets itself so as to reduce the magnitude of the motion vector supplied from the motion detection circuit 344 to ½.

  In step S313, the signal processing chip 334 performs signal processing to be performed by the internal structure switched in step S312 on the input image data supplied from the tuner unit 311 (FIG. 32), that is, here for the normal screen. The process of detecting the motion vector of the image or the process of detecting the motion vector for the multi-screen is performed, and the process proceeds to step S314.

  Here, in the normal screen mode in which the command sequence generation unit 330 transmits the normal screen mode command sequence in FIG. 36, since there is no missing frame in the image data of the channel CH1, for example, as the input image data, the channel CH1 If the frame of interest in the image data is the nth frame, the motion vector of the nth frame can be detected by referring to the next n + 1th frame, for example.

  On the other hand, in the multi-screen mode in which the command sequence generation unit 330 transmits the multi-screen mode command sequence shown in the lower part of FIG. 36, the input image data, for example, image data of channels CH1 to CH4 has 3 frames for 4 frames. If the non-missing frame of interest in the image data of the channels CH1 to CH4 is the nth frame, the motion vector of the nth frame is, for example, the next non-missing frame. It is necessary to detect by referring to the (n + 4) th frame.

  Therefore, in the normal screen mode, the signal processing chip 334 detects the motion vector for the normal screen mode, that is, detects the motion vector of the nth frame by referring to the next n + 1th frame. In the multi-screen mode, motion vector detection processing for the multi-screen mode, that is, the n + 4th frame which is the next non-missing frame is used as the motion vector of the n-th frame which is a non-missing frame. The detection process is performed by referring to.

  That is, when the operation mode is the normal screen mode, the signal processing chip 334 is supplied with, for example, image data of the channel CH1 output from the tuner unit 311 of FIG. 32. The image data of the channel CH1 is stored in the frame memory 341. And supplied to the motion vector detection circuit 344.

  The frame memory 341 temporarily stores the frame of the image data of the channel CH1 supplied thereto, thereby delaying it by one frame and supplying it to the delay circuit 342 and the selector 343. In the normal screen mode, the selector 343 uses the output of the frame memory 341 as an input to the motion detection circuit 344 in accordance with the normal screen motion vector detection command. The frame of the image data is supplied to the motion detection circuit 344 via the selector 343.

  From the above, when the image data of the channel CH1 of the (n + 4) th frame is supplied to the motion detection circuit 344, the frame memory 341 sends to the selector 343 the channel CH1 of the (n + 3) th frame before that frame. Image data is supplied. In this case, since the selector 343 supplies the frame CH1 image data frame from the frame memory 341 to the motion detection circuit 344, the image data of the channel CH1 of the (n + 3) th frame and the (n + 4) th frame, that is, the channel Assuming that a frame having image data of CH1 is an attention frame, the attention frame and a frame next to the attention frame are supplied to the motion detection circuit 344.

  The motion detection circuit 344 detects the motion vector of each pixel of the image data of the channel CH1 of the frame of interest by referring to the image data of the channel CH1 of the next frame of the frame of interest, and supplies it to the selection unit 345.

  In the normal screen mode, the selection unit 345 sets itself so as to output the motion vector supplied from the motion detection circuit 344 in response to the normal screen motion vector detection command. The motion vector supplied from 344 is output as it is as the motion vector for the normal screen mode.

  On the other hand, when the operation mode is the multi-screen mode, the signal processing chip 334 outputs to the signal processing chip 334 image data having a missing frame, for example, each of the channels CH1 to CH4 (hereinafter referred to as a missing frame as appropriate). The image data with missing frames of the channels CH1 to CH4 is supplied to the frame memory 341 and the motion vector detection circuit 344.

  The frame memory 341 temporarily stores the frames of the image data with the missing frames of the channels CH1 to CH4 supplied thereto, thereby delaying it by one frame and supplying it to the delay circuit 342 and the selector 343. The delay circuit 342 further delays the frames of image data with missing frames of the channels CH1 to CH4 supplied from the frame memory 341 by three frames, and supplies the delayed frames to the selector 343.

  In the multi-screen mode, the selector 343 uses the output of the delay circuit 342 as an input to the motion detection circuit 344 in response to the multi-screen motion vector detection command. The frame of image data with a missing frame of CH4 to CH4 is supplied to the motion detection circuit 344 via the selector 343.

  Here, as described above, when an image of the (n + 4) th frame is supplied to the motion detection circuit 344, an image of the nth frame four frames before that is supplied from the delay circuit 342 to the selector 343. In this case, since the selector 343 supplies the nth frame from the delay circuit 342 to the motion detection circuit 344, the image data of the nth frame and the n + 4th frame is supplied to the motion detection circuit 344. .

  In the multi-screen mode, the image data with missing frames of the channels CH1 to CH4 is supplied from the tuner unit 311 to the signal processing chip 334 as described above. The images with missing frames of the channels CH1 to CH4 are images in which there are non-missing frames every four frames if attention is paid to a certain channel. Therefore, the motion detection circuit 344 has a certain channel among the channels CH1 to CH4. If the non-defective frame is the target frame, the target frame that is the non-defective frame and the next non-defective frame (four frames after the target frame) are supplied to the motion detection circuit 344 for the channel. Is done.

  The motion detection circuit 344 detects the motion vector of each pixel of the frame of interest by referring to the next non-defective frame in the channel of the frame of interest, and supplies it to the selection unit 345.

  In the multi-screen mode, the selection unit 345 sets itself so as to reduce the motion vector supplied from the motion detection circuit 344 to ½ in response to the multi-screen motion vector detection command. Therefore, the motion vector supplied from the motion detection circuit 344 is reduced to ½, and the reduced motion vector is output as a motion vector for the multi-screen mode.

  In addition, when the operation mode is the multi-screen mode, the reason why the motion vector detected by the selection unit 345 for the pixels of the non-defective frame is halved is as follows.

  That is, in the multi-screen mode, the command sequence generation unit 330 transmits the multi-screen mode command sequence in the lower part of FIG. In this case, in the signal processing chip 331, as described above, the reduced image generation processing corresponding to the reduced image generation command in the multi-screen mode command sequence is performed on the image data with missing frames of the channels CH1 to CH4. Then, the reduced image data of the channels CH1 to CH4 obtained by reducing the size of the image data with the missing frame to the above-described small screen size (size obtained by equally dividing one frame into 2 × 2) is obtained.

  Note that, in the reduced image data of channels CH1 to CH4, there is a missing frame, similar to the image data with missing frames of channels CH1 to CH4 before the reduced image generation processing is performed. Hereinafter, the reduced image data having the missing frame is appropriately referred to as reduced image data having the missing frame.

  In the signal processing chip 331, the reduced image data with missing frames of the channels CH1 to CH4 obtained as described above is supplied to the signal processing chip 332. In the multi-screen mode, in the signal processing chip 332, the reduced image data with missing frames of the channels CH1 to CH4 supplied from the signal processing chip 331 is targeted for the reduced image data in the multi-screen mode command sequence (lower part of FIG. 36). Time resolution creation processing is performed in accordance with the time resolution creation command, and reduced image data with no missing frames (hereinafter, referred to as reduced image data without missing frames) of channels CH1 to CH4 is obtained.

  As described above, in the multi-screen mode, the signal processing chips 331 and 332 reduce the reduced image data of the small screen size, that is, the number of horizontal and vertical pixels of the image data of one frame to 1/2. The signal processing for handling the reduced image data of the (thinned out) size is performed.

  On the other hand, in the signal processing chip 334, since a motion vector is detected for image data with a missing frame of channels CH1 to CH4 output from the tuner unit 311 (unreduced image), the size of the motion vector is Theoretically, it is twice the size when a motion vector is detected for reduced image data. For this reason, in the multi-screen mode, the selection unit 345 of the signal processing chip 334 detects the motion vector detected for the image data (non-reduced image) data of the channels CH1 to CH4 for the reduced image data. In order to correspond to the motion vector, the signal is reduced to ½ and supplied to the signal processing chips 331 and 332.

  In step S314, the signal processing chip 334 determines whether there is image data to be subjected to signal processing. If it is determined in step S314 that there is image data to be subjected to signal processing in the signal processing chip 334, that is, if input image data is supplied from the tuner 311 to the signal processing chip 334, step S314 is performed. Returning to S313, the process of detecting the motion vector for the normal screen or the motion vector for the multi-screen is continued.

  If it is determined in step S314 that there is no image data to be subjected to signal processing in the signal processing chip 334, that is, input image data is no longer supplied from the tuner unit 311 to the signal processing chip 334. If so, the process ends.

  Note that the signal processing chip 334 temporarily terminates the processing according to the flowchart of FIG. 43 even when, for example, a new command sequence is transmitted from the command sequence generation unit 330, and starts again from step S311. To start.

  Here, as a motion vector detection method in the motion vector detection circuit 344, for example, a block matching method, a gradient method, or any other method can be employed.

  Further, in the above-described case, the motion vector is detected in units of pixels to obtain the motion vector for each pixel. However, the motion vector may be detected by, for example, a predetermined number of pixels or a predetermined number of pixels. This can be done in units of blocks composed of these pixels. In this case, a motion vector for an arbitrary pixel can be substituted with a motion vector detected for another nearby pixel or a motion vector detected for a block including the pixel.

  Next, FIG. 44 shows a configuration example of the signal processing chip 331 of FIG.

  The signal processing chip 331 switches the internal structure in accordance with commands composing the command sequence supplied from the command sequence generation unit 330 (FIG. 35), and performs spatial resolution creation processing or reduction using the class classification adaptive processing described above. Image generation processing (reduced image generation processing as resizing processing) is performed.

  That is, the signal processing chip 331 includes a target pixel selection unit 351, tap selection units 352 and 353, a class classification unit 354, a coefficient output unit 355, a prediction calculation unit 356, a reception unit 357, and a motion determination unit 358. Of these, the pixel-of-interest selection unit 351, tap selection units 352, 353, class classification unit 354, coefficient output unit 355, and prediction calculation unit 356 are the pixel-of-interest selection unit 11, tap selection units 12, 13 and class of FIG. These correspond to the classification unit 14, the coefficient output unit 15, and the prediction calculation unit 16, respectively.

  Therefore, in the signal processing chip 331, the first image data of the class classification adaptive process as the input image data supplied from the tuner unit 311 (FIG. 32) is converted into the second image data of the class classification adaptive process. Is output.

However, in FIG. 44, the coefficient output unit 355 includes coefficient memories 361 ₁ and 361 ₂ and a selection unit 362.

In the coefficient memories 361 ₁ and 361 ₂ , tap coefficients for spatial resolution creation processing and tap coefficients for reduced image generation processing as resizing processing, which are obtained by learning performed in advance, are stored. Here, the tap coefficient for the reduced image generation processing is used to reduce the size of the image data output from the tuner unit 311 (FIG. 32) to the size of a small screen (size obtained by equally dividing one frame into 2 × 2). It is.

The class (class code) output from the class classification unit 354 is supplied to the coefficient memories 361 ₁ and 361 ₂ . Each of the coefficient memories 361 ₁ and 361 ₂ reads the tap coefficient of the class from the class classification unit 354 and outputs it to the selection unit 362.

As described above, the selection unit 362 is supplied with commands from the reception unit 357 in addition to the tap coefficients read from the coefficient memories 361 ₁ and 361 ₂ . The selection unit 362 selects one of the output ends of the coefficient memories 361 ₁ or 361 ₂ according to the command supplied from the reception unit 357, and uses the selected output end as the input end of the prediction calculation unit 356. By connecting to, the internal structure of the signal processing chip 331 is switched.

Here, when the selection unit 362 selects, for example, the output end of the coefficient memory 361 ₁ out of the coefficient memory 361 ₁ or 361 ₂ and connects the selected output end to the input end of the prediction calculation unit 356 The tap coefficient for spatial resolution creation processing read from the coefficient memory 361 ₁ is supplied to the prediction calculation unit 356, and as a result, the signal processing chip 331 functions as an IC that performs spatial resolution creation processing. become.

On the other hand, when the selection unit 362 selects the output end of the coefficient memory 361 _{2 in} the coefficient memory 361 ₁ or 361 ₂ and connects the selected output end to the input end of the prediction calculation unit 356, the prediction calculation is performed. the section 356, becomes the tap coefficients for the reduced image generation processing to be read out from the coefficient memory 361 ₂ is supplied, as a result, the signal processing chip 331 will serve as an IC for performing reduced image generation processing.

  In FIG. 44, the class classification unit 354 is supplied with class determination from the tap selection unit 353 and motion determination information described later from the movement determination unit 358. The class classification unit 354 adds a bit representing the motion determination information from the motion determination unit 358 as an upper bit or a lower bit of a class code obtained by performing class classification based on the class tap selected for the target pixel. The bit string obtained as a result is supplied to the coefficient output unit 355 as a final class code.

  Further, in FIG. 44, the receiving unit 357 receives the command sequence transmitted wirelessly from the command sequence generating unit 330 (FIG. 35), and from the command sequence received from the command sequence generating unit 330, the signal processing chip. A signal processing command that can be executed by the 331 is recognized and supplied to the selection unit 362.

That is, when the receiving unit 357 receives the normal screen mode command sequence shown in FIG. 36, the receiving unit 357 recognizes the spatial resolution creation command therein as a signal processing command that can be executed by the signal processing chip 331 and selects it. Supplied to the unit 362. In this case, the selection unit 362 selects the output end of the coefficient memory 361 ₁ and connects it to the input end of the prediction calculation unit 356.

On the other hand, when receiving the multi-screen mode command sequence shown in the lower part of FIG. 36, the receiving unit 357 recognizes and selects a reduced image generation command as a signal processing command that can be executed by the signal processing chip 331. Supplied to the unit 362. In this case, selection section 362 selects the output terminal of the coefficient memory 361 ₂ are connected to the input terminal of the prediction computation unit 356.

  In FIG. 44, the motion determination unit 358 receives the motion vector of each pixel constituting the first image data as the input image data supplied from the tuner unit 311 (FIG. 32) from the signal processing chip 334 (FIG. 44). 35). The motion determination unit 358 includes, from the signal processing chip 334, the pixel of the first image data corresponding to the pixel of interest (for example, in the frame of the pixel of interest) out of the motion vector of each pixel constituting the first image data. The magnitude or direction of the motion vector of the pixel of the first image data that is closest to the spatio-temporal position of the pixel of interest is determined, and 1 bit or several bits representing the size or direction is used as motion determination information The data is supplied to the class classification unit 354.

  Next, the processing of the signal processing chip 331 in FIG. 44 will be described with reference to the flowchart in FIG.

  When the command sequence is transmitted from the command sequence generation unit 330 in FIG. 35, in step S321, the reception unit 357 receives the command sequence. Further, the reception unit 357 recognizes a signal processing command that can be executed by the signal processing chip 331 from the command sequence, supplies the command to the selection unit 362, and proceeds to step S322.

  In step S322, the selection unit 362 switches the internal structure of the signal processing chip 331 in accordance with a command from the reception unit 357.

In other words, the selection unit 362 selects one of the output ends of the coefficient memories 361 ₁ or 361 ₂ according to the command from the reception unit 357, and selects the selected output end as the prediction calculation unit 356 (FIG. 44). ), The internal structure of the signal processing chip 331 is switched.

  Thereafter, the process proceeds to step S323, and thereafter, in steps S323 to S328, the same processing as in steps S11 to S16 of FIG. 2 is performed.

  That is, in step S323, the pixel-of-interest selecting unit 351 selects one of the pixels constituting the second image data corresponding to the first image data input to the signal processing chip 331 that has not yet been set as the pixel of interest. Are selected as the target pixel, and the process proceeds to step S324.

  In step S324, the tap selection units 352 and 353 respectively select the prediction tap and the class tap for the target pixel from the first image data supplied thereto. The prediction tap is supplied from the tap selection unit 352 to the prediction calculation unit 356, and the class tap is supplied from the tap selection unit 353 to the class classification unit 354.

  Further, in step S324, the motion determination unit 358 moves the pixel of the first image data corresponding to the pixel of interest among the motion vectors of each pixel constituting the first image data from the signal processing chip 334. The size or direction of the vector is determined, and 1 bit or several bits representing the size or direction is supplied to the class classification unit 354 as motion determination information for the pixel of interest.

  The class classification unit 354 receives the class tap for the target pixel from the tap selection unit 353, and also receives the motion determination information for the target pixel from the motion determination unit 358. In step S325, the class tap and the motion determination are received. Based on the information, the target pixel is classified. Furthermore, the class classification unit 354 outputs the class of the target pixel obtained as a result of the class classification to the coefficient output unit 355, and the process proceeds to step S326.

In step S326, the coefficient output unit 355 outputs the class tap coefficient supplied from the class classification unit 354. That is, the coefficient output unit 355 reads the tap coefficient of the class supplied from the class classification unit 354 from the coefficient memory 361 ₁ or 361 ₂ from which the selection unit 362 has selected the output end, and performs the prediction calculation. Output to the unit 356. In step S326, the prediction calculation unit 356 acquires the tap coefficient output by the coefficient output unit 355, and the process proceeds to step S327.

  In step S327, the prediction calculation unit 356 performs the prediction calculation of Expression (1) using the prediction tap output from the tap selection unit 352 and the tap coefficient acquired from the coefficient output unit 355. Thereby, the prediction calculation unit 356 calculates and outputs the pixel value of the target pixel, and proceeds to step S328.

  In step S328, the pixel-of-interest selecting unit 351 determines whether there is second image data that has not yet been set as the pixel of interest. If it is determined in step S328 that there is still the second image data that is not the pixel of interest, the process returns to step S323, and the same processing is repeated thereafter.

  If it is determined in step S328 that there is no second image data that has not yet been set as the pixel of interest, the process ends.

When receiving the normal screen mode command sequence shown in FIG. 36 from the command sequence generation unit 330, the reception unit 357 supplies the spatial resolution creation command therein to the selection unit 362. In this case, selection unit 362, to choose from the output terminal of the coefficient memory 361 _1, the tap coefficients for the spatial resolution creation processing stored in the coefficient memory 361 ₁ is supplied to the prediction computation unit 356.

  Further, the case where the command sequence generation unit 330 transmits the normal screen mode command sequence shown in FIG. 36 is a case where the operation mode is the normal screen mode. In this case, the tuner unit 311 (FIG. 32) As a predetermined channel, for example, the channel CH1 is selected, whereby the signal processing chip 331 is supplied with image data having no missing frame of the channel CH1.

  Therefore, in this case, the signal processing chip 331 executes the spatial resolution creation process using the image data without the missing frame of the channel CH1 supplied thereto as the first image data, and the spatial resolution creation process performs the spatial resolution creation process. Is output to the subsequent signal processing chip 332 (FIG. 35) as the second image data.

On the other hand, when receiving the multi-screen mode command sequence shown in the lower part of FIG. 36 from the command sequence generation unit 330, the reception unit 357 supplies the reduced image generation command therein to the selection unit 362. In this case, selection unit 362, to choose from the output terminal of the coefficient memory 361 _2, the tap coefficients for the reduced image generation processing, which is stored in the coefficient memory 361 ₂ are supplied to the prediction computation unit 356.

  Further, the case where the command sequence generation unit 330 transmits the multi-screen mode command sequence shown in the lower part of FIG. 36 is a case where the operation mode is the multi-screen mode. In this case, the tuner unit 311 (FIG. 32) For example, channels CH1 to CH4 as a plurality of channels are selected in a time-sharing manner, whereby the image data with missing frames of channels CH1 to CH4 is supplied to the signal processing chip 331.

  Therefore, in this case, the signal processing chip 331 executes reduced image generation processing using the image data with missing frames of the channels CH1 to CH4 supplied thereto as the first image data, and performs size reduction by the reduced image generation processing. Is reduced (the number of pixels is reduced), and the reduced image data with missing frames of channels CH1 to CH4 is output as second image data to the signal processing chip 332 in the subsequent stage (FIG. 35).

  Next, FIG. 46 shows a configuration example of the signal processing chip 332 of FIG.

  The signal processing chip 332 switches its internal structure in accordance with the commands constituting the command sequence supplied from the command sequence generation unit 330 (FIG. 35), and performs noise removal processing or time resolution using the above-described class classification adaptive processing. Perform creative processing.

  That is, the signal processing chip 332 includes a memory 370, a target pixel selection unit 371, tap selection units 372 and 373, a class classification unit 374, a coefficient output unit 375, a prediction calculation unit 376, a reception unit 377, and tap determination units 378 and 379. It is configured. Of these, the target pixel selection unit 371, tap selection units 372 and 373, class classification unit 374, coefficient output unit 375, and prediction calculation unit 376 are the target pixel selection unit 11, tap selection units 12 and 13, and class in FIG. These correspond to the classification unit 14, the coefficient output unit 15, and the prediction calculation unit 16, respectively.

  Therefore, in the signal processing chip 332, the high-resolution image data without the missing frame of the channel CH1 or the channels CH1 to CH4 as the first image data of the class classification adaptive processing supplied from the signal processing chip 331 (FIG. 35). The reduced image data with missing frames is converted into second image data of the class classification adaptive processing and output.

However, in FIG. 46, the coefficient output unit 375 includes coefficient memories 381 ₁ and 381 ₂ and a selection unit 382.

The coefficient memories 381 ₁ and 381 ₂ store tap coefficients for noise removal processing and tap coefficients for time resolution creation processing, which are obtained by learning performed in advance. Here, the tap coefficient for the time resolution creation processing is such that reduced image data with missing frames of the channels CH1 to CH4 supplied from the signal processing chip 331 is reduced image data without missing frames.

The class (class code) output from the class classification unit 374 is supplied to the coefficient memories 381 ₁ and 381 ₂ . Each of the coefficient memories 381 ₁ and 381 ₂ reads out the tap coefficient of the class from the class classification unit 374 and outputs it to the selection unit 382.

As described above, the selection unit 382 is supplied with commands from the reception unit 377 in addition to the tap coefficients read from the coefficient memories 381 ₁ and 381 ₂ . The selection unit 382 selects one of the output ends of the coefficient memories 3811 ₁ or 3812 ₂ in accordance with the command supplied from the reception unit 377, and selects the selected output end as the input end of the prediction calculation unit 376. By connecting to, the internal structure of the signal processing chip 332 is switched.

Here, when the selection unit 382 selects, for example, the output end of the coefficient memory 3811 ₁ out of the coefficient memory 381 ₁ or 381 ₂ and connects the selected output end to the input end of the prediction calculation unit 376 , the prediction computation unit 376, it is possible to tap coefficients for noise removal processing which is read out from the coefficient memory 381 ₁ is supplied, as a result, the signal processing chip 332, will function as an IC for performing a noise removal process .

On the other hand, when the selection unit 382 selects the output terminal of the coefficient memory 3812 _{2 in} the coefficient memory 381 ₁ or 381 ₂ and connects the selected output terminal to the input terminal of the prediction calculation unit 376, the prediction calculation is performed. the section 376, becomes the tap coefficients for the temporal resolution creation processing which is read out from the coefficient memory 381 ₂ is supplied, as a result, the signal processing chip 332, will function as an IC for performing time resolution creation processing.

  In FIG. 46, the receiving unit 377 receives the command sequence transmitted wirelessly from the command sequence generating unit 330 (FIG. 35), and from the command sequence received from the command sequence generating unit 330, the signal processing chip. A signal processing command that can be executed by the 332 is recognized and supplied to the selection unit 382.

That is, when receiving the normal screen mode command sequence shown in FIG. 36, the receiving unit 377 recognizes the noise removal command therein as a signal processing command that can be executed by the signal processing chip 332, and selects the selecting unit. 382. In this case, selection section 382 selects the output terminal of the coefficient memory 381 ₁ is connected to the input of the prediction computation unit 376.

On the other hand, when receiving the multi-screen mode command sequence shown in the lower part of FIG. 36, the receiving unit 377 recognizes the time resolution creation command therein as a signal processing command that can be executed by the signal processing chip 332 and selects it. Supplied to the unit 382. In this case, selection section 382 selects the output terminal of the coefficient memory 381 ₂ are connected to the input terminal of the prediction computation unit 376.

  Further, in FIG. 46, the tap determination units 378 and 379 include movements of pixels constituting the input image data supplied from the signal processing chip 334 (FIG. 35) to the signal processing unit 314 by the tuner unit 311 (FIG. 32). Vectors are supplied.

  The tap determination unit 378 or 379 determines a pixel to be a prediction tap or a class tap for the target pixel based on the motion vector supplied from the signal processing chip 334, respectively.

  That is, what kind of pixel is selected and configured from the first image data of the class classification adaptation process for the prediction tap and the class tap (hereinafter, collectively referred to as “tap” as appropriate, as appropriate) for the target pixel. Is not particularly limited.

  However, in the signal processing chip 332 of FIG. 46, reduction without missing frames is performed by performing class classification adaptive processing using tap coefficients for temporal resolution creation processing using reduced image data with missing frames as first image data. Temporal resolution creation processing may be performed to obtain image data as second image data.

In the temporal resolution creation process, reduced image data of each frame of the frame period T ₁ is predicted (generated) for each of the four channels CH1 to CH4. For the prediction, only the pixels of one non-defective frame are used for the prediction. Than the non-defective frame of a plurality of frames, for example, two non-defective frames closer to the frame of the target pixel (the pixel generated by prediction), that is, the non-defective frame immediately before the frame of the target pixel (however, If the frame of the pixel of interest is a non-defective frame, it is desirable to use the non-defective frame) and the pixel of the non-defective frame immediately after that. Furthermore, it is desirable to consider the movement of the image (subject) in the prediction of the pixel of interest in the time resolution creation process.

  Therefore, for example, the tap determination unit 378 selects a pixel corresponding to the target pixel from among the pixels constituting the reduced image data of the non-defective frame immediately before the frame of the target pixel (hereinafter, referred to as the previous non-defective frame as appropriate). Based on the motion vector, from the pixels constituting the reduced image data of the immediately preceding non-missing frame and the pixels constituting the reduced image data of the non-missing frame immediately after the frame of the target pixel (hereinafter referred to as the immediately following non-missing frame). Then, a pixel to be a prediction tap for the target pixel is determined.

  That is, the tap determination unit 378 arranges the immediately preceding non-missing frame, the frame of the target pixel, and the immediately following non-missing frame in time series so that the motion vector of the pixel of the immediately preceding non-missing frame corresponding to the target pixel passes through the target pixel. When corrected, some pixels of the immediately preceding non-missing frame centered on the start point of the corrected motion vector and some pixels of the immediately following non-missing frame centered on the end point of the corrected motion vector The pixel to be a prediction tap for the target pixel is determined.

  Here, when the time resolution creation processing is performed in the signal processing chip 332, the motion vector supplied from the signal processing chip 334 is not the motion vector of each pixel of the reduced image data of the immediately preceding non-defective frame or the immediately following non-defective frame. The motion vector of the image data before being reduced to the reduced image data (hereinafter referred to as pre-reduction image data as appropriate) (the size of which is halved). Therefore, the tap determination unit 378 sets the motion vector of the pixel of the pre-reduction image data corresponding to the pixel of the reduced image data to the same positional relationship as the pixel of the reduced image data (the position of the pixel of the reduced image data in the image frame). Used as the motion vector of the pixel at the position of the image frame of the pre-reduction image data.

  The tap determination unit 378 determines a pixel to be a prediction tap for the pixel of interest, and then supplies information representing the pixel (hereinafter, appropriately referred to as prediction tap information) to the tap selection unit 372. Then, the tap selection unit 372 selects a pixel to be a prediction tap from the image data stored in the memory 370 according to the prediction tap information from the tap determination unit 378.

  In the same manner as the tap determination unit 378, the tap determination unit 379 also determines a pixel to be a class tap for the pixel of interest, and transmits information representing the pixel (hereinafter referred to as class tap information as appropriate) to the tap selection unit 373. Supply. Then, the tap selection unit 373 selects a pixel to be a class tap from the image data stored in the memory 370 according to the class tap information from the tap determination unit 379.

  Here, in the signal processing chip 332 of FIG. 46, from the above, a tap is selected from two frames of image data supplied from the signal processing chip 331. For this reason, the signal processing chip 332 in FIG. 46 is provided with a memory 370 for temporarily storing frames of image data supplied from the signal processing chip 331.

  Further, in the signal processing chip 332 of FIG. 46, the time resolution creation processing is performed for each of the four channels CH1 to CH4. In this case, while the time resolution creation process is being performed for any one of the channels CH1 to CH4, it is necessary to hold the image data of the other three channels. The memory 370 is provided to hold the image data.

  In addition to the time resolution creation process, the signal processing chip 332 in FIG. 46 performs the noise removal process as described above. Even when the noise removal process is performed, the tap determination units 378 and 379 determine the pixel to be the tap, as in the case where the time resolution creation process is performed.

  However, since the noise removal processing is performed on high-resolution image data without a missing frame of channel CH1, the immediately preceding non-missing frame is the frame of the target pixel, and the immediately following non-missing frame is the frame of the target pixel. Next frame.

  Next, processing of the signal processing chip 332 in FIG. 46 will be described with reference to the flowchart in FIG.

  When the command sequence is transmitted from the command sequence generation unit 330 in FIG. 35, in step S331, the reception unit 377 receives the command sequence. Further, the reception unit 377 recognizes a signal processing command that can be executed by the signal processing chip 332 from the command sequence, supplies the command to the selection unit 382, and proceeds to step S332.

  In step S <b> 332, the selection unit 382 switches the internal structure of the signal processing chip 332 according to the command from the reception unit 377.

In other words, the selection unit 382 selects one of the output ends of the coefficient memories 381 ₁ or 381 ₂ according to the command from the reception unit 377, and selects the selected output end as the prediction calculation unit 376 (FIG. 46). ), The internal structure of the signal processing chip 332 is switched.

  Thereafter, the process proceeds to step S333. Thereafter, in steps S333 to S338, the same processing as in steps S11 to S16 of FIG. 2 is performed.

  That is, the high-resolution image data without the missing frame of the channel CH1 input from the signal processing chip 331 to the signal processing chip 332 or the reduced image data with the missing frames of CH1 to CH4 is the first image data of the class classification adaptive processing. Are sequentially supplied to and stored in the memory 370.

  In step S333, the pixel-of-interest selecting unit 371 selects, as a pixel of interest, one of the pixels constituting the second image data of the class classification adaptation process that has not yet been selected as the pixel of interest. The process proceeds to S334.

  In step S334, the tap determination unit 378 or 379 determines a pixel of the first image data to be a prediction tap or a class tap based on the motion vector supplied from the signal processing chip 334 (FIG. 35), and the pixel The prediction tap information or the class tap information representing is supplied to the tap selection unit 372 or 373, respectively.

  Further, in step S334, the tap selection units 372 and 373 determine the target pixel from the first image data stored in the memory 370 according to the prediction tap information and the class tap information supplied from the tap determination units 378 and 379. Select the prediction taps and class taps. The prediction tap is supplied from the tap selection unit 372 to the prediction calculation unit 376, and the class tap is supplied from the tap selection unit 373 to the class classification unit 374.

  The class classification unit 374 receives the class tap for the target pixel from the tap selection unit 373, and classifies the target pixel based on the class tap in step S335. Further, the class classification unit 374 outputs the class of the target pixel obtained as a result of the class classification to the coefficient output unit 375, and the process proceeds to step S336.

In step S336, the coefficient output unit 375 outputs the tap coefficient of the class supplied from the class classification unit 374. That is, the coefficient output unit 375 reads out the tap coefficient of the class supplied from the class classification unit 374 from the coefficient memory 381 ₁ or 381 ₂ from which the selection unit 382 has selected the output terminal, and performs prediction calculation To the unit 376. In step S336, the prediction calculation unit 376 acquires the tap coefficient output by the coefficient output unit 375, and the process proceeds to step S337.

  In step S <b> 337, the prediction calculation unit 376 performs the prediction calculation of Expression (1) using the prediction tap output from the tap selection unit 372 and the tap coefficient acquired from the coefficient output unit 375. Thereby, the prediction calculation part 376 calculates | requires and outputs the pixel value of an attention pixel, and progresses to step S338.

  In step S338, the pixel-of-interest selection unit 371 determines whether there is second image data that has not yet been set as the pixel of interest. If it is determined in step S338 that there is still second image data that is not the pixel of interest, the process returns to step S333, and the same processing is repeated thereafter.

  If it is determined in step S338 that there is no second image data that has not yet been set as the target pixel, the process ends.

When receiving the normal screen mode command sequence shown in FIG. 36 from the command sequence generation unit 330, the reception unit 377 supplies the noise removal command therein to the selection unit 382. In this case, selection unit 382, to choose from the output terminal of the coefficient memory 381 _1, the tap coefficients for noise removal processing which is stored in the coefficient memory 381 ₁ is supplied to the prediction computation unit 376.

  Further, the case where the command sequence generation unit 330 transmits the normal screen mode command sequence shown in FIG. 36 is a case where the operation mode is the normal screen mode. In this case, the tuner unit 311 (FIG. 32) For example, channel CH1 is selected as a predetermined channel. Then, in the signal processing chip 331 (FIG. 35), the image data of the channel CH1 is converted into high-resolution image data, whereby the signal processing chip 332 is supplied with the high-resolution image data of the channel CH1.

  Therefore, in this case, the signal processing chip 332 executes the noise removal process using the high-resolution image data of the channel CH1 supplied thereto as the first image data, and the S / N is improved by the noise removal process. The high-resolution image data of the channel CH1 is output to the subsequent memory unit 333 (FIG. 35) as the second image data.

On the other hand, when receiving the multi-screen mode command sequence shown in the lower part of FIG. 36 from the command sequence generation unit 330, the reception unit 377 supplies the time resolution creation command therein to the selection unit 382. In this case, selection unit 382, to choose from the output terminal of the coefficient memory 381 _2, the tap coefficients for the temporal resolution creation processing, which is stored in the coefficient memory 381 ₂ are supplied to the prediction computation unit 376.

  Further, the case where the command sequence generation unit 330 transmits the multi-screen mode command sequence shown in the lower part of FIG. 36 is a case where the operation mode is the multi-screen mode. In this case, the tuner unit 311 (FIG. 32) For example, channels CH1 to CH4 as a plurality of channels are selected in a time division manner. Then, in the signal processing chip 331 (FIG. 35), the image data with the missing frame of the channels CH1 to CH4 is converted into the reduced image data with the missing frame, whereby the signal processing chip 332 has the channels CH1 to CH4. Reduced image data with missing frames is supplied.

  Therefore, in this case, the signal processing chip 332 executes the time resolution creation process using the reduced image data with missing frames of the channels CH1 to CH4 supplied thereto as the first image data, and the time resolution creation process Reduced image data of channels CH1 to CH4 having no missing frame is obtained as second image data, and is output to the subsequent memory unit 333 (FIG. 35).

  As described above, in each of the signal processing chips 331, 332, and 334 of FIG. 35, the internal structure is switched in accordance with at least one command in the command sequence, and the image data input thereto is subjected to signal processing. Since the image data as a result of the signal processing is output, a plurality of functions can be easily realized with a single hardware.

  Furthermore, since the signal processing chip 332 performs signal processing on the signal processing result of the signal processing chip 331, the signal processing unit 314 (FIG. 35) as a whole can easily realize more functions.

  Note that the signal processing unit 314 can be configured by providing an arbitrary number of signal processing chips similar to the signal processing chips 331 and 332.

  In the above case, the coefficient output unit 355 of the signal processing chip 331 (FIG. 44) stores the tap coefficient. However, instead of the tap coefficient, the coefficient seed data is stored. It is possible to generate a tap coefficient from the coefficient seed data. The same applies to the signal processing chip 332.

  Next, a third embodiment of the present invention will be described.

  FIG. 48 shows a configuration example of a television receiver as the third embodiment of the present invention.

  The tuner 601 is supplied with a broadcast signal (transmission signal) of a digital broadcast received by an antenna (not shown). The broadcast signal of this digital broadcast is digital data defined by MPEG (Moving Picture Experts Group) -2, etc., and is transmitted as a transport stream composed of a plurality of TS (Transport Stream) packets. come. Based on the control of the controller 613, the tuner 601 selects (selects) a broadcast signal of a predetermined channel (frequency) from broadcast signals of a plurality of channels supplied from the antenna, and broadcasts the selected channel. The signal is supplied to the demodulator 602.

  Based on the control of the controller 613, the demodulator 602 demodulates the transport signal of the broadcast signal of the predetermined channel supplied from the tuner 601 by, for example, QPSK (Quadrature Phase Shift Keying) demodulation and the like. The transport stream is supplied to the error correction unit 603.

  The error correction unit 603 detects and corrects an error in the transport stream supplied from the demodulation unit 602 based on the control of the controller 613. Then, the corrected transport stream is supplied to a demultiplexer (DEM) 604.

  The demultiplexer 604 descrambles the transport stream supplied from the error correction unit 603 as necessary based on the control of the controller 613. Further, the demultiplexer 604 extracts a TS packet of a predetermined program from the transport stream supplied from the error correction unit 603 by referring to the TS packet PID (Packet Identifier) based on the control of the controller 613. Then, the TS packet of video data is supplied to the video decoder 605, and the TS packet of audio data is supplied to the audio decoder 610, respectively.

  The video decoder 605 decodes the video data (its TS packet) supplied from the demultiplexer 604 using the MPEG-2 system, and supplies the decoded data to the LSI 606 or the combining unit 607.

  The LSI 606 receives the command sequence supplied from the controller 613, and switches the internal structure in a reconfigurable manner according to the command sequence, similar to the R-IC 212 of the first embodiment. The LSI 606 performs signal processing on the image data (video data) output from the video decoder 605 and supplies (outputs) the image data obtained by the signal processing to the combining unit 607.

  When image data is supplied from the LSI 606, the combining unit 607 selects the image data. On the other hand, when image data is not supplied from the LSI 606, the synthesis unit 607 selects image data supplied from the video decoder 605. The synthesizing unit 607 superimposes the image data supplied from the OSD (On Screen Display) unit 608 on the selected one of the image data supplied from the video decoder 605 or the LSI 606, and displays an LCD (Liquid Crystal Display). 609 to display.

  When image data is not supplied from the OSD unit 608, the synthesizing unit 607 supplies the selected one of the image data supplied from the video decoder 605 or the LSI 606 to the LCD 609 as it is.

  Here, the OSD unit 608 generates, for example, image data such as the number and volume of the currently selected channel based on the control of the controller 613, and supplies the image data to the combining unit 607.

  On the other hand, the audio decoder 610 decodes the audio data (its TS packet) supplied from the demultiplexer 604 by the MPEG-2 system, supplies it to an output terminal (not shown), and supplies it to the speaker 611 for output.

  The controller 613 controls the tuner 601, the demodulator 602, the error corrector 603, the demultiplexer 604, the video decoder 605, the audio decoder 610, and the OSD unit 608. Further, the controller 613 executes various processes based on operation signals corresponding to user operations supplied from the key input unit 614 and the remote control I / F 617.

  For example, the controller 613 generates a command sequence including one or more commands in response to an operation signal supplied from the remote control I / F 617 and corresponding to a user operation, and transmits (supplies) to the LSI 606.

  The key input unit 614 includes, for example, a switch button, receives an operation when the user selects a desired channel, and supplies an operation signal corresponding to the user's operation to the controller 613. The display unit 615 displays, for example, the channel selected by the tuner 601 based on the control signal supplied from the controller 613.

  A remote control I / F (InterFace) 617 supplies an operation signal supplied from the light receiving unit 616 corresponding to a user operation to the controller 613. The light receiving unit 616 receives (receives) an operation signal corresponding to a user operation transmitted from the remote controller (remote commander 618), and supplies the received signal to the remote controller I / F 617.

  Next, FIG. 49 is a plan view showing a configuration example of the remote control 618 of FIG.

  The remote control 618 is provided with a user interface unit 621 and an LCD panel 622.

  The user interface unit 621 is operated when the power button 612A is operated when turning on / off the power of the television receiver in FIG. 48, and when the input to the television receiver is switched to television broadcasting or external input. An input button 621B, a channel button 621C that is operated when selecting a channel, and a function button 621D that is operated when a predetermined function is requested to the television receiver (LSI 606).

  In FIG. 49, four buttons of functions A, B, C, and D are provided as function buttons 621D.

  The LCD panel 622 displays predetermined information, for example, information indicating a button of the user interface unit 621 operated immediately before.

  Next, FIG. 50 shows an example of the electrical configuration of the remote control 618 of FIG.

  The remote control 618 includes a user interface unit 621, an LCD panel 622, a control unit 632, a storage unit 633, and a transmission unit 634 that are connected to each other via a bus 631.

  The control unit 632 is configured by, for example, a CPU and controls each block configuring the remote controller 618 by executing a program stored in the storage unit 633. The storage unit 633 stores programs executed by the control unit 632 and necessary data. The storage unit 633 stores data necessary for the control unit 632 and the like to perform processing.

  When the user operates the user interface unit 621, the transmission unit 634 transmits an operation signal corresponding to the operation by infrared rays or radio waves. The operation signal transmitted by the transmission unit 634 is received by the light receiving unit 616 (FIG. 48).

  Next, the processing of the controller 613 in FIG. 48 will be outlined with reference to the flowchart in FIG.

  In step S601, the controller 613 determines whether the user interface unit 621 (FIG. 49) of the remote controller 618 has been operated by the user. If it is determined in step S601 that the user interface unit 621 has not been operated, the process returns to step S601.

  If it is determined in step S601 that the user interface unit 621 has been operated, that is, the user operates the user interface unit 621, and an operation signal corresponding to the operation causes the light receiving unit 616 and the remote control I / F 617 to operate. In step S602, the controller 613 determines whether the power button 621A (FIG. 49) has been operated based on the operation signal.

  When it is determined in step S602 that the power button 621A has been operated, that is, the user operates the power button 621A, and an operation signal corresponding to the operation is transmitted to the controller via the light receiving unit 616 and the remote control I / F 617. If the power is supplied to 613, the process proceeds to step S603, where the controller 613 performs power processing for turning on or off the power of the television receiver in FIG. 48, and returns to step S601.

  If it is determined in step S602 that the power button 621A is not operated, that is, if a button other than the power button 621A is operated, the process proceeds to step S604, and the controller 613 corresponds to the operated button. To return to step S601.

  Here, when the function button 621D of the remote control 618 (FIG. 49) is operated and the controller 613 receives an operation signal corresponding to the operation of the function button 621D via the light receiving unit 616 and the remote control I / F 617, the controller In response to the operation signal (operated function button 621D), 613 generates a command string and transmits it to the LSI 606.

  FIG. 52 shows the format of a command string generated by the controller 613.

  For example, the command string is configured by sequentially arranging a header, one or more commands, and an EOC (End Of Command) code.

  Next, with reference to the flowchart of FIG. 53, the process of step S604 of FIG. 51 by the controller 613 will be further described.

  In step S611, the controller 613 determines whether or not the function A button (hereinafter referred to as A button as appropriate) of the function buttons 621D (FIG. 49) has been operated. If it is determined in step S611 that the A button has been operated, the process proceeds to step S612, where the controller 613 generates a one-dimensional spatial resolution creation command or a two-dimensional spatial resolution creation command as one or more commands in the command sequence shown in FIG. A command sequence in which commands are arranged is generated, transmitted to the LSI 606, and the process returns.

  If it is determined in step S611 that the A button has not been operated, the process proceeds to step S613, where the controller 613 calls the function B button (hereinafter referred to as the B button as appropriate) of the function buttons 621D (FIG. 49). ) Has been operated.

  If it is determined in step S613 that the B button has been operated, the process proceeds to step S614, where the controller 613 generates a command sequence in which a noise removal command is arranged as one or more commands in the command sequence shown in FIG. Transmit to LSI 606 and return.

  If it is determined in step S613 that the B button has not been operated, the process proceeds to step S615, and the controller 613 determines that the function C button of the function buttons 621D (FIG. 49) (hereinafter referred to as C button as appropriate). ) Has been operated.

  If it is determined in step S615 that the C button has been operated, the process proceeds to step S616, where the controller 613 performs a one-dimensional spatial resolution creation command, a noise removal command, and one or more commands in the command sequence shown in FIG. Is generated, transmitted to the LSI 606, and the process returns.

  If it is determined in step S615 that the C button has not been operated, the process proceeds to step S617, where the controller 613 selects the function D button (hereinafter referred to as the D button as appropriate) of the function buttons 621D (FIG. 49). ) Has been operated.

  If it is determined in step S617 that the D button has been operated, the process proceeds to step S618, where the controller 613 performs a two-dimensional spatial resolution creation command, a noise removal command, and one or more commands in the command sequence illustrated in FIG. Is generated, transmitted to the LSI 606, and the process returns.

  If it is determined in step S617 that the D button has not been operated, that is, if any button other than the power button 621A and the function button 621D has been operated, the process proceeds to step S619, and the controller 613 has been operated. Performs processing corresponding to the button and returns.

  Next, FIG. 54 shows a command sequence generated by the controller 613 in response to the operation of the function button 621D and transmitted to the LSI 606.

  When the A button is operated, the controller 613 has a command in which a one-dimensional spatial resolution creation command or a two-dimensional spatial resolution creation command as one command is arranged, as shown first or second from the top on the left side of FIG. A button command sequence 641 or 642 as a sequence is generated and transmitted to the LSI 606.

  When the B button is operated, the controller 613 generates a B button command sequence 643 that is a command sequence in which a noise removal command that is one command is arranged, as shown in the third from the upper left in FIG. , Transmitted to the LSI 606.

  Further, when the C button is operated, the controller 613 is a command sequence in which two commands, a one-dimensional spatial resolution creation command and a noise removal command, are arranged as shown first from the upper right in FIG. A C button command string 644 is generated and transmitted to the LSI 606.

  When the D button is operated, the controller 613 is a command sequence in which two commands, a two-dimensional spatial resolution creation command and a noise removal command, are arranged as shown in the second from the top in FIG. A D button command string 645 is generated and transmitted to the LSI 606.

  Next, FIG. 55 shows a configuration example of the LSI 606 of FIG.

  The receiving unit 650 receives a command sequence transmitted from the controller 613 (FIG. 48) and supplies the command sequence to the SW (switch) circuits 654 and 655 and the signal processing circuit 656.

  The frame memory 651 stores the image data (image input signal) supplied from the video decoder 605 (FIG. 48), for example, in units of one frame, and supplies it to the input terminal 654A of the SW circuit 654.

  The frame memory 652 stores the image data (image output signal) supplied from the signal processing circuit 656 via the output terminal 655A of the SW circuit 655, for example, in units of one frame, and supplies it to the synthesis unit 607 (FIG. 48). To do.

  The frame memory 653 stores the image data supplied from the output terminal 655B of the SW circuit 655, for example, in units of one frame, and supplies the image data to the input terminal 654B of the SW circuit 654.

  The SW circuit 654 has two input terminals 654A and 654B. The input terminal 654A is supplied with image data stored in the frame memory 651, and the input terminal 654B is stored in the frame memory 653. Image data is supplied. For example, the SW circuit 654 selects one of the input terminals 654A or 654B in accordance with a command sequence supplied from the receiving unit 650, and the image data supplied from the selected input terminal is converted into a signal. This is supplied to the processing circuit 656.

  The SW circuit 655 has two output terminals 655A and 655B. The output terminal 655A is connected to the frame memory 652, and the output terminal 655B is connected to the frame memory 653. The SW circuit 655 is supplied with the image data after the signal processing from the signal processing circuit 656, and the SW circuit 655 outputs, for example, according to the command string supplied from the receiving unit 650 or the like. One of the terminals 655A and 655B is selected, and the image data from the signal processing circuit 656 is output to the selected output terminal.

  The signal processing circuit 656 switches its internal structure to reconfigurable according to the command string from the receiving unit 650, performs signal processing on the image data supplied from the SW circuit 654, and obtains image data obtained by the signal processing. Is supplied to the SW circuit 655.

  Next, processing of the LSI 606 in FIG. 55 will be described with reference to the flowchart in FIG.

  When a command string is transmitted from the controller 613 (FIG. 48) to the LSI 606, the receiving unit 650 receives the command string and supplies it to the SW circuits 654 and 655 and the signal processing circuit 656 in step S631. Then, the process proceeds to step S632.

  Here, the command sequence received by the receiving unit 650 is assumed to be composed of N commands. However, N is an integer of 1 or more.

  In step S632, the frame memory 651 stores the image data of one frame (field) supplied from the video decoder 605 (FIG. 48), and the process proceeds to step S633.

  In step S633, the SW circuits 654 and 655 determine whether the command sequence supplied from the receiving unit 650 is one or a plurality of commands.

  If it is determined in step S633 that the number of commands constituting the command string is one, the process proceeds to step S634, where the SW circuit 654 selects the input terminal 654A and the SW circuit 655 selects the output terminal 655A. The process proceeds to S635.

  In step S635, the signal processing circuit 656 selects a command to be a command of interest from commands composing a command string from the receiving unit 650. Here, in this case, since there is one command constituting the command string, that one command is selected as the attention command.

  Further, in step S635, the signal processing circuit 656 switches the internal structure in accordance with the command of interest, so that the signal processing corresponding to the command of interest can be executed, and the process proceeds to step S636. .

  In step S636, the signal processing circuit 656 reads out one frame of image data from the frame memory 651 via the input terminal 654A selected by the SW circuit 654, and uses the one frame of image data as a target command. Perform the corresponding signal processing.

  Here, in step S636, the signal processing circuit 656 performs signal processing at an N-times speed equal to the number N of commands constituting the command sequence supplied from the receiving unit 650. In this case, since the command sequence is composed of one command, the signal processing circuit 656 has a 1 × speed, that is, a processing speed corresponding to the frame rate (or field rate) (within a time equal to the frame period, 1 The image data of one frame is signal-processed at a processing speed of completing the signal processing of the image data of the frame.

  The image data obtained by the signal processing by the signal processing circuit 656 in step S636 is supplied to the SW circuit 655, and the process proceeds to step S637.

  Here, since the SW circuit 655 selects the output terminal 655A in step S634, the image data supplied from the signal processing circuit 656 to the SW circuit 655 is supplied from the output terminal 655A to the frame memory 652.

  In step S637, the frame memory 652 stores the image data supplied from the signal processing circuit 656 via the SW circuit 655, and outputs the stored image data to the synthesis circuit 607 (FIG. 48). The process proceeds to S650.

  On the other hand, if it is determined in step S633 that there are a plurality of commands constituting the command sequence, the process proceeds to step S638, the SW circuit 654 selects the input terminal 654A, and the SW circuit 655 selects the output terminal 655B. The process proceeds to step S639.

  In step S639, the signal processing circuit 656 selects a command to be a command of interest from commands composing a command string from the receiving unit 650. That is, for example, the signal processing circuit 656 selects, as the attention command, the command closest to the head that has not yet been regarded as the attention command among the plurality of commands constituting the command string.

  Further, in step S639, the signal processing circuit 656 switches the internal structure in accordance with the command of interest, thereby enabling the signal processing corresponding to the command of interest to be executed, and the process proceeds to step S640. .

  In step S640, the signal processing circuit 656 reads one frame of image data from the frame memory 651 via the terminal 654A selected by the SW circuit 654, and responds to the command of interest for the one frame of image data. The signal processing is performed at N times speed equal to the number N of commands constituting the command sequence.

  In step S640, the image data obtained by the signal processing by the signal processing circuit 656 is supplied to the SW circuit 655, and the process proceeds to step S641.

  Here, since the SW circuit 655 selects the output terminal 655B in step S638, the image data supplied from the signal processing circuit 656 to the SW circuit 655 is supplied from the output terminal 655B to the frame memory 653.

  In step S641, the frame memory 653 stores the image data supplied from the signal processing circuit 656 via the SW circuit 655, and the process proceeds to step S642.

  In step S642, the SW circuit 654 switches the selection from the input terminal 654A to 654B, and proceeds to step S643.

  In step S643, the signal processing circuit 656 newly selects a command to be a command of interest from the commands constituting the command sequence from the reception unit 650, as in step S639.

  Further, in step S643, the signal processing circuit 656 switches the internal structure in accordance with the new attention command, thereby enabling the signal processing corresponding to the attention command to be executed, and step S644. Proceed to

  In step S644, the signal processing circuit 656 determines whether the command of interest is the last command constituting the command string.

  If it is determined in step S644 that the command of interest is not the last command constituting the command string, that is, if there is a command that has not yet been noticed in the command string, the process proceeds to step S645, where the signal processing circuit Reference numeral 656 reads out image data from the frame memory 653 via the input terminal 654B selected by the SW circuit 654, that is, reads out image data obtained by the previous signal processing by the signal processing circuit 656. , The signal processing corresponding to the command of interest is performed at an N-times speed equal to the number N of commands constituting the command sequence.

  In step S645, the image data obtained by the signal processing by the signal processing circuit 656 is supplied to the SW circuit 655, and the process proceeds to step S646.

  Here, since the SW circuit 655 keeps selecting the output terminal 655B in step S638, the image data supplied from the signal processing circuit 656 to the SW circuit 655 is supplied from the output terminal 655B to the frame memory 653. The

  In step S646, the frame memory 653 stores (overwrites) the image data supplied from the signal processing circuit 656 via the SW circuit 655. Then, the process returns to step S643, and the same processing is repeated thereafter.

  If it is determined in step S644 that the command of interest is the last command constituting the command string, the process proceeds to step S647, and the SW circuit 655 switches the selection from the output terminal 655B to 655A, and then proceeds to step S648. move on.

  In step S648, the signal processing circuit 656 reads the image data from the frame memory 653 via the terminal 654B selected by the SW circuit 654, that is, the image data obtained by the previous signal processing by the signal processing circuit 656. And the signal processing corresponding to the command of interest is performed at the N-times speed equal to the number N of commands constituting the command sequence.

  In step S648, the image data obtained by the signal processing by the signal processing circuit 656 is supplied to the SW circuit 655, and the process proceeds to step S649.

  Here, since the SW circuit 655 has selected the output terminal 655A in step S647, the image data supplied from the signal processing circuit 656 to the SW circuit 655 is supplied from the output terminal 655A to the frame memory 652.

  In step S649, the frame memory 652 stores the image data supplied from the signal processing circuit 656 via the SW circuit 655, and outputs the stored image data to the synthesis circuit 607 (FIG. 48). The process proceeds to S650.

  In step S650, it is determined whether the image data of the next frame has been supplied to the frame memory 651. If it is determined in step S650 that the image data of the next frame has been supplied to the frame memory 651, the process returns to step S632, the frame memory 651 stores the image data, and the same processing is repeated thereafter. At this time, it is assumed that all commands constituting the command sequence have not yet been set as the command of interest.

  On the other hand, if it is determined in step S650 that the image data of the next frame has not been supplied to the frame memory 651, the process ends.

  As described above, when the command string is composed of a plurality of commands, the LSI 606 switches the internal structure according to the command of interest, executes signal processing corresponding to the command of interest on the image data, and thereafter Further, the internal structure is switched according to the new attention command, and the signal processing corresponding to the new attention command is executed on the image data obtained by the previous signal processing. Therefore, a plurality of functions can be easily realized with a single hardware.

  Next, FIG. 57 shows a configuration example of the signal processing circuit 656 of FIG.

  Image data is supplied from the SW circuit 654 (FIG. 55) to the scanning line conversion circuit 521A. The scanning line conversion circuit 521B converts the scanning direction of the image data supplied thereto and supplies it to the switching circuit 662.

  Image data after signal processing by the signal processing circuit 661 is supplied from the switching circuit 664 to the scanning line conversion circuit 521B. The scanning line conversion circuit 521B converts the scanning direction of the image data supplied thereto and supplies it to the switching circuit 663.

  Image data is supplied to the frame memory 523 from the SW circuit 654 (FIG. 55). The frame memory 523 stores the image data supplied thereto, delays it by a time corresponding to one frame, and supplies it to the terminal io2 ′ of the signal processing circuit 661.

  The signal processing circuit 661 has terminals io1, io2, io2 ′, io3, io4, and io5. Image data is supplied from the switching circuit 662 to the terminal io1, and image data is supplied from the switching circuit 663 to the terminal io2. Image data is supplied from the frame memory 523 to the terminal io2 ′. From the terminal io3 or io4, image data obtained by performing signal processing in the signal processing circuit 661 is output, and this image data is supplied to the switching circuit 664 or 665, respectively. A command string is supplied to the terminal io5 from the receiving unit 650 (FIG. 55).

  The signal processing circuit 661 switches its internal structure in accordance with the command string supplied to the terminal io5, performs signal processing on the image data supplied to the terminal io1, io2, or io2 ′, and obtains the resulting image data as the result. , Output from terminal io3 or io4.

  A command sequence is supplied to the switching circuit 662 from the receiving unit 650 (FIG. 55). The switching circuit 662 is supplied with image data from the SW circuit 654 (FIG. 55) and the scanning line conversion circuit 521A. The switching circuit 662 selects the image data from the SW circuit 654 or the image data from the scanning line conversion circuit 521A in accordance with the command string from the receiving unit 650, and supplies it to the terminal io1 of the signal processing circuit 661.

  A command sequence is supplied to the switching circuit 663 from the receiving unit 650 (FIG. 55). The switching circuit 663 is supplied with image data from the SW circuit 654 (FIG. 55) and the scanning line conversion circuit 521B. The switching circuit 663 selects the image data from the SW circuit 654 or the image data from the scanning line conversion circuit 521B according to the command sequence from the receiving unit 650, and supplies the selected image data to the terminal io2 of the signal processing circuit 661.

  A command sequence is supplied to the switching circuit 664 from the receiving unit 650 (FIG. 55). The switching circuit 664 is supplied with image data from a terminal io3 of the signal processing circuit 661. The switching circuit 664 selects the switching circuit 665 or the scanning line conversion circuit 521B in accordance with the command string from the receiving unit 650, and supplies image data from the terminal io3 to the selected circuit.

  A command sequence is supplied to the switching circuit 665 from the receiving unit 650 (FIG. 55). The switching circuit 665 is supplied with image data from the switching circuit 664 and the terminal io4 of the signal processing circuit 661. The switching circuit 665 selects the image data from the switching circuit 664 or the image data from the terminal io4 of the signal processing circuit 661 according to the command sequence from the receiving unit 650, and supplies it to the SW circuit 655 (FIG. 55). .

  FIG. 58 shows a configuration example of the signal processing circuit 661 of FIG.

  Note that the signal processing circuit 661 is provided with terminals io1, io2, io2 ′, io3, io4, and io5, as well as a power supply terminal and the like, although not shown.

  58, a signal processing circuit 661 includes arithmetic circuit groups 411A and 411B as a plurality of circuit groups, memories 412A and 412B, product-sum arithmetic circuit groups 413A and 413B, adders 414A and 414B, multipliers 415A and 415B, and registers It consists of groups 416A and 416B.

  Then, for these circuit groups or circuits, the input / output or mutual connection state (which means both circuit groups or interconnections between circuits and interconnections between circuits within the circuit groups) is switched. A switching circuit for this purpose is provided in the signal processing circuit 661.

  In other words, the flow of the digital signal in the signal processing circuit 661 and the function of each circuit group can be controlled by the control signal. That is, a switching circuit 421A or 421B is provided in association with the arithmetic circuit group 411A or 411B, respectively. Further, a switching circuit 422A or 422B is provided in association with the memory 412A or 412B, respectively. Further, a switching circuit 423A or 423B is provided in association with the product-sum operation circuit group 413A or 413B, respectively. Further, a switching circuit 424 is provided in association with the adders 414A and 414B, the multipliers 415A and 415B, and the register groups 416A and 416B.

  A command string com is supplied from the terminal io5 to the switching circuits 421A, 421B, 422A, 422B, 423A, 423B, and 424. The switching circuits 421A, 421B, 422A, 422B, 423A, 423B, and 424 are arranged according to the command string com as arithmetic circuit groups 411A and 411B, memories 412A and 412B, product-sum arithmetic circuit groups 413A and 413B, adders 414A and 414B, The connection state between the multipliers 415A and 415B and the register groups 416A and 416B is switched, thereby switching the internal structure of the signal processing circuit 661.

  FIG. 59 shows another configuration example of the signal processing circuit 661 in FIG.

  Image data from the terminal io1 is supplied to the class classification circuit 511A, the delay selection circuit 512A, and the line delay circuit 517.

  The class classification circuit 511A corresponds to the tap selection unit 13 and the class classification unit 14 of FIG. The class classification circuit 511A selects some pixels (pixel values thereof) from the image data from the terminal io1 or the image data from the line delay circuit 517, classifies the pixels as class taps, and performs class classification. The resulting class code is supplied to the switching circuit 513A.

  However, the class classification circuit 511A can select a plurality of pixels in a one-dimensional array and a plurality of pixels in a two-dimensional array as class taps. Here, as the plurality of pixels in the one-dimensional array, for example, there are some pixels arranged in the horizontal direction and some pixels arranged in the vertical direction. In addition, as the plurality of pixels in the two-dimensional array, for example, there is a pixel in a rectangular region having X pixels in the horizontal direction and Y pixels in the vertical direction with a certain pixel as the center.

  Accordingly, the class classification performed by the class classification circuit 511A includes a classification performed by using a plurality of pixels in a one-dimensional array as a class tap and a classification performed by using a plurality of pixels in a two-dimensional array as a class tap. Class classification performed using a plurality of pixels in a one-dimensional array as class taps is hereinafter referred to as one-dimensional class classification as appropriate. Further, class classification performed using a plurality of pixels in a two-dimensional array as class taps is hereinafter referred to as two-dimensional class classification as appropriate.

  Note that the class tap can also be composed of a plurality of pixels in a three-dimensional array. As the plurality of pixels in the three-dimensional array, for example, there is a pixel in a rectangular parallelepiped region having X pixels in the horizontal direction, Y pixels in the vertical direction, and T pixels in the time direction centering on a certain pixel. . Class classification performed using a plurality of pixels in a three-dimensional array as class taps is hereinafter referred to as three-dimensional class classification as appropriate.

  The class classification circuit 511A performs one-dimensional class classification and two-dimensional class classification, and supplies two class codes obtained by the one-dimensional class classification and the two-dimensional class classification to the switching circuit 513A.

  The delay selection circuit 512A corresponds to the tap selection unit 12 in FIG. The delay selection circuit 512A selects some pixels (pixel values thereof) from the image data from the terminal io1 or the image data from the line delay circuit 517, and supplies these pixels as prediction taps to the switching circuit 514A. .

  However, the delay selection circuit 512A can select a plurality of pixels in a one-dimensional array and a plurality of pixels in a two-dimensional array as prediction taps. A prediction tap composed of a plurality of pixels in a one-dimensional array is hereinafter referred to as a one-dimensional prediction tap as appropriate, and a prediction tap composed of a plurality of pixels in a two-dimensional array is appropriately referred to below as two-dimensional prediction This is called a tap.

  Note that the prediction tap can be composed of a plurality of pixels in a three-dimensional array, and a prediction tap composed of a plurality of pixels in a three-dimensional array is hereinafter referred to as a three-dimensional prediction tap as appropriate.

  The delay selection circuit 512A obtains a one-dimensional prediction tap and a two-dimensional prediction tap and supplies them to the switching circuit 514A.

  Note that the delay selection circuit 512A uses a one-dimensional prediction tap and a two-dimensional prediction in order to adjust the timing at which the prediction tap and the tap coefficient to be calculated by the prediction calculation circuit 516A are supplied to the prediction calculation circuit 516A. The tap is output after being delayed by a predetermined time.

  The switching circuit 513A selects one of the two output terminals from which the class classification circuit 511A outputs two class codes according to the command string from the terminal io5, and connects it to the coefficient output circuit 515A. As a result, either the class code as the result of the one-dimensional class classification or the class code as the result of the two-dimensional class classification is transmitted from the class classification circuit 511A to the coefficient output circuit 515A via the switching circuit 513A. Supplied.

  The switching circuit 514A selects one of the two output terminals from which the delay selection circuit 512A outputs two prediction taps according to the command string from the terminal io5, and connects it to the prediction calculation circuit 516A. As a result, one of the one-dimensional prediction tap and the two-dimensional prediction tap is supplied from the delay selection circuit 512A to the prediction calculation circuit 516A via the switching circuit 514A.

  The coefficient output circuit 515A corresponds to the coefficient output unit 15 in FIG. 1 and stores several different types of tap coefficients obtained by learning performed in advance. The coefficient output circuit 515A selects a tap coefficient of a type corresponding to the command string from the terminal io5, and further corresponds to the class code supplied from the class classification circuit 511A via the switching circuit 513A. The tap coefficient of the class to be read is read and supplied to the prediction calculation circuit 516A.

  The coefficient output circuit 515A stores, as each type of tap coefficient, what should be used when the prediction tap is a one-dimensional prediction tap and what should be used when the prediction tap is a two-dimensional prediction tap. Further, as the tap coefficients, those to be used when the one-dimensional class classification is performed as the class classification and those to be used when the two-dimensional class classification is performed are stored. As described above, a tap coefficient of a plurality of patterns may exist as a certain type of tap coefficient, but the description of this point will be omitted below.

  The prediction calculation circuit 516A corresponds to the prediction calculation unit 16 in FIG. Prediction operation circuit 516A performs the operation of equation (1) using the prediction tap supplied from delay selection circuit 512A via switching circuit 514A and the tap coefficient supplied from coefficient output circuit 515A, and the operation The result is supplied to the product-sum operation circuit 518 and the switching circuit 519 as the pixel of interest (pixel value thereof).

  Here, the calculation of the prediction tap and the tap coefficient according to the equation (1) is equivalent to the calculation (filtering) by the FIR (Finite Impulse Response) filter. Therefore, the calculation of the expression (1) performed using the one-dimensional prediction tap, the two-dimensional prediction tap, or the three-dimensional prediction tap is hereinafter appropriately referred to as a one-dimensional filter operation, a two-dimensional filter operation, or a three-dimensional filter operation. Each.

  The class classification circuit 511B, the delay selection circuit 512B, the switching circuits 513B and 514B, the coefficient output circuit 515B, and the prediction calculation circuit 516B are the above-described class classification circuit 511A, delay selection circuit 512A, switching circuits 513A and 514A, coefficient output circuit 515A, Each is configured in the same manner as the prediction calculation circuit 516A. The class classification circuits 511B to 516B have the same connection relationship as the class classification circuits 511A to 516A.

  However, the image data from the terminal io2, the image data from the terminal io2 ′, and the image data from the line delay circuit 517 are supplied to the class classification circuit 511B and the delay selection circuit 512B.

  The class classification circuit 511B configures a class tap by selecting a pixel from the image data supplied thereto, and performs one-dimensional class classification, two-dimensional class classification, and three-dimensional class classification based on the class tap. . Furthermore, the class classification circuit 511B supplies three class codes obtained by the three class classifications to the switching circuit 513B.

  The class classification circuit 511B also detects the motion of the image data supplied from the terminal io2 or io2 ′, and supplies motion information representing the motion to the product-sum operation circuit 518.

  The delay selection circuit 512B selects a pixel from the image data supplied thereto, thereby forming a one-dimensional prediction tap, a two-dimensional prediction tap, and a three-dimensional prediction tap, and supplies the switching circuit 514B.

  The switching circuit 513B selects one of the three class codes from the class classification circuit 511B according to the command string from the terminal io5, and supplies it to the coefficient output circuit 515B.

  The switching circuit 514B selects one of the three prediction taps from the delay selection circuit 512B according to the command string from the terminal io5, and supplies the selected prediction tap to the prediction calculation circuit 516B.

  Similar to the coefficient output circuit 515A, the coefficient output circuit 515B stores several different types of tap coefficients. The coefficient output circuit 515B selects a tap coefficient of a type corresponding to the command string from the terminal io5, and further corresponds to the class code supplied from the class classification circuit 511B via the switching circuit 513B from the tap coefficient. The tap coefficient of the class to be read is read and supplied to the prediction calculation circuit 516B.

  Prediction operation circuit 516A performs the operation of equation (1) using the prediction tap supplied from delay selection circuit 512B via switching circuit 514B and the tap coefficient supplied from coefficient output circuit 515B, and the operation The result is supplied to the terminal io4 and the product-sum operation circuit 518 as the pixel of interest (pixel value thereof).

  The line delay circuit 517 is a circuit that generates a delay of one to several lines configured by a memory, delays image data from the terminal io1 by one to several lines, class classification circuits 511A and 511B, and a delay This is supplied to the selection circuits 512A and 512B.

  The product-sum operation circuit 518 performs weighted addition between the outputs of the prediction operation circuits 516A and 516B using the motion information supplied from the class classification circuit 511B as a weight, and supplies the addition result to the switching circuit 519.

  The switching circuit 519 selects one of the output of the prediction calculation circuit 516A or the output of the product-sum calculation circuit 518 in accordance with the command string from the terminal io5, and supplies it to the terminal io3.

  As described above, in FIG. 59, the class classification circuits 511A and 511B correspond to the tap selection unit 13 and the class classification unit 14 in FIG. 1, and the delay selection circuits 512A and 512B correspond to the tap selection unit 12 in FIG. To do. The coefficient output circuits 515A and 515B correspond to the coefficient output unit 15 in FIG. 1, and the prediction calculation circuits 516A and 516B correspond to the prediction calculation unit 16 in FIG.

  Therefore, in the class classification circuit 511A to the prediction calculation circuit 516A, the image data supplied from the terminal io1 and the image data supplied from the line delay circuit 517 are used as the first image data as the class classification adaptive processing (image conversion as The image data output from the prediction calculation circuit 516A is the second image data as a result of the class classification adaptation process.

  The class classification circuit 511B through the prediction calculation circuit 516B also use the image data supplied from the terminal io2, the image data supplied from the terminal io2 ′, and the image data supplied from the line delay circuit 517 as the first image data. As described above, the class classification adaptive process (the image conversion process) can be performed, and the image data output from the prediction calculation circuit 516B is the second image data as a result of the class classification adaptive process.

  Next, image conversion processing as class classification adaptive processing performed by the class classification circuits 511A to 516A and the class classification circuits 511B to 516B will be described with reference to the flowchart of FIG.

  In the following description, class classification circuits 511A and 511B are switched as appropriate, class selection circuit 511, delay selection circuits 512A and 512B as delay selection circuit 512, switching circuits 513A and 513B as switching circuit 513, and switching circuits 514A and 514B as appropriate. The circuit 514, the coefficient output circuits 515A and 515B are described as the coefficient output circuit 515, and the prediction calculation circuits 516A and 516B are described as the prediction calculation circuit 516, respectively.

  In step S 661, image data is input to the class classification circuit 511 and the delay selection circuit 512. Note that the image data input to the class classification circuit 511 and the delay selection circuit 512 is the first image data of the class classification adaptation process, and in the class classification adaptation process, the first image data is the second image data. Converted to data.

  Thereafter, a pixel that has not yet been set as the target pixel in the second image data is selected as the target pixel, and the process proceeds to step S662. The class classification circuit 511 selects a pixel that is a class tap for the target pixel as the first pixel. After selecting (extracting) from the image data, the process proceeds to step S663. In step S663, the class classification circuit 511 performs class classification based on the class tap, and outputs the class code obtained as a result to the coefficient output circuit 515 via the switching circuit 513.

  Then, the process proceeds to step S664, and the delay selection circuit 512 selects (extracts) the pixel that becomes the prediction tap for the target pixel from the first image data, supplies the selected pixel to the prediction calculation circuit 516, and proceeds to step S665.

  In step S665, the coefficient output circuit 515 outputs the class tap coefficient corresponding to the class code output from the class classification circuit 511 in step S663 to the prediction calculation circuit 516, and proceeds to step S666.

  In step S666, the prediction calculation circuit 516 performs the calculation (filter calculation) of Expression (1) using the prediction tap from the delay selection circuit 512 and the tap coefficient from the coefficient output circuit 515, and the calculation result is The target pixel (its pixel value) is output, and the process ends.

  Note that the processing in steps S662 to S666 in FIG. 60 is performed using all pixels of the second image data as the target pixel.

  Next, processing of the signal processing circuit 656 in FIG. 57 having the signal processing circuit 661 having the configuration shown in FIG. 59 will be described.

  First, the A button of the remote controller 618 (FIG. 49) is operated, whereby the A button command string 641 in which the one-dimensional spatial resolution creation command, which is one command shown in FIG. The processing when received at 650 and supplied to the signal processing circuit 656 of FIG. 57 will be described.

  The A button command string 641 supplied to the signal processing circuit 656 (FIG. 57) is supplied to the switching circuits 662 to 665 and the terminal io5 of the signal processing circuit 661.

  The switching circuit 662 selects the output of the scanning line conversion circuit 512A in accordance with the one-dimensional spatial resolution creation command in the A button command string 641, and connects it to the terminal io1 of the signal processing circuit 661. The switching circuit 663 selects the output of the scanning line conversion circuit 512B in accordance with the one-dimensional spatial resolution creation command in the A button command string 641, and connects it to the terminal io2 of the signal processing circuit 661.

  Further, the switching circuit 664 selects an input to the scanning line conversion circuit 512B in accordance with the one-dimensional spatial resolution creation command in the A button command string 641, and converts the terminal io3 of the signal processing circuit 661 to the scanning line conversion. Connect to circuit 512B. The switching circuit 665 selects the terminal io4 of the signal processing circuit 661 in accordance with the one-dimensional spatial resolution creation command in the A button command string 641, and connects it to the SW circuit 655 (FIG. 55).

  Further, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 (FIG. 59) change the signal line according to the one-dimensional spatial resolution creation command in the A button command row 641. The selection state (connection state of each block) is switched to perform the one-dimensional spatial resolution creation process.

  Here, the one-dimensional spatial resolution creation process means a spatial resolution creation process that uses a one-dimensional prediction tap as a prediction tap.

  61 shows the signal processing circuit 656 of FIG. 57 including the signal processing circuit 661 in which the selection state of the signal line is switched to perform the one-dimensional spatial resolution creation processing.

  In FIG. 61, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 are selected according to the one-dimensional spatial resolution creation command in the A button command string 641. Are switched as indicated by a solid line in FIG. 61, whereby a one-dimensional spatial resolution creation process is performed as a signal process.

  In FIG. 61, a signal line indicated by a dotted line means a signal line that is wired but is not related to signal processing by the current signal processing circuit 661.

  In FIG. 61, the frame memory 523 and the switching circuits 662 to 665 in FIG. 57 are not shown.

  In the signal processing circuit 656 of FIG. 61, for example, standard resolution image data (hereinafter referred to as SD image data as appropriate) input as the first image data of the class classification adaptive processing is input in the horizontal direction. The spatial resolution creation process (hereinafter referred to as HD image data as appropriate) is converted to high-resolution image data as second image data of the class classification adaptive process, in which the number of pixels in the vertical direction is doubled. One-dimensional spatial resolution creation processing) is performed.

  Here, for example, first, the number of pixels in the vertical direction of the SD image data (first image data) is doubled, and then the number of pixels in the horizontal direction is doubled, so that the number of pixels is four. It is assumed that double HD image data (second image data) is obtained.

  In FIG. 61, SD image data is supplied to the scanning line conversion circuit 521A. The scanning line conversion circuit 521A performs conversion from horizontal scanning (television raster scanning order) to vertical scanning for the SD image data supplied thereto.

  The SD image data whose scanning direction is converted by the scanning line conversion 521A is supplied to the terminal io1 of the signal processing circuit 661.

  The SD image data supplied to the terminal io1 is supplied to the class classification circuit 511A and the delay selection circuit 512A.

  The class classification circuit 511A selects a class tap from the SD image data from the terminal io1, and performs one-dimensional class classification based on the class tap. Then, the class classification circuit 511A supplies the class code obtained by the one-dimensional class classification to the coefficient output circuit 515A via the switching circuit 513A.

  The coefficient output circuit 515A stores at least a plurality of types of tap coefficients including tap coefficients for one-dimensional spatial resolution creation processing, and the one-dimensional spatial resolution creation command in the A button command string 641 supplied from the terminal io5. Accordingly, the tap coefficient for the one-dimensional spatial resolution creation process is selected. Then, the coefficient output circuit 515A reads out the tap coefficient of the class corresponding to the class code supplied via the switching circuit 513A from the class classification circuit 511A, and supplies it to the prediction calculation circuit 516A.

  On the other hand, the delay selection circuit 512A selects a one-dimensional prediction tap from the SD image data from the terminal io1, and supplies it to the prediction calculation circuit 516A via the switching circuit 514A.

  Prediction operation circuit 516A uses a plurality of pixel data of SD image data as one-dimensional prediction taps supplied from delay selection circuit 512A via switching circuit 514A, and tap coefficients supplied from coefficient output circuit 515A. Then, the filter calculation (one-dimensional filter calculation) of Expression (1) is performed, and thereby, the image data (pixel value of the pixel) in which the number of pixels in the vertical direction is twice the original SD image data is obtained and output. .

  The image data output by the prediction calculation circuit 516A is output from the terminal io3 through the switching circuit 119, and is further supplied to the scanning line conversion circuit 521B through the switching circuit 664 (FIG. 57).

  The scanning line conversion circuit 521B performs conversion from vertical scanning to horizontal scanning for the image data supplied thereto. Then, the scanning line conversion circuit 521B supplies the image data that has become a scanning signal similar to the television raster to the terminal io2 by the conversion.

  The image data supplied to the terminal io2 is supplied to the class classification circuit 511B and the delay selection circuit 512B.

  The class classification circuit 511B selects a class tap from the image data from the terminal io2, and performs one-dimensional class classification based on the class tap. Then, the class classification circuit 511B supplies the class code obtained by the one-dimensional class classification to the coefficient output circuit 515B via the switching circuit 513B.

  The coefficient output circuit 515B stores a plurality of types of tap coefficients including tap coefficients for at least one-dimensional spatial resolution creation processing, and a one-dimensional spatial resolution creation command in the A button command string 641 supplied from the terminal io5. Accordingly, the tap coefficient for the one-dimensional spatial resolution creation process is selected. Then, the coefficient output circuit 515B reads the tap coefficient of the class corresponding to the class code supplied via the switching circuit 513B from the class classification circuit 511B, and supplies the read tap coefficient to the prediction calculation circuit 516B.

  On the other hand, the delay selection circuit 512B selects a one-dimensional prediction tap from the image data from the terminal io2, and supplies it to the prediction calculation circuit 516B via the switching circuit 514B.

  The prediction calculation circuit 516B uses a plurality of pixel data of image data as a one-dimensional prediction tap supplied from the delay selection circuit 512B via the switching circuit 514B, and a tap coefficient supplied from the coefficient output circuit 515B. The filter operation (one-dimensional filter operation) of Expression (1) is performed, whereby HD image data (pixel value of a pixel) in which the number of pixels in the vertical direction and the horizontal direction are both double the original image data. Output.

  The image data output by the prediction calculation circuit 516B is output from the terminal io4, and further supplied to the SW circuit 655 (FIG. 55) via the switching circuit 665 (FIG. 57).

  Next, the A button of the remote control 618 (FIG. 49) is operated, whereby the A button command string 642 in which the two-dimensional spatial resolution creation command as one command shown in FIG. 54 is arranged is received in FIG. A process when received by the unit 650 and supplied to the signal processing circuit 656 of FIG. 57 will be described.

  The A button command string 642 supplied to the signal processing circuit 656 (FIG. 57) is supplied to the switching circuits 662 to 665 and the terminal io5 of the signal processing circuit 661.

  The switching circuit 662 selects the output of the scanning line conversion circuit 512A in accordance with the two-dimensional spatial resolution creation command in the A button command string 642 and connects it to the terminal io1 of the signal processing circuit 661. Further, the switching circuit 663 selects the output of the scanning line conversion circuit 512B in accordance with the two-dimensional spatial resolution creation command in the A button command string 642 and connects it to the terminal io2 of the signal processing circuit 661.

  Further, the switching circuit 664 selects an input to the scanning line conversion circuit 512B in accordance with the two-dimensional spatial resolution creation command in the A button command string 642, and converts the terminal io3 of the signal processing circuit 661 to the scanning line conversion. Connect to circuit 512B. The switching circuit 665 selects the terminal io4 of the signal processing circuit 661 in accordance with the two-dimensional spatial resolution creation command in the A button command string 642, and connects it to the SW circuit 655 (FIG. 55).

  Further, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 (FIG. 59) change the signal line according to the two-dimensional spatial resolution creation command in the A button command row 642. The selection state is switched to perform the two-dimensional spatial resolution creation process.

  Here, the two-dimensional spatial resolution creation process means a spatial resolution creation process that uses a two-dimensional prediction tap as a prediction tap.

  FIG. 62 shows the signal processing circuit 656 of FIG. 57 including the signal processing circuit 661 in which the signal line selection state is switched to perform the two-dimensional spatial resolution creation processing.

  In FIG. 62, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 change the signal line selection state according to the two-dimensional spatial resolution creation command in the A button command string 642. Are switched as indicated by a solid line in FIG. 62, whereby a two-dimensional spatial resolution creation process is performed as a signal process.

  In FIG. 62, a signal line indicated by a dotted line means a signal line that is wired but is not related to signal processing by the current signal processing circuit 661.

  In FIG. 62, the frame memory 523 and the switching circuits 662 to 665 in FIG. 57 are not shown.

  In the signal processing circuit 656 of FIG. 62, for example, as in the case of FIG. 61, the SD image data as the first image data of the class classification adaptive processing inputted thereto is converted into pixels in the horizontal and vertical directions. A spatial resolution creation process is performed in which the number is doubled and converted into HD image data as the second image data of the class classification adaptive process. However, in FIG. 62, the two-dimensional class classification is adopted as the class classification, and the two-dimensional prediction tap is adopted as the prediction tap.

  Here, for example, first, the number of pixels in the vertical direction of the SD image data (first image data) is doubled, and then the number of pixels in the horizontal direction is doubled. It is assumed that double HD image data (second image data) is obtained.

  In FIG. 62, SD image data is supplied to the scanning line conversion circuit 521A. The scanning line conversion circuit 521A converts the scanning direction of the SD image data supplied thereto from horizontal scanning to vertical scanning, and supplies it to the terminal io1 of the signal processing circuit 661.

  The SD image data supplied to the terminal io1 is supplied to the class classification circuit 511A, the delay selection circuit 512A, and the line delay circuit 517.

  The line delay circuit 517 delays the SD image data supplied thereto by one to several lines and supplies it to the class classification circuits 511A and 511B and the delay selection circuits 512A and 512B.

  The class classification circuit 511A selects a class tap from the SD image data from the terminal io1 and the SD image data from the line delay circuit 517A, and performs two-dimensional class classification based on the class tap. Then, the class classification circuit 511A supplies the class code obtained by the two-dimensional class classification to the coefficient output circuit 515A via the switching circuit 513A.

  The plurality of types of tap coefficients stored in the coefficient output circuit 515A include tap coefficients for two-dimensional spatial resolution creation processing, and the coefficient output circuit 515A receives the A button command string 642 supplied from the terminal io5. The tap coefficient for 2D spatial resolution creation processing is selected in accordance with the 2D spatial resolution creation command. Then, the coefficient output circuit 515A reads out the tap coefficient of the class corresponding to the class code supplied via the switching circuit 513A from the class classification circuit 511A, and supplies it to the prediction calculation circuit 516A.

  On the other hand, the delay selection circuit 512A selects a two-dimensional prediction tap from the SD image data from the terminal io1 and the SD image data from the line delay circuit 517A, and supplies it to the prediction calculation circuit 516A via the switching circuit 514A. .

  Prediction operation circuit 516A uses a plurality of pixel data of SD image data as two-dimensional prediction taps supplied from delay selection circuit 512A via switching circuit 514A, and tap coefficients supplied from coefficient output circuit 515A. Then, the filter operation (two-dimensional filter operation) of Expression (1) is performed, and thereby, the image data (pixel value of the pixel) in which the number of pixels in the vertical direction is twice the original SD image data is obtained and output. .

  The image data output by the prediction calculation circuit 516A is output from the terminal io3 through the switching circuit 119, and is further supplied to the scanning line conversion circuit 521B through the switching circuit 664 (FIG. 57).

  The scanning line conversion circuit 521B converts the scanning direction of the image data supplied thereto from vertical scanning to horizontal scanning, and supplies the image data, which is a scanning signal similar to a television raster, to the terminal io2.

  The image data supplied to the terminal io2 is supplied to the class classification circuit 511B and the delay selection circuit 512B.

  The class classification circuit 511B selects a class tap from the image data from the terminal io2 and the image data from the line delay circuit 517, and performs two-dimensional class classification based on the class tap. Then, the class classification circuit 511B supplies the class code obtained by the two-dimensional class classification to the coefficient output circuit 515B via the switching circuit 513B.

  The plurality of types of tap coefficients stored in the coefficient output circuit 515B include tap coefficients for two-dimensional spatial resolution creation processing, and the coefficient output circuit 515B receives the A button command string 642 supplied from the terminal io5. The tap coefficient for 2D spatial resolution creation processing is selected in accordance with the 2D spatial resolution creation command. Then, the coefficient output circuit 515B reads the tap coefficient of the class corresponding to the class code supplied via the switching circuit 513B from the class classification circuit 511B, and supplies the prediction calculation circuit 516B.

  On the other hand, the delay selection circuit 512B selects a two-dimensional prediction tap from the image data from the terminal io2 and the image data from the line delay circuit 517, and supplies it to the prediction calculation circuit 516B via the switching circuit 514B.

  The prediction calculation circuit 516B uses a plurality of pixel data of image data as a two-dimensional prediction tap supplied from the delay selection circuit 512B via the switching circuit 514B, and the tap coefficient supplied from the coefficient output circuit 515B, The filter operation (two-dimensional filter operation) of Expression (1) is performed, whereby HD image data (pixel values of pixels) in which the number of pixels in the vertical direction and the horizontal direction are both double the original image data. Output.

  The image data output by the prediction calculation circuit 516B is output from the terminal io4, and further supplied to the SW circuit 655 (FIG. 55) via the switching circuit 665 (FIG. 57).

  Next, the B button of the remote control 618 (FIG. 49) is operated, and as a result, the B button command string 643 in which the noise removal command as one command shown in FIG. 54 is arranged is received by the receiving unit 650 in FIG. A process when received and supplied to the signal processing circuit 656 of FIG. 57 will be described.

  The B button command string 643 supplied to the signal processing circuit 656 is supplied to the switching circuits 662 to 665 and the terminal io5 of the signal processing circuit 661.

  The switching circuit 662 selects the input to the signal processing circuit 656 according to the noise removal command in the B button command string 643 and connects it to the terminal io1 of the signal processing circuit 661. The switching circuit 663 also selects an input to the signal processing circuit 656 according to the noise removal command in the B button command string 643 and connects it to the terminal io2 of the signal processing circuit 661.

  Further, the switching circuit 664 selects an input to the switching circuit 665 in accordance with the noise removal command in the B button command string 643 and connects the terminal io3 of the signal processing circuit 661 to the switching circuit 665. Further, the switching circuit 665 selects the output of the switching circuit 664 according to the noise removal command in the B button command string 643 and connects it to the SW circuit 655 (FIG. 55).

  Further, switching circuits 513A and 513B, switching circuits 514A and 514B, and switching circuit 519 of signal processing circuit 661 (FIG. 59) change the signal line selection state in accordance with the noise removal command in B button command sequence 643. Then, switch to perform noise removal processing.

  Here, the noise removal processing in the signal processing circuit 661 is performed by adding the processing result of the two-dimensional noise removal processing and the processing result of the three-dimensional noise removal processing according to the motion of the image. The two-dimensional noise removal process or the three-dimensional noise removal process means a noise removal process using a two-dimensional prediction tap or a three-dimensional prediction tap as a prediction tap. Two-dimensional noise removal processing is effective when there is motion in the image, and three-dimensional noise removal processing is effective when there is no motion in the image (small motion).

  FIG. 63 shows the signal processing circuit 656 of FIG. 57 including the signal processing circuit 661 in which the signal line selection state is switched to perform the noise removal processing.

  In FIG. 63, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 change the signal line selection state according to the noise removal command in the B button command string 643. As indicated by the solid line in FIG. 63, switching is performed, whereby noise removal processing is performed as signal processing.

  In FIG. 63, a signal line indicated by a dotted line means a signal line that is wired but is not related to signal processing by the current signal processing circuit 661.

  In FIG. 63, the scanning line conversion circuits 521A and 521B and the switching circuits 662 to 665 in FIG. 57 are not shown.

  In the signal processing circuit 656 shown in FIG. 63, the first image data of the class classification adaptive processing input thereto is converted into the second image data of the class classification adaptive processing whose S / N is improved. A noise removal process and a three-dimensional noise removal process are performed, and the result of the two-dimensional noise removal process and the result of the three-dimensional noise removal process are added in accordance with the motion of the image, thereby obtaining a final result. Noise removal processing is performed to obtain high S / N image data as a result of simple signal processing.

  In FIG. 63, the image data input to the signal processing circuit 656 is supplied to the frame memory 523 and the terminals io1 and io2 of the signal processing circuit 661.

  The frame memory 523 stores the image data supplied thereto, delays it by one frame (or field), and supplies it to the terminal io2 ′ of the signal processing circuit 661.

  Here, the frame of the image data supplied to the terminals io1 and io2 is hereinafter referred to as a current frame as appropriate. Further, a frame one frame before the current frame supplied from the frame memory 523 to the terminal io2 ′ is referred to as a previous frame.

  The image data of the current frame supplied to the terminal io1 is supplied to the class classification circuit 511A, the delay selection circuit 512A, and the line delay circuit 517. The image data of the current frame supplied to the terminal io2 is supplied to the class classification circuit 511B and the delay selection circuit 512B. Further, the image data of the previous frame supplied to the terminal io2 ′ is also supplied to the class classification circuit 511B and the delay selection circuit 512B.

  The line delay circuit 517 delays the image data of the current frame supplied thereto by one to several lines and supplies it to the class classification circuits 511A and 511B and the delay selection circuits 512A and 512B.

  The class classification circuit 511A selects a class tap from the image data of the current frame from the terminal io1 and the image data of the current frame from the line delay circuit 517A, and performs two-dimensional class classification based on the class tap. Then, the class classification circuit 511A supplies the class code obtained by the two-dimensional class classification to the coefficient output circuit 515A via the switching circuit 513A.

  The plurality of types of tap coefficients stored in the coefficient output circuit 515A include a tap coefficient for two-dimensional noise removal processing, and the coefficient output circuit 515A stores the B button command string 643 supplied from the terminal io5. A tap coefficient for two-dimensional noise removal processing is selected according to the noise removal command in the middle. Then, the coefficient output circuit 515A reads out the tap coefficient of the class corresponding to the class code supplied via the switching circuit 513A from the class classification circuit 511A, and supplies it to the prediction calculation circuit 516A.

  The delay selection circuit 512A selects a two-dimensional prediction tap from the current frame image data from the terminal io1 and the current frame image data from the line delay circuit 517A, and the prediction calculation circuit via the switching circuit 514A. Supply to 516A.

  The prediction calculation circuit 516A receives a plurality of pixel data of image data of the current frame as a two-dimensional prediction tap supplied from the delay selection circuit 512A via the switching circuit 514A, and a tap coefficient supplied from the coefficient output circuit 515A. And performing the filter operation (two-dimensional filter operation) of the expression (1), thereby obtaining the image data (pixel value of the pixel) of the current frame subjected to the two-dimensional noise removal processing, and the product-sum operation circuit Output to 518.

  On the other hand, the class classification circuit 511B selects a class tap from the image data of the current frame from the terminal io2, the image data of the current frame from the line delay circuit 517, and the image data of the previous frame from the terminal io2 ′, and the class. Based on the tap, three-dimensional classification is performed. Then, the class classification circuit 511B supplies the class code obtained by the three-dimensional class classification to the coefficient output circuit 515B via the switching circuit 513B.

  Further, the class classification circuit 511B moves the image data of the current frame from the image data of the current frame from the terminal io2, the image data of the current frame from the line delay circuit 517, and the image data of the previous frame from the terminal io2 ′. , And motion information K representing the motion is supplied to the product-sum operation circuit 118.

  Here, as a motion detection method in the class classification circuit 511B, for example, a so-called gradient method can be adopted. In the gradient method, the amount of motion is obtained using frame difference and tilt information (sampling difference in the horizontal direction and line difference in the vertical direction) for the pixels in the motion region.

  That is, according to the gradient method, the frame difference ΔF (the pixel value of the current frame minus the corresponding pixel value of the previous frame) and the sampling difference ΔE (the value of a pixel in the current frame, E) is obtained. Then, from the integrated value Σ | ΔF | in the motion region of the absolute value | ΔF | of the frame difference ΔF and the integrated value Σ | ΔE | in the motion region of the absolute value | ΔE | The magnitude | v1 | of the direction component v1 is obtained by the expression | v1 | = Σ | ΔF | / Σ | ΔE |. The direction (right or left) of the horizontal component v1 of the motion is obtained from the relationship between the polarity of the frame difference ΔF and the polarity of the sampling difference ΔE. The vertical component v2 of motion can be obtained in the same manner.

  The class classification circuit 511B obtains the motion of the image from the horizontal component v1 and the vertical component v2 of the motion. For example, information representing the magnitude of the motion in a predetermined number of bits is used as the motion information K. This is supplied to the sum calculation circuit 118.

  As described above, the class classification circuit 511B can perform class classification using the horizontal component v1 and the vertical component v2 of motion in addition to performing three-dimensional class classification.

  The plurality of types of tap coefficients stored in the coefficient output circuit 515B include a tap coefficient for three-dimensional noise removal processing, and the coefficient output circuit 515B uses the B button command string 643 supplied from the terminal io5. A tap coefficient for three-dimensional noise removal processing is selected according to the noise removal command in the middle. Then, the coefficient output circuit 515B reads the tap coefficient of the class corresponding to the class code supplied via the switching circuit 513B from the class classification circuit 511B, and supplies the prediction calculation circuit 516B.

  The delay selection circuit 512B selects a three-dimensional prediction tap from the current frame image data from the terminal io2, the current frame image data from the line delay circuit 517, and the previous frame image data from the terminal io2 ′. This is supplied to the predictive arithmetic circuit 516B via the switching circuit 514B.

  The prediction calculation circuit 516B uses a plurality of pixel data of image data as a three-dimensional prediction tap supplied from the delay selection circuit 512B via the switching circuit 514B, and a tap coefficient supplied from the coefficient output circuit 515B. The filter operation (three-dimensional filter operation) of Expression (1) is performed, thereby obtaining the image data (pixel value of the pixel) of the current frame on which the three-dimensional noise removal processing has been performed, and outputting to the product-sum operation circuit 518 To do.

  The product-sum operation circuit 518 includes the current frame image data subjected to the two-dimensional noise removal processing from the prediction operation circuit 516A, and the current frame image data subjected to the three-dimensional noise removal processing from the prediction operation circuit 516B. Are added using the motion information K supplied from the class classification circuit 511B as a weighting coefficient.

  That is, assuming that the image data of the current frame that has been subjected to the two-dimensional noise removal process or the three-dimensional noise removal process is P2 or P3, respectively, and the motion information K has a value in the range of 0 to 1. The product-sum operation circuit 518 obtains image data P as a final noise removal processing result according to the equation P = K × P2 + (1−K) × P3.

  The image data output from the product-sum operation circuit 518 is output from the terminal io3 via the switching circuit 519, and further supplied to the SW circuit 655 (FIG. 55) via the switching circuits 664 and 665 (FIG. 57). The

  Next, the C button of the remote control 618 (FIG. 49) is operated, whereby the C button command string 644 in which the one-dimensional spatial resolution creation command and the noise removal command shown in FIG. The processing when received at 650 and supplied to the signal processing circuit 656 of FIG. 57 will be described.

  The C button command string 644 supplied to the signal processing circuit 656 (FIG. 57) is supplied to the switching circuits 662 to 665 and the terminal io5 of the signal processing circuit 661.

  The switching circuit 662 selects the output of the scanning line conversion circuit 512A in accordance with the first one-dimensional spatial resolution creation command in the C button command string 644 and connects it to the terminal io1 of the signal processing circuit 661. Further, the switching circuit 663 selects the output of the scanning line conversion circuit 512B in accordance with the first one-dimensional spatial resolution creation command in the C button command string 644 and connects it to the terminal io2 of the signal processing circuit 661. .

  Further, the switching circuit 664 selects the input to the scanning line conversion circuit 512B in response to the first one-dimensional spatial resolution creation command in the C button command string 644, and the terminal io3 of the signal processing circuit 661 is selected. Connected to the scanning line conversion circuit 512B. Further, the switching circuit 665 selects the terminal io4 of the signal processing circuit 661 in accordance with the first one-dimensional spatial resolution creation command in the C button command string 644 and connects it to the SW circuit 655 (FIG. 55). .

  Further, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 (FIG. 59), according to the first one-dimensional spatial resolution creation command in the C button command sequence 644, The selection state of the signal line is switched to perform the one-dimensional spatial resolution creation process.

  That is, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 change the signal line selection state in accordance with the one-dimensional spatial resolution creation command in the C button command row 644. Switching is performed as shown by the solid line in FIG.

  Therefore, as described with reference to FIG. 61, the signal processing circuit 656 performs one-dimensional resolution creation processing on the image data input thereto, and the image data whose spatial resolution of the image data is improved is The signal is output from the terminal io4 and further supplied to the SW circuit 655 (FIG. 55) via the switching circuit 665 (FIG. 57).

  As described with reference to the flowchart of FIG. 56, when the one-dimensional spatial resolution creation process, which is a signal process corresponding to the first one-dimensional spatial resolution creation command of the C button command string 644, is performed in the LSI 606 of FIG. The SW circuit 655 supplies the image data supplied from the signal processing circuit 656 to the frame memory 653 for storage. Further, the SW circuit 654 supplies the image data stored in the frame memory 654 to the signal processing circuit 656.

  Therefore, the image data obtained by performing the one-dimensional resolution creation processing in the signal processing circuit 656 (FIG. 55) is re-transmitted to the signal processing circuit 656 via the switching circuit 665, the frame memory 653, and the SW circuit 654. input.

  Thereafter, in the signal processing circuit 656 (FIG. 57), the switching circuit 662 selects the input to the signal processing circuit 656 in accordance with the second noise removal command in the C button command string 644, and the signal processing circuit 661 is connected to terminal io1. The switching circuit 663 also selects an input to the signal processing circuit 656 according to the second noise removal command in the C button command string 644 and connects it to the terminal io2 of the signal processing circuit 661.

  Further, the switching circuit 664 also selects the input to the switching circuit 665 in accordance with the second noise removal command in the C button command string 644 and connects the terminal io3 of the signal processing circuit 661 to the switching circuit 665. To do. The switching circuit 665 also selects the output of the switching circuit 664 according to the second noise removal command in the C button command string 644 and connects it to the SW circuit 655 (FIG. 55).

  Furthermore, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 (FIG. 59) change the signal line in accordance with the second noise removal command in the C button command string 644. The selected state is switched to perform noise removal processing.

  That is, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 change the signal line selection state in FIG. 63 according to the noise removal command in the C button command string 644. Switch as shown by the solid line.

  Therefore, as described with reference to FIG. 63, the signal processing circuit 656 performs noise removal processing on the image data whose spatial resolution has been improved by the one-dimensional spatial resolution creation processing that is re-input to the signal processing circuit 656. Image data whose data S / N is improved is output from the terminal io3, and further supplied to the SW circuit 655 (FIG. 55) via the switching circuits 664 and 665 (FIG. 57).

  As described with reference to the flowchart of FIG. 56, when the noise removal processing, which is the signal processing corresponding to the second noise removal command of the C button command sequence 644, that is, the last noise removal command is performed in the LSI 606 of FIG. 655 supplies the image data supplied from the signal processing circuit 656 to the synthesizing unit 607 (FIG. 48) via the frame memory 652.

  Therefore, image data whose spatial resolution has been improved by the one-dimensional spatial resolution creation process and whose S / N has been improved by the noise removal process is supplied to the synthesis unit 607.

  Next, the D button of the remote control 618 (FIG. 49) is operated, whereby the D button command string 644 in which the two-dimensional spatial resolution creation command and the noise removal command shown in FIG. The processing when received at 650 and supplied to the signal processing circuit 656 of FIG. 57 will be described.

  The D button command string 644 supplied to the signal processing circuit 656 (FIG. 57) is supplied to the switching circuits 662 to 665 and the terminal io5 of the signal processing circuit 661.

  The switching circuit 662 selects the output of the scanning line conversion circuit 512A in accordance with the first two-dimensional spatial resolution creation command in the D button command string 644 and connects it to the terminal io1 of the signal processing circuit 661. Further, the switching circuit 663 selects the output of the scanning line conversion circuit 512B in accordance with the first two-dimensional spatial resolution creation command in the D button command string 644 and connects it to the terminal io2 of the signal processing circuit 661. .

  Further, the switching circuit 664 selects an input to the scanning line conversion circuit 512B in response to the first two-dimensional spatial resolution creation command in the D button command string 644, and the terminal io3 of the signal processing circuit 661 is selected. Connected to the scanning line conversion circuit 512B. Further, the switching circuit 665 selects the terminal io4 of the signal processing circuit 661 in accordance with the first two-dimensional spatial resolution creation command in the D button command string 644 and connects it to the SW circuit 655 (FIG. 55). .

  Further, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 (FIG. 59), according to the first two-dimensional spatial resolution creation command in the D button command sequence 644, The selection state of the signal line is switched to perform the two-dimensional spatial resolution creation process.

  That is, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 change the signal line selection state according to the two-dimensional spatial resolution creation command in the D button command row 644. Switching is performed as shown by the solid line in FIG.

  Therefore, in the signal processing circuit 656, as described with reference to FIG. 62, two-dimensional resolution creation processing is performed on the image data input thereto, and image data whose spatial resolution of the image data is improved is obtained. The signal is output from the terminal io4 and further supplied to the SW circuit 655 (FIG. 55) via the switching circuit 665 (FIG. 57).

  As described with reference to the flowchart of FIG. 56, when the two-dimensional spatial resolution creation process, which is the signal processing corresponding to the first two-dimensional spatial resolution creation command of the D button command string 644, is performed in the LSI 606 of FIG. The SW circuit 655 supplies the image data supplied from the signal processing circuit 656 to the frame memory 653 for storage. Further, the SW circuit 654 supplies the image data stored in the frame memory 654 to the signal processing circuit 656.

  Therefore, the image data obtained by performing the two-dimensional resolution creation processing in the signal processing circuit 656 (FIG. 55) is re-transmitted to the signal processing circuit 656 via the switching circuit 665, the frame memory 653, and the SW circuit 654. input.

  Thereafter, in the signal processing circuit 656 (FIG. 57), the switching circuit 662 selects the input to the signal processing circuit 656 in accordance with the second noise removal command in the D button command string 644, and the signal processing circuit 661 is connected to terminal io1. The switching circuit 663 also selects an input to the signal processing circuit 656 according to the second noise removal command in the D button command string 644 and connects it to the terminal io2 of the signal processing circuit 661.

  Further, the switching circuit 664 also selects the input to the switching circuit 665 in response to the second noise removal command in the D button command string 644 and connects the terminal io3 of the signal processing circuit 661 to the switching circuit 665. To do. The switching circuit 665 also selects the output of the switching circuit 664 in accordance with the second noise removal command in the D button command string 644 and connects it to the SW circuit 655 (FIG. 55).

  Further, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 (FIG. 59) change the signal line according to the second noise removal command in the D button command string 644. The selected state is switched to perform noise removal processing.

  That is, the switching circuits 513A and 513B, the switching circuits 514A and 514B, and the switching circuit 519 of the signal processing circuit 661 change the signal line selection state in FIG. 63 according to the noise removal command in the D button command string 644. Switch as shown by the solid line.

  Therefore, as described with reference to FIG. 63, the signal processing circuit 656 performs noise removal processing on the image data whose spatial resolution has been improved by the two-dimensional spatial resolution creation processing that is re-input to the signal processing circuit 656. Image data whose data S / N is improved is output from the terminal io3, and further supplied to the SW circuit 655 (FIG. 55) via the switching circuits 664 and 665 (FIG. 57).

  As described with reference to the flowchart of FIG. 56, when the noise removal processing, which is the signal processing corresponding to the second noise removal command in the D button command sequence 644, is performed in the LSI 606 of FIG. 655 supplies the image data supplied from the signal processing circuit 656 to the synthesizing unit 607 (FIG. 48) via the frame memory 652.

  Therefore, the image data whose spatial resolution is improved by the two-dimensional spatial resolution creation process and whose S / N is improved by the noise removal process is supplied to the synthesis unit 607.

  As described above, in the LSI 606, when the command sequence has a plurality of commands, the command sequence corresponds to, for example, the first command of the first and second commands as the plurality of commands in the command sequence. Then, the internal structure is switched to a state in which the first signal processing corresponding to the first command is executed, the first signal processing is executed, and then the second command of the plurality of commands in the command sequence is further executed. In response, the internal structure is switched to a state in which the second signal processing corresponding to the second command is executed, and the second signal processing is executed. Therefore, according to the LSI 606, a plurality of functions can be easily realized with a single hardware.

  Next, a fourth embodiment of the present invention will be described.

  FIG. 64 shows an example of the configuration of an IC 700 as the fourth embodiment of the present invention.

  Note that the IC 700 in FIG. 64 can be used in place of the LSI 606 in the television receiver in FIG. 48, for example.

  The IC 700 receives a command sequence including one or more commands supplied from the outside (for example, the controller 613 in FIG. 48), and reconfigurable the internal structure according to each of the one or more commands in the command sequence. Switch to.

  For example, when the command sequence is composed of a plurality of commands, the IC 700 executes the first signal processing corresponding to the first command in the command sequence, and then further performs the second command in the command sequence. The second signal processing corresponding to is performed.

  That is, in FIG. 64, the class classification circuit 711A, the delay selection circuit 712A, the switching circuits 713A and 714A, the coefficient output circuit 715A, or the prediction calculation circuit 716A are the same as the class classification circuit 511A, the delay selection circuit 512A, and the switching circuit 513A in FIG. 514A, the coefficient output circuit 515A, or the prediction calculation circuit 516A.

  64, the class classification circuit 711B, the delay selection circuit 712B, the switching circuits 713B and 714B, the coefficient output circuit 715B, or the prediction calculation circuit 716B are the same as the class classification circuit 511B, the delay selection circuit 512B, and the switching circuit 513B in FIG. , 514B, the coefficient output circuit 515B, or the prediction calculation circuit 516B.

  Further, in FIG. 64, the line delay circuit 717, the product-sum operation circuit 718, or the switching circuit 719 is configured similarly to the line delay circuit 517, the product-sum operation circuit 518, or the switching circuit 519 of FIG.

  However, the class classification circuit 511B of FIG. 59 performs one-dimensional class classification, two-dimensional class classification, and three-dimensional class classification, and the switching circuit 513B selects one of the class codes obtained by the three class classifications. In FIG. 64, the class classification circuit 711B performs the two-dimensional class classification and the three-dimensional class classification, and the switching circuit 713B performs the two class classifications. Any one of the class codes obtained by the above is selected and output to the coefficient output circuit 715B.

  Also, the delay selection circuit 512B of FIG. 59 constitutes a one-dimensional prediction tap, a two-dimensional prediction tap, and a three-dimensional prediction tap, and the switching circuit 514B selects any one of the three prediction taps, 64. In FIG. 64, the delay selection circuit 712B configures a two-dimensional prediction tap and a three-dimensional prediction tap, and the switching circuit 714B includes the two prediction taps. Either one is selected and supplied to the prediction calculation circuit 716B.

  The receiving unit 720 receives a command sequence including one or more commands supplied from the outside (for example, the controller 613 in FIG. 48) and supplies the command string to necessary ones of the blocks constituting the IC 700.

  Image data input to the IC 700 is supplied to the frame memory 721. The frame memory 721 stores the image data supplied thereto, for example, delays by one frame, and supplies it to the class classification circuit 711B and the delay selection circuit 712B.

  Note that image data input to the IC 700 (hereinafter referred to as input image data as appropriate) is supplied not only to the frame memory 721 but also to the class classification circuit 711A, the delay selection circuit 712A, and the switching circuit 724. Yes.

  Image data is supplied from the switching circuit 724 to the blocking circuit 722. The blocking circuit 722 cuts out (extracts) a rectangular block having a predetermined point (pixel) as the center (center of gravity) from the image data supplied thereto, and outputs the extracted block to the switching circuit 726.

  The determination circuit 723 is supplied with image data obtained by performing the prediction calculation of Expression (1) from the prediction calculation circuit 716B. The determination circuit 723 performs determination processing described later on the image data supplied from the prediction calculation circuit 716B, and outputs the image data from the prediction calculation circuit 716B to the switching circuit 727 according to the result of the determination processing. Alternatively, the processing result in the IC 700 is output to the outside.

  The switching circuit 724 is supplied with input image data and image data output from the switching circuit 725. The switching circuit 724 selects input image data or image data output from the switching circuit 725 according to the command sequence supplied from the receiving unit 720 and outputs the selected image data to the blocking circuit 722.

  Further, the switching circuit 724 outputs the input image data supplied thereto to the switching circuit 725 according to the command sequence supplied from the receiving unit 720.

  As described above, the switching circuit 725 is supplied not only with the image data output from the switching circuit 724 but also with the image data output from the switching circuit 719. The switching circuit 725 supplies the image data output from the switching circuit 724 to the switching circuit 726 in accordance with the command sequence supplied from the receiving unit 720.

  The switching circuit 725 selects either the switching circuit 724 or 726 according to the command sequence supplied from the receiving unit 720, and the image data supplied from the switching circuit 719 is selected for the selected one. Is output.

  The switching circuit 726 selects either the image data supplied from the blocking circuit 722 or the image data supplied from the switching circuit 725 according to the command sequence supplied from the receiving unit 720, and classifies it. This is supplied to the circuit 711B and the delay selection circuit 712B.

  The switching circuit 727 is supplied with image data output from the line delay circuit 717 and image data output from the determination circuit 723. The switching circuit 727 selects either the image data from the line delay circuit 717 or the image data from the determination circuit 723 according to the command sequence supplied from the receiving unit 720, and selects the class classification circuit 711B and the delay The data is output to the selection circuit 712B.

  Next, FIG. 65 shows an example of commands constituting a command string received by the receiving unit 720 of FIG. 64 from the outside and supplied to necessary blocks of the IC 700.

  In the fourth embodiment, for example, “Kaizodo 1dimentional”, “Kaizodo 2dimentional”, “Kaizodo 3dimentional”, “Zoom ver1”, “Zoom ver2”, etc., are prepared as shown in the upper part of FIG. .

  The commands “Kaizodo 1dimentional”, “Kaizodo 2dimentional”, or “Kaizodo 3dimentional” are respectively a one-dimensional spatial resolution creation process, a two-dimensional spatial resolution creation process, or a three-dimensional spatial resolution creation process, that is, a one-dimensional prediction tap, This command requests a spatial resolution creation process using a three-dimensional prediction tap or a three-dimensional prediction tap.

  The command “Zoom ver1” is a command for requesting an enlargement process for obtaining image data enlarged to a desired magnification by repeatedly (recursively) performing a resizing process for enlarging at a certain constant magnification. Further, the command “Zoom ver2” is a command for requesting an enlargement process for obtaining image data enlarged to the desired magnification by performing the resizing process for enlarging the desired magnification only once.

  Here, the enlargement process corresponding to the command “Zoom ver1” is hereinafter referred to as a recursive enlargement process as appropriate. The enlargement process corresponding to the command “Zoom ver2” is hereinafter referred to as a once enlargement process as appropriate.

  The receiving unit 720 receives a command sequence including one or more of the above commands from the outside, and supplies the command block to a necessary block of the IC 700.

  That is, for example, as shown in the lower left part of FIG. 65, the receiving unit 720 receives a command sequence including one command “Kaizodo 2 dimension” from the outside and supplies the command block to a necessary block of the IC 700.

  In addition, the receiving unit 720 receives a command sequence including two commands “Kaizodo 2dimentional” and “Zoom ver1” from the outside as shown in the lower right of FIG. 65, for example, and supplies them to the necessary blocks of the IC 700 To do.

  Next, FIG. 66 shows a configuration example of the coefficient output unit 715A of FIG.

  In FIG. 66, the coefficient output unit 715 </ b> A includes a coefficient memory group 731 and a selection unit 732.

  The coefficient memory group 731 includes a plurality of coefficient memories, and each coefficient memory includes a tap coefficient for a one-dimensional spatial resolution creation process and a tap coefficient for a two-dimensional spatial resolution creation process, which are obtained by learning performed in advance. A tap coefficient for three-dimensional spatial resolution creation processing and a tap coefficient for resizing processing for resizing the image size to various magnifications are stored.

  A class code is supplied to each coefficient memory of the coefficient memory group 731 from the class classification circuit 711A in FIG. 64 via the switching circuit 713A. Each coefficient memory reads the tap coefficient of the class corresponding to the class code supplied thereto and outputs it to the selection unit 732.

  As described above, the tap coefficient read from each coefficient memory of the coefficient memory group 731 is supplied to the selection unit 732, and the command string from the reception unit 720 is also supplied. The selection unit 732 selects one of tap coefficients supplied from each coefficient memory of the coefficient memory group 731 according to the command string supplied from the reception unit 211, and supplies the selected tap coefficient to the prediction calculation circuit 716A.

  Note that the coefficient output unit 715B of FIG. 64 is configured in the same manner as the coefficient output circuit 715A of FIG. However, the coefficient memories of the coefficient memory group 731 constituting the coefficient output units 715A and 715B do not have to be the same. That is, the coefficient output units 715A and 715B can store tap coefficients that are partially different or all different.

  Next, with reference to the flowchart of FIG. 67, the processing of the IC 700 of FIG. 64 when a command sequence including one command “Kaizodo 1 dimension” is transmitted from the outside will be described.

  First, in step S701, the receiving unit 720 receives a command sequence from the outside, supplies it to the necessary blocks constituting the IC 700, and proceeds to step S702.

  In step S702, the IC 700 switches its internal structure in accordance with the command string received by the receiving unit 720, and the process proceeds to step S703.

  That is, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 (FIG. 64) are connected to the signal line according to one command “Kaizodo 1dimentional” constituting the command string. Is switched to perform one-dimensional spatial resolution creation processing corresponding to the command “Kaizodo 1dimentional”.

  FIG. 68 shows an IC 700 in which the signal line selection state is switched to perform the one-dimensional spatial resolution creation process.

  In FIG. 68, the switching circuits 713A and 713B, switching circuits 714A and 714B, switching circuit 719, and switching circuits 724 to 727 of the IC 700 select a signal line according to one command “Kaizodo 1dimentional” that constitutes a command string. The state is switched as indicated by a solid line in FIG. 68, whereby a one-dimensional spatial resolution creation process is performed as signal processing.

  In FIG. 68, a signal line indicated by a dotted line means a signal line that is wired but has nothing to do with signal processing by the current IC 700.

  In the IC 700 of FIG. 68, the input image data input to the IC 700 is used as the first image data of the class classification adaptive process, and the spatial resolution is improved, and the high resolution as the second image data of the class classification adaptive process A one-dimensional spatial resolution creation process for obtaining the image data is performed.

  Hereinafter, the processing after step S703 in the flowchart of FIG. 67 is performed in the IC 700 whose internal structure is switched as shown in FIG.

  That is, in step S703, one of the pixels constituting the high-resolution image data as the second image data in the class classification adaptation process that has not yet been set as the target pixel is selected as the target pixel, and the process proceeds to step S704.

  In step S704, the class classification circuit 711A selects a class tap for the pixel of interest from the input image data as the first image data of the class classification adaptation process, and performs one-dimensional class classification based on the class tap. . Then, the class classification circuit 711A supplies the class code obtained by the one-dimensional class classification to the coefficient output circuit 715A via the switching circuit 713A, and proceeds from step S704 to S705.

  The coefficient output circuit 715A is for one-dimensional spatial resolution creation processing among a plurality of types of tap coefficients described with reference to FIG. 66 in response to one command “Kaizodo 1dimentional” constituting the command sequence supplied from the receiving unit 720. The tap coefficient is selected. In step S705, the coefficient output circuit 715A reads the tap coefficient of the class corresponding to the class code supplied from the class classification circuit 711A via the switching circuit 713A among the selected tap coefficients. Acquired and supplied to the prediction calculation circuit 716A, the process proceeds to step S706.

  In step S706, the delay selection circuit 712A selects a one-dimensional prediction tap for the pixel of interest from the input image data as the first image data of the class classification adaptation process, and the prediction calculation circuit 716A via the switching circuit 714A. The process proceeds to step S707.

  In step S707, the prediction calculation circuit 716A uses the one-dimensional prediction tap supplied from the delay selection circuit 712A via the switching circuit 714A and the tap coefficient supplied from the coefficient output circuit 715A, and uses Equation (1). A filter operation (one-dimensional filter operation) is performed, thereby obtaining and outputting a pixel of interest (pixel value) of the second image data in which the spatial resolution is improved as compared with the first image data.

  Therefore, in this case, the IC 700 functions as a one-dimensional digital filter that performs filtering on a one-dimensional tap (prediction tap).

  The image data output by the prediction calculation circuit 716A is output to the outside of the IC 700 via the switching circuit 719.

  Then, the process proceeds to step S708, and it is determined whether or not all the pixels constituting the frame of the target pixel (hereinafter referred to as the target frame as appropriate) have been set as the target pixel. If it is determined in step S708 that all the pixels constituting the target frame have not yet been set as the target pixel, the process returns to step S703, and the same processing is repeated thereafter.

  If it is determined in step S708 that all of the pixels constituting the target frame are the target pixels, the process ends.

  If input image data of a frame next to the frame of interest is supplied to the IC 700, the process of steps S703 to S708 is repeated with that frame as a new frame of interest.

  Next, the processing of the IC 700 in FIG. 64 when a command sequence including one command “Kaizodo 2 dimension” is transmitted from the outside will be described with reference to the flowchart in FIG. 69.

  First, in step S711, the receiving unit 720 receives a command string from the outside, supplies it to the necessary blocks constituting the IC 700, and proceeds to step S712.

  In step S712, the IC 700 switches its internal structure according to the command string received by the receiving unit 720, and the process proceeds to step S713.

  That is, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 (FIG. 64) Is switched to perform the two-dimensional spatial resolution creation process corresponding to the command “Kaizodo 2dimentional”.

  FIG. 70 shows an IC 700 in which the signal line selection state is switched to perform a two-dimensional spatial resolution creation process.

  In FIG. 70, switching circuits 713A and 713B, switching circuits 714A and 714B, switching circuit 719, and switching circuits 724 to 727 of IC 700 select a signal line in accordance with one command “Kaizodo 2dimentional” constituting a command sequence. The state is switched as indicated by a solid line in FIG. 70, whereby a two-dimensional spatial resolution creation process is performed as a signal process.

  In FIG. 70, a signal line indicated by a dotted line means a signal line that is wired but is not related to signal processing by the current IC 700.

  In the IC 700 of FIG. 70, the input image data input to the IC 700 is used as the first image data of the class classification adaptive process, and the spatial resolution is improved, and the high resolution as the second image data of the class classification adaptive process is improved. Two-dimensional spatial resolution creation processing for obtaining the image data is performed.

  Hereinafter, the processing after step S713 in the flowchart of FIG. 69 is performed in the IC 700 whose internal structure is switched as shown in FIG.

  That is, in step S713, one of the pixels constituting the high-resolution image data as the second image data of the class classification adaptive process that has not yet been set as the target pixel is selected as the target pixel, and the process proceeds to step S714.

  In step S714, the class classification circuit 711A determines the target pixel from the input image data as the first image data of the class classification adaptation process and the image data output by the line delay circuit 717 after delaying the input image data. Class taps are selected, and two-dimensional classification is performed based on the class taps. Then, the class classification circuit 711A supplies the class code obtained by the two-dimensional class classification to the coefficient output circuit 715A via the switching circuit 713A, and proceeds from step S714 to S715.

  The coefficient output circuit 715A is for two-dimensional spatial resolution creation processing among a plurality of types of tap coefficients described with reference to FIG. 66 in response to one command “Kaizodo 2dimentional” constituting the command sequence supplied from the receiving unit 720. The tap coefficient is selected. In step S715, the coefficient output circuit 715A reads the tap coefficient of the class corresponding to the class code supplied from the class classification circuit 711A via the switching circuit 713A among the selected tap coefficients. Acquired and supplied to the prediction calculation circuit 716A, the process proceeds to step S716.

  In step S716, the delay selection circuit 712A determines the pixel of interest from the input image data as the first image data of the class classification adaptation process and the image data output by the line delay circuit 717 by delaying the input image data. The two-dimensional prediction tap is selected and supplied to the prediction calculation circuit 716A via the switching circuit 714A, and the process proceeds to step S717.

  In step S717, the prediction calculation circuit 716A uses the two-dimensional prediction tap supplied from the delay selection circuit 712A via the switching circuit 714A and the tap coefficient supplied from the coefficient output circuit 715A, and uses Equation (1). A filter calculation (two-dimensional filter calculation) is performed, thereby obtaining and outputting a pixel of interest (pixel value) of the second image data having a spatial resolution improved over that of the first image data.

  The image data output by the prediction calculation circuit 716A is output to the outside of the IC 700 via the switching circuit 719.

  Therefore, in this case, the IC 700 functions as a two-dimensional digital filter that performs filtering on a two-dimensional tap (prediction tap).

  Then, the process proceeds to step S718, where it is determined whether or not all the pixels constituting the target pixel frame (target frame) have been set as the target pixel. If it is determined in step S718 that all the pixels constituting the target frame have not yet been set as the target pixel, the process returns to step S713, and the same processing is repeated thereafter.

  If it is determined in step S718 that all the pixels constituting the target frame are the target pixels, the process ends.

  If the input image data of the frame next to the target frame is supplied to the IC 700, the process of steps S713 to S718 is repeated with that frame as a new target frame.

  Next, processing of the IC 700 in FIG. 64 when a command sequence including one command “Kaizodo 3 dimension” is transmitted from the outside will be described with reference to the flowchart in FIG. 71.

  First, in step S721, the receiving unit 720 receives a command string from the outside, supplies it to the necessary blocks constituting the IC 700, and proceeds to step S722.

  In step S722, the IC 700 switches its internal structure according to the command string received by the receiving unit 720, and the process proceeds to step S723.

  That is, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 (FIG. 64) Is switched to perform the 3D spatial resolution creation process corresponding to the command “Kaizodo 3dimentional”.

  FIG. 72 shows an IC 700 in which the signal line selection state is switched to perform a three-dimensional spatial resolution creation process.

  In FIG. 72, the switching circuits 713A and 713B, switching circuits 714A and 714B, switching circuit 719, and switching circuits 724 to 727 of the IC 700 select a signal line according to one command “Kaizodo 3dimentional” that constitutes a command string. The state is switched as indicated by a solid line in FIG. 72, whereby a three-dimensional spatial resolution creation process is performed as signal processing.

  In FIG. 72, a signal line indicated by a dotted line means a signal line that is wired but has nothing to do with signal processing by the current IC 700.

  In the IC 700 of FIG. 72, the input image data input to the IC 700 is used as the first image data of the class classification adaptation process, and the spatial resolution is improved, and the high resolution as the second image data of the class classification adaptation process is improved. A three-dimensional spatial resolution creation process for obtaining the image data is performed.

  Hereinafter, the processing after step S723 in the flowchart of FIG. 71 is performed in the IC 700 whose internal structure is switched as shown in FIG.

  That is, in step S723, one of the pixels constituting the high-resolution image data as the second image data of the class classification adaptive process that has not yet been set as the target pixel is selected as the target pixel, and the process proceeds to step S724.

  In step S724, the class classification circuit 711B selects a class tap for the pixel of interest, and performs three-dimensional class classification based on the class tap.

  Here, in FIG. 72, the input image data as the first image data of the class classification adaptive processing is supplied to the class classification circuit 711B via the switching circuits 724, 725, and 726. Further, the input image data is delayed by one or several lines by the line delay circuit 717 and supplied to the class classification circuit 711 B via the switching circuit 727. Also, the input image data is supplied to the class classification circuit 711B after being delayed by one frame in the frame memory 721.

  The class classification circuit 711B selects a class tap from these image data, and performs three-dimensional class classification based on the class tap.

  Then, the class classification circuit 711B supplies the class code obtained by the three-dimensional class classification to the coefficient output circuit 715B via the switching circuit 713B, and proceeds from step S724 to S725.

  The coefficient output circuit 715B is for three-dimensional spatial resolution creation processing among a plurality of types of tap coefficients described in FIG. 66 in response to one command “Kaizodo 3dimentional” constituting the command sequence supplied from the receiving unit 720. The tap coefficient is selected. In step S725, the coefficient output circuit 715B reads out the tap coefficient of the class corresponding to the class code supplied from the class classification circuit 711B via the switching circuit 713B among the selected tap coefficients. Acquired and supplied to the prediction calculation circuit 716B, the process proceeds to step S726.

  In step S726, the delay selection circuit 712B selects a three-dimensional prediction tap for the pixel of interest, supplies it to the prediction calculation circuit 716B via the switching circuit 714B, and proceeds to step S727.

  Here, in FIG. 72, the delay selection circuit 712B is supplied with the input image data as the first image data of the class classification adaptive processing via the switching circuits 724, 725, and 726. Further, the input image data is delayed by one or several lines by the line delay circuit 717 and supplied to the delay selection circuit 712 B via the switching circuit 727. The input image data is supplied to the delay selection circuit 712B after being delayed by one frame in the frame memory 721.

  The delay selection circuit 712B selects a three-dimensional prediction tap from these image data, and supplies it to the prediction calculation circuit 716B via the switching circuit 714B.

  In step S727, the prediction calculation circuit 716B uses the three-dimensional prediction tap supplied from the delay selection circuit 712B via the switching circuit 714B and the tap coefficient supplied from the coefficient output circuit 715B, and uses Equation (1). A filter calculation (three-dimensional filter calculation) is performed, thereby obtaining and outputting a pixel of interest (pixel value) of the second image data in which the spatial resolution is improved as compared with the first image data.

  The image data output by the prediction calculation circuit 716B is output to the outside of the IC 700 via (bypass) the product-sum calculation circuit 718 and the switching circuit 719.

  Therefore, in this case, the IC 700 functions as a three-dimensional digital filter that performs filtering on a three-dimensional tap (prediction tap).

  Then, the process proceeds to step S728, and it is determined whether or not all the pixels constituting the target pixel frame (target frame) are set as the target pixels. If it is determined in step S728 that all the pixels constituting the target frame have not yet been set as the target pixel, the process returns to step S723, and the same processing is repeated thereafter.

  If it is determined in step S728 that all the pixels constituting the target frame are the target pixels, the process ends.

  If the input image data of the frame next to the frame of interest is supplied to the IC 700, the process of steps S723 to S728 is repeated with that frame as a new frame of interest.

  Next, the processing of the IC 700 in FIG. 64 when a command sequence including two commands “Kaizodo 2dimentional” and “Kaizodo 3dimentional” is transmitted from the outside will be described with reference to the flowchart in FIG.

  First, in step S731, the receiving unit 720 receives a command string from the outside, supplies it to the necessary blocks that constitute the IC 700, and proceeds to step S732.

  In step S732, the IC 700 switches its internal structure according to the command string received by the receiving unit 720, and the process proceeds to step S733.

  That is, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 (FIG. 64) Correspondingly, the 2D spatial resolution creation process corresponding to the command “Kaizodo 3dimentional” and the 2D spatial resolution creation process corresponding to the command “Kaizodo 3dimentional” are performed for the selection state of the signal line. A process of weighting and adding the image data obtained as a result of the processing and the image data obtained as a result of the three-dimensional spatial resolution creation process with a weight according to the motion (hereinafter referred to as adaptive spatial resolution creation process as appropriate) Switch to.

  FIG. 74 shows an IC 700 in which the selection state of signal lines is switched to perform adaptive spatial resolution creation processing.

  In FIG. 74, the switching circuits 713A and 713B, switching circuits 714A and 714B, switching circuit 719, and switching circuits 724 to 727 of the IC 700 correspond to two commands “Kaizodo 2dimentional” and “Kaizodo 3dimentional” constituting the command sequence. The selection state of the signal line is switched as indicated by a solid line in FIG. 74, whereby adaptive spatial resolution creation processing is performed as signal processing.

  In FIG. 74, a signal line indicated by a dotted line means a signal line that is wired but is not related to the current signal processing by the IC 700.

  In the IC 700 of FIG. 74, the input image data input to the IC 700 is used as the first image data of the class classification adaptation process, and the spatial resolution is improved, and the high resolution as the second image data of the class classification adaptation process Two-dimensional spatial resolution creation processing and three-dimensional spatial resolution creation processing for obtaining the image data are performed simultaneously. Then, the image data obtained as a result of the two-dimensional spatial resolution creation process and the image data obtained as a result of the three-dimensional spatial resolution creation process are weighted and added with a weight according to the motion of the image.

  Hereinafter, the processing after step S733 in the flowchart of FIG. 73 is performed in the IC 700 whose internal structure is switched as shown in FIG.

  That is, in step S733, the target pixel is selected, and the process proceeds to step S734.

  In step S734, the two-dimensional spatial resolution creation process described with reference to FIGS. 69 and 70 and the three-dimensional spatial resolution creation process described with reference to FIGS. 71 and 72 are performed on the target pixel. Then, the pixel value of the target pixel obtained by the two-dimensional spatial resolution creation process is supplied from the prediction arithmetic circuit 716A to the product-sum operation circuit 718, and the pixel value of the target pixel obtained by the three-dimensional spatial resolution creation process is The prediction operation circuit 716 </ b> B supplies the product-sum operation circuit 718.

  In step S734, the class classification circuit 711B detects the movement of the input image data in the same manner as in the class classification circuit 511B of FIG. 59 described in the third embodiment, and motion information K representing the movement is obtained. Is supplied to the product-sum operation circuit 718.

  Thereafter, the process proceeds from step S734 to S735, and the product-sum operation circuit 718 generates the pixel value of the target pixel obtained by the two-dimensional spatial resolution creation processing from the prediction calculation circuit 716A and the three-dimensional spatial resolution creation from the prediction calculation circuit 716B. Weighted addition with the pixel value of the target pixel obtained by the processing is performed using the pixel motion information K of the input image data corresponding to the target pixel supplied from the class classification circuit 711B as a weighting coefficient.

  That is, if the pixel values obtained by the two-dimensional spatial resolution creation process or the three-dimensional spatial resolution creation process are P2 and P3, respectively, and the motion information K has a value in the range of 0 to 1, The sum calculation circuit 718 obtains a pixel value (pixel value of the target pixel) P as a result of the adaptive spatial resolution creation process according to the equation P = K × P2 + (1−K) × P3.

  The pixel value obtained by the product-sum operation circuit 718 is output to the outside of the IC 700 via the switching circuit 719.

  Then, the process proceeds from step S735 to S736, and it is determined whether or not all the pixels constituting the target pixel frame (target frame) are the target pixels. If it is determined in step S736 that all the pixels constituting the target frame have not yet been set as the target pixel, the process returns to step S733, and the same processing is repeated thereafter.

  If it is determined in step S736 that all the pixels constituting the target frame are the target pixels, the process ends.

  If input image data of a frame next to the frame of interest is supplied to the IC 700, the process of steps S733 to S736 is repeated with that frame as a new frame of interest.

  Next, the processing of the IC 700 in FIG. 64 when a command sequence consisting of one command “Zoom ver1” is transmitted from the outside will be described with reference to the flowchart in FIG.

  First, in step S741, the receiving unit 720 receives a command string from the outside, supplies it to the necessary blocks that constitute the IC 700, and proceeds to step S742.

  In step S742, the IC 700 switches the internal structure according to the command string received by the receiving unit 720, and the process proceeds to step S743.

  That is, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 (FIG. 64) Is switched to perform recursive enlargement processing corresponding to the command “Zoom ver1”.

  FIG. 76 shows an IC 700 in which the signal line selection state is switched to perform recursive enlargement processing.

  In FIG. 76, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 select the signal line according to one command “Zoom ver1” that constitutes the command sequence. The state is switched as indicated by a solid line in FIG. 76, whereby recursive enlargement processing is performed as signal processing.

  In FIG. 76, a signal line indicated by a dotted line means a signal line that is wired but has nothing to do with signal processing by the current IC 700.

  In the IC 700 of FIG. 76, the input image data input to the IC 700 is used as the first image data of the classification adaptation process, and a part of the blocks is resized to a magnification (desired magnification) desired by the user. Then, a recursive enlargement process is performed to obtain image data having an enlarged size as the second image data in the class classification adaptation process.

  It is assumed that the desired magnification is included in the command string, for example, as the parameter p1 of the command “Zoom ver1”. Further, it is assumed that the position that becomes the center of enlargement when the input image data is enlarged is also included in the command string as the parameter p2 of the command “Zoom ver1”.

  Hereinafter, the processing after step S743 in the flowchart of FIG. 75 is performed in the IC 700 whose internal structure is switched as shown in FIG.

  That is, in step S743, the blocking circuit 722 extracts a block that is an object of enlargement of input image data supplied via the switching circuit 724.

  That is, the blocking circuit 722 expands by the magnification represented by the parameter p1 from the magnification represented by the parameter “p1” of the command “Zoom ver1” included in the command sequence supplied from the receiving unit 720 and the size of the input image data. Thus, the block size (hereinafter referred to as the extraction size as appropriate) that is the size of the input image data is recognized. Further, the blocking circuit 722 converts the image data of the extracted size block into the input image data with the position (center of gravity) represented by the parameter “p2” of the command “Zoom ver1” included in the command sequence supplied from the receiving unit 720 as the center (center of gravity). Extract from In this case, for example, both the horizontal and vertical directions of the block are enlarged at a magnification represented by the parameter p1.

  Image data of the extracted size block is supplied from the blocking circuit 722 to the class classification circuit 711B and the delay selection circuit 712B via the switching circuit 726.

  Thereafter, the process proceeds from step S743 to S744, and among the pixels constituting the enlarged image data as the second image data of the class classification adaptive process obtained by enlarging the image data of the extracted size block, it is still a target pixel. One that does not exist is selected as the target pixel, and the process proceeds to step S745.

  In step S745, the class classification circuit 711B selects a class tap for the pixel of interest from the image data of the extracted size block supplied from the blocking circuit 722 via the switching circuit 726, and based on the class tap, Perform two-dimensional classification.

  Then, the class classification circuit 711B supplies the class code obtained by the two-dimensional class classification to the coefficient output circuit 715B via the switching circuit 713B, and proceeds from step S745 to S746.

  The coefficient output circuit 715B selects a tap coefficient for resizing processing from among a plurality of types of tap coefficients described with reference to FIG. 66 in response to one command “Zoom ver1” that configures a command sequence supplied from the receiving unit 720. Selected.

  The coefficient output circuit 715B selects a resizing process tap coefficient selected according to one command “Zoom ver1” constituting the command sequence supplied from the receiving unit 720, and the image is obtained by multiplying an image with a predetermined magnification (for example, , Relatively low magnification such as 1.1 times). Therefore, the tap coefficient for resizing processing selected by the coefficient output circuit 715B according to the command “Zoom ver1” does not necessarily expand the image to the magnification represented by the parameter p1.

  However, in the coefficient output circuit 715B, a plurality of sets of tap coefficients for resizing processing for enlarging each of a plurality of magnifications are prepared, and the command “Zoom ver1” is selected from the plurality of sets of tap coefficients. One set of tap coefficients (for example, one that enlarges the magnification closest to the magnification represented by the parameter p1) may be selected.

  In step S746, the coefficient output circuit 715B selects the tap coefficient of the class corresponding to the class code supplied from the class classification circuit 711B via the switching circuit 713B according to the command “Zoom ver1”. Is obtained by reading from the tap coefficients, and is supplied to the prediction calculation circuit 716B, and the process proceeds to step S747.

  In step S747, the delay selection circuit 712B selects a two-dimensional prediction tap for the pixel of interest from the image data of the extracted size block supplied from the blocking circuit 722 via the switching circuit 726, and passes through the switching circuit 714B. Is supplied to the prediction calculation circuit 716B, and the process proceeds to step S748.

  In step S748, the prediction calculation circuit 716B uses the two-dimensional prediction tap supplied from the delay selection circuit 712B via the switching circuit 714B and the tap coefficient supplied from the coefficient output circuit 715B, and uses Equation (1). A filter operation (two-dimensional filter operation) is performed, and thereby a target pixel (pixel value) of the enlarged image data is obtained and output.

  The enlarged image data output from the prediction calculation circuit 716B is supplied to the determination circuit 723.

  Then, the process proceeds to step S749, where it is determined whether or not all the pixels constituting the enlarged image data are the target pixel. If it is determined in step S749 that all the pixels constituting the enlarged image data have not yet been set as the target pixel, the process returns to step S744, and the same processing is repeated thereafter.

  If it is determined in step S749 that all the pixels constituting the enlarged image are the target pixels, the process proceeds to step S750, and the determination circuit 723 extracts the enlarged image data supplied from the prediction calculation circuit 716B. It is determined whether or not the size of the image data of the block is enlarged to the magnification indicated by the parameter p1.

  In step S750, when it is determined that the enlarged image data from the prediction calculation circuit 716B does not have a size obtained by enlarging the image data of the extracted size block to the magnification represented by the parameter p1, that is, from the prediction calculation circuit 716B. When the size of the enlarged image data is smaller than the size obtained by enlarging the image data of the extracted size block to the magnification represented by the parameter p1, the process proceeds to step S751, and the determination circuit 723 receives the enlarged image data from the prediction arithmetic circuit 716B. Is fed back (supplied) to the class classification circuit 711B and the selection delay circuit 712B via the switching circuit 727, and the process returns to step S744.

  Then, the processing from step S744 onward is repeated for the enlarged image data fed back from the determination circuit 723 to the class classification circuit 711B and the selection delay circuit 712B (hereinafter referred to as feedback image data as appropriate).

  That is, in this case, in step S744, among the pixels constituting the enlarged image data as the second image data of the class classification adaptive processing obtained by enlarging the feedback image data by the magnification represented by the parameter p1, One of the pixels that is not the target pixel is selected as the target pixel, and the process proceeds to step S745.

  In step S745, the class classification circuit 711B selects a class tap for the target pixel from the feedback image data from the determination circuit 723, performs two-dimensional class classification based on the class tap, and performs the two-dimensional class classification. The obtained class code is supplied to the coefficient output circuit 715B via the switching circuit 713B, and the process proceeds from step S745 to S746.

  In step S746, the coefficient output circuit 715B selects the tap coefficient of the class corresponding to the class code supplied from the class classification circuit 711B via the switching circuit 713B according to the command “Zoom ver1”. Is obtained by reading from the tap coefficients for use, supplied to the prediction calculation circuit 716B, and the process proceeds to step S747.

  In step S747, the delay selection circuit 712B selects a two-dimensional prediction tap for the target pixel from the feedback image data from the determination circuit 723, and supplies the selected two-dimensional prediction tap to the prediction calculation circuit 716B via the switching circuit 714B. Proceed to

  In step S748, the prediction calculation circuit 716B uses the two-dimensional prediction tap supplied from the delay selection circuit 712B via the switching circuit 714B and the tap coefficient supplied from the coefficient output circuit 715B, and uses Equation (1). A filter operation (two-dimensional filter operation) is performed, whereby a target pixel (pixel value) of the enlarged image data is obtained and output to the determination circuit 723.

  Then, the process proceeds to step S749, where it is determined whether or not all the pixels constituting the enlarged image data are the target pixel. If it is determined in step S749 that all the pixels constituting the enlarged image data have not yet been set as the target pixel, the process returns to step S744, and the same processing is repeated thereafter.

  If it is determined in step S749 that all of the pixels constituting the enlarged image are the target pixel, the process proceeds to step S750, and as described above, in the determination circuit 723, the enlargement supplied from the prediction arithmetic circuit 716B. It is determined whether or not the image data has a size obtained by enlarging the image data of the extracted size block to the magnification represented by the parameter p1.

  If it is determined in step S750 that the enlarged image data from the prediction calculation circuit 716B has a size obtained by enlarging the image data of the extracted size block to the magnification represented by the parameter p1, the determination circuit 723 The enlarged image data from the prediction calculation circuit 716B is output to the outside of the IC 700, and the process is terminated.

  Therefore, in this case, the IC 700 functions as an enlargement device that enlarges an image.

  If input image data of the next frame is supplied to the IC 700, the processes of steps S743 to S751 are repeated for the next frame.

  Next, the processing of the IC 700 in FIG. 64 when a command sequence consisting of one command “Zoom ver2” is transmitted from the outside will be described with reference to the flowchart in FIG. 77.

  First, in step S761, the receiving unit 720 receives a command string from the outside, supplies it to the necessary blocks constituting the IC 700, and proceeds to step S762.

  In step S762, the IC 700 switches the internal structure in accordance with the command string received by the receiving unit 720, and the process proceeds to step S763.

  That is, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 (FIG. 64) are connected to the signal line according to one command “Zoom ver2” constituting the command sequence. The selection state of is switched so that the one-time enlargement process corresponding to the command “Zoom ver2” is performed.

  FIG. 78 shows the IC 700 in which the signal line selection state is switched to perform the enlargement process once.

  In FIG. 78, the switching circuits 713A and 713B, switching circuits 714A and 714B, switching circuit 719, and switching circuits 724 to 727 of the IC 700 select the signal line according to one command “Zoom ver2” that constitutes a command string. The state is switched as indicated by a solid line in FIG. 78, and as a result, enlargement processing is performed once.

  In FIG. 78, a signal line indicated by a dotted line means a signal line that is wired but is not related to the current signal processing by the IC 700.

  In the IC 700 of FIG. 78, the input image data input to the IC 700 is used as the first image data of the class classification adaptation process, and a part of the blocks is resized at a desired magnification (request magnification) requested by the user. A one-time enlargement process is performed to obtain image data having an enlarged size as the second image data in the class classification adaptive process.

  Note that the desired magnification is included in the command string, for example, as the parameter p1 of the command “Zoom ver2”. Further, it is assumed that the position that becomes the center of enlargement when the input image data is enlarged is also included in the command string as the parameter p2 of the command “Zoom ver2”.

  Hereinafter, the processing after step S763 in the flowchart of FIG. 77 is performed in the IC 700 whose internal structure is switched as shown in FIG.

  That is, in step S763, the blocking circuit 722 extracts a block that is an object of enlargement of input image data supplied via the switching circuit 724, as in step S743 of FIG.

  That is, the blocking circuit 722 enlarges by the magnification represented by the parameter p1 from the magnification represented by the parameter “p1” of the command “Zoom ver2” included in the command sequence supplied from the receiving unit 720 and the size of the input image data. Thus, the block size (extraction size) that is the size of the input image data is recognized. Further, the blocking circuit 722 converts the image data of the extracted size block into the input image data with the position (center of gravity) indicated by the parameter “p2” of the command “Zoom ver2” included in the command sequence supplied from the receiving unit 720 as the center (center of gravity). Extract from

  Image data of the extracted size block is supplied from the blocking circuit 722 to the class classification circuit 711B and the delay selection circuit 712B via the switching circuit 726.

  After that, the process proceeds from step S763 to S764, and among the pixels constituting the enlarged image data as the second image data of the class classification adaptive processing obtained by enlarging the image data of the extracted size block, it is still a target pixel. One that does not exist is selected as the target pixel, and the process proceeds to step S765.

  In step S765, the class classification circuit 711B selects a class tap for the pixel of interest from the image data of the extracted size block supplied from the blocking circuit 722 via the switching circuit 726, and based on the class tap, Perform two-dimensional classification.

  Then, the class classification circuit 711B supplies the class code obtained by the two-dimensional class classification to the coefficient output circuit 715B via the switching circuit 713B, and proceeds from step S765 to S766.

  The coefficient output circuit 715B, according to one command “Zoom ver2” configuring the command sequence supplied from the receiving unit 720, the magnification that the parameter p1 represents the image of the plural types of tap coefficients described in FIG. The tap coefficient for the resizing process to be enlarged is selected.

  In step S766, the coefficient output circuit 715B selects the tap coefficient of the class corresponding to the class code supplied from the class classification circuit 711B via the switching circuit 713B according to the command “Zoom ver2”. Is obtained by reading from the tap coefficients for use, supplied to the prediction calculation circuit 716B, and proceeds to step S767.

  In step S767, the delay selection circuit 712B selects a two-dimensional prediction tap for the pixel of interest from the image data of the extracted size block supplied from the blocking circuit 722 via the switching circuit 726, and passes through the switching circuit 714B. Is supplied to the prediction calculation circuit 716B, and the process proceeds to step S768.

  In step S768, the prediction calculation circuit 716B uses the two-dimensional prediction tap supplied from the delay selection circuit 712B via the switching circuit 714B and the tap coefficient supplied from the coefficient output circuit 715B, and uses Equation (1). A filter operation (two-dimensional filter operation) is performed, thereby obtaining and outputting a pixel of interest (pixel value) of enlarged image data obtained by enlarging the image data of the extracted size block at a magnification represented by the parameter p1.

  The enlarged image data output by the prediction calculation circuit 716B is output to the outside of the IC 700 via the determination circuit 723.

  Proceeding to step S769, it is determined whether all the pixels constituting the enlarged image data have been set as the target pixel. If it is determined in step S769 that all the pixels constituting the enlarged image data have not yet been set as the target pixel, the process returns to step S764, and the same processing is repeated thereafter.

  If it is determined in step S769 that all the pixels constituting the enlarged image are the target pixels, the process is terminated.

  Therefore, in this case as well, the IC 700 functions as an enlargement device that enlarges an image.

  If input image data of the next frame is supplied to the IC 700, the processes of steps S764 to S769 are repeated for the next frame.

  Next, with reference to the flowchart of FIG. 79, the processing of the IC 700 of FIG. 64 when a command sequence including two commands “Kaizodo 2 dimension” and “Zoom ver1” is transmitted from the outside will be described.

  First, in step S781, the receiving unit 720 receives a command sequence from the outside, supplies it to the necessary blocks that constitute the IC 700, and proceeds to step S782.

  In step S782, the IC 700 switches the internal structure in accordance with the command string received by the receiving unit 720, and the process proceeds to step S783.

  That is, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 (FIG. 64) use two commands “Kaizodo 2dimentional” and “Zoom ver1” that constitute the command sequence. In response, the signal line selection state is switched to perform a two-dimensional spatial resolution creation process corresponding to the command “Kaizodo 2 dimension” and a recursive enlargement process corresponding to the command “Zoom ver1”.

  FIG. 80 shows an IC 700 in which the signal line selection state is switched to perform a two-dimensional spatial resolution creation process and to perform a recursive enlargement process on the result of the process.

  In FIG. 80, switching circuits 713A and 713B, switching circuits 714A and 714B, switching circuit 719, and switching circuits 724 to 727 of IC 700 correspond to two commands “Kaizodo 2dimentional” and “Zoom ver1” that constitute a command string. Then, the signal line selection state is switched as indicated by a solid line in FIG. 80, whereby a two-dimensional spatial resolution creation process is performed as a signal process, and a recursive enlargement process is performed on the process result.

  In FIG. 80, a signal line indicated by a dotted line means a signal line that is wired but has nothing to do with signal processing by the current IC 700.

  In the IC 700 of FIG. 80, the input image data input to the IC 700 is used as the first image data of the class classification adaptation process, and the spatial resolution is improved, and the high resolution as the second image data of the class classification adaptation process is improved. Two-dimensional spatial resolution creation processing for obtaining the image data is performed. Further, in the IC 700 of FIG. 80, the high-resolution image data is used as the first image data for the class classification adaptation process, and the enlarged image data as the second image data for the class classification adaptation process is enlarged. The resulting recursive expansion process is performed.

  Hereinafter, the processing after step S783 in the flowchart of FIG. 79 is performed in the IC 700 whose internal structure is switched as shown in FIG.

  That is, in step S783, the two-dimensional spatial resolution creation process described in steps S713 to S718 of FIG. 69 is performed. High-resolution image data obtained by the two-dimensional spatial resolution creation processing is supplied from the prediction calculation circuit 716A to the blocking circuit 722 via the switching circuits 719, 725, and 724.

  In step S784, the recursive enlargement process described in steps S743 to S751 in FIG. 75 is performed on the high-resolution image data supplied to the blocking circuit 722. The enlarged image data obtained by the recursive enlargement process is output from the determination circuit 723 to the outside of the IC 700.

  In this case, the IC 700 functions as a two-dimensional digital filter that performs filtering on a two-dimensional tap (prediction tap) and an enlargement device that performs image enlargement.

  If input image data of the next frame is supplied to the IC 700, the processes of steps S783 and S784 are repeated for the next frame.

  Next, with reference to the flowchart of FIG. 81, the processing of the IC 700 of FIG. 64 when a command sequence including two commands “Kaizodo 2 dimension” and “Zoom ver2” is transmitted from the outside will be described.

  First, in step S791, the receiving unit 720 receives an external command string, supplies it to the necessary blocks constituting the IC 700, and proceeds to step S792.

  In step S792, the IC 700 switches its internal structure according to the command string received by the receiving unit 720, and the process proceeds to step S793.

  That is, the switching circuits 713A and 713B, the switching circuits 714A and 714B, the switching circuit 719, and the switching circuits 724 to 727 of the IC 700 (FIG. 64) use two commands “Kaizodo 2dimentional” and “Zoom ver2” that constitute the command sequence. Accordingly, the signal line selection state is switched so as to perform a two-dimensional spatial resolution creation process corresponding to the command “Kaizodo 2 dimension” and a one-time enlargement process corresponding to the command “Zoom ver2”.

  FIG. 82 shows an IC 700 in which the signal line selection state is switched to perform a two-dimensional spatial resolution creation process and to perform a one-time enlargement process for the processing result.

  In FIG. 82, switching circuits 713A and 713B, switching circuits 714A and 714B, switching circuit 719, and switching circuits 724 to 727 of IC 700 correspond to two commands “Kaizodo 2dimentional” and “Zoom ver2” that constitute a command string. The signal line selection state is switched as shown by a solid line in FIG. 82, whereby a two-dimensional spatial resolution creation process is performed as a signal process, and an enlargement process is performed once for the process result.

  In FIG. 82, a signal line indicated by a dotted line means a signal line that is wired but has nothing to do with signal processing by the current IC 700.

  In the IC 700 of FIG. 82, the input image data input to the IC 700 is used as the first image data of the class classification adaptive process, and the spatial resolution is improved, and the high resolution as the second image data of the class classification adaptive process is improved. Two-dimensional spatial resolution creation processing for obtaining the image data is performed. Further, in the IC 700 of FIG. 82, the high-resolution image data is used as the first image data for the class classification adaptation process, and the enlarged image data as the second image data for the class classification adaptation process is enlarged. The obtained one-time enlargement process is performed.

  Hereinafter, the processing after step S793 in the flowchart of FIG. 81 is performed in the IC 700 whose internal structure is switched as shown in FIG.

  That is, in step S793, the two-dimensional spatial resolution creation process described in steps S713 to S718 of FIG. 69 is performed. High-resolution image data obtained by the two-dimensional spatial resolution creation processing is supplied from the prediction calculation circuit 716A to the blocking circuit 722 via the switching circuits 719, 725, and 724.

  Then, the process proceeds to step S794, and the one-time enlargement process described in steps S743 to S751 in FIG. 77 is performed on the high-resolution image data supplied to the blocking circuit 722. The enlarged image data obtained by this one-time enlargement process is output from the determination circuit 723 to the outside of the IC 700.

  Also in this case, the IC 700 functions as a two-dimensional digital filter that performs filtering on a two-dimensional tap (prediction tap) and an enlargement device that performs image enlargement.

  If input image data of the next frame is supplied to the IC 700, the processes of steps S793 and S794 are repeated for the next frame.

  As described above, in the IC 700, when the command sequence includes, for example, the first and second commands as a plurality of commands, the internal structure is changed according to the plurality of commands in the command sequence. The first signal processing corresponding to the first command and the second signal processing corresponding to the second command are switched to the execution state, the first signal processing is performed, and then the second signal processing is further performed. Execute. Therefore, according to the IC 700, a plurality of functions can be easily realized with a single hardware.

  In this specification, the processing of each step described with reference to the flowchart is not necessarily performed in time series in the order described in the flowchart, and is performed in parallel if possible. Also good.

  In the present embodiment, the signal processing is performed on the image data. However, the signal processing target is not limited to the image data, and may be audio data, for example.

It is a block diagram which shows the structural example of the image conversion apparatus 1 which performs the image conversion process by a class classification adaptive process. 4 is a flowchart for explaining image conversion processing by the image conversion apparatus 1. It is a block diagram which shows the structural example of the learning apparatus 21 which learns a tap coefficient. 3 is a block diagram illustrating a configuration example of a learning unit 36 of the learning device 21. FIG. It is a figure for demonstrating various image conversion processes. It is a flowchart explaining the learning process by the learning apparatus. It is a block diagram which shows the structural example of the image conversion apparatus 51 which performs the image conversion process by a class classification adaptive process. 3 is a block diagram illustrating a configuration example of a coefficient output unit 55 of the image conversion apparatus 51. FIG. It is a block diagram which shows the structural example of the learning apparatus 71 which learns coefficient seed data. It is a block diagram which shows the structural example of the learning part 76 of the learning apparatus 71. FIG. It is a flowchart explaining the learning process by the learning apparatus 71. FIG. It is a block diagram which shows the structural example of the learning apparatus 101 which learns coefficient seed data. 3 is a block diagram illustrating a configuration example of a learning unit 106 of the learning device 101. FIG. It is a block diagram which shows the structural example of one Embodiment of the AV system to which this invention is applied. It is a flowchart explaining operation | movement of the main body 200 of an AV system. It is a figure which shows the example of a command sequence. It is a flowchart explaining operation | movement of the main body 200 of an AV system. It is a figure which shows the example of a command sequence. It is a flowchart explaining operation | movement of the main body 200 of an AV system. It is a figure which shows the example of a command sequence. It is a flowchart explaining operation | movement of the main body 200 of an AV system. It is a figure which shows the example of a command sequence. It is a flowchart explaining operation | movement of the main body 200 of an AV system. It is a figure which shows the example of a command sequence. It is a flowchart explaining operation | movement of the main body 200 of an AV system. It is a figure which shows the example of a command sequence. 3 is a block diagram illustrating a configuration example of an R-IC 212. FIG. 4 is a block diagram illustrating a configuration example of a coefficient output unit 255. FIG. It is a flowchart explaining the process of R-IC212. It is a block diagram which shows the other structural example of R-IC212. 4 is a block diagram illustrating a configuration example of a coefficient output unit 275. FIG. It is a block diagram which shows the structural example of one Embodiment of the television receiver 301 to which this invention is applied. 6 is a diagram for explaining control of a tuner unit 311 by a system controller 318. FIG. It is a figure for demonstrating the screen display of normal screen mode, and the screen display of multi-screen mode. 3 is a block diagram illustrating a configuration example of a signal processing unit 314. FIG. It is a figure which shows the command sequence which the command sequence production | generation part 330 produces | generates. It is a figure explaining the image data input / output with respect to the signal processing chip 331. It is a figure explaining the image data input / output with respect to the signal processing chip 332. It is a figure explaining the image data input / output with respect to the signal processing chip 331. It is a figure explaining the image data input / output with respect to the signal processing chip 332. It is a flowchart explaining the process of the signal processing part 314. FIG. 3 is a block diagram illustrating a configuration example of a signal processing chip 334. FIG. 3 is a flowchart illustrating processing of a signal processing chip 334. 3 is a block diagram illustrating a configuration example of a signal processing chip 331. FIG. 4 is a flowchart illustrating processing of a signal processing chip 331. 3 is a block diagram illustrating a configuration example of a signal processing chip 332. FIG. It is a flowchart explaining the process of the signal processing chip 332. It is a block diagram which shows the structural example of one Embodiment of the television receiver to which this invention is applied. FIG. 38 is a plan view showing an example of an external configuration of a remote control 618. 3 is a block diagram illustrating an example of an electrical configuration of a remote control 618. FIG. 6 is a flowchart illustrating processing of a controller 613. It is a figure which shows the format of the command sequence which the controller 613 produces | generates. 6 is a flowchart illustrating processing of a controller 613. It is a figure which shows the example of the command sequence which the controller 613 produces | generates according to operation of the function button 621D. 2 is a block diagram illustrating a configuration example of an LSI 606. FIG. 5 is a flowchart for explaining processing of an LSI 606. 6 is a block diagram illustrating a configuration example of a signal processing circuit 656. FIG. FIG. 10 is a block diagram illustrating a configuration example of a signal processing circuit 661. FIG. 10 is a block diagram illustrating another configuration example of the signal processing circuit 661. It is a flowchart explaining the image conversion process as a class classification adaptive process performed with the class classification circuit 511A thru | or the prediction arithmetic circuit 516A, and the class classification circuit 511B thru | or the prediction arithmetic circuit 516B. It is a block diagram which shows the internal structure of the signal processing circuit 656 containing the signal processing circuit 661 which switched the selection state of the signal line | wire so that a one-dimensional spatial resolution creation process might be performed. It is a block diagram which shows the internal structure of the signal processing circuit 656 containing the signal processing circuit 661 which switched the selection state of the signal line | wire so that a two-dimensional spatial resolution creation process might be performed. It is a block diagram which shows the internal structure of the signal processing circuit 656 containing the signal processing circuit 661 which switched the selection state of the signal line | wire so that a noise removal process might be performed. It is a block diagram which shows the structural example of one Embodiment of IC700 to which this invention is applied. FIG. 65 is a diagram for explaining a command sequence received by the receiving unit 720 in FIG. 64 from the outside. It is a block diagram which shows the structural example of coefficient output circuit 715A. It is a flowchart explaining the process of IC700 when the command sequence which consists of command "Kaizodo 1dimentional" is transmitted. It is a block diagram which shows the internal structure of IC700 which switched the selection state of the signal line | wire so that a one-dimensional spatial resolution creation process may be performed. It is a flowchart explaining the process of IC700 when the command sequence which consists of command "Kaizodo 2dimentional" is transmitted. It is a block diagram which shows the internal structure of IC700 which switched the selection state of the signal line | wire so that a two-dimensional spatial resolution creation process might be performed. It is a flowchart explaining the process of IC700 when the command sequence which consists of command "Kaizodo 3dimentional" is transmitted. It is a block diagram which shows the internal structure of IC700 which switched the selection state of the signal line | wire so that a three-dimensional spatial resolution creation process might be performed. It is a flowchart explaining the process of IC700 when the command sequence which consists of command "Kaizodo 2dimentional" and "Kaizodo 3dimentional" is transmitted. It is a block diagram which shows the internal structure of IC700 which switched the selection state of the signal line | wire so that an adaptive spatial resolution creation process may be performed. It is a flowchart explaining the process of IC700 when the command sequence which consists of command "Zoom ver1" is transmitted. It is a block diagram which shows the internal structure of IC700 which switched the selection state of the signal line | wire so that a recursive expansion process might be performed. It is a flowchart explaining the process of IC700 when the command sequence which consists of command "Zoom ver2" is transmitted. It is a block diagram which shows the internal structure of IC700 which changed the selection state of the signal line | wire so that it might enlarge once. It is a flowchart explaining the process of IC700 when the command sequence which consists of command "Kaizodo 2dimentional" and "Zoom ver1" is transmitted. It is a block diagram which shows the internal structure of IC700 which switched the selection state of a signal line | wire to perform a two-dimensional spatial resolution creation process, and also to perform a recursive expansion process for the process result. It is a flowchart explaining the process of IC700 when the command sequence which consists of command "Kaizodo 2dimentional" and "Zoom ver2" is transmitted. FIG. 11 is a block diagram showing an internal structure of an IC 700 in which a signal line selection state is switched to perform a two-dimensional spatial resolution creation process and to perform a one-time enlargement process for the processing result.

Explanation of symbols

200 main body, 201 remote controller, 201A operation unit, 201B transmission unit, 202 control device, 202A transmission / reception unit, 202B control unit, 203 DVD recorder unit, 204 HDD recorder unit, 205 television receiver unit, 211 reception unit, 212 R- IC, 213 recording unit, 214 DVD, 215 playback unit, 221 reception unit, 222 R-IC, 223 recording unit, 224 HDD, 225 playback unit, 231 reception unit, 232 R-IC, 233 display, 251 pixel-of-interest selection unit , 252, 253 tap selection unit, 254 class classification unit, 255 coefficient output unit, 256 prediction calculation unit, 261 _{1 to} 261 ₄ coefficient memory, 262 selection unit, 275 coefficient output unit, 281 coefficient generation unit, 282 coefficient seed output unit , 283 parameter memory 284 coefficient memory, 291 ₁ to 291 ₄ coefficients Memory, 292 selection unit, 301 television receiver, 302 antenna, 303 remote controller, 311 tuner unit, 312 amplifier circuit, 313 speaker, 314 to 316 signal processing unit, 317 display, 318 controller, 319 remote control reception unit, 330 command sequence Generation unit, 331, 332 signal processing chip, 333 memory unit, 334 signal processing chip, 340 reception unit, 341 frame memory, 342 delay circuit, 343 selector, 344 motion detection circuit, 345 selection unit, 351 attention pixel selection unit, 352 , 353 tap selector, 354 classification unit, 355 the coefficient output unit, 356 prediction calculation unit, 357 reception unit, 358 motion determination unit, 361 _1, 361 ₂ coefficient memory, 362 selector, 371 a target pixel selecting section, 372 373 tap selector, 374 classification unit, 375 the coefficient output unit, 376 prediction calculation unit, 377 reception unit, 378, 379 tap determination unit, 381 _1, 381 ₂ coefficient memory, 382 selector, 411A, 411B arithmetic circuit group, 412A, 412B memory, 413A, 413B product-sum operation circuit group, 414A, 414B adder, 415A, 415B multiplier, 416A, 415B register group, 421A to 423A, 421B to 423B, 424 switching circuit, 511A, 511B class classification circuit, 512A, 512B delay selection circuit, 513A, 513B, 514A, 514B switching circuit, 515A, 515B coefficient output circuit, 516A, 516B prediction operation circuit, 517 line delay circuit, 518 product-sum operation circuit, 519 switching circuit, 521 , 521B scanning line conversion circuit, 523 frame memory, 601 tuner, 602 demodulation unit, 603 error correction unit, 604 demultiplexer, 605 video decoder, 606 LSI, 607 synthesis unit, 608 OSD unit, 609 LCD, 610 audio decoder, 611 Speaker, 613 controller, 614 key input unit, 615 display unit, 616 light receiving unit, 617 remote control I / F, 618 remote control, 621 user interface unit, 621A power button, 621B input button, 621C channel button, 621D function button, 622 LCD Panel, 631 bus, 632 control unit, 633 storage unit, 634 transmission unit, 650 reception unit, 651 to 653 frame memory, 654 SW circuit, 654A, 654B input terminal, 655 SW circuit, 55A, 655B output terminal, 656, 661 signal processing circuit, 662 to 665 switching circuit, 711A, 711B class classification circuit, 712A, 712B delay selection circuit, 713A, 713B, 714A, 714B switching circuit, 715A, 715B coefficient output circuit, 716A, 716B Prediction operation circuit, 717 line delay circuit, 718 product-sum operation circuit, 719 switching circuit, 721 frame memory, 722 block circuit, 723 determination circuit, 724 to 727 switching circuit, 731 memory group, 732 selection unit

Claims (17)

In a signal processing apparatus comprising first and second signal processing means,
The first signal processing means switches the internal structure in accordance with at least one command of a command string made up of a plurality of commands, performs signal processing on the first signal, and outputs a second signal. ,
The second signal processing means switches an internal structure in accordance with at least one command in the command sequence, performs signal processing on the second signal, and outputs a third signal. A signal processing device. The signal processing apparatus according to claim 1, wherein the first or second signal processing means is a one-chip IC (Integrated Circuit). The signal processing apparatus according to claim 1, further comprising command sequence generation means for generating the command sequence in response to an external signal. The first or second signal processing means switches the internal structure, thereby removing noise contained in the signal, removing distortion generated in the signal, and improving the spatial resolution of the image. The signal processing apparatus according to claim 1, wherein a spatial resolution creation process to be performed or a temporal resolution creation process to improve the temporal resolution of an image is performed. The first signal processing means includes a class tap coefficient obtained by classifying a signal of interest of interest in the second signal into one of a plurality of classes, and the attention signal The signal processing apparatus according to claim 1, wherein the signal of interest is obtained by an operation using the first signal selected for a signal. The first signal processing means includes
Class tap selection means for selecting, from the first signal, a class tap used for classifying the signal of interest into any one of the plurality of classes;
Class classification means for classifying the signal of interest based on the class tap;
A prediction tap selection means for selecting, from the first signal, a prediction tap used for calculation with the tap coefficient when obtaining the signal of interest;
Tap coefficient output means for outputting a tap coefficient of the class of the signal of interest;
6. An arithmetic unit that obtains the target signal by performing a calculation using a tap coefficient of the class of the target signal and the prediction tap selected for the target signal. A signal processing device according to 1. The signal processing apparatus according to claim 6, wherein the first signal processing unit switches an internal structure so that a type of a tap coefficient output from the tap coefficient output unit is changed. The second signal processing means includes a class tap coefficient obtained by classifying a signal of interest of interest in the third signal into one of a plurality of classes, and the attention The signal processing apparatus according to claim 1, wherein the signal of interest is obtained by an operation using the second signal selected for the signal. The second signal processing means includes
Class tap selection means for selecting a class tap used for classifying the signal of interest into any one of the plurality of classes from the second signal;
Class classification means for classifying the signal of interest based on the class tap;
A prediction tap selection means for selecting a prediction tap used for calculation with the tap coefficient when obtaining the signal of interest from the second signal;
Tap coefficient output means for outputting a tap coefficient of the class of the signal of interest;
The calculation unit for calculating the signal of interest by performing calculation using the tap coefficient of the class of the signal of interest and the prediction tap selected for the signal of interest. A signal processing device according to 1. The signal processing apparatus according to claim 9, wherein the second signal processing unit switches an internal structure so that a type of a tap coefficient output from the tap coefficient output unit is changed. The first to third signals are image signals,
The first signal processing means switches the internal structure to change the spatial resolution creation process for improving the spatial resolution or the image reduction process for reducing the size of the first image signal that is the first signal. And outputting a second image signal which is the second signal,
The second signal processing means performs a noise removal process for removing noise or a time resolution creation process for improving time resolution on the second image signal by switching an internal structure, and the third signal processing means. The signal processing apparatus according to claim 1, wherein a third image signal that is a signal of the first signal is output. The signal processing apparatus according to claim 11, further comprising an image signal output unit that receives a broadcast signal and outputs the first image signal obtained from the broadcast signal. A motion vector detection means for performing a motion vector detection process according to one command of the command sequence;
The signal processing apparatus according to claim 11, wherein the first or second signal processing unit performs signal processing using a motion vector detected by the motion vector detection processing. The signal processing apparatus according to claim 11, further comprising an image signal storage unit that stores the third image signal in response to one command in the command sequence. In a signal processing method of a signal processing apparatus comprising first and second signal processing means,
The first signal processing means switches the internal structure in accordance with at least one command in a command string made up of a plurality of commands, performs signal processing on the first signal, and outputs a second signal. A first signal processing step;
The second signal processing means switches an internal structure in accordance with at least one command in the command sequence, performs signal processing on the second signal, and outputs a third signal. A signal processing method comprising: a signal processing step. A first signal processing means for switching the internal structure in accordance with at least one command of a command sequence of a plurality of commands, performing signal processing on the first signal, and outputting a second signal;
The signal processing apparatus, wherein the second signal is signal-processed by second signal processing means for switching an internal structure in accordance with at least one command in the command sequence. The first signal processing means for switching the internal structure in response to at least one command in the command sequence made up of a plurality of commands processes the second signal output by processing the first signal. Comprising second signal processing means;
The second signal processing means switches an internal structure in accordance with at least one command in the command sequence, performs signal processing on the second signal, and outputs a third signal. A signal processing device.

Images