JP2003169350A - Image processor and image processing method, storage medium, and its program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate a component video signal precisely from a composite video signal under consideration of a motion vector. <P>SOLUTION: A motion vector detecting part 2 detects a motion vector based on a featured amount to be calculated from a composite video signal of a point under consideration and the periphery of the point under consideration and the phase information of the composite video signal. An arithmetic part 4 classifies the composite video signal of the point under consideration into one of a plurality of classes based on at least one of the composite video signal, the featured amount, the phase information, and the detected motion vector. The arithmetic part 4 extracts a prediction tap including at least one of the composite video signal and the featured amount, and generates a component video signal corresponding to the composite video signal under consideration based on a prediction coefficient and the extracted prediction tap. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, a recording medium, and a program, and more particularly, to a component video signal based on a composite video signal or a process for generating a component video signal. The present invention relates to an image processing device and method, a recording medium, and a program for generating a specified coefficient.

[0002]

2. Description of the Related Art In so-called three-dimensional YC separation, a motion amount is detected, and based on the detected motion amount, either one of two-dimensional filter processing and three-dimensional filter processing is selected and executed. NTSC (National Television Sys
tem Committee) A composite video signal is separated into component video signals including a luminance signal and a color signal.

[0003]

[Problems to be Solved by the Invention] However, in the past,
It has not been considered to separate the composite video signal into the component video signals in consideration of the motion vector.

The present invention has been made in view of such circumstances, and an object of the present invention is to enable a component video signal to be generated from a composite video signal more accurately in consideration of a motion vector.

[0005]

A first image processing apparatus of the present invention, based on a first feature amount obtained from a point of interest and a composite video signal around the point of interest, and phase information of the composite video signal, At least one of a detection means for detecting a motion vector, a composite video signal, a first feature amount obtained from a point of interest and a composite video signal around the point of interest, phase information of the composite video signal, and a detected motion vector. Class classifying means for classifying the composite video signal of interest into one of a plurality of classes based on the above, and at least one of the composite video signal and the first feature quantity corresponding to the classified class. Based on the extraction means that extracts the prediction taps that include the prediction coefficient and the extracted prediction taps, Characterized in that it comprises a generating means for generating a component video signal corresponding to Ttobideo signal.

[0006] The extracting means may further extract the prediction tap based on the dynamic range of the composite video signal or the dynamic range of the first characteristic amount corresponding to the composite video signal.

The class classification means individually classifies the composite video signal of the point of interest into one of a plurality of classes corresponding to the luminance signal or the chrominance signal forming the component video signal. You can

The extracting means may individually extract the prediction taps corresponding to the luminance signal or the chrominance signal forming the component video signal.

The detecting means, based on the composite video signal, corresponds to the phase of the color signal at the point of interest and generates the second characteristic quantity by the first characteristic quantity generating means and the second characteristic quantity based on the second characteristic quantity. Corresponding to a target point based on first vector detecting means for detecting a first vector approximate to the motion vector, and a target point which is a target of calculation of motion vector and a composite video signal around the target point. Second feature amount generating means for generating a third feature amount, and second vector detecting means for detecting a motion vector as the second vector by calculating the third feature amount corresponding to the first vector. And can be provided.

The first vector detecting means includes a second feature amount of a target screen which is a screen of a moving image to which the target point belongs, and a second feature amount of an adjacent screen which is a screen of a moving image adjacent to the target screen. It is possible to detect the first vector that is close to the motion vector based on the correlation with.

The second vector detecting means is the screen of the moving image which is adjacent to the target screen and the third feature amount of the first range including the target point in the target screen which is the screen of the moving image to which the target point belongs. It is possible to detect the motion vector as the second vector by calculating the correlation with the third feature amount in the second range corresponding to the first vector in the adjacent screen.

The second vector detecting means may apply the class classification adaptive processing to the third feature quantity and detect the motion vector as the second vector.

The second vector detecting means may classify the composite video signal of the point of interest based on the phase of the color signal at the point of interest.

The second vector detecting means can apply the adaptive processing to the third feature quantity corresponding to the second vector detected in the past.

A first image processing method of the present invention comprises a detection step of detecting a motion vector based on a feature point obtained from a point of interest and a composite video signal around the point of interest and phase information of the composite video signal, Based on at least one of the composite video signal, the feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, the composite video signal of the point of interest is determined. A class classification step of classifying into one of a plurality of classes; an extraction step of extracting a prediction tap corresponding to the classified class, the prediction tap including at least one of a composite video signal and a feature amount; Supports the composite video signal of interest based on the predicted taps Characterized in that it comprises a generation step of generating a component video signal that.

The program of the first recording medium of the present invention is
A detection step of detecting a motion vector based on the feature amount obtained from the attention point and the composite video signal around the attention point, and the phase information of the composite video signal,
Based on at least one of the composite video signal, the feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, the composite video signal of the point of interest is determined. A class classification step of classifying into one of a plurality of classes; an extraction step of extracting a prediction tap corresponding to the classified class, the prediction tap including at least one of a composite video signal and a feature amount; Generating a component video signal corresponding to the composite video signal of the point of interest based on the prediction tap.

A first program of the present invention comprises a detection step of detecting a motion vector based on a feature amount obtained from a point of interest and a composite video signal around the point of interest, and phase information of the composite video signal, and a composite video. Based on at least one of the signal, the feature point obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, a plurality of composite video signals of the point of interest are selected. A class classification step of classifying into one of the classes, an extraction step of extracting a prediction tap corresponding to the classified class and including at least one of a composite video signal and a feature amount, a prediction coefficient and the extracted prediction Supports the composite video signal of interest based on the tap. Characterized in that to execute a generating step of generating a component video signal to the computer.

The second image processing apparatus of the present invention detects the motion vector based on the first feature quantity obtained from the point of interest and the composite video signal around the point of interest and the phase information of the composite video signal. And at least one of the composite video signal, the first feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector. Class classification means for classifying the point composite video signal into one of a plurality of classes, and corresponding to the classified class,
An extraction unit that extracts a prediction tap including at least one of the composite video signal and the first feature amount; and a calculation unit that calculates a coefficient based on the component video signal and the extracted prediction tap. To do.

The extracting means may further extract the prediction taps based on the dynamic range of the composite video signal or the dynamic range of the first characteristic amount corresponding to the composite video signal.

The class classification means individually classifies the composite video signal of the attention point into one of a plurality of classes corresponding to the luminance signal or the chrominance signal forming the component video signal. You can

The extraction means can individually extract the prediction taps corresponding to the luminance signal or the chrominance signal forming the component video signal.

The detecting means, based on the composite video signal, corresponds to the phase of the color signal at the point of interest, and generates the second characteristic quantity by the first characteristic quantity generating means and the second characteristic quantity based on the second characteristic quantity. Corresponding to a target point based on first vector detecting means for detecting a first vector approximate to the motion vector, and a target point which is a target of calculation of motion vector and a composite video signal around the target point. Second feature amount generating means for generating a third feature amount, and second vector detecting means for detecting a motion vector as the second vector by calculating the third feature amount corresponding to the first vector. And can be provided.

The first vector detecting means includes a second feature amount of a target screen which is a screen of a moving image to which the target point belongs, and a second feature amount of an adjacent screen which is a screen of a moving image adjacent to the target screen. It is possible to detect the first vector that is close to the motion vector based on the correlation with.

The second vector detecting means is a moving image screen adjacent to the target screen, and a third feature amount of the first range including the target point in the target screen which is the screen of the moving image to which the target point belongs. It is possible to detect the motion vector as the second vector by calculating the correlation with the third feature amount in the second range corresponding to the first vector in the adjacent screen.

The second vector detecting means may apply the class classification adaptive processing to the third feature quantity and detect the motion vector as the second vector.

The second vector detecting means can classify the composite video signal of the point of interest based on the phase of the color signal at the point of interest.

The second vector detecting means can apply the adaptive processing to the third feature quantity corresponding to the second vector detected in the past.

A second image processing method of the present invention comprises a detection step of detecting a motion vector based on a feature point obtained from a point of interest and a composite video signal around the point of interest and phase information of the composite video signal, Based on at least one of the composite video signal, the feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, the composite video signal of the point of interest is determined. A class classification step of classifying the class into one of a plurality of classes; an extraction step of extracting a prediction tap corresponding to the classified class and including at least one of a composite video signal and a feature amount; a component video signal; A calculation that calculates the coefficient based on the extracted prediction taps Characterized in that it comprises a step.

The program of the second recording medium of the present invention is
A detection step of detecting a motion vector based on the feature amount obtained from the attention point and the composite video signal around the attention point, and the phase information of the composite video signal,
Based on at least one of the composite video signal, the feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, the composite video signal of the point of interest is determined. A class classification step of classifying the class into one of a plurality of classes; an extraction step of extracting a prediction tap corresponding to the classified class and including at least one of a composite video signal and a feature amount; a component video signal; And a calculation step for calculating a coefficient based on the extracted prediction taps.

A second program of the present invention comprises a detection step of detecting a motion vector based on a feature point obtained from a point of interest and a composite video signal around the point of interest and phase information of the composite video signal, and a composite video. Based on at least one of the signal, the feature point obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, a plurality of composite video signals of the point of interest are selected. A class classification step of classifying into one of the classes; an extraction step of extracting a prediction tap corresponding to the classified class, the prediction tap including at least one of a composite video signal and a feature amount; Based on the prediction taps Characterized in that to execute a-up the computer.

A first image processing apparatus and method of the present invention;
In the recording medium and the program, based on the feature amount obtained from the attention point and the composite video signal around the attention point, and the phase information of the composite video signal,
A motion vector is detected, and based on at least one of the composite video signal, the feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, The point composite video signal is classified into one of a plurality of classes, the prediction tap corresponding to the classified class and including the composite video signal and / or the feature amount is extracted, the prediction coefficient and the extracted A component video signal corresponding to the composite video signal of interest is generated based on the prediction taps.

A second image processing apparatus and method of the present invention,
In the recording medium and the program, based on the feature amount obtained from the attention point and the composite video signal around the attention point, and the phase information of the composite video signal,
A motion vector is detected, and based on at least one of the composite video signal, the feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, A point composite video signal is classified into one of a plurality of classes, a prediction tap corresponding to the classified class and including at least one of the composite video signal and a feature is extracted, a component video signal, and an extraction Coefficients are calculated based on the predicted taps.

[0033]

1 is a block diagram showing the configuration of an embodiment of an image processing apparatus according to the present invention.

The NTSC decoder 1 generates an image signal which is digital data corresponding to the image signal based on the input image signal which is the composite video signal of the NTSC system, and converts the image signal into digital image signal. Generate corresponding subcarrier phase information. NTSC decoder 1
The image signal and the subcarrier phase information, which are the generated digital data, are supplied to the motion vector detection unit 2, the class tap extraction unit 3, and the calculation unit 4.

The image signal which is digital data generated by the NTSC decoder 1 is, for example, the result of subtracting the value of the I signal indicating the color from the value of the Y signal indicating the brightness, and the other value indicating the color from the value of the Y signal. The result of subtracting the value of the Q signal, which is a signal, the result of adding the value of the I signal to the value of the Y signal, and Y
It consists of any of the values obtained by adding the value of the Q signal to the value of the signal.

The subcarrier phase information generated by the NTSC decoder 1 is the result of subtracting the value of the I signal from the value of the Y signal, the value included in the image signal being the digital data generated by the NTSC decoder 1. Is it the result of subtracting the value of the Q signal from the value of the Y signal?
This is information indicating whether the result is the result of adding the value of the signal or the result of adding the value of the Q signal to the value of the Y signal.

FIG. 2 is a diagram for explaining an image signal and subcarrier phase information which are digital data generated by the NTSC decoder 1.

Hereinafter, in this specification, field #
-1 indicates a field before field # 0, and field # -2 indicates a field before field # -1. Field # + 1 is (after) field # 0
, And field # + 2 indicates the field next to field # + 1.

The field # 0 and the field whose absolute value is an even number are so-called top fields. A field having an odd number absolute value is a so-called bottom field.

For example, field # -1 is a bottom field and contains image signals corresponding to line 2, line 4, line 6 and the like. Field # 0 is a so-called top field and includes image signals corresponding to line 1, line 3, line 5 and the like.

For example, frame # -1 is field #
-2 and a field # -3 (not shown). Frame # 0 is composed of field # 0 and field # -1. Frame # + 1 is composed of field # + 2 and field # + 1.

Therefore, for example, in frame # 0, line 1 of field # 0, line 2 of field # -1,
Line # 3 of field # 0, line 4 of field # -1, line 5 of field # 0, and field #-
The lines 6 of 1 are arranged in order from the top of the screen, and the lines of field # 0 and the lines of field # -1 are alternately arranged.

The digital data generated by the NTSC decoder 1 corresponds to the period in which the phase of the portion of the composite video signal of the NTSC system that is balanced-modulated by the luminance signal and the chrominance signal changes, and the spatial direction in the field. It is a discrete data. Individual values corresponding to so-called pixels, which form the digital data generated by the NTSC decoder 1, are generated by quantizing an NTSC composite video signal and correspond to one luminance and one color.

In FIG. 2, white squares, white circles, black squares, and black circles constitute an image signal which is digital data, and each represent one value corresponding to one pixel.

A white square indicates that the value corresponds to the result of adding the value of the I signal indicating the color to the value of the Y signal indicating the luminance. A white circle indicates that the value corresponds to the result of adding the value of the Q signal, which is another signal indicating color, to the value of the Y signal.

A black square indicates that the value corresponds to the result of subtracting the value of the I signal from the value of the Y signal. A black circle indicates that the value corresponds to the result of subtracting the value of the Q signal from the value of the Y signal.

The addition or subtraction of the value of the Y signal and the value of the I signal or the addition or subtraction of the value of the Y signal and the value of the Q signal in the value of the image signal which is digital data generated by the NTSC decoder 1 It corresponds to the inversion of the phase of the I signal and the Q signal corresponding to the color signal component in which the subcarriers are balanced-modulated in the NTSC composite video signal input to the NTSC decoder 1.

The subcarrier phase information is sent to the NTSC decoder 1
Each of the values of the image signal, which is digital data generated by, is a value obtained by subtracting the value of the I signal from the value of the Y signal,
Indicates which of a value obtained by subtracting the value of the Q signal from the value of the Y signal, a value obtained by adding the value of the I signal to the value of the Y signal, and a value obtained by adding the value of the Q signal to the value of the Y signal. Information.

The motion vector detecting section 2 includes a parameter A and a parameter B supplied from the outside of the image processing apparatus.
Also, based on the image signal and the subcarrier phase information supplied from the NTSC decoder 1, an upper layer characteristic amount and a lower layer characteristic amount are generated. The motion vector detection unit 2 detects the motion between the field of interest and the next field corresponding to each value of the image signal supplied from the NTSC decoder 1 based on the upper layer feature amount and the lower layer feature amount. Generate a vector. Details of the upper layer feature amount and the lower layer feature amount will be described later.

The motion vector detection unit 2 supplies the generated upper layer feature amount, lower layer feature amount, and motion vector to the class tap extraction unit 3 and the calculation unit 4. The motion vector detection unit 2 uses the upper layer feature amount, the lower layer feature amount,
And other motion vectors such as residuals calculated in the motion vector block matching process are supplied to the class tap extraction unit 3 and the calculation unit 4.

The parameters A and B are parameters that specify the contents of the processing in the motion vector detecting unit 2, and, for example, a correlation calculation method in the motion vector detecting process of the motion vector detecting unit 2 and a block to be matched. Specify the size of, or the size of the search area.

The class tap extraction unit 3 is the NTSC decoder 1
A class tap composed of image signal data, upper layer feature amount data, and lower layer feature amount data is extracted based on the subcarrier phase information supplied from To do. Although the details will be described later, the class tap is data for classifying the class according to the data of interest.

The class tap extracting section 3 supplies the extracted class taps to the calculating section 4.

The calculation unit 4 classifies the class based on the class taps supplied from the class tap extraction unit 3, and the subcarrier phase information supplied from the NTSC decoder 1 and the motion supplied from the motion vector detection unit 2. Based on the vector, adaptive processing is applied to the image signal supplied from the NTSC decoder 1 and the upper layer feature amount and the lower layer feature amount supplied from the motion vector detection unit 2 to obtain a composite video signal image signal. A corresponding component video signal consisting of, for example, a Y signal, a U signal, and a V signal is generated. The calculation unit 4 outputs the generated component video signal. Details of the class classification process and the adaptation process will be described later.

As described above, the image processing apparatus according to the present invention generates the component video signal corresponding to the input image signal which is the composite video signal, and outputs the generated component video signal.

FIG. 3 is a block diagram showing the configuration of the motion vector detecting section 2. The image signal and the subcarrier phase information supplied from the NTSC decoder 1 are stored in the buffer 21,
It is input to the feature amount conversion unit 23 and the feature amount conversion unit 24.

The buffer 21 stores the image signal and subcarrier phase information supplied from the NTSC decoder 1,
The stored image signal and subcarrier phase information are supplied to the feature amount conversion unit 23 and the buffer 22. That is, the buffer 21 delays the input image signal and subcarrier phase information for a period corresponding to one field, and supplies the delayed image signal and subcarrier phase information to the feature amount conversion unit 23 and the buffer 22.

The buffer 22 stores the image signal and subcarrier phase information supplied from the buffer 21, and supplies the stored image signal and subcarrier phase information to the feature amount conversion section 24. That is, the buffer 22 is
The image signal and subcarrier phase information supplied from the buffer 21 are delayed for a period corresponding to one field, and the delayed image signal and subcarrier phase information are supplied to the feature amount conversion unit 24.

The feature amount conversion unit 23 generates a higher-layer feature amount based on the image signal and subcarrier phase information input from the NTSC decoder 1 and the image signal and subcarrier phase information supplied from the buffer 21. The generated upper layer feature amount is output to the outside of the motion vector detection unit 2, and the upper layer feature amount is supplied to the upper layer motion vector detection unit 25.

The feature amount conversion unit 24 generates a lower layer feature amount based on the image signal and subcarrier phase information input from the NTSC decoder 1 and the image signal and subcarrier phase information supplied from the buffer 22. , Sub-carrier phase information and the generated upper layer feature amount are supplied to the lower layer motion vector detection unit 26. Further, the feature amount conversion unit 24 outputs the generated lower layer feature amount to the outside of the motion vector detection unit 2.

The feature amount conversion unit 23 and the feature amount conversion unit 24 generate an upper layer feature amount or a lower layer feature amount in real time based on the input image signal and subcarrier phase signal, and sequentially output them. Even in this case, the generated upper layer feature amount or lower layer feature amount is temporarily stored for each field, and the upper layer feature amount or the lower layer feature amount is collectively output for each field. Good.

The upper layer motion vector detection unit 25 approximates to the finally detected motion vector based on the upper layer feature amount supplied from the feature amount conversion unit 23 by the calculation method specified by the parameter A or the like. Then, an upper layer vector indicating a rough motion between fields is generated, and the generated upper layer vector is supplied to the lower layer motion vector detection unit 26.

The lower layer motion vector detecting section 26 uses the calculation method specified by the parameter B or the like to supply the lower layer characteristic amount and subcarrier phase information supplied from the characteristic amount converting section 24, and the upper layer motion vector detecting section 25. Based on the upper layer vector supplied from the above, the higher layer vector is compared with the upper layer vector to generate a more accurate lower layer vector, and the generated lower layer vector is output as a motion vector.

FIG. 4 shows the upper layer motion vector detecting unit 25.
3 is a block diagram showing the configuration of FIG. Vector generator 31
Based on the data included in the parameter A, which indicates the range of the magnitude and the direction of the vector to be generated, sequentially generates the vector having the size and the direction of the predetermined range, and the generated vector is used as the correlation value calculation unit 32. And the determination unit 33.

The correlation value calculation unit 32 is designated by the vector supplied from the vector generation unit 31 based on the data included in the parameter A designating the correlation value calculation method.
Calculate the correlation value between the upper layer feature amount corresponding to one field and the upper layer feature amount corresponding to another field,
The calculated correlation value is supplied to the determination unit 33. For example, the correlation value is the sum of the absolute values of the differences between the upper layer feature amounts. The correlation value can be the sum of the squares of the absolute values of the differences between the upper layer feature amounts. Alternatively, the correlation value is a correlation in which an upper layer feature amount corresponding to one field and an upper layer feature amount corresponding to another field are a sequence of two random variables, that is, a cross correlation of the upper layer feature amount. Can be

The correlation value calculation unit 32 executes the processing of calculating the correlation value by using the sparse block using the upper layer feature amount that is sparse as compared with the original image signal data. Therefore, the amount of calculation in the upper layer motion vector detection unit 25 can be reduced.

The determination unit 33 stores the correlation value supplied from the correlation value calculation unit 32 in association with the vector supplied from the vector generation unit 31. When the correlation value corresponding to the vector having the size and direction of the predetermined range is calculated, the determination unit 33 determines the stored correlation value based on the data included in the parameter A that specifies the calculation method of the correlation value. Among the values, the correlation value having the strongest correlation is selected, and the vector corresponding to the selected correlation value is output as the upper layer vector. For example, when the correlation value is the sum of the squares of the absolute values of the differences in the upper layer feature values, the determination unit 33 outputs the vector corresponding to the minimum correlation value as the upper layer vector.

FIG. 5 shows the lower layer motion vector detecting section 26.
3 is a block diagram showing the configuration of FIG. Vector generator 41
Is the size and direction of the predetermined range based on the data included in the parameter B, which indicates the range of the size and direction of the upper layer vector supplied from the upper layer motion vector detection unit 25 and the generated vector. The generated vector is sequentially generated, and the generated vector is supplied to the correlation value calculation unit 42 and the determination unit 43.

The correlation value calculation unit 42, based on the data included in the parameter B that specifies the calculation method of the correlation value, the lower layer feature corresponding to one field specified by the vector supplied from the vector generation unit 41. The correlation value between the amount and the lower layer feature amount corresponding to another field is calculated, and the calculated correlation value is supplied to the determination unit 43. For example, the correlation value is the sum of the absolute values of the differences between the lower layer feature amounts. The correlation value can be the sum of squares of the absolute values of the differences in the lower layer feature amounts. Alternatively, the correlation value is a correlation in which a lower layer feature amount corresponding to one field and a lower layer feature amount corresponding to another field are two random variable sequences,
That is, it can be a cross-correlation of the lower layer feature amount.

The correlation value calculating unit 42 executes the processing of calculating the correlation value by using the dense blocks using the lower layer feature amount having the same density as the data of the original image signal.

The determination unit 43 stores the correlation value supplied from the correlation value calculation unit 42 in association with the vector supplied from the vector generation unit 41. When the correlation value corresponding to the vector having the size and direction of the predetermined range is calculated, the determination unit 43 determines the stored correlation based on the data included in the parameter B that specifies the calculation method of the correlation value. Among the values, the correlation value having the strongest correlation is selected, and the vector corresponding to the selected correlation value is output as the lower layer vector. For example, when the correlation value is the sum of the absolute values of the differences of the lower layer feature amounts, the determination unit 43 outputs the vector corresponding to the minimum correlation value as the lower layer vector.

A detailed example of the process executed by the motion vector detecting section 2 will be described with reference to FIGS. 6 to 9.

FIG. 6 shows the upper layer motion vector detecting section 25.
Of the upper layer feature amount used for the process of detecting the upper layer vector by the lower layer motion vector detecting unit 26, the lower layer feature amount used for the process of detecting the lower layer vector by the lower layer motion vector detecting unit 26, and the lower layer motion vector detecting unit 26. It is a figure which shows the example of a search area.

The numbers in FIG. 6 represent weights. For example, when all the weights are 1, the feature amount conversion unit 23 outputs the data of the predetermined phase included in the image signal as it is as the upper layer feature amount.

For example, the upper layer motion vector detecting section 25
Is one block including the attention data corresponding to the attention point for which the motion vector is to be detected, and the six data around the attention data, among the higher-layer feature quantities made up of the data having the same phase as the attention data, Matching is performed with data having the same phase as the target data in other fields. That is, the higher-layer motion vector detection unit 25 determines the arrangement of the predetermined number of feature quantities belonging to one block of the field of interest and the feature quantities of other fields whose values are closest to each other. Detect the position.
The upper hierarchy vector indicates the position of the feature amount of the other field that has the closest arrangement and value from the position of the data of interest of the field of interest.

Here, the fact that the data phases are the same means
It means that the signals that are the basis of data calculation are the same and that the data calculation method of addition or subtraction is the same.

For example, one piece of data is converted from the value of the Y signal to I.
When the value of the signal is subtracted and the value of the other data is the value of the Y signal minus the value of the I signal, the phases of the two data are the same. When one data has a value obtained by subtracting the value of the Q signal from the value of the Y signal and another data has a value obtained by subtracting the value of the Q signal from the value of the Y signal, the phases of the two data are the same. Is.

Similarly, when one data has a value obtained by adding the value of the I signal to the value of the Y signal and another data has a value obtained by adding the value of the I signal to the value of the Y signal, the two The phases of the two data are the same. One data is Q in the value of Y signal
When it has a value obtained by adding the value of the signal, and the other data has a value obtained by adding the value of the Y signal to the value of the Q signal,
The phases of these two data are the same.

On the other hand, data having a value obtained by subtracting the value of the I signal from the value of the Y signal, data having a value obtained by adding the value of the I signal to the value of the Y signal, and subtracting the value of the Q signal from the value of the Y signal. The data having the above value and the data having the value obtained by adding the value of the Q signal to the value of the Y signal have mutually different phases.

The feature quantity conversion section 23 can know whether or not the phases of two data are the same based on the subcarrier phase information.

As shown in FIG. 7, for example, in the upper layer motion vector detection unit 25, in the field # -1 which is the field of interest, the phase of the data of interest and the phase of the data of interest around the data of interest indicated by a white circle are Matching is performed with a corresponding field, field # 0, in units of blocks of higher layer feature amounts, which are composed of the same six data items corresponding to white circles located at the vertices of a hexagon. More specifically, the upper layer motion vector detection unit 25
Is the target data corresponding to the target point to be searched in field # 0, which is the field corresponding to the block of the upper hierarchy feature amount of the field of interest, and the target data around the target data indicated by white circles. , The correlation with the block of the upper layer feature amount, which is composed of the six data corresponding to the white circles located at the vertices of the hexagon and having the same phase as the target data, is calculated, and the matching is determined.

Next, returning to FIG. 6, an example of the lower layer feature amount will be described.

The feature amount conversion unit 24 calculates the lower layer feature amount corresponding to the target data, for example, based on the data around the target data in the target field. The feature amount conversion unit 24 multiplies each of the four pieces of data adjacent to the target data of the field of interest by a weight of 1, the target data by a weight of 4, and adds the results of the multiplication. As the lower layer feature amount.

Similarly, the feature quantity conversion unit 24 converts the target data corresponding to the target point to be searched for in the field corresponding to the field of interest into the target data based on the data around the target data. The corresponding lower layer feature amount is calculated.

Further, the feature quantity conversion unit 24 is, for example, as shown in FIG.
As shown in, in the field # -1 of interest, two data above the attention data, two data below the attention data, and two data right from the attention data,
Then, each of the two data on the left side of the target data is multiplied by a weight of 1, the target data is multiplied by a weight of 4, and the multiplication result is added to obtain a lower layer feature amount.

In the corresponding field # 0, the feature quantity conversion section 24 determines that the data two levels above the target data, the data two levels below the target data, the two data right from the target data, and two from the target data. Each of the data on the left side is multiplied by a weight of 1, the target data is multiplied by a weight of 4, and the multiplication results are added to obtain a lower layer feature amount.

The lower layer motion vector detecting section 26 searches for a predetermined range of the characteristic amount of the corresponding field centered on the position of the data designated by the upper layer vector supplied from the upper layer motion vector detecting section 25. In the area, the position of the block including the predetermined number of lower layer feature amounts of the field of interest and the position of the block including the predetermined number of lower layer feature amounts having the highest correlation are detected.
The lower layer motion vector detection unit 26 outputs, as a motion vector, a lower layer vector indicating the center position of the detected block of the corresponding field from the position of the data of interest of the field of interest.

FIG. 9 shows the lower layer motion vector detecting section 26.
It is a figure which shows the example of the area | region and block which are searched.
In FIG. 9, triangles indicate lower layer feature amounts.

In the example shown in FIG. 9, the area to be searched is a rectangular area in which seven lower layer characteristic amounts are arranged horizontally and seven lower layer characteristic amounts are arranged vertically. The block to be searched is a rectangular area in which five lower layer feature amounts are arranged horizontally and five lower layer feature amounts are arranged vertically.

FIG. 10 is a diagram showing an example of another block of the upper layer characteristic amount for judging matching. In FIG. 10, triangles indicate data having the same phase as the target data.

As shown in FIG. 10, for example, the upper layer motion vector detection unit 25, in the field # -1 which is the field of interest, shows the data of interest indicated by a black triangle and the data of interest indicated by a white triangle. The highest correlation in the corresponding field, field # 0, in units of blocks of the higher-level feature quantity, which is composed of the data of interest and the six data corresponding to the white triangles located at the vertices of the hexagon Detects strong blocks.

When the feature quantity conversion section 23 outputs the upper layer feature quantity consisting of the image signal data having the same phase as the data of interest, the number of the upper layer feature quantity data is 4 times the original image signal data. It will be 1.

FIG. 11 and FIG. 12 are diagrams showing other examples of the higher layer feature amount.

As shown in FIG. 11, the feature quantity conversion unit 23
For example, calculates an average value of four data of different phases on one line of a predetermined field, and outputs the calculated average value as an upper layer feature amount.

As shown in FIG. 12, the feature quantity conversion unit 23
Is, for example, an average value of four pieces of data adjacent to each other vertically and horizontally on two lines adjacent to each other above and below a predetermined field and having different phases. Output as a quantity.

When the feature quantity conversion unit 23 calculates the average value of four data having different phases and outputs the upper hierarchy feature quantity, the number of data of the upper hierarchy feature quantity is the same as that of the original image signal data. It becomes 1/4.

13 to 17 are diagrams showing other examples of the lower layer feature amount.

As shown in FIG. 13, the feature amount conversion unit 24
Is based on the data above and below the data of interest, and the second data to the left of the data of interest and the second data to the right of the data of interest in field # -1 of interest. The feature amount is calculated. For example, let p2 be the data of interest, p0 be the upper data, p1 be the second data on the left, and p3 be the second data on the right.
When the lower data is p4, the lower layer feature amount y corresponding to the target data p2 is calculated by the equation (1). y = p0 + p1 + 4 * p2 + p3 + p4 (1)

The feature quantity conversion unit 24 outputs the upper data and the lower data of the target data of the corresponding field # 0, the second data on the left side of the target data and the second data on the right side of the target data. Based on this, the lower layer feature amount is calculated. For example, if the target data is p2, the upper data is p0, the second data is p1 on the left, the second data is p3 on the right, and the lower data is p4. The lower layer feature amount y to be calculated is calculated by Expression (1).

The lower layer feature amount described with reference to FIG. 13 is calculated on the basis of four data having different phases around the target data or the target data, so that the influence of the variation component of the subcarrier is reduced. Therefore, it becomes possible to detect a motion vector with higher accuracy.

As shown in FIG. 14, the feature quantity conversion unit 24
Is the line of the target data and the field # -2, which is the field preceding the target field, when the lower layer feature amount corresponding to the target data of the target field, field # -1, is calculated. Based on the data above and below the data of interest on the lines above and below the line to be recorded, and the second data to the left of the data of interest and the second data to the right of the data of interest Calculate the hierarchical feature amount. For example, let p2 be the target data, p0 be the data above the target data in line 3 of the previous field # -2 above line 4 where the target data is located, and the second data to the left of line 4. Let p1 be the second data on the right side of line 4 be p3, and let the data below the target data in line 5 of the previous field # -2 below line 4 where the target data is located be p4, The lower layer feature amount y corresponding to the attention data p2 is calculated by the equation (2). y = p0 + p1 + 4 * p2 + p3 + p4 (2)

When calculating the lower layer feature quantity corresponding to the target data of the field # 0 which is the corresponding field, the feature quantity conversion unit 24 calculates the line of the field # + 1 which is the field next to the target data and field # 0. In addition, on the upper and lower lines of the line where the target data is located, the upper data and the lower data of the target data, and the second data on the left side of the target data and the second data on the right side of the target data. The lower layer feature amount is calculated based on the third data. For example, the target data is p2, the data above the target data in line 2 of the next field # + 1 above line 3 where the target data is located is p0, and the second data on the left side of line 3 is p1. age,
The second data on the right side of line 3 is p3, and the lower data of the target data in line 4 of the next field # + 1 on the lower side of line 3 where the target data is located is p4. The corresponding lower layer feature amount y is calculated by equation (2).

As shown in FIG. 15, the feature quantity conversion unit 24
Is based on the data of interest, the data above and below the data of interest, and the data adjacent to the left side of the data of interest and the data adjacent to the right side of the data of interest in field # -1 of interest. The lower layer feature amount corresponding to the data is calculated. For example, if the attention data is p2,
When the data above the data of interest is p0, the data adjacent to the left of the data of interest is p1, the data adjacent to the right of the data of interest is p3, and the data below the data of interest is p4, the data of interest p2 The lower layer feature amount y corresponding to
It is calculated by the equation (3). y = p0 + p1 + 4 * p2 + p3 + p4 (3)

The feature quantity conversion unit 24 determines the target data, the data above and below the target data, and the data adjacent to the left side of the target data and the right side of the target data of the corresponding field # 0. The lower layer feature amount corresponding to the target data is calculated based on the data.
For example, the target data is p2, the data above the target data is p0, the data adjacent to the left side of the target data is p1, the data adjacent to the right side of the target data is p3, and the data below the target data is When p4 is set, the lower layer feature amount y corresponding to the target data p2 is calculated by Expression (3).

As shown in FIG. 16, the feature amount conversion unit 24
Is the data of interest, the data adjacent to the left side of the data of interest and the data adjacent to the right of the data of interest, and the second data to the left of the data of interest and 2 to the right of the data of interest. The lower layer feature amount corresponding to the data of interest is calculated based on the third data. For example, let the data of interest be p2, the second data to the left of the data of interest be p0, the data adjacent to the left of the data of interest be p1, and the data adjacent to the right of the data of interest.
When p3 is set and p2 is the second data on the right side of the target data, the lower layer feature amount y corresponding to the target data p2 is calculated by the equation (4). y = p0 + p1 + 2 * p2 + 2 * p3 + p4 (4)

The feature amount conversion unit 24 determines the target data of the target field # 0, the data adjacent to the left side of the target data and the data adjacent to the right side of the target data, and the second data to the left side of the target data. And the lower layer feature amount corresponding to the target data is calculated based on the second data on the right side of the target data. For example, target data
p2, the second data on the left side of the target data is p0,
When the data adjacent to the left side of the target data is p1, the data adjacent to the right side of the target data is p3, and the second data is p4 on the right side of the target data, the lower layer feature amount y corresponding to the target data p2 Is calculated by the equation (4).

As shown in FIG. 17, the feature quantity conversion unit 24
When calculating the lower layer feature amount of the field # -1 which is the attention field, the attention data, the second data on the left side of the attention data, the second data on the right side of the attention data, and the front of the attention field The lines of the field # -2, which is a field, on the lines above and below the line on which the target data is located, the target data, the second data to the left of the target data, and the second data to the right of the target data. The lower layer feature amount is calculated based on the upper data and the lower data of each of the above data.

For example, the target data is p4, the second data on the left side is p3, and the second data on the right side is p5.
In line 3 of the previous field # -2 above line 4 where the data of interest is located, the data above the data of interest is p1, the data above p3 is p0, and the data above p5 is p2, In line 5 of the previous field # -2 below line 4 where the data of interest is located, the data below the data of interest is p7, the data below p3 is p6, and the data below p5. Is set to p8, and the lower layer feature amount y corresponding to the attention data p4 is calculated by the equation (5). y = p0 + (-2) * p1 + p2 + (-2) * p3 + 4 * p4 + (-2) * p5 + p6 + (-2) * p7 + p8 (5)

When calculating the lower layer feature quantity of the corresponding field, field # 0, the feature quantity conversion unit 24 selects the target data, the second data on the left side of the target data, and the second data on the right side of the target data. Field # which is the next field of data and target field
+1 line, on the upper and lower lines of the line where the target data is located, the upper side of each of the target data, the second data to the left of the target data, and the second data to the right of the target data. The lower layer feature amount is calculated on the basis of the above data and the lower data.

For example, the target data is p4, the second data on the left side is p3, the second data on the right side is p5,
In line 2 of the next field # + 1 above line 3 where the target data is located, the upper data of the target data is p1, the upper data of p3 is p0, and the upper data of p5 is p2. In line 4 of the next field # + 1 below line 3 where the data is located, the lower data of the target data is p7, the lower data of p3 is p6, and the lower data of p5 is p8. Then, the lower layer feature amount y corresponding to the target data p4 is calculated by the equation (5).

The upper layer feature amount and the lower layer feature amount exemplified above are not normalized because the respective feature amounts are calculated. Of course, the upper layer feature amount and the lower layer feature amount may be normalized.

Next, the detection of the lower hierarchy vector by the class classification adaptive processing will be described.

Here, the class classification process will be briefly described.

Now, for example, in the lower layer feature quantity,
A certain tapped data and three data adjacent to it form a class tap consisting of 2 × 2 data, and
Each data is expressed by 1 bit (takes either 0 or 1 level). In this case, the 2 × 2 4-data block including the target data can be classified into 16 (= (2 ¹ ) ⁴ ) patterns according to the level distribution of each data. Therefore, in the present case, the data of interest can be classified into 16 patterns, and such pattern classification is the class classification processing.

Here, normally, for example, about 8 bits are assigned to each data. Also, class tap 3
With 9 data of × 3, if the class classification processing is performed on such a class tap, the class will be classified into a huge number of classes (2 ⁸ ) ⁹ .

Therefore, in the present embodiment, an ADRC (Adaptive Dynamic Range Co
ding) processing is performed, and thereby, the number of classes is reduced by reducing the number of bits of the data forming the class tap.

To simplify the explanation, consider a class tap composed of four data of lower layer feature values arranged on one line. In ADRC processing, the maximum value MAX and the minimum value of the data values are considered. MIN is detected. Then, DR = MAX-MIN is set as a local dynamic range of the block composed of the class taps, and the data value of the data composing the block of the class taps is requantized into K bits based on the dynamic range DR. To be done.

That is, the minimum value MIN is subtracted from each data value in the block, and the subtracted value is divided by DR / 2 ^K. Then, it is converted into a code (ADRC code) corresponding to the division value obtained as a result. Specifically, for example, when K = 2, it is determined to which range the divided value is obtained by equally dividing the dynamic range DR into 4 (= 2 ² ), and the divided value is the lowest. Range of levels of
If it belongs to the range of the second lowest level, the range of the third lowest level, or the range of the highest level,
For example, 00B, 01B, 10B, or 1 respectively
It is encoded into 2 bits such as 1B (B represents a binary number). Then, on the decoding side, ADRC
The code 00B, 01B, 10B, or 11B is obtained by dividing the dynamic range DR into four equal to the central value L ₀₀ of the range of the lowest level, the central value L ₀₁ of the range of the second lowest level, and from the bottom. Decoding is performed by converting the value to the central value L ₁₀ of the range of the third level or the central value L ₁₁ of the range of the uppermost level and adding the minimum value MIN to the value.

Here, such ADRC processing is called non-edge matching.

Details of the ADRC processing are disclosed in, for example, Japanese Patent Application Laid-Open No. 3-53778 filed by the applicant of the present application.

As described above, the number of classes can be reduced by performing the ADRC process of requantizing with the number of bits smaller than the number of bits assigned to the data forming the class tap.

In this embodiment, the class classification process is performed based on the ADRC code. However, the class classification process may be performed by other methods such as DPCM (predictive coding) and
It is also possible to perform data for which BTC (Block Truncation Coding), VQ (Vector Quantization), DCT (Discrete Cosine Transform), Hadamard Transform and the like have been performed.

In the present embodiment, the adaptive process is executed for each class thus classified. As the adaptive processing, there are a method of performing a prediction calculation using a prediction coefficient learned in advance and a method of learning a predicted value by the center of gravity method.
Further, as a condition for learning, it is necessary to prepare a target teacher signal including an image signal and a motion vector.

Next, an adaptive process for performing a prediction calculation using a prediction coefficient for each class generated in advance by learning using this teacher signal will be described. For example, consider that a prediction tap is configured by 25 taps of 5 × 5 lower layer feature amounts E0 to E24 and the motion evaluation value E ′ is predicted.

The predicted value E'of the motion evaluation value is calculated by the equation (6).

[0126]

[Equation 1] Ei indicates a lower layer feature amount. wi indicates a prediction coefficient.

For example, when the above-mentioned 1-bit ADRC is applied to nine pieces of data and the data is classified into 512 classes, the motion evaluation value E is calculated by multiplying and summing the prediction coefficient and the lower layer feature quantity generated for each class. 'Is predicted.

For example, two orthogonal motion evaluation values E'can be calculated and used as the x and y components of the motion vector to calculate the motion vector. Further, for example, n motion evaluation values E ′ are calculated, and the magnitudes of n unit vectors having different directions are shown, and each of the n motion evaluation values E ′ is converted into n units. The motion vector can be calculated by multiplying each of the vectors and adding the multiplied results.

Further, not only the motion vector but also the evaluation value E'corresponding to the Y signal and the evaluation corresponding to the U signal are calculated by the calculation based on the equation (6) when predicting not only the motion vector but also the component video signal by the adaptive processing. The value E ′ and the evaluation value E ′ corresponding to the V signal can be calculated individually.

Since the above-described prediction coefficient is generated by learning in advance, the learning will be described here.

An example of generating the prediction coefficient based on the model of equation (6) by the method of least squares will be shown.

The least squares method is applied as follows. As a generalized example, Equation (7) is considered, where X is input data, W is a prediction coefficient, and Y is a prediction value. Observation equation: XW = Y (7)

[0133]

[Equation 2]

The least squares method is applied to the data collected by the above observation equation. In the example of Expression (6), n
Is 25 and m is the number of training data.

Consider the residual equation of equation (9) based on the observation equation of equation (7).

[0136]

[Equation 3]

From the residual equation of equation (9), the most probable value of each wi corresponds to the case where the condition for minimizing the value shown in equation (10) is satisfied.

[0138]

[Equation 4]

That is, the condition of equation (11) may be taken into consideration.

[0140]

[Equation 5]

N conditions based on i in equation (11) are satisfied.
It is sufficient to calculate w1 to wn. Therefore, the residual equation (9)
Equation (12) is obtained from

[0142]

[Equation 6]

From equations (11) and (12), equation (13) is obtained.

[0144]

[Equation 7]

A normal equation (14) is obtained from the equations (10) and (13).

[0146]

[Equation 8]

Since it is possible to set up the same number of equations as the number n of unknowns in the normal equation of the equation (14), the most probable value of the certainty wi can be obtained. For example, using the sweeping method (Gauss-Jordan elimination method), equation (14)
Is solved.

As described above, the optimum prediction coefficient w is obtained for each class, and the prediction value w is used to calculate a prediction value E'indicating a vector close to the motion vector of the teacher signal by the equation (6). It is the adaptive processing that is required.

When predicting the component video signal,
It is necessary to find the optimum prediction coefficient w for each class.

The adaptive processing is the same as the interpolation processing using a so-called interpolation filter as far as only the equation (6) is seen, but the prediction coefficient w corresponding to the tap coefficient of the interpolation filter is the teacher signal. It is possible to reproduce the original motion vector because it is obtained by learning, so to speak. From this, it can be said that the adaptive process is, so to speak, a process having a creative action.

FIG. 18 is a block diagram showing another configuration of the lower hierarchical motion vector detecting unit 26 for detecting the lower hierarchical vector by the class classification adaptive processing.

The class classification unit 51 classifies the class based on the subcarrier phase information and the lower layer feature amount, and predicts the class code indicating the classified class into the prediction tap extraction unit 5.
Supply to 2. For example, the class classification unit 51 classifies the class according to the phase of the data of interest. Also, for example,
The class classification unit 51 applies ADRC processing to a predetermined number of lower layer feature amounts corresponding to the data of interest to perform class classification.

The predictive tap extracting unit 52 determines a predetermined number at a predetermined position based on the class indicated by the class code, the lower layer vector corresponding to the previous field and the upper layer vector supplied from the memory 55. The lower layer feature amount of is extracted, and the extracted lower layer feature amount is supplied to the calculation unit 53 as a prediction tap.

The calculation unit 53 predicts the lower hierarchy vector by executing, for example, the calculation shown in the equation (6) based on the coefficient set including a predetermined number of prediction coefficients supplied from the prediction coefficient memory 54. To do. The calculation unit 53 supplies the predicted lower layer vector to the memory 55 and outputs it to the outside.

The memory 55 stores the lower layer vector supplied from the calculation unit 53, and supplies the stored lower layer vector to the prediction tap extracting unit 52.

As described above, the motion vector detecting section 2 can detect a highly accurate motion vector and supply the detected motion vector to the class tap extracting section 3 and the calculating section 4.

The class tap extraction unit 3 is the NTSC decoder 1
Based on the subcarrier phase information supplied from the motion vector detection unit 2 and the motion vector supplied from the motion vector detection unit 2, the data of the image signal, the data of the upper layer feature quantity, and the lower layer which are a set of data for class classification. A class tap composed of hierarchical feature data is extracted.

FIG. 19 is a block diagram showing the structure of the arithmetic unit 4.

The class classification unit 71-1 has a motion vector supplied from the motion vector detection unit 2, an upper layer feature amount,
Based on the lower layer feature amount, the class tap supplied from the class tap extraction unit 3, and the subcarrier phase information supplied from the NTSC decoder 1, the class code indicating the result of the class classification is predicted tap extracted. Part 7
2-1 and coefficient memory 73-1.

For example, the class classification unit 71-1 uses the subcarrier phase information supplied from the NTSC decoder 1 as the basis.
Classify. Also, the class classification unit 71-1 is NTSC.
Based on the data of interest indicated by the subcarrier phase information supplied from the decoder 1, the class is classified based on the phase of the data designated by the motion vector. Therefore, the number of classified classes is a power of four.

Further, the class classification unit 71-1 classifies the class based on the upper layer feature amount and the lower layer feature amount supplied from the motion vector detection unit 2. That is, for example, the class classification unit 71-1 applies ADRC processing to the upper layer feature amount supplied from the motion vector detection unit 2 and quantizes the upper layer feature amount into a class code having a predetermined number of bits. The class classification unit 71-1 applies ADRC processing to the lower layer feature amount supplied from the motion vector detection unit 2, and quantizes the lower layer feature amount into a class code having a predetermined number of bits. When quantizing by threshold judgment, quantizing by threshold judgment is equivalent to class coding the reliability of a vector.

FIG. 20 is a diagram for explaining an example of class taps and prediction taps corresponding to the upper layer feature amount.

When the data of interest has a phase indicated by a white circle as shown in FIG. 20A, for example, the upper layer feature amount is calculated from only the data of the same phase as the data of interest indicated by a white circle as shown in FIG. 20B. Composed. At this time, for example, the class tap corresponding to the higher-level feature amount is the attention data p5 and the data p two higher than the attention data p5.
1, data p2 on the one side above the attention data p5, data p2 on the two left side, data p3 on the one side above the attention data p5, and data p3 on the four left side of the attention data p5
4, data p6 on the four right side of the data of interest p5, one data below the data of interest p5, two data p7 on the left side and one data below the data of interest p5, two data p on the right side
8 and the data p9 that is two lower than the target data p5.

For example, as shown in FIG. 20C, the upper layer feature amount is composed of an average value of a total of four data, which are indicated by triangles, that is, two vertically and two horizontally. The vertical position of the upper layer feature amount composed of the average value is
With respect to the position of the original image signal data, the horizontal position of the upper layer feature amount, which is displaced from the original image signal data by half the line width, and is composed of the average value , It is shifted by half the data interval.

When the attention data is p11, the class tap corresponding to the higher layer feature amount is, for example, two lines and half upper side with respect to the attention data p11, and is on the left side by half the interval of the original data. 2 lines and half upper side of the data A1 of FIG.
Data A2 on the right side of one and half of the original data interval,
The line is one and half above and the data A3 on the left one and half of the original data interval, the line one and half above and the data A4 and line A4 on the right one half the original data interval One half above, the data A5 to the right of two and half of the original data interval, the line half above, and half the original data left data A6, the line half above. Then, one and half of the original data interval is on the right side of the data A7, the line is half down, and one and half of the original data interval is on the left side of data A8, and the line is half down. The data A9 on the right side by half of the interval of the original data, the line is half down, and the data A10 on the right side by two and half of the interval of the original data, and one and half down on the line of original data Half the data interval Data A11 on the left side, one line and one half lower side, and one half of the original data interval and half data on the right side A12, two lines and half lower side, and one original data interval Data A on the left half
13, data is two and half down and data A14 on the right side of half of the original data interval, two and half lines on the right side, and two and half of the original data interval on the left It is composed of data A15.

FIG. 21 is a diagram for explaining an example of class taps and prediction taps corresponding to the lower layer feature amount.

The class tap corresponding to the current field # -1 is 3 × 3 data centered on the lower layer feature amount data corresponding to the target data, that is, the lower layer feature amount corresponding to the target data. Data, data above the lower layer feature amount data corresponding to the target data, above the lower layer feature amount data corresponding to the target data, left side data, lower layer feature amount data corresponding to the target data Data on the right side, data on the left side of the lower layer feature amount data corresponding to the target data, data on the right side of the lower layer feature amount data corresponding to the target data, and the lower layer feature corresponding to the target data It is composed of the data on the left side of the amount data, the data on the left side, and the data on the right side of the lower layer feature amount data corresponding to the target data.

The class tap of the corresponding field # 0 is 3 × 3 data centered on the data of the lower layer feature amount designated by the motion vector from the target data, that is, designated by the motion vector from the target data. The lower layer feature amount data, the upper data of the lower layer feature amount data specified by the motion vector from the target data, the upper layer data of the lower layer feature amount specified by the motion vector from the target data, From the data, the data on the right side of the lower layer feature amount specified by the motion vector from the attention data, and the data on the left side of the data of the lower layer feature amount specified by the motion vector from the attention data, from the attention data Data on the right side of the lower layer feature amount data specified by the motion vector, lower floor specified by the motion vector from the data of interest A lower data of the feature quantity,
It is composed of the data on the right side, which is below the data on the left side and the data of the lower layer feature amount designated by the motion vector from the data of interest.

The class classification unit 71-1 uses the attention data of the image signal and the peripheral data around the attention data as a basis.
Classify into spatial classes. That is, for example, the class classification unit 71-1 applies the ADRC processing to the target data of the image signal and the peripheral data, and quantizes the target data and the peripheral data into a class code having a predetermined number of bits.

The class classification unit 71-1 performs class classification based on the target data of the corresponding field indicated by the motion vector from the target data of the target field and the peripheral data around the target data. That is, for example, the class classification unit 71-1 uses the target data of the corresponding field, which is indicated by the motion vector from the target data of the target field,
And ADRC processing is applied to peripheral data around the target data, and the target data and peripheral data are quantized into a class code having a predetermined number of bits.

The class classification unit 71-1 uses the amount of movement as a basis.
You may classify.

The class classification unit 71-1 supplies the dynamic range, which is the difference between the maximum value and the minimum value of the data values of the image data of interest and the peripheral data belonging to a predetermined block, to the prediction tap extraction unit 72-1. To do.

The class classification unit 71-1 may calculate the dynamic range based on the class tap of the lower layer feature amount of the field of interest, an example of which is shown in FIG.

Further, the class classification unit 71-1 may classify the class based on the upper layer feature amount included in the class tap.

The predictive tap extraction unit 72-1 is based on the motion vector supplied from the motion vector detection unit 2 and the feature amount such as the residual, and the class code and the dynamic range supplied from the class classification unit 71-1. Further, predetermined data is extracted from the image signal data, the upper layer feature amount, and the lower layer feature amount, and the extracted data is supplied to the prediction calculation unit 74-1 as a prediction tap.

When the dynamic range supplied from the class classification unit 71-1 is large, the prediction tap extraction unit 72-1 includes the edge of the image in the vicinity, and thus the prediction tap is composed of the data in the vicinity. Extract prediction taps with a small spatial spread. On the other hand, the prediction tap extraction unit 7
In 2-1, when the dynamic range supplied from the class classification unit 71-1 is small, since the image is a flat portion, the prediction taps are composed of a wide range of data, and thus the prediction taps having a large spatial spread are extracted.

The prediction tap extraction unit 72-1 has a large possibility that the motion vector is erroneous when the residual which is the feature amount supplied from the motion vector detection unit 2 is large. Extract taps. On the other hand, the prediction tap extracting unit 72-1 has a small possibility that the motion vector is erroneous when the residual, which is the feature amount supplied from the motion vector detecting unit 2, is small, and thus the prediction tap having a small spatial spread. To extract.

The predictive tap extracting unit 72-1 may extract predictive taps based on only the class code.

When the data of interest has a phase indicated by a white circle as shown in FIG. 20A, for example, the upper layer feature amount is calculated from only the data of the same phase as the data of interest indicated by a white circle as shown in FIG. 20B. Composed. At this time, for example, the prediction tap corresponding to the upper layer feature amount is the attention data p5, and the data p two higher than the attention data p5.
1, data p2 on the one side above the attention data p5, data p2 on the two left side, data p3 on the one side above the attention data p5, and data p3 on the four left side of the attention data p5
4, data p6 on the four right side of the data of interest p5, one data below the data of interest p5, two data p7 on the left side and one data below the data of interest p5, two data p on the right side
8 and the data p9 that is two lower than the target data p5.

For example, as shown in FIG. 20C, the upper layer feature amount is composed of an average value of a total of four data, which are indicated by triangles, two vertically and two horizontally. The upper and lower positions of the upper layer feature amount formed by the average value are displaced from the position of the data of the original image signal by half the line width. Further, the left and right positions of the upper layer feature amount formed of the average value are displaced from the position of the data of the original image signal by half the data interval.

At this time, when the attention data is p11, the prediction tap corresponding to the higher layer feature amount is, for example,
The data A is two lines and half above the target data p11, and is on the left side by half the interval of the original data.
1. The data A2 is two and a half upper side of the target data p11, and the data A2 is one and a half right of the original data interval, and the line is one and a half upper side of the original data interval. One and half left data A3, one line and one half upper side, and half the original data interval on the right side data A4, one line and one half upper side, and two original data intervals Data A on the right half
5, the line is half upper side, the data A6 on the left side by half the interval of the original data, the line is half upper side,
Data A7 on the right side of one and half of the original data interval,
The line is half down, one half of the original data interval and half the left data A8, the line is half down, half the original data interval on the right data A9, half the line is half down Data A10 on the right side by two and half of the original data interval and data A1 on the left side by one and half lines and half the original data interval
1, data is one and a half downside, and data A12 on the right side of one and half of the original data interval, and two and half lines on the right side of the original data interval, and left side is one and a half of the original data interval Data A13, two lines and two half lines on the lower side, and data A14 on the right side by half the interval of the original data,
The line is two and half down and consists of data A15 two and half the original data spacing to the left.

As shown in FIG. 21, the prediction tap corresponding to the current field # -1 in the lower layer feature amount is 5 × 5 data centered on the lower layer feature amount data corresponding to the data of interest. Composed of.

The predictive tap of the corresponding field # 0 in the lower layer feature amount is centered on the data of the lower layer feature amount designated by the motion vector from the target data, and is 5
It consists of × 5 data.

The coefficient memory 73-1 includes a class classification unit 71.
Based on the class code supplied from -1, the coefficient set corresponding to the class specified by the class code is selected from the coefficient sets stored in advance, and the selected class code is sent to the prediction calculation unit 74-1. Supply.

The prediction calculation unit 74-1 corresponds to, for example, the equation (6) based on the prediction tap supplied from the prediction tap extraction unit 72-1 and the coefficient set supplied from the coefficient memory 73-1. The Y signal is predicted by calculation, and the predicted Y signal is output.

The class classification unit 71-2 includes the motion vector supplied from the motion vector detection unit 2, the upper layer feature amount,
Based on the lower layer feature amount, the class tap supplied from the class tap extraction unit 3, and the subcarrier phase information supplied from the NTSC decoder 1, the class code indicating the result of the class classification is predicted tap extracted. Part 7
2-2 and coefficient memory 73-2.

For example, the class classification unit 71-2 uses the subcarrier phase information supplied from the NTSC decoder 1 as the basis.
Classify. Also, the class classification unit 71-2 is NTSC.
Based on the data of interest indicated by the subcarrier phase information supplied from the decoder 1, the class is classified based on the phase of the data designated by the motion vector.

The class classifying unit 71-2 classifies the class based on the upper layer feature amount and the lower layer feature amount supplied from the motion vector detecting unit 2. That is, for example, the class classification unit 71-2 applies the ADRC process to the upper layer feature amount supplied from the motion vector detection unit 2, and quantizes the upper layer feature amount into a class code having a predetermined number of bits. The class classification unit 71-2 applies the ADRC process to the lower layer feature amount supplied from the motion vector detection unit 2, and quantizes the lower layer feature amount into a class code having a predetermined number of bits.

The class classification unit 71-2 determines, based on the attention data of the image signal and the peripheral data around the attention data,
Classify into spatial classes. That is, for example, the class classification unit 71-2 applies the ADRC processing to the target data of the image signal and the peripheral data, and quantizes the target data and the peripheral data into a class code having a predetermined number of bits.

The class classification unit 71-2 performs class classification based on the target data of the corresponding field indicated by the motion vector from the target data of the target field and the peripheral data around the target data. That is, for example, the class classification unit 71-2 uses the target data of the corresponding field, which is indicated by a motion vector from the target data of the target field,
And ADRC processing is applied to peripheral data around the target data, and the target data and peripheral data are quantized into a class code having a predetermined number of bits.

The class classification unit 71-2, based on the amount of movement,
You may classify.

The class classification unit 71-2 supplies the dynamic range, which is the difference between the maximum value and the minimum value of the data values of the target data of the image signal and the peripheral data of the predetermined block, to the prediction tap extraction unit 72-2. .

Note that the class classification unit 71-2 may calculate the dynamic range based on the class tap of the attention field lower layer feature amount, an example of which is shown in FIG.

Further, the class classification unit 71-2 may classify the class based on the higher layer feature amount included in the class tap.

The predictive tap extracting unit 72-2 is based on the motion vector supplied from the motion vector detecting unit 2 and the feature amount such as the residual, and the class code and the dynamic range supplied from the class classifying unit 71-2. Further, predetermined data is extracted from the image signal data, the upper layer feature amount, and the lower layer feature amount, and the extracted data is supplied to the prediction calculation unit 74-2 as a prediction tap.

The predictive tap extracting section 72-2 may extract predictive taps based on only the class code.

The coefficient memory 73-2 includes a class classification unit 71.
-2, a coefficient set corresponding to the class specified by the class code is selected from the coefficient sets stored in advance based on the class code supplied from -2, and the selected class code is selected by the prediction calculation unit 74-2. Supply.

The prediction calculation unit 74-2 corresponds to, for example, the expression (6) based on the prediction tap supplied from the prediction tap extraction unit 72-2 and the coefficient set supplied from the coefficient memory 73-2. The operation is used to predict the U signal, and the predicted U signal is output.

The class classification unit 71-3 has the motion vector supplied from the motion vector detection unit 2, the upper layer feature amount,
Based on the lower layer feature amount, the class tap supplied from the class tap extraction unit 3, and the subcarrier phase information supplied from the NTSC decoder 1, the class code indicating the result of the class classification is predicted tap extracted. Part 7
2-3 and the coefficient memory 73-3.

For example, the class classification unit 71-3, based on the subcarrier phase information supplied from the NTSC decoder 1,
Classify. Also, the class classification unit 71-3 is an NTSC
Based on the data of interest indicated by the subcarrier phase information supplied from the decoder 1, the class is classified based on the phase of the data designated by the motion vector.

The class classifying unit 71-3 classifies the class based on the upper layer characteristic amount and the lower layer characteristic amount supplied from the motion vector detecting unit 2. That is, for example, the class classification unit 71-3 applies the ADRC process to the upper layer feature amount supplied from the motion vector detection unit 2, and quantizes the upper layer feature amount into a class code having a predetermined number of bits. The class classification unit 71-3 applies the ADRC process to the lower layer feature amount supplied from the motion vector detection unit 2, and quantizes the lower layer feature amount into a class code having a predetermined number of bits.

The class classification unit 71-3, based on the attention data of the image signal and the peripheral data around the attention data,
Classify into spatial classes. That is, for example, the class classification unit 71-3 applies the ADRC processing to the target data of the image signal and the peripheral data, and quantizes the target data and the peripheral data into a class code having a predetermined number of bits.

The class classification unit 71-3 performs class classification based on the target data of the corresponding field indicated by the motion vector from the target data of the target field and the peripheral data around the target data. That is, for example, the class classification unit 71-3, the target data of the corresponding field, which is indicated by the motion vector from the target data of the target field,
And ADRC processing is applied to peripheral data around the target data, and the target data and peripheral data are quantized into a class code having a predetermined number of bits.

The class classification unit 71-3, based on the amount of movement,
You may classify.

The class classification unit 71-3 supplies the prediction tap extraction unit 72-3 with the dynamic range which is the difference between the maximum value and the minimum value of the data values of the image data of interest and the peripheral data of the predetermined block. .

Note that the class classification unit 71-3 may calculate the dynamic range based on the class tap of the attention field lower layer feature amount, an example of which is shown in FIG.

Further, the class classification unit 71-3 may classify the class based on the higher layer feature amount included in the class tap.

The prediction tap extraction unit 72-3 is based on the motion vector supplied from the motion vector detection unit 2, the feature amount such as the residual, and the class code and the dynamic range supplied from the class classification unit 71-3. Then, predetermined data is extracted from the image signal data, the upper layer feature amount, and the lower layer feature amount, and the extracted data is supplied to the prediction calculation unit 74-3 as a prediction tap.

The predictive tap extracting section 72-3 may extract predictive taps based on only the class code.

The coefficient memory 73-3 includes a class classification unit 71.
-3, the coefficient set corresponding to the class specified by the class code is selected from the coefficient sets stored in advance based on the class code supplied from -3, and the selected class code is selected by the prediction calculation unit 74-3. Supply.

The prediction calculation unit 74-3 corresponds to, for example, the expression (6) based on the prediction tap supplied from the prediction tap extraction unit 72-3 and the coefficient set supplied from the coefficient memory 73-3. The V signal is predicted by calculation, and the predicted V signal is output.

As described above, the operation unit 4 predicts the Y signal, the U signal, and the V signal, which are more accurate component video signals than the conventional one, by the class classification adaptive processing, and predicts the predicted Y. Signals, U signals, and V signals can be output.

Below, the class classification units 71-1 to 71-3
When there is no need to distinguish each of them, they are simply referred to as a class classification unit 71.

Hereinafter, the prediction tap extracting units 72-1 to 72
-3 is simply referred to as a prediction tap extracting unit 72 when it is not necessary to individually distinguish it.

Hereinafter, when it is not necessary to individually distinguish the coefficient memories 73-1 to 73-3, the coefficient memories 73-1 to 73-3 are simply used.
Called.

Hereinafter, when it is not necessary to individually distinguish the prediction calculation units 74-1 to 74-3, the prediction calculation unit 74 is simply used.
Called.

FIG. 22 is a block diagram showing another configuration of the arithmetic unit 4. The same parts as those shown in FIG. 19 are designated by the same reference numerals, and the description thereof will be omitted.

The selector 81 selects one of the three class codes and the dynamic range individually supplied from the class classification units 71-1 to 71-3, and selects the selected class code and dynamic range. The range is supplied to the prediction tap extracting unit 82.

The selector 81 selects from among the three coefficient sets individually supplied from the coefficient memories 73-1 to 73-3.
One coefficient set is selected, and the selected coefficient set is supplied to the prediction calculation unit 83.

The predictive tap extracting unit 82 receives the motion vector supplied from the motion vector detecting unit 2, the upper layer feature amount,
A prediction tap is extracted based on the lower layer feature amount, the class code and the dynamic range supplied from the selector 81, and the extracted prediction tap is calculated by the prediction calculation unit 8
Supply to 3.

When the class code and the dynamic range output from the class classification unit 71-1 are supplied via the selector 81 to the prediction tap extraction unit 82, Y is output.
The prediction tap corresponding to the signal is extracted, and the extracted prediction tap is supplied to the prediction calculation unit 83. Prediction tap extraction unit 82
When the class code and the dynamic range output from the class classification unit 71-2 are supplied through the selector 81, the prediction tap corresponding to the U signal is extracted, and the extracted prediction tap is calculated by the prediction calculation unit 83. Supply to.
The prediction tap extraction unit 82 extracts the prediction tap corresponding to the V signal when the class code and the dynamic range output from the class classification unit 71-3 are supplied via the selector 81, and the extracted prediction tap. Is supplied to the prediction calculation unit 83.

The prediction calculation unit 83 has a prediction tap extraction unit 82.
When the prediction tap corresponding to the Y signal is supplied from, the Y signal is predicted based on the coefficient set supplied from the coefficient memory 73-1 via the selector 81, and the predicted Y signal is output. The prediction calculation unit 83 includes the prediction tap extraction unit 8
2 when the prediction tap corresponding to the U signal is supplied,
The U signal is predicted via the selector 81 based on the coefficient set supplied from the coefficient memory 73-2, and the predicted U signal is calculated.
Output a signal. When the prediction tap extraction unit 82 supplies the prediction tap corresponding to the V signal, the prediction calculation unit 83 predicts the V signal based on the coefficient set supplied from the coefficient memory 73-3 via the selector 81. Then, the predicted V signal is output.

FIG. 23 is a block diagram showing another structure of the arithmetic unit 4. The same parts as those shown in FIG. 19 are designated by the same reference numerals, and the description thereof will be omitted.

The class classification unit 91 supplies the motion vector supplied from the motion vector detection unit 2, the upper layer feature amount and the lower layer feature amount, the class tap supplied from the class tap extraction unit 3, and the NTSC decoder 1. Based on the generated subcarrier phase information, the class is classified, and the class code indicating the result of the class classification is predicted tap extraction unit 72-
1 to 72-3 and coefficient memories 73-1 to 73-3
Supply to.

Prediction tap extraction units 72-1 to 72-3
For example, different prediction taps are extracted based on different thresholds.

FIG. 24 is a block diagram showing another structure of the arithmetic unit 4. The same parts as those shown in FIG. 19 are designated by the same reference numerals, and the description thereof will be omitted. The same parts as those shown in FIG. 22 are designated by the same reference numerals, and the description thereof will be omitted.

The class classification unit 91 supplies the motion vector supplied from the motion vector detection unit 2, the upper layer feature amount and the lower layer feature amount, the class tap supplied from the class tap extraction unit 3, and the NTSC decoder 1. Classifying is performed based on the obtained subcarrier phase information, and a class code indicating the result of classifying is supplied to the prediction tap extracting unit 82 and the coefficient memories 73-1 to 73-3.

The selector 101 selects one coefficient set from the three coefficient sets individually supplied from the coefficient memories 73-1 to 73-3, and supplies the selected coefficient set to the prediction calculation section 83.

FIG. 25 is a block diagram showing details of an example of the configuration of the embodiment of the image processing apparatus according to the present invention.
The NTSC composite signal input to the image processing device is supplied to the subcarrier phase information detection circuit 121, the field memory 122-1, the motion vector detection circuit 123, and the delay circuit 124-1.

The subcarrier phase information detection circuit 121 has
The phase is detected from the input NTSC composite signal, and the information on the detected phase is supplied to the motion vector detection circuit 123, the subcarrier phase information detection circuit 125, and the class classification unit 127.

The field memory 122-1 stores the input NTSC composite signal, delays it for a period corresponding to one field, and stores the stored NTSC composite signal in the field memory 122-2, the motion vector detection circuit 123, And the delay circuit 124-2.

The field memory 122-2 stores the NTSC composite signal supplied from the field memory 122-1 and delays it for a period corresponding to one field, and stores the stored NTSC composite signal in the motion vector detection circuit 123. And the delay circuit 124-3.

Therefore, the motion vector detection circuit 123 simultaneously receives the signal of the reference field, the signal of the next (future) field of the reference field, and the signal of the previous (past) field of the reference field. Supplied. The motion vector detection circuit 123 corresponds to the reference field and the previous field based on the signal of the reference field, the signal of the field next to the reference field, and the signal of the field before the reference field. Vector (t-1) and the vector (t +) corresponding to the reference field and the next field.
1) is generated.

The motion vector detection circuit 123 determines the vector (t) corresponding to the reference field and the previous field.
−1), and a vector (t + 1) corresponding to the reference field and the next field are supplied to the subcarrier phase information detection circuit 125.

The motion vector detection circuit 123 uses the vector (t) corresponding to the reference field and the previous field.
-1) is the area cutting circuit 126-3 and the area cutting circuit 1
28-3, and the vector (t + 1) corresponding to the reference field and the next field is supplied to the area cutting circuit 126-1 and the area cutting circuit 128-1.

The motion vector detection circuit 123 supplies the feature amount calculated by the process of detecting the vector (t-1) and the vector (t + 1) to the class classification unit 127.

The delay circuit 124-1 delays the signal of the field next to the reference field in accordance with the processing time of the motion vector detection circuit 123, and outputs the signal of the field next to the delayed reference field. It is supplied to the area cutout circuit 126-1.

The delay circuit 124-2 delays the signal of the reference field in correspondence with the processing time of the motion vector detection circuit 123, and sends the delayed signal of the reference field to the area cutout circuit 126-1. Supply.

The delay circuit 124-3 delays the signal of the field before the reference field in accordance with the processing time of the motion vector detection circuit 123, and outputs the signal of the field before the delayed reference field. It is supplied to the area cutting circuit 126-1.

The subcarrier phase information detection circuit 125
The phase information supplied from the subcarrier phase information detection circuit 121, the reference field supplied from the motion vector detection circuit 123 and the vector (t-1) corresponding to the previous field, and the reference field and the next field. Based on the vector (t + 1) corresponding to the field of
The phase information corresponding to the previous field and the phase information corresponding to the next field are generated. The subcarrier phase information detection circuit 125 supplies the phase information corresponding to the previous field and the phase information corresponding to the next field to the class classification unit 127.

The area extraction circuit 126-1 is supplied from the delay circuit 124-1 based on the vector (t + 1) corresponding to the reference field and the next field supplied from the motion vector detection circuit 123. , The predetermined data included in the signal of the field next to the reference field is cut out, and the cut-out data is classified into
Supply to 7.

The area cutout circuit 126-2 is supplied from the delay circuit 124-2 based on the vector (t-1) corresponding to the reference field and the previous field supplied from the motion vector detection circuit 123. Cut out the specified data included in the signal of the reference field,
The cut out data is supplied to the class classification unit 127.

The area cutout circuit 126-3 is the delay circuit 12
The predetermined data included in the signal of the field before the reference field supplied from 4-3 is cut out,
The cut out data is supplied to the class classification unit 127.

The class classification unit 127 includes the phase information supplied from the subcarrier phase information detection circuit 121, the feature amount supplied from the motion vector detection circuit 123, and the previous field supplied from the subcarrier phase information detection circuit 125. , The phase information corresponding to the next field, and the phase information corresponding to the next field, supplied from the area cutting circuit 126-1,
Data cut out from the signal of the field next to the reference field, data cut out from the signal of the reference field supplied from the area cutting circuit 126-2, and supplied from the area cutting circuit 126-3 Based on the data cut out from the signal of the field before the reference field, the class code indicating the result of the class classification is extracted by the area cutting circuit 128-1 and the area cutting circuit 128-2. , Region extraction circuit 128-3, coefficient memory 129, and prediction unit 130.

The class classifying section 127 classifies into four classes based on the phase information corresponding to the previous field supplied from the phase information detecting circuit 125, for example. The class classification unit 127, for example, based on the phase information corresponding to the reference field supplied from the phase information detection circuit 121,
Classify into 4 classes. The class classification unit 127, for example,
Based on the phase information corresponding to the next field supplied from the phase information detection circuit 125, it is classified into four classes.

The class categorizing unit 127, for example, outputs the data cut out from the signal in the field next to the reference field, the data cut out from the signal in the reference field, and the signal in the field before the reference field. 5 data that are data cut out from
ADRC processing is applied to classify into 32 classes.

The class classification unit 127, for example, based on the feature amount supplied from the motion vector detection circuit 123, compares and determines with a predetermined threshold value and classifies the field next to the reference field into two classes. Then, the field before the reference field is classified into two classes.

The class classification unit 127, for example, multiplies the above class classifications and finally classifies them into 8192 classes.

The class classification unit 127 has the phase information supplied from the subcarrier phase information detection circuit 121, the feature quantity supplied from the motion vector detection circuit 123, and the phase corresponding to the field before the subcarrier phase information detection circuit 125. Information and phase information corresponding to the next field, data extracted from the signal of the field next to the reference field supplied from the area extraction circuit 126-1, and supplied from the area extraction circuit 126-2 The structure of the prediction tap is based on the data cut out from the signal of the reference field and the data cut out from the signal of the field before the reference field, which is supplied from the region cutting circuit 126-3. The structure variable information to be designated is generated, and the generated structure variable information is subjected to the area extraction circuit 128-
1, area cutout circuit 128-2, and area cutout circuit 12
8-3.

The area cutout circuit 128-1 supplies the vector (t + 1) corresponding to the reference field and the next field supplied from the motion vector detection circuit 123, and the class code and the class code supplied from the class classification unit 127. Based on the structure variable information, predetermined data included in the signal of the field next to the reference field supplied from the delay circuit 124-1 is cut out, and the cut out data is supplied to the prediction unit 130.

The area cutout circuit 128-2 includes a predetermined code contained in the signal of the reference field supplied from the delay circuit 124-2 based on the class code and the structure variable information supplied from the class classification unit 127. Data is cut out and the cut out data is supplied to the prediction unit 130.

The area cutout circuit 128-3 supplies the vector (t-1) corresponding to the reference field and the previous field supplied from the motion vector detection circuit 123, and the class supplied from the class classification unit 127. Based on the code and the structure variable information, predetermined data included in the signal of the field before the reference field supplied from the delay circuit 124-3 is cut out, and the cut out data is supplied to the prediction unit 130.

The coefficient memory 129 predicts a predetermined coefficient set corresponding to the classified class on the basis of the prediction mode setting signal supplied from the outside and the class code.
Supply to 0.

The predicting unit 130 selects the class code supplied from the class classifying unit 127, the coefficient set supplied from the coefficient memory 129, and the field next to the reference field supplied from the area extracting circuit 128-1. The data cut out from the signal, the data cut out from the signal of the reference field supplied from the area cutout circuit 128-2, and the data before the reference field supplied from the area cutout circuit 128-3 Based on the data cut out from the field signal, for example, Y signal, U signal, and V signal
Predict a component video signal consisting of signals.

As described above, the image processing apparatus according to the present invention can generate a more accurate component video signal corresponding to the input composite signal.

FIG. 26 is a flow chart for explaining the process of generating a component video signal by the image processing apparatus according to the present invention.

In step S1, the motion vector detecting section 2 detects a motion vector based on the image signal and subcarrier phase information supplied from the NTSC decoder 1. Details of the motion vector detection process will be described later.

In step S2, the class tap extracting unit 3 determines the image signal data and the upper layer feature based on the subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detecting unit 2. Extract class taps consisting of quantity data and lower layer feature quantity data.

At step S3, the operation section 4 determines the NTSC
The image signal and subcarrier phase information supplied from the decoder 1, the motion vector supplied from the motion vector detection unit 2, the data of the upper layer feature amount and the data of the lower layer feature amount, and the class tap extraction unit 3 are supplied. Classify based on the class tap. For example, in FIG.
9, the class classification units 71-1 to 71- of the calculation unit 4 are shown.
Reference numeral 3 denotes the image signal and subcarrier phase information supplied from the NTSC decoder 1, the motion vector supplied from the motion vector detection unit 2, the upper layer feature amount data, the lower layer feature amount data, and the class tap extraction unit. Classifying is performed based on the class taps supplied from 3.

In step S4, the calculation unit 4 uses the classified classes and the motion vector supplied from the motion vector detection unit 2 to calculate the image signal data, the upper layer feature amount data, and the lower layer feature amount. Extract the prediction tap consisting of the data of. For example, the prediction tap extracting units 72-1 to 72-3 illustrated in FIG. 19 are based on the class code indicating the classified class, the dynamic range, and the motion vector supplied from the motion vector detecting unit 2,
A prediction tap composed of image signal data, upper layer feature amount data, and lower layer feature amount data is extracted.

In step S5, the operation section 4 predicts the component video signal based on the extracted prediction tap and the coefficient set stored in advance, and the processing ends. For example, the prediction calculation units 74-1 to 74-3 of the calculation unit 4 illustrated in FIG. 19 are based on the coefficient set supplied from the prediction taps and the coefficient memories 73-1 to 73-3, respectively. Predict the signal and the V signal.

FIG. 27 is a flow chart for explaining the motion vector detection processing by the motion vector detection unit 2 corresponding to step S1. In step S11, the motion vector detection unit 2 acquires the phase of the subcarrier of the data of interest based on the subcarrier phase information supplied from the NTSC decoder 1 via the NTSC decoder 1.

In step S12, the feature quantity conversion unit 2
3 calculates the characteristic amount of the upper layer for the field of interest and the corresponding field based on the image signal and the acquired phase of the subcarrier, and outputs the calculated characteristic amount as the upper layer characteristic amount.

In step S13, the upper layer motion vector detection unit 25 detects the upper layer vector, which is an approximate vector approximate to the motion vector, based on the upper layer feature amount supplied from the feature amount conversion unit 23. Details of the process of detecting the upper layer vector will be described later.

In step S14, the feature quantity conversion unit 2
4 calculates the characteristic amount of the lower layer for the field of interest and the corresponding field based on the image signal and the acquired phase of the subcarrier, and outputs the calculated characteristic amount as the lower layer characteristic amount.

In step S15, the lower layer motion vector detecting unit 26 detects the lower layer characteristic amount supplied from the characteristic amount converting unit 24 and the upper layer motion vector detecting unit 2.
The lower layer vector is detected based on the upper layer vector supplied from No. 5. Details of the process of detecting the lower layer vector will be described later.

In step S16, the lower layer motion vector detection unit 26 outputs the motion vector, and the process ends. That is, the lower layer motion vector detection unit 26
Outputs the detected lower layer motion vector as a motion vector.

FIG. 28 is a flow chart for explaining the details of the process of detecting the upper layer vector by the upper layer motion vector detecting unit 25, which corresponds to step S13. In step S21, the vector generation unit 31 of the higher layer motion vector detection unit 25 generates a predetermined vector based on the parameter A and supplies it to the correlation value calculation unit 32 and the determination unit 33. In step S22, the correlation value calculation unit 32 determines, from the vector generation unit 31 of a block including a predetermined number of upper layer feature values centering on the upper layer feature value corresponding to the attention data of the attention field, and the corresponding field. A correlation value with a block composed of a predetermined number of upper layer feature values centering on the position specified by the supplied vector is calculated. The correlation value calculation unit 32 calculates, for example, as the correlation value, the sum of the absolute values of the differences between the upper layer feature amount belonging to the block of the field of interest and the upper layer feature amount belonging to the block of the corresponding field.

At step S23, the judging section 33
The correlation value supplied from the correlation value calculation unit 32 is stored in association with the vector supplied from the vector generation unit 31.

In step S24, the upper layer motion vector detection unit 25 specifies the parameter A,
It is determined whether or not the correlation value in the predetermined range has been calculated, and when it is determined that the correlation value in the predetermined range has not been calculated, the process returns to step S21, and the correlation value is calculated corresponding to the next vector. The process of is repeated.

If it is determined in step S24 that the correlation value in the predetermined range designated by the data included in the parameter A has been calculated, the process proceeds to step S25, and the determination unit 33 determines the stored correlation value. Of these, the vector corresponding to the strongest correlation is selected. For example, the correlation value calculation unit 3
2 calculates, as a correlation value, the sum of the absolute values of the differences between the upper layer feature amount belonging to the block of the field of interest and the upper layer feature amount belonging to the block at the position specified by the vector of the corresponding field, The determination unit 33 selects the smallest correlation value.

At step S26, the judgment section 33
The vector selected in the process of step S25 is output as the upper layer vector, and the process ends.

FIG. 29 is a flow chart for explaining the details of the process of detecting the lower hierarchy vector corresponding to step S15. In step S31, the vector generation unit 41 of the lower layer motion vector detection unit 26 generates a predetermined vector based on the upper layer vector and the parameter B, and supplies it to the correlation value calculation unit 42 and the determination unit 43.
In step S32, the correlation value calculation unit 42 has a block composed of a predetermined number of lower layer feature values centering on the lower layer feature values corresponding to the attention data in the attention field,
A correlation value with a block composed of a predetermined number of lower layer feature values centering on the position designated by the upper layer vector of the corresponding field and the vector supplied from the vector generation unit 41 is calculated. The correlation value calculation unit 42 calculates, for example, as the correlation value, the cross-correlation between the lower layer feature amount belonging to the block of the field of interest and the lower layer feature amount belonging to the block specified by the vector of the corresponding field. .

At step S33, the judging section 43
The correlation value supplied from the correlation value calculation unit 42 is stored in association with the vector supplied from the vector generation unit 41.

[0275] In step S34, the lower hierarchical motion vector detection unit 26 determines whether or not the correlation value in the predetermined range designated by the data included in the parameter B has been calculated, and the correlation value in the predetermined range is determined. If it is determined that the correlation value has not been calculated, the process returns to step S31 and the correlation value calculation process is repeated.

If it is determined in step S34 that the correlation value within the predetermined range has been calculated, the process proceeds to step S35, and the determination section 43 selects the vector corresponding to the strongest correlation among the stored correlation values. To do. For example, the correlation value calculation unit 42 calculates, as the correlation value, the cross-correlation between the lower layer feature amount belonging to the block of the field of interest and the lower layer feature amount belonging to the block specified by the vector of the corresponding field. At this time, the determination unit 33 selects the maximum correlation value.

At step S36, the judging section 43
The vector selected in the process of step S35 is output as the lower layer vector, and the process ends.

In this way, the motion vector detecting section 2 can detect a motion vector with higher accuracy.

Since the motion vector detecting section 2 is adapted to execute the process of detecting the motion vector in the two layers, it is possible to increase the accuracy without increasing the circuit scale and the complicated operation. Of high motion vector can be detected.

FIG. 30 corresponds to the processing of step S15, and the lower hierarchical motion vector detecting section 2 having the configuration shown in FIG.
11 is a flowchart illustrating details of another process of detecting a lower layer vector according to No. 6;

In step S41, the class classification unit 5
1 classifies the data corresponding to the data of interest based on the subcarrier phase information and the lower layer feature amount.

In step S42, the prediction tap extracting unit 52 determines a predetermined number of classified classes, which corresponds to the lower layer vector corresponding to the previous field and the upper layer vector supplied from the memory 55. The prediction tap which is the lower layer feature amount of is extracted.

In step S43, the calculation section 53
The lower hierarchy vector is calculated based on the coefficient set and the prediction tap stored in the prediction coefficient memory 54.

At step S44, the calculation section 53
The lower layer vector is stored in the memory 55, the lower layer vector is output, and the process ends.

As described above, the lower layer motion vector detecting unit 26 having the configuration shown in FIG. 18 can apply the class classification adaptive processing to the lower layer feature amount to generate a more accurate motion vector.

As described above, the image processing apparatus according to the present invention can generate a more accurate component video signal from a composite video signal in consideration of a motion vector as compared with the conventional one.

FIG. 31 is a block diagram showing the configuration of the embodiment of the image processing apparatus according to the present invention for generating the prediction coefficient used for the prediction processing of the component video signal.

The same parts as those of the image processing apparatus shown in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted.

The component video signal input to the image processing apparatus is supplied to the NTSC encoder 201 and the coefficient calculation section 202.

[0290] The NTSC encoder 201 generates an NTSC composite signal based on the input component video signal, and supplies the generated NTSC composite signal to the NTSC decoder 1.

The coefficient calculation unit 202 inputs based on the subcarrier phase information supplied from the NTSC decoder 1, the motion vector supplied from the motion vector detection unit 2, and the class tap supplied from the class tap extraction unit 3. The component video signal that is generated, the image signal that is supplied from the NTSC decoder 1, and the upper layer feature amount and the lower layer feature amount that are supplied from the motion vector detection unit 2 are given by the formula (1
4) is applied to generate prediction coefficients for predicting a component video signal. Arithmetic unit 4
Supplies the generated prediction coefficient to the coefficient memory 203.

The coefficient memory 203 stores the prediction coefficient supplied from the coefficient calculation unit 202.

FIG. 32 is a block diagram showing the structure of the coefficient calculation unit 202. The same parts as those shown in FIG. 19 are designated by the same reference numerals, and the description thereof will be omitted.

The prediction coefficient calculation unit 221-1 converts the prediction tap supplied from the prediction tap extraction unit 72-1 and the Y signal included in the input component video signal into
For example, the calculation shown in Expression (14) is applied to calculate the prediction coefficient for predicting the Y signal, and the calculated prediction coefficient is output.

The prediction coefficient calculation unit 221-2 adds the prediction tap supplied from the prediction tap extraction unit 72-2 and the U signal included in the input component video signal to
For example, the calculation shown in Expression (14) is applied to calculate the prediction coefficient for predicting the U signal, and the calculated prediction coefficient is output.

The prediction coefficient calculation unit 221-3 converts the prediction tap supplied from the prediction tap extraction unit 72-3 and the V signal included in the input component video signal into
For example, the calculation shown in Expression (14) is applied to calculate the prediction coefficient for predicting the V signal, and the calculated prediction coefficient is output.

FIG. 33 is a block diagram showing another structure of the coefficient calculation unit 202. The same parts as those shown in FIG. 22 are designated by the same reference numerals, and the description thereof will be omitted.

The selector 231 is the class classification unit 71-1.
One of the three class codes and the dynamic range individually supplied from the to 71-3 is selected, and the selected class code and dynamic range are supplied to the prediction tap extracting unit 82.

[0299] When the prediction tap extraction unit 82 supplies the prediction tap corresponding to the Y signal, the prediction coefficient calculation unit 232 supplies the prediction tap supplied from the prediction tap extraction unit 82.
And a prediction coefficient for predicting the Y signal is calculated based on the Y signal included in the component video signal, and the calculated prediction coefficient is output. The prediction coefficient calculation unit 232, when the prediction tap corresponding to the U signal is supplied from the prediction tap extraction unit 82, based on the prediction tap supplied from the prediction tap extraction unit 82 and the U signal included in the component video signal. , A prediction coefficient for predicting the U signal is calculated, and the calculated prediction coefficient is output. The prediction coefficient calculation unit 232, when the prediction tap corresponding to the V signal is supplied from the prediction tap extraction unit 82, based on the prediction tap supplied from the prediction tap extraction unit 82 and the V signal included in the component video signal. , V signal, a prediction coefficient for predicting the V signal is calculated, and the calculated prediction coefficient is output.

FIG. 34 is a block diagram showing still another configuration of the coefficient calculation unit 202. The same parts as those shown in FIG. 23 are designated by the same reference numerals, and the description thereof will be omitted.

The prediction coefficient calculation unit 221-1 converts the prediction tap supplied from the prediction tap extraction unit 72-1 and the Y signal included in the input component video signal into
For example, the calculation shown in Expression (14) is applied to calculate the prediction coefficient for predicting the Y signal, and the calculated prediction coefficient is output.

The prediction coefficient calculation unit 221-2 adds the prediction tap supplied from the prediction tap extraction unit 72-2 and the U signal included in the input component video signal to
For example, the calculation shown in Expression (14) is applied to calculate the prediction coefficient for predicting the U signal, and the calculated prediction coefficient is output.

The prediction coefficient calculation unit 221-3 converts the prediction tap supplied from the prediction tap extraction unit 72-3 and the V signal included in the input component video signal into
For example, the calculation shown in Expression (14) is applied to calculate the prediction coefficient for predicting the V signal, and the calculated prediction coefficient is output.

FIG. 35 is a block diagram showing still another configuration of the coefficient calculation unit 202. The same parts as those shown in FIG. 24 are designated by the same reference numerals, and the description thereof will be omitted.

When the prediction tap extraction unit 82 supplies the prediction tap corresponding to the Y signal, the prediction coefficient calculation unit 232 supplies the prediction tap supplied from the prediction tap extraction unit 82.
And a prediction coefficient for predicting the Y signal is calculated based on the Y signal included in the component video signal, and the calculated prediction coefficient is output. The prediction coefficient calculation unit 232, when the prediction tap corresponding to the U signal is supplied from the prediction tap extraction unit 82, based on the prediction tap supplied from the prediction tap extraction unit 82 and the U signal included in the component video signal. , A prediction coefficient for predicting the U signal is calculated, and the calculated prediction coefficient is output. The prediction coefficient calculation unit 232, when the prediction tap corresponding to the V signal is supplied from the prediction tap extraction unit 82, based on the prediction tap supplied from the prediction tap extraction unit 82 and the V signal included in the component video signal. , V signal, a prediction coefficient for predicting the V signal is calculated, and the calculated prediction coefficient is output.

FIG. 36 is a block diagram showing details of an example of the configuration of the embodiment of the image processing apparatus for calculating the prediction coefficient. The same parts as those shown in FIG. 25 are designated by the same reference numerals, and the description thereof will be omitted.

An HD (High-Definition) or progressive component video signal which is an input signal is supplied to the thinning filter 261 and the selection circuit 263.

The decimation filter 261 generates an SD (Standard-Definition) interlaced component video signal from an HD or progressive component video signal which is an input signal. Thinning filter 26
1 supplies the generated component video signal to the NTSC encoder 262 and the selection circuit 263.

The NTSC encoder 262 uses the input component video signal as the student image
Generate a composite video signal. NTSC encoder 2
A subcarrier phase information detection circuit 121 and a field memory 1 62 generate the generated NTSC composite video signal.
22-1, the motion vector detection circuit 123, and the delay circuit 124-1.

The selection circuit 263 receives the learning mode setting signal supplied from the outside, and inputs it to the image processing apparatus.
HD or progressive component video signal,
And one of the SD interlaced component video signals supplied from the thinning filter 261 is selected, and the selected signal is supplied to the normal equation calculation circuit 264 as a teacher image.

The normal equation operation circuit 264 uses the class code and area cutout circuit 1 supplied from the class classification unit 127.
28-1, the data cut out from the signal of the field next to the reference field supplied from 28-1, the data cut out from the signal of the reference field supplied from the area cutting circuit 128-2, the area cut Output circuit 128-3
Data extracted from the signal of the field before the reference field, which is supplied from
3 based on the teacher image supplied from
A prediction coefficient for predicting a signal and a V signal is calculated. The normal equation calculation circuit 264 supplies the calculated prediction coefficient to the coefficient memory 265.

The coefficient memory 265 stores the prediction coefficient supplied from the normal equation calculating circuit 264.

As described above, the image processing apparatus according to the present invention can calculate the prediction coefficient for predicting the component video signal based on the composite video signal.

FIG. 37 is a flow chart for explaining a learning process for calculating a prediction coefficient for predicting a component video signal based on a composite video signal by the image processing apparatus according to the present invention.

[0315] In step S201, the NTSC encoder 201 generates a composite video signal based on the input component video signal.

In step S202, the motion vector detecting section 2 detects a motion vector based on the image signal which is the composite video signal and the subcarrier phase information. For details of the processing in step S202, see step S1.
Since the processing is the same as that of step 1, the description thereof will be omitted.

In step S203, the class tap extracting unit 3 determines the image signal data and the upper layer feature based on the subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detecting unit 2. Extract class taps consisting of quantity data and lower layer feature quantity data.

In step S204, the coefficient calculation unit 2
Reference numeral 02 denotes the image signal and subcarrier phase information supplied from the NTSC decoder 1, the motion vector supplied from the motion vector detection unit 2, the upper layer feature amount data, the lower layer feature amount data, and the class tap extraction unit. Three
Classify based on the class tap supplied from.
For example, the class classification units 71-1 to 71-3 of the coefficient calculation unit 202 illustrated in FIG. 32 include the image signal and subcarrier phase information supplied from the NTSC decoder 1 and the motion vector supplied from the motion vector detection unit 2. , The upper layer feature amount data, the lower layer feature amount data, and the class tap supplied from the class tap extracting unit 3 are used for classifying.

In step S205, the coefficient calculation unit 2
Reference numeral 02 extracts a prediction tap composed of image signal data, upper layer feature amount data, and lower layer feature amount data, based on the classified classes and the motion vector supplied from the motion vector detection unit 2. . For example, in FIG.
Prediction tap extraction unit 72-1 of coefficient calculation unit 202 shown in FIG.
To 72-3 are a class code indicating a classified class, a dynamic range, and a motion vector detection unit 2
A prediction tap composed of image signal data, upper layer feature amount data, and lower layer feature amount data is extracted based on the motion vector supplied from.

In step S206, the coefficient calculation unit 2
02 calculates a prediction coefficient for predicting the component video signal based on the extracted prediction tap and the input component video signal, and the process ends. For example, the prediction coefficient calculation units 221-1 to 221-3 of the coefficient calculation unit 202 illustrated in FIG. 32 predict the Y signal, the U signal, or the V signal based on the prediction tap and the component video signal, respectively. Calculate the prediction coefficient of.

As described above, the image processing apparatus according to the present invention can calculate the prediction coefficient for predicting the component video signal based on the composite video signal.

FIG. 38 is a block diagram showing a structure of an image processing apparatus for generating a prediction coefficient used in a process of calculating a lower layer vector in the lower layer motion vector detecting unit 26 having the structure shown in FIG.

The NTSC decoder 301 receives the input NTSC
Based on the image signal which is the composite video signal of the system,
An image signal which is digital data corresponding to the image signal is generated, and subcarrier phase information corresponding to the image signal which is digital data is generated. NTSC decoder 3
01 supplies the generated image data, which is digital data, and subcarrier phase information to the image memory 302.

The image memory 302 is the NTSC decoder 301.
The image signal and the subcarrier phase information supplied from are stored, and the stored image signal and subcarrier phase information are supplied to the coefficient generation unit 303.

The coefficient generator 303 is provided with a parameter A supplied from outside the image processing apparatus and the image memory 302.
A coefficient set corresponding to the image signal and the motion vector supplied from the image memory 302 is generated based on the subcarrier phase information supplied from, and the generated coefficient set is output. The coefficient set is composed of a predetermined number of prediction coefficients corresponding to each class.

The parameter A is a parameter for designating the content of the processing in the coefficient generator 303, and designates, for example, the correlation calculation method, the size of the block to be matched, the size of the search area, or the like.

FIG. 39 is a block diagram showing the structure of the coefficient generator 303. The same parts as those of the motion vector detecting unit 2 shown in FIG. 3 are designated by the same reference numerals, and the description thereof will be omitted.

The learning unit 321 includes the motion vector, the lower layer feature amount and subcarrier phase information supplied from the feature amount conversion unit 24, and the upper layer motion vector detection unit 25.
A coefficient set for calculating a motion vector is calculated based on the higher-layer vector supplied from, and the calculated coefficient set is output.

FIG. 40 is a block diagram showing the structure of the learning section 321.

The class classification unit 351 classifies the class based on the subcarrier phase information and the lower layer feature amount, and supplies a class code indicating the classified class to the prediction tap extraction unit 352.

The prediction tap extraction unit 352 extracts a predetermined lower layer feature amount based on the class indicated by the class code, the motion vector predictor corresponding to the previous field supplied from the memory 354, and the upper layer vector. do it,
The extracted lower layer feature amount is supplied to the coefficient calculation unit 353 as a prediction tap.

The coefficient calculation unit 353 is the prediction tap extraction unit 3
The prediction coefficient and the motion vector predictor are calculated based on the prediction tap that is supplied from 52 and that is composed of predetermined data of the lower layer feature amount. The coefficient calculation unit 353 supplies the calculated coefficient to the prediction coefficient memory 355 and the calculated motion vector predictor to the memory 354.

The memory 354 stores the predicted motion vector supplied from the coefficient calculation unit 353, and supplies the stored predicted motion vector to the prediction tap extraction unit 352. That is, the prediction tap extraction unit 352 is supplied with the prediction motion vector corresponding to the immediately preceding field.

The prediction coefficient memory 355 is composed of the coefficient calculation unit 35.
The prediction coefficient supplied from No. 3 is stored, and the stored prediction coefficient is output as a coefficient set.

Next, with reference to the flowchart in FIG. 41, the learning process of the image processing apparatus for generating the prediction coefficient used in the process of calculating the lower hierarchy vector will be described.

In step S301, the coefficient generator 3
03 obtains the phase of the subcarrier of the data of interest based on the subcarrier phase information supplied from the NTSC decoder 1 via the NTSC decoder 1.

At step S302, the coefficient generator 3
The feature amount conversion unit 23 of 03 calculates the feature amount of the upper layer and outputs the calculated feature amount as the upper layer feature amount.

At step S303, the coefficient generator 3
The upper layer motion vector detection unit 25 of 03 detects the upper layer vector based on the upper layer feature amount supplied from the feature amount conversion unit 23. The details of the process of step S303 are the same as the processes described with reference to the flowchart of FIG. 28, and thus description thereof will be omitted.

In step S304, the coefficient generator 3
The feature quantity conversion unit 24 of 03 calculates the feature quantity of the lower hierarchy and outputs the calculated feature quantity as the lower hierarchy feature quantity.

At step S305, the learning section 321.
Is the motion vector, the lower layer feature amount supplied from the feature amount conversion unit 24, and the upper layer motion vector detection unit 25.
A coefficient set is calculated based on the upper layer vector supplied from the above, and the process ends.

FIG. 42 is a flow chart for explaining the details of the coefficient set calculation process by the learning unit 321 corresponding to the process of step S305.

In step S321, the class classification unit 351 classifies the class corresponding to the data of interest based on the subcarrier phase information and the lower layer feature amount.

In step S322, the prediction tap extracting unit 352 determines that a predetermined number of classified classes corresponding to the predicted motion vector corresponding to the previous field and the upper layer vector supplied from the memory 354 are stored.
A prediction tap that is a predetermined lower layer feature amount is extracted.

In step S323, the coefficient calculation unit 3
53 is a prediction tap supplied from the prediction tap extraction unit 352, and a motion vector supplied from the outside,
Calculate the coefficient. The coefficient calculation unit 353 supplies the calculated coefficient to the prediction coefficient memory 355.

In step S324, the prediction coefficient memory 355 stores the coefficient, and the process ends.

As described above, the image processing device having the configuration shown in FIG. 38 can generate the prediction coefficient used in the process of calculating the lower hierarchy vector.

In the above description, the image signal which is digital data has been described as a signal corresponding to the Y signal, the I signal or the Q signal, but it may be a signal including the Y signal, the I signal or the Q signal. Without limitation, Y signal, U signal,
And other types of image signals such as V signal, Y signal, Pb signal, and Pr signal, or Y signal, Cb signal, and Cr signal.

Also, although the component video signals generated are described as being Y signals, U signals, and V signals, they are not limited to Y signals, U signals, and V signals, and Y signals,
Pb signal and Pr signal, or Y signal, Cb signal,
It may be an image signal of another system such as a Cr signal and a Cr signal.

The image processing apparatus according to the present invention is an NTSC
Based on the image signal which is the composite video signal of the system,
I explained that it generates a component video signal, but NT
The component video signal may be generated based on a composite video signal of another system such as the PAL (Phase Alternation by Line) system as well as the SC system.

Although it has been described that the image processing apparatus according to the present invention detects the motion vector of the attention field of interest and the field next to the attention field, the attention field of attention and the attention field of The motion vector with respect to the previous field may be detected.

Although it has been described that the image processing apparatus according to the present invention detects a motion vector in units of fields, it may be possible to detect motion vectors in units of frames.

The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processes is executed by software, a program that constitutes the software can execute various functions by installing a computer in which dedicated hardware is installed or various programs. It is installed from a recording medium into a possible personal computer such as a general-purpose computer.

FIG. 43 is a diagram illustrating an example of a recording medium and a computer. CPU (Central Processing Uni
t) 501 is various application programs and OS
(Operating System) is actually executed. ROM (Read-onl
The y Memory) 502 generally stores basically fixed data of a program used by the CPU 501 and parameters for calculation. RAM (Random-Access Memory)
503 stores a program used in the execution of the CPU 501 and parameters that change appropriately in the execution. These are connected to each other by a host bus 504 including a CPU bus and the like.

The host bus 504 is connected to the PCI (Peripheral Component Interconnect / Interf) via the bridge 505.
ace) bus or other external bus 506.

The keyboard 508 is operated by the user when inputting various commands to the CPU 501. The mouse 509 is operated by the user when pointing or selecting points on the screen of the display 510. The display 510 is a liquid crystal display device or a CRT (Cathode Ray).
Tube), etc., and displays various information as text and images. An HDD (Hard Disk Drive) 511 drives hard disks and causes them to record or reproduce programs and information executed by the CPU 501.

The drive 512 is mounted on the magnetic disk 551, the optical disk 552, and the magneto-optical disk 55.
3 or the data or program recorded in the semiconductor memory 554 is read, and the data or program is read by the interface 507, the external bus 506,
The RAM 503 connected via the bridge 505 and the host bus 504 is supplied.

These keyboard 508 to drive 5
12 is connected to the interface 507, and the interface 507 includes the external bus 506 and the bridge 5.
05, and the host bus 504 to the CPU 501.

The video interface 513 acquires the supplied image signal, and the external bus 506 and bridge 50
5, and the acquired image signal is supplied to the RAM 503 or the CPU 501 via the host bus 504. The video interface 513 outputs the predicted component video signal.

As shown in FIG. 43, the recording medium is a magnetic disk 551 (on which a program is recorded, which is distributed in order to provide a user with a program for executing the processing corresponding to the block diagram, separately from the computer. Floppy disk (including registered trademark), optical disk 552 (CD
-ROM (Compact Disc-Read Only Memory), DVD (Digita
(including Versatile Disc)), magneto-optical disc 553
In addition to being configured by a removable medium (including MD (Mini-Disc) (trademark)) or a semiconductor memory 554, a program provided to a user in a state where the computer is pre-installed is recorded. The ROM 502 and the HDD 511 are included.

The program for executing the process corresponding to the block diagram for the user may be supplied to the computer via a wired or wireless communication medium.

Further, in the present specification, the steps for writing the program stored in the recording medium are not limited to the processing performed in time series according to the described order, but are not necessarily performed in time series. It also includes processing executed in parallel or individually.

[0362]

According to the first image processing apparatus and method, the recording medium, and the program of the present invention, the feature amount obtained from the point of interest and the composite video signal around the point of interest and the phase information of the composite video signal are obtained. Based on at least one of the composite video signal, the feature amount obtained from the attention point and the composite video signal around the attention point, the phase information of the composite video signal, and the detected motion vector. , The composite video signal of interest is classified into one of a plurality of classes, a prediction tap corresponding to the classified class and including at least one of the composite video signal and the feature amount is extracted, and the prediction coefficient and Based on the extracted prediction taps, the composite video signal of the point of interest is supported. Since such a component video signal is generated, taking into account the motion vector,
It becomes possible to generate a component video signal from a composite video signal with higher accuracy.

A second image processing apparatus and method of the present invention;
According to the recording medium and the program, based on the feature amount obtained from the attention point and the composite video signal around the attention point, and the phase information of the composite video signal,
A motion vector is detected, and based on at least one of the composite video signal, the feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected motion vector, A point composite video signal is classified into one of a plurality of classes, a prediction tap corresponding to the classified class and including at least one of the composite video signal and a feature is extracted, a component video signal, and an extraction Since the coefficient is calculated based on the predicted tap, the coefficient can be used to generate the component video signal from the composite video signal more accurately in consideration of the motion vector. .

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention.

FIG. 2 is a diagram illustrating image signal and subcarrier phase information.

FIG. 3 is a block diagram showing a configuration of a motion vector detection unit 2.

FIG. 4 is a block diagram showing a configuration of a higher layer motion vector detection unit 25.

5 is a block diagram showing a configuration of a lower layer motion vector detection unit 26. FIG.

FIG. 6 is a diagram showing an example of an upper layer characteristic amount, a lower layer characteristic amount, and a search area.

[Fig. 7] Fig. 7 is a diagram illustrating a block of a higher-level feature amount.

FIG. 8 is a diagram illustrating calculation of a lower layer feature amount.

FIG. 9 is a diagram showing an example of areas and blocks searched by a lower hierarchical motion vector detecting unit 26.

FIG. 10 is a diagram showing an example of another block of a higher layer feature amount.

FIG. 11 is a diagram showing another example of upper layer feature values.

FIG. 12 is a diagram showing another example of upper layer feature values.

FIG. 13 is a diagram showing another example of lower layer feature amounts.

FIG. 14 is a diagram showing another example of lower layer feature amounts.

FIG. 15 is a diagram showing another example of lower layer feature amounts.

FIG. 16 is a diagram showing another example of lower layer feature amounts.

FIG. 17 is a diagram showing another example of lower layer feature amounts.

FIG. 18 is a block diagram showing another configuration of the lower layer motion vector detection unit 26.

FIG. 19 is a block diagram showing a configuration of a calculation unit 4.

FIG. 20 is a diagram illustrating an example of a class tap and a prediction tap corresponding to a higher layer feature amount.

FIG. 21 is a diagram illustrating an example of class taps and prediction taps corresponding to lower layer feature amounts.

FIG. 22 is a block diagram showing another configuration of the arithmetic unit 4.

FIG. 23 is a block diagram showing another configuration of the arithmetic unit 4.

FIG. 24 is a block diagram showing another configuration of the arithmetic unit 4.

FIG. 25 is a block diagram showing details of an example of the configuration of the embodiment of the image processing apparatus according to the present invention.

FIG. 26 is a flowchart illustrating a process of generating a component video signal.

FIG. 27 is a flowchart illustrating a process of detecting a motion vector.

[Fig. 28] Fig. 28 is a flowchart illustrating details of processing for detecting an upper layer vector.

[Fig. 29] Fig. 29 is a flowchart illustrating details of a process of detecting a lower layer vector.

FIG. 30 is a flowchart illustrating details of another process of detecting a lower layer vector.

FIG. 31 is a block diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention that generates prediction coefficients.

FIG. 32 is a block diagram showing the configuration of a coefficient calculation unit 202.

FIG. 33 is a block diagram showing another configuration of the coefficient calculation unit 202.

FIG. 34 is a block diagram showing another configuration of the coefficient calculation unit 202.

FIG. 35 is a block diagram showing another configuration of the coefficient calculation unit 202.

FIG. 36 is a block diagram illustrating details of an example of a configuration of an embodiment of an image processing apparatus that calculates a prediction coefficient.

FIG. 37 is a flowchart illustrating a learning process for calculating a prediction coefficient.

[Fig. 38] Fig. 38 is a block diagram illustrating a configuration of an image processing device that generates a prediction coefficient used in a process of calculating a lower layer vector.

FIG. 39 is a block diagram showing a configuration of a coefficient generation unit 303.

FIG. 40 is a block diagram showing a configuration of a learning unit 321.

FIG. 41 is a flowchart illustrating a learning process of the image processing device.

FIG. 42 is a flowchart illustrating the details of the coefficient set calculation process.

FIG. 43 is a diagram illustrating an example of a recording medium and a computer.

[Explanation of symbols]

1 NTSC decoder, 2 motion vector detector, 3
Class tap extraction unit, 4 calculation unit, 23 feature amount conversion unit, 24 feature amount conversion unit, 25 upper layer motion vector detection unit, 26 lower layer motion vector detection unit,
31 vector generation unit, 32 correlation value calculation unit, 33
Determination unit, 41 vector generation unit, 42 correlation value calculation unit, 43 determination unit, 51 class classification unit, 52
Prediction tap extraction unit, 53 calculation unit, 54 prediction coefficient memory, 55 memory, 71-1 to 71-3 class classification unit, 72-1 to 72-3 prediction tap extraction unit, 73-1 to 73-3 coefficient memory, 74-1
To 74-3 prediction calculation unit, 81 selector, 82
Prediction tap extraction unit, 83 prediction calculation unit, 91 class classification unit, 101 selector, 201 NTSC encoder, 202 coefficient calculation unit, 203 coefficient memory,
211-1 to 211-3 Prediction coefficient calculator, 23
1 selector, 232 prediction coefficient calculation unit, 303
Coefficient generation unit, 321 learning unit, 351 class classification unit, 352 prediction tap extraction unit, 353 coefficient calculation unit, 354 memory, 355 prediction coefficient memory, 5
01 CPU, 502 ROM, 503 RAM,
511 HDD, 551 magnetic disk, 552
Optical disk, 553 Magneto-optical disk, 554 Semiconductor memory

Continued front page    F-term (reference) 5B056 BB23 BB33 BB36 BB71 HH03                 5C057 AA06 AA13 DB06 EA02 EA03                       EA05 EA06 EA07 ED07 EG08                       EG10 GC04                 5C066 AA03 AA13 CA09 DB07 DC01                       DD01 GA02 GA04 GA05 KC05                       KD06 KG02

Claims (26)

[Claims] 1. An image processing device for generating a component video signal based on a composite video signal, comprising: a first characteristic amount obtained from a point of interest and the composite video signal around the point of interest; Detecting means for detecting a motion vector based on phase information; the composite video signal, the first feature amount obtained from the attention point and the composite video signal around the attention point, and the phase of the composite video signal Class classification means for classifying the composite video signal at the point of interest into one of a plurality of classes based on information and at least one of the detected motion vectors; Correspondingly, the composite video signal and the first feature amount are less And extraction means for extracting a prediction tap including one of the two, and generation means for generating the component video signal corresponding to the composite video signal at the point of interest based on a prediction coefficient and the extracted prediction tap. An image processing device characterized by: 2. The extracting means further extracts the prediction tap based on a dynamic range of the composite video signal or a dynamic range of the first feature amount corresponding to the composite video signal. The image processing apparatus according to claim 1. 3. The class classification means individually assigns the composite video signal of the attention point to one of a plurality of classes corresponding to a luminance signal or a chrominance signal forming the component video signal. The image processing apparatus according to claim 1, wherein the image processing apparatus is classified. 4. The extracting means corresponds to a luminance signal or a chrominance signal forming the component video signal,
The image processing apparatus according to claim 1, wherein the prediction taps are individually extracted. 5. The first detecting means generates a second characteristic amount corresponding to the phase of the color signal at the point of interest based on the composite video signal.
Feature amount generating means, first vector detecting means for detecting a first vector that is approximate to the motion vector based on the second feature amount, and an object that is a calculation target of the motion vector. A second feature amount that generates a third feature amount corresponding to the target point based on the point and the composite video signal around the target point
And a second vector detecting unit for detecting the motion vector as a second vector by calculating the third characteristic amount corresponding to the first vector. The image processing apparatus according to claim 1. 6. The first vector detecting means includes the second feature amount of a target screen which is a screen of a moving image to which the target point belongs, and an adjacency which is a screen of the moving image adjacent to the target screen. The image processing apparatus according to claim 5, wherein the first vector that is approximate to the motion vector is detected based on the correlation with the second feature amount on the screen. 7. The second vector detecting means, in the attention screen which is a screen of a moving image to which the attention point belongs, the third feature amount in the first range including the attention point, and the attention screen. By calculating the correlation with the third feature amount of the second range corresponding to the first vector in the adjacent screen that is the screen of the moving image adjacent to the motion vector, the motion vector is set as the second vector. The image processing apparatus according to claim 5, wherein the image processing apparatus detects the image. 8. The method according to claim 5, wherein the second vector detecting means applies a class classification adaptive process to the third feature quantity to detect the motion vector as the second vector. The image processing device described. 9. The method according to claim 8, wherein the second vector detecting means classifies the composite video signal of the attention point based on the phase of the color signal at the attention point. Image processing device. 10. The second vector detecting means applies adaptive processing to the third feature quantity corresponding to the second vector detected in the past.
The image processing device according to item 1. 11. An image processing method for generating a component video signal based on a composite video signal, comprising: a feature amount obtained from the attention point and the composite video signal around the attention point; and phase information of the composite video signal. A detection step of detecting a motion vector, the composite video signal, the feature amount obtained from the point of interest and the composite video signal around the point of interest, the phase information of the composite video signal, and the detected A class classification step of classifying the composite video signal of the point of interest into one of a plurality of classes based on at least one of the motion vectors; and the composite video corresponding to the classified class. At least one of a signal and the feature amount And a generating step of generating the component video signal corresponding to the composite video signal of the point of interest based on a prediction coefficient and the extracted prediction tap. Image processing method. 12. A program for image processing for generating a component video signal based on a composite video signal, the feature amount being obtained from a point of interest and the composite video signal around the point of interest, and the composite video signal. Based on the phase information of the detection step of detecting a motion vector, the composite video signal, the feature point obtained from the composite video signal around the point of interest and the point of interest, the phase information of the composite video signal, And a classifying step of classifying the composite video signal of the point of interest into one of a plurality of classes based on at least one of the detected motion vectors; , The composite video signal and the feature amount An extracting step of extracting a prediction tap including at least one; and a generating step of generating the component video signal corresponding to the composite video signal of the attention point based on a prediction coefficient and the extracted prediction tap A recording medium having a computer-readable program recorded thereon. 13. A computer for controlling an image processing device which generates a component video signal based on a composite video signal, a feature amount obtained from the attention point and the composite video signal around the attention point, and the composite video signal. Based on the phase information of the detection step of detecting a motion vector, the composite video signal, the feature point obtained from the composite video signal around the point of interest and the point of interest, the phase information of the composite video signal, And a classifying step of classifying the composite video signal of the point of interest into one of a plurality of classes based on at least one of the detected motion vectors; , The composite video signal and the characteristic amount An extraction step of extracting a prediction tap including at least one of the following, and a generation step of generating the component video signal corresponding to the composite video signal of the attention point based on the prediction coefficient and the extracted prediction tap. Program to let. 14. An image processing apparatus for generating a coefficient used in a process of predicting a component video signal based on a composite video signal, the first point being obtained from a point of interest and the composite video signal around the point of interest. A detection unit that detects a motion vector based on a feature amount and phase information of the composite video signal, and the first feature obtained from the composite video signal, the point of interest, and the composite video signal around the point of interest. A class that classifies the composite video signal of the point of interest into one of a plurality of classes based on at least one of a quantity, the phase information of the composite video signal, and the detected motion vector. A classification means and a composite video signal corresponding to the classified class. No. and a prediction tap including at least one of the first feature quantity, and a calculation means for calculating the coefficient based on the component video signal and the extracted prediction tap. A characteristic image processing device. 15. The extraction means further extracts the prediction tap based on a dynamic range of the composite video signal or a dynamic range of the first feature amount corresponding to the composite video signal. The image processing device according to claim 14. 16. The class classification means individually sets the composite video signal of the attention point to one of a plurality of classes corresponding to a luminance signal or a chrominance signal forming the component video signal. The image processing device according to claim 14, wherein the image processing device is classified. 17. The image processing apparatus according to claim 14, wherein the extraction unit individually extracts the prediction taps corresponding to a luminance signal or a chrominance signal forming the component video signal. 18. The first detecting means generates a second characteristic amount corresponding to the phase of the color signal at the point of interest based on the composite video signal.
Feature amount generating means, first vector detecting means for detecting a first vector approximate to the motion vector based on the second feature amount, and an object to be a target of calculation of the motion vector detection. A second feature amount that generates a third feature amount corresponding to the target point based on the point and the composite video signal around the target point
And a second vector detecting unit for detecting the motion vector as a second vector by calculating the third characteristic amount corresponding to the first vector. The image processing apparatus according to claim 14. 19. The second vector of the attention screen, which is a screen of a moving image to which the attention point belongs,
Detecting the first vector that is approximate to the motion vector on the basis of the correlation between the feature amount and the second feature amount of the adjacent screen that is the screen of the moving image adjacent to the target screen. The image processing apparatus according to claim 18, which is characterized in that: 20. The second vector detecting means, in a target screen which is a screen of a moving image to which the target point belongs,
The third characteristic amount in the first range including the attention point and the second range in the second range corresponding to the first vector in the adjacent screen which is the screen of the moving image adjacent to the attention screen. The image processing apparatus according to claim 18, wherein the motion vector is detected as the second vector by calculating a correlation with the feature amount of 3. 21. The second vector detection means applies class classification adaptation processing to the third feature quantity,
19. The image processing apparatus according to claim 18, wherein the motion vector is detected as the vector of. 22. The second vector detection means classifies the composite video signal at the point of interest based on the phase of the color signal at the point of interest, according to claim 21. Image processing device. 23. The second vector detecting means applies an adaptive process to the third feature quantity corresponding to the second vector detected in the past.
1. The image processing device according to 1. 24. In an image processing method for generating a coefficient used in a process of predicting a component video signal based on a composite video signal, a feature amount obtained from an attention point and the composite video signal around the attention point, And a detection step of detecting a motion vector based on phase information of the composite video signal, the composite video signal, the point of interest, and the feature amount obtained from the composite video signal around the point of interest, the composite video signal A classifying step of classifying the composite video signal of the attention point into one of a plurality of classes based on at least one of the phase information and the detected motion vector of It corresponds to the class And an extraction step of extracting a prediction tap including at least one of the feature amount, and an operation step of operating the coefficient based on the component video signal and the extracted prediction tap. Processing method. 25. A program for image processing, which generates a coefficient used in a process of predicting a component video signal based on a composite video signal, comprising: a point of interest and the composite video signal around the point of interest. A feature amount obtained, and a detection step of detecting a motion vector based on phase information of the composite video signal, the composite video signal, the feature point and the feature amount obtained from the composite video signal around the focus point, A class classification step of classifying the composite video signal of the attention point into one of a plurality of classes based on at least one of the phase information of the composite video signal and the detected motion vector. Corresponding to the classified class, An extraction step of extracting a prediction tap including at least one of a video signal and the feature amount; and a calculation step of calculating the coefficient based on the component video signal and the extracted prediction tap. A recording medium on which a computer-readable program is recorded. 26. A computer controlling an image processing device for generating a coefficient used in a process of predicting a component video signal based on a composite video signal, the method comprising: A feature amount obtained, and a detection step of detecting a motion vector based on phase information of the composite video signal, the composite video signal, the feature point and the feature amount obtained from the composite video signal around the focus point, A class classification step of classifying the composite video signal of the attention point into one of a plurality of classes based on at least one of the phase information of the composite video signal and the detected motion vector. Corresponding to the classified classes, And a calculation step of calculating the coefficient on the basis of the component video signal and the extracted prediction tap.