JP2009272713A - Image processor, image processing method, coefficient data generator and coefficient data generation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To steeply erect the edge of an image and to reduce ringing. <P>SOLUTION: From an interpolated curved surface generated by an interpolated curved surface generation circuit 1, a feature amount indicating the phase of the edge in the image signals of a high resolution and a feature amount indicating the direction of the edge are calculated. From the calculated feature amounts indicating the phase and direction of the edge, a class to which the pixel data of a position under consideration belongs is detected. Thus, when generating the image signals of the high resolution, the edge is steeply erected and ringing is reduced. Especially, during high density (zoom), the image signals of the high resolution for which the edge is more steeply erected than before are obtained. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, a coefficient data generation apparatus, and a coefficient data generation method that are suitable for use in converting a standard resolution image into a high resolution image. Specifically, a feature amount indicating the phase of the edge in the high-resolution image signal is calculated from the interpolation curved surface generated by the interpolation curved surface generation unit, and the pixel data of the target position is calculated from the feature amount indicating the calculated phase of the edge. By detecting the class to which the image belongs, when generating a high-resolution image signal, the edge can be made to stand sharply and the ringing can be reduced.

  Conventionally, processing of converting a standard resolution or low resolution image (hereinafter also referred to as an SD image) into a high resolution image (hereinafter also referred to as an HD image) or enlarging the image has been performed. In this case, the interpolation (compensation) of the pixel values of the missing pixels is performed by a so-called interpolation filter or the like.

  However, even if pixel interpolation is performed using an interpolation filter, it is difficult to obtain a high-resolution image because the HD image component (high-frequency component) that is not included in the SD image cannot be restored.

  An image processing apparatus that has solved this problem performs high-frequency components that are not included in the SD image by performing an adaptive process for obtaining a predicted value of a pixel of the HD image by linear combination of the SD image and predetermined coefficient data. Restore.

  For example, the predicted value E [y] of the pixel value y of a pixel constituting the HD image (hereinafter also referred to as HD pixel) is set to the pixel value (hereinafter referred to as learning data) of several SD pixels (pixels constituting the SD image). (Also referred to as) x1, x2,..., Xi and predetermined coefficient data w1, w2,. In this case, the predicted value E [y] can be expressed by the following formula (1).

  Therefore, in order to generalize, a matrix W composed of a set of coefficient data w, a matrix X composed of a set of learning data, and a matrix Y ′ composed of a set of predicted values E [y] are expressed by the following equation (2). Define.

  As a result, an observation equation such as the following equation (3) is established.

  Then, it is considered that the least square method is applied to the observation equation shown in Expression (3) to obtain a predicted value E [y] close to the pixel value y of the HD pixel. In this case, a matrix Y composed of a set of true pixel values y of HD pixels serving as teacher data and a matrix E composed of a set of residuals e of predicted values E [y] with respect to the pixel values y of HD pixels are expressed by the following equations: It is defined in (4).

  Thereby, the residual equation shown in the following formula (5) is established from the formula (3).

  In this case, the coefficient data wi for obtaining the predicted value E [y] close to the pixel value y of the HD pixel can be obtained, for example, by minimizing the square error shown in the following equation (6).

  Accordingly, when the square error of the equation (6) is differentiated by the coefficient data wi is 0, that is, the coefficient data wi satisfying the following equation (7) is the predicted value E [ This is an optimum value for obtaining y].

  Therefore, first, the following equation (8) is established by differentiating the equation (7) with the coefficient data wi.

  From the equations (7) and (8), the following equation (9) is obtained.

  Further, considering the relationship between the learning data x, the coefficient data w, the teacher data y, and the residual e in the residual equation of the equation (5), the normal equation shown in the following equation (10) is obtained from the equation (9). Obtainable.

  The normal equation of equation (10) can be formed by the same number as the number of coefficient data w to be obtained. Therefore, by solving equation (10) (however, to solve equation (10), equation (10) 10), the matrix composed of the coefficients related to the coefficient data w needs to be regular), and the optimum coefficient data w can be obtained. In solving the equation (10), for example, a sweeping method (Gauss-Jordan elimination method) or the like can be applied.

  As described above, the optimum coefficient data w is obtained, and further, the adaptive data is obtained by using the coefficient data w to obtain the predicted value E [y] close to the pixel value y of the HD pixel using Equation 1. is there. Note that the adaptive processing is different from the interpolation processing in that a component included in the HD image that is not included in the SD image is reproduced. That is, the adaptive process is the same as the interpolation process using a so-called interpolation filter as long as only Expression (1) is seen, but the coefficient data w corresponding to the tap coefficient of the interpolation filter uses the teacher data y. In other words, since it is obtained by learning, the components included in the HD image can be reproduced. That is, a high-resolution image can be easily obtained. From this, it can be said that the adaptive process is a process having an effect of creating an image (resolution).

  FIG. 18 is a block diagram illustrating a configuration example of the image processing apparatus 400 that converts the SD image data Din into the HD image data Dout by the adaptive processing as described above. The SD image data Din is supplied as an input to the prediction tap selection circuit 6 and the class tap selection circuit 10.

  The class tap selection circuit 10 selects an SD pixel (hereinafter also referred to as a class tap) corresponding to an HD pixel (hereinafter also referred to as a target pixel) for which a predicted value is to be obtained by adaptive processing. The class detection circuit 11 detects a level pattern (pixel value distribution) of pixel values of SD pixels constituting the class tap. For example, the class detection circuit 11 obtains the feature amount of the SD pixel constituting the class tap using ADRC (Adaptive Dynamic Range Coding) and classifies the feature amount into a predetermined class. The class code generation circuit 4C calculates a value assigned in advance to the level pattern as the class code of the target pixel and outputs it to the coefficient memory 5C.

  The coefficient memory 5C outputs coefficient data to the estimated prediction calculation circuit 7 based on this class code. That is, the coefficient memory 5C stores coefficient data obtained by learning in advance for each class. When a class code is supplied, the coefficient data corresponding to the class code is read, and an estimated prediction calculation circuit 7 is supplied.

  As a result, the prediction prediction calculation circuit 7 is supplied with the prediction tap corresponding to the target pixel from the prediction tap selection circuit 6 and the coefficient data for the class of the target pixel. Then, the estimated prediction calculation circuit 7 uses the coefficient data w1, w2,..., Wi and the SD pixels x1, x2,. The calculation shown in (1) is performed. Thereby, the predicted value E [y] of the pixel value y of the target pixel (HD pixel) is obtained, and this is output as the pixel value of the HD pixel.

  In relation to such a conventional example, Patent Document 1 discloses an image signal conversion device that converts SD pixels into HD pixels. According to this image signal conversion apparatus, the class is determined based on the space class obtained by ADRC and the motion class obtained from the inter-frame difference value.

JP 2001-238185 A

  By the way, the class detection circuit according to the conventional example and Patent Document 1 obtains the feature amount of the SD pixel constituting the class tap using ADRC and classifies the feature amount into a predetermined class. As described above, since the feature amount with pixel accuracy (pixel unit) is used, there is a problem that the waveform of the edge does not appear when performing high-density (zoom) conversion. There is also a problem that ringing occurs.

  Therefore, the present invention solves such a problem, and an image processing apparatus, an image processing method, and coefficient data generation capable of reducing the ringing while allowing the edges of the image to stand sharply. An object is to provide an apparatus and a coefficient data generation method.

  In order to solve the above-described problem, an image processing apparatus according to the present invention includes a plurality of image processing devices that are located around a target position in a second image signal having a higher resolution than the first image signal. A feature amount indicating the phase of the edge in the second image signal is calculated from an interpolation curved surface generation unit that selects the first pixel data and generates an interpolation curved surface, and the interpolation curved surface generated by the interpolation curved surface generation unit A first class detection unit that detects a class to which the pixel data of the target position belongs, from the feature quantity indicating the phase of the edge calculated by the feature quantity calculation unit, and the first class A coefficient data selection unit that selects coefficient data with reference to the class detected by the detection unit; and a plurality of second pixels located around the target position in the second image signal from the first image signal Day Calculating a first pixel data selection unit that selects the plurality of second pixel data selected by the first pixel data selection unit and the coefficient data selected by the coefficient data selection unit, An estimation calculation unit for obtaining pixel data of the target position.

  According to the image processing device of the present invention, the interpolation curved surface generation unit includes a plurality of positions located around the position of interest in the second image signal having a higher resolution than the first image signal from the first image signal. The first pixel data is selected to generate an interpolation curved surface. The feature amount calculation unit calculates a feature amount indicating the phase of the edge in the second image signal from the interpolation curved surface. The first class detection unit detects the class to which the pixel data of the target position belongs from the feature quantity indicating the phase of the edge. The coefficient data selection unit selects coefficient data with reference to the detected class. The estimation calculation unit calculates a plurality of second pixel data and coefficient data to obtain pixel data of the target position.

  In this example, the feature amount calculation unit described above calculates a feature amount indicating the direction of the edge in the second image signal from the interpolation curved surface. The first class detection unit detects a class to which the pixel data of the target position belongs, from the feature quantity indicating the edge phase and the feature quantity indicating the edge direction. As a result, when a high-resolution image signal is generated, edges can be sharply raised and ringing can be reduced.

  In order to solve the above-described problem, an image processing method according to the present invention includes a plurality of images that are located around a target position in a second image signal having a higher resolution than the first image signal. A feature amount indicating the phase of an edge in the second image signal is calculated from a first step of generating an interpolation curved surface by selecting the first pixel data and an interpolation curved surface generated in the first step. A second step of detecting, a third step of detecting a class to which the pixel data of the target position belongs from the feature amount indicating the phase of the edge calculated in the second step, and detecting in the third step A fourth step of selecting coefficient data with reference to the determined class, and selecting a plurality of second pixel data located around the position of interest in the second image signal from the first image signal First And a sixth step of calculating the plurality of second pixel data selected in the fifth step and the coefficient data selected in the fourth step to obtain the pixel data of the target position It has.

  In order to solve the above-described problem, the coefficient data generation device according to the present invention includes a student corresponding to the first image signal from a teacher signal corresponding to the second image signal having a higher resolution than the first image signal. A student signal generation unit that generates a signal, and a plurality of first pixel data located around a target position in the teacher signal are selected from the student signal generated by the student signal generation unit to generate an interpolation curved surface An interpolated curved surface generation unit, a feature amount calculating unit that calculates a feature amount indicating a phase of an edge in the teacher signal from the interpolated curved surface generated by the interpolated curved surface generation unit, and an edge calculated by the feature amount calculating unit A first class detection unit that detects a class to which the pixel data of the target position belongs from a feature amount indicating a phase of the target position, and the teacher signal from the student signal generated by the student signal generation unit. A first pixel data selection unit that selects a plurality of second pixel data located around the target position in FIG. 1, a class detected by the first class detection unit, and a selection by the first pixel data selection unit Coefficient data used when converting the first image signal into the second image signal by calculation using the plurality of second pixel data and the pixel data of the target position in the teacher signal for each class And an arithmetic unit to be obtained.

  According to the coefficient data generation apparatus of the present invention, the interpolation curved surface generation unit generates a interpolation curved surface by selecting a plurality of first pixel data located around the target position in the teacher signal from the student signal. The feature amount calculation unit calculates a feature amount indicating the phase of the edge in the teacher signal from the interpolation curved surface. The first class detection unit detects the class to which the pixel data of the target position belongs from the feature quantity indicating the phase of the edge. The calculation unit performs calculation using the class detected by the first class detection unit, the plurality of second pixel data selected by the first pixel data selection unit, and the pixel data of the target position in the teacher signal. Coefficient data used when converting one image signal into a second image signal is obtained for each class.

  In this example, the feature amount calculation unit described above calculates a feature amount indicating the direction of the edge in the teacher signal from the interpolation curved surface. The first class detection unit detects a class to which the pixel data of the target position belongs, from the feature quantity indicating the edge phase and the feature quantity indicating the edge direction. As a result, when generating a high-resolution image signal, it is possible to provide coefficient data that can make edges sharply and reduce ringing.

  According to the image processing device and the image processing method of the present invention, the feature amount indicating the phase of the edge in the high-resolution image signal is calculated from the interpolation curved surface generated by the interpolation curved surface generation unit, and the calculated edge The class to which the pixel data of the target position belongs is detected from the feature quantity indicating the phase.

  With this configuration, when a high-resolution image signal is generated, an edge can be made to be steep and ringing can be reduced. In particular, at the time of high-density (zoom), it is possible to obtain a high-resolution image signal having edges sharper than ever.

  Further, according to the coefficient data generation device and the coefficient data generation method according to the present invention, the feature amount indicating the phase of the edge in the high-resolution image signal is calculated from the interpolation curved surface generated by the interpolation curved surface generation unit, and the calculation is performed. The class to which the pixel data of the target position belongs is detected from the feature amount indicating the phase of the edge, and is used when converting the first image signal into the high-resolution second image signal for each class. Coefficient data is obtained.

  With this configuration, when generating a high-resolution image signal, it is possible to provide coefficient data that can make edges sharp and can reduce ringing.

  Next, a first embodiment of an image processing apparatus, an image processing method, a coefficient data generation apparatus, and a coefficient data generation method according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of an image processing apparatus 100 according to the first embodiment. The image processing apparatus 100 shown in FIG. 1 performs class classification adaptive processing using information finer than pixel accuracy, that is, information below a pixel, instead of class classification adaptive processing (for example, ADRC) using feature quantities based on pixel accuracy. It is. In the present invention, feature quantities specialized for edges are used. Specifically, the first and second differential values at the target pixel position obtained from the interpolation curved surface of the input image for the edge phase and the edge direction are classified into classes.

  An image processing apparatus 100 shown in FIG. 1 includes an interpolation curved surface generation circuit 1, a feature amount calculation determination circuit 2, a class detection circuit 3, a class code generation circuit 4A, a coefficient memory 5A, a prediction tap selection circuit 6, an estimated prediction calculation circuit 7, An input terminal 8 and an output terminal 9 are provided.

  The SD image data Din is input to the input terminal 8. The interpolated curved surface generating circuit 1 is an example of an interpolated curved surface generating unit, and selects a plurality of pixel data located around the target position in the HD image data Dout having higher resolution than the SD image data Din from the SD image data Din. To generate an interpolated curved surface. For example, the interpolation curved surface generation circuit 1 generates a spline interpolation curved surface as the interpolation curved surface. The interpolation curved surface may use cubic interpolation, Lagrange interpolation, Lanczos interpolation, or the like. However, this interpolated curved surface does not include a curved surface created by DRC (Digital Reality Creation) because performance is difficult to obtain unless it is smoothly connected. This is because the continuity of the class cannot be maintained unless it is connected smoothly. A method of generating the spline interpolation curved surface will be described in detail with reference to FIG. The interpolation curved surface generation circuit 1 outputs the generated spline interpolation curved surface information to the feature amount calculation determination circuit 2.

  The feature amount calculation determination circuit 2 is an example of a feature amount calculation unit, and calculates a feature amount indicating an edge phase and an edge direction in the HD image data Dout from a spline interpolation curved surface. FIG. 2 is a conceptual diagram illustrating an example of an edge phase and an edge direction. The feature amount calculation determination circuit 2 obtains a primary differential value and a secondary differential value from the spline interpolation curved surface at the target pixel position. Here, the primary differential value indicates a tangent vector on the spline interpolation curved surface, and the secondary differential value indicates a curvature on the spline interpolation curved surface. The primary differential value and the secondary differential value are used to determine the edge phase and the edge direction shown in FIG. The calculation method of the primary differential value and the secondary differential value will be described in detail with reference to FIG.

  The class detection circuit 3 is an example of a first class detection unit, and detects a class to which pixel data (a pixel of interest) of a target position belongs from a feature amount indicating an edge phase and a feature amount indicating an edge direction. For example, the class detection circuit 3 detects the class to which the pixel of interest belongs from the primary differential value and the secondary differential value of the spline interpolation curved surface. In this example, the class detection circuit 3 divides the class in the circumferential direction on the plane composed of the primary differential value and the secondary differential value of the spline interpolation curved surface, and the primary differential value and the secondary differential value of the spline interpolation curved surface. The value is developed on the plane to obtain the edge phase class. The class detection circuit 3 divides the class in the circumferential direction on a plane composed of the first derivative value of x and the first derivative value of y of the spline interpolated curved surface, and the first derivative of x and y of the spline interpolated curved surface. The value is developed on the plane to obtain the edge direction class. The class detection circuit 3 outputs the obtained edge phase class and edge direction class (differential value class) to the class code generation circuit 4A.

  The class code generation circuit 4A generates a class code based on the class to which the target pixel belongs and outputs it to the coefficient memory 5A. For example, the class code generation circuit 4A outputs 3-bit phase class data and 4-bit direction class data as class codes. The coefficient memory 5 </ b> A is an example of a coefficient data selection unit, selects coefficient data with reference to this class code, and outputs the selected coefficient data to the estimated prediction calculation circuit 7. That is, the coefficient memory 5A stores coefficient data obtained by learning in advance for each class. When a class code is supplied, the coefficient data corresponding to the class code is read and an estimated prediction calculation circuit 7 is supplied.

  The prediction tap selection circuit 6 is an example of a first pixel data selection unit, and selects a prediction tap corresponding to the target pixel from the SD image data Din and outputs the prediction tap to the estimated prediction calculation circuit 7. The estimated prediction calculation circuit 7 is an example of an estimation calculation unit, and uses the coefficient data and the SD pixels that form the prediction tap from the prediction tap selection circuit 6 to perform the calculation shown in Expression (1). Thereby, the predicted value E [y] of the pixel value y of the target pixel (HD pixel) is obtained, and this is output from the output terminal 9 as HD image data Dout composed of the pixel value of the HD pixel.

  As described above, the image processing apparatus 100 illustrated in FIG. 1 classifies the class of the pixel of interest using the feature values of the edge phase and the edge direction. Thereby, for example, when the SD pixel is converted to the HD pixel, the position of the edge portion of the HD pixel can be clearly grasped. As a result, when high-density conversion is performed, an edge waveform rises and ringing can be prevented from occurring.

FIG. 3 is a conceptual diagram illustrating an example of generating a spline interpolation curved surface. As the spline interpolation curved surface, a B-spline curved surface is generated from, for example, 4 × 4 16 points (Q _{00 to} Q ₃₃ ) input pixels shown in FIG. At this time, the cubic expression (11) for generating between pixels is as follows. In Expression (11), P is a pixel value (target pixel) to be created, Q is an input pixel value, and t is a phase (0-1) to be created. Note that when the position of the pixel of interest P changes, the positions of the 16 input pixels Q _{00 to} Q ₃₃ shown in FIG. 3 also change.

  When the equation (11) is first-order differentiated with respect to x, the following equation (12) is obtained. Further, when the equation (11) is first-order differentiated with respect to y, the following equation (13) is obtained.

  Further, when the equation (11) is second-order differentiated with respect to x, the following equation (14) is obtained. Further, when the equation (11) is second-order differentiated with respect to y, the following equation (15) is obtained.

  Next, an example of calculating the edge phase class will be described. FIG. 4 is a conceptual diagram illustrating an example of calculating the edge phase class. A primary differential value and a secondary differential value are obtained at the position of the target pixel P1 on the spline interpolation curved surface obtained from the input image. For example, at the position of the target pixel P1 shown in FIG. 4, the first differential value of x is obtained by the above equation (12), and the first differential value of y is obtained by the above equation (13). Further, at the position of the target pixel P1, the secondary differential value of x is obtained by the above-described equation (14), and the secondary differential value of y is obtained by the above-described equation (15). Then, the obtained primary differential value and secondary differential value are developed on a plane composed of the primary differential value and the secondary differential value, and classes are divided in the circumferential direction on the plane.

  For example, the length d1 of the first differential value of x and y is obtained by the following equation (16). Further, the length d2 of the secondary differential value of x and y is obtained by the following equation (17).

  Since the lengths d1 and d2 are both positive, the inner product is obtained by the following equation (18) to obtain the sign sgn. For example, when the inner product obtained by Expression (18) is negative, −1 is set to the sign sgn. Further, when the inner product obtained by Expression (18) is positive, +1 is set to the code sgn.

For example, when the number of phase classes is 8, the following equation (19) is used. In this example, when the value of tan ⁻¹ (x) is π / 2, for example, and −π / 2, the class is treated as the same class. That is, the classes belonging to the areas Q1 and Q2 shown in FIG. 4 are treated as the same class.

When hardware implementation is considered, processing is performed by having a conversion table as shown in FIG. FIG. 5 is an explanatory diagram showing a configuration example of a tan ⁻¹ -class code conversion table. The conversion table shown in FIG. 5 shows the relationship between the value of tan (d2 × sgn / d1 in Expression (19)) and the class code. For example, when the value of tan is in the range of “0.0 to 0.41421,” the class code is “0”. When the tan value is in the range of “0.41421 to 1.0”, the class code is “1”. Similarly, class codes “2” to “7” are set.

  Next, an example of calculating the edge direction class will be described. FIG. 6 is a conceptual diagram illustrating an example of calculating an edge direction class. First-order differential values of x and y are obtained at the position of the target pixel P2 on the spline interpolation curved surface obtained from the input image. For example, at the position of the pixel of interest P2 shown in FIG. 6, the first differential value of x is obtained by the above equation (12), and the first differential value of y is obtained by the above equation (13). Then, the obtained first differential values of x and y are developed on a plane composed of the first differential value in the x direction and the first differential value in the y direction, and the class is divided in the circumferential direction on the plane. .

For example, when the number of classes in the direction is 16, it is obtained as in the following equation (20). In this example, when the value of tan ⁻¹ (x) is π / 2, for example, and −π / 2, the class is treated as the same class. That is, the classes belonging to the region Q3 and the region Q4 shown in FIG. 6 are treated as the same class.

When hardware implementation is considered, processing is performed by having a conversion table as shown in FIG. FIG. 7 is an explanatory diagram showing a configuration example of a tan ⁻¹ -class code conversion table. The conversion table shown in FIG. 7 shows the relationship between the value of tan (dy / dx in Expression (20)) and the class code. For example, when the value of tan is in the range of “0.0 to 0.19891”, the class code is “0”. When the tan value is in the range of “0.19891 to 0.41421,” the class code is “1”. Similarly, class codes “2” to “15” are set.

  Next, the effect of the edge direction class will be described in detail. 8A and 8B are schematic diagrams illustrating an example of an edge of a target (teacher) image. FIG. 8A shows the edge in the vertical direction and the level of the pixel value in the cross section. FIG. 8B shows the edge in the oblique direction and the level of the pixel value in the cross section. At this time, even if the directions of the edges shown in FIGS. 8A and 8B are different, the waveforms shown in FIGS. 8A and 8B when viewed in the horizontal direction are the same, and the levels of the pixel values of the edges are the same. .

  9A and 9B are schematic diagrams illustrating an example of an edge of an input (student) image. FIG. 9A shows the edge in the vertical direction and the level of the pixel value in the cross section. FIG. 9B shows the edge in the oblique direction and the level of the pixel value in the cross section. At this time, if the directions of the edges shown in FIGS. 9A and 9B are different, the waveforms shown in FIGS. 9A and 9B when viewed in the horizontal direction are different, and the pixel value levels of the edges are different. In this case, if the edge direction is different, the pixel data used in the prediction tap is different. That is, the pixel data of the tap used for prediction illustrated in FIG. 9A is three pieces of pixel data that belong between the black boundary and the white boundary. On the other hand, the pixel data of the tap used for prediction shown in FIG. 9B is five pieces of pixel data that belong between the black boundary and the white boundary.

  In this way, even if the edge phase is the same value, the tap used for prediction is always the same place (direction) regardless of the edge direction, so the tap differs depending on the edge direction. Therefore, it is necessary to classify in the edge direction. As taps used for prediction, for example, 15 taps shown in FIG. 10 are used. FIG. 10 is a schematic diagram illustrating a configuration example of a prediction tap.

  If the edge phase has the same value and the tap used for prediction is always in the same place (direction) with respect to the edge direction, for example, the arrow direction R in FIG. 9B, the tap is the same depending on the edge direction. Therefore, the edge direction class is not necessary.

  Subsequently, the functions of the image processing apparatus 400 according to the conventional example shown in FIG. 18 and the image processing apparatus 100 shown in FIG. 1 according to the present invention will be compared. FIG. 11 is a schematic diagram illustrating a configuration example of a prediction tap and a class tap. As the prediction tap and the class tap shown in FIG. 11, five taps are selected in the horizontal direction. In this comparative example, the number of pixels only in the horizontal direction is multiplied by 8 (horizontal 8 times processing).

  12A and 12B are explanatory diagrams illustrating an example of a result of HD conversion processing in which the number of pixels only in the horizontal direction is increased by eight times. 12A and 12B, the vertical axis represents the pixel value level, and the horizontal axis represents the coordinates. In this example, the differential value class uses only the phase class and does not use the direction class.

  FIG. 12A is an example in which a pixel is interpolated at a 1/8 position between input pixels. The input image data (two-dot chain line) shown in FIG. 12A has no waveform, and has a distorted waveform. On the other hand, the image data using the differential value class has a rectangular waveform with standing waveforms. At this time, the image data using the differential value class (solid line) is substantially equal to the true waveform shown by the broken line. Further, the image data (one-dot chain line) using the ADRC class has a waveform deviating from the true value waveform.

  FIG. 12B is an example in which a pixel is interpolated at a position of 8/8 between input pixels. The input image data (two-dot chain line) shown in FIG. 12B has no waveform and has a distorted waveform. On the other hand, the image data (solid line) using the differential value class has a waveform with a rectangular shape. At this time, the image data using the differential value class is substantially equal to the true waveform shown by the broken line. Further, the image data (one-dot chain line) using the ADRC class has a waveform deviating from the true value waveform.

  FIG. 13A is an explanatory diagram illustrating an example of qualitative evaluation of the image processing apparatus 400 according to the related art. FIG. 13B is an explanatory diagram showing an example of qualitative evaluation of the image processing apparatus 100 according to the present invention. 13A and 13B show an example in which SD data is converted into 8 × 8 times HD data. At this time, it can be seen that the ringing is reduced in the image using the differential value class of the image processing apparatus 100 as compared with the image using the ADRC class of the image processing apparatus 400.

  Subsequently, an operation example of the image processing apparatus 100 will be described with reference to FIGS. 1 and 14. FIG. 14 is a flowchart showing an operation example of the image processing apparatus 100 according to the present invention. In this example, mapping processing for obtaining an HD image from an SD image is performed using coefficient data obtained by learning. Further, the coefficient memory 5A shown in FIG. 1 stores coefficient data obtained by learning in advance for each class, and reads the coefficient data corresponding to the class code when the class code is supplied. The estimated prediction calculation circuit 7 is supplied.

  In step ST1 shown in FIG. 14, the image processing apparatus 100 inputs the SD image data Din from the input terminal 8 and proceeds to step ST2. In step ST2, the interpolation curved surface generation circuit 1 generates an interpolation curved surface from the SD image data Din. For example, the interpolation curved surface generation circuit 1 generates a spline interpolation curved surface as the interpolation curved surface. Subsequently, the process proceeds to step ST3.

  In step ST3, the feature amount calculation determination circuit 2 calculates the feature amount of the edge phase and the feature amount of the edge direction from the spline interpolation curved surface. For example, the feature amount calculation determination circuit 2 obtains a primary differential value indicating the tangent vector and a secondary differential value indicating the curvature from the spline interpolation curved surface at the target pixel position. The primary differential value and the secondary differential value are used to determine the edge phase and the edge direction. Subsequently, the process proceeds to step ST4.

  In step ST4, the class detection circuit 3 detects the class to which the pixel of interest belongs from the primary differential value and the secondary differential value of the spline interpolation curved surface. For example, the class detection circuit 3 divides the class in the circumferential direction on the plane composed of the primary differential value and the secondary differential value of the spline interpolation curved surface, and obtains the primary differential value and the secondary differential value of the spline interpolation curved surface. The edge phase class is obtained by expanding on the plane. The class detection circuit 3 divides the class in the circumferential direction on a plane composed of the first derivative value of x and the first derivative value of y of the spline interpolated curved surface, and the first derivative of x and y of the spline interpolated curved surface. The value is developed on the plane to obtain the edge direction class. Subsequently, the process proceeds to step ST5.

  In step ST5, the class code generation circuit 4A generates a class code based on the class to which the target pixel belongs and outputs it to the coefficient memory 5A. The coefficient memory 5A outputs coefficient data to the estimated prediction calculation circuit 7 based on this class code. Subsequently, the process proceeds to step ST6.

  In step ST6, the prediction tap selection circuit 6 selects a prediction tap (see FIG. 10) corresponding to the target pixel from the SD image data Din input from the input terminal 8, and outputs the prediction tap to the estimated prediction calculation circuit 7 to output the step ST7. Migrate to

  In step ST7, the estimated prediction calculation circuit 7 uses the coefficient data and the prediction tap from the prediction tap selection circuit 6 to perform the product-sum calculation shown in the above equation (1). Thereby, the predicted value E [y] of the pixel value y of the target pixel (HD pixel) is obtained, and this is output as the pixel value of the HD pixel. Subsequently, the process proceeds to step ST8.

  In step ST8, the image processing apparatus 100 determines whether or not the SD image data Din input from the input terminal 8 is finished. When the SD image data Din input from the input terminal 8 is finished, the process is finished. If the SD image data Din input from the input terminal 8 has not been completed, the process returns to step ST1.

  As described above, according to the image processing device 110 and the image processing method according to the present invention, the feature amount indicating the phase of the edge in the high-resolution image signal is calculated from the interpolation curved surface generated by the interpolation curved surface generation circuit 1, The class to which the pixel data at the target position belongs is detected from the calculated feature quantity indicating the phase of the edge.

  With this configuration, when a high-resolution image signal is generated, an edge can be made to be steep and ringing can be reduced. In particular, at the time of high-density (zoom), it is possible to obtain a high-resolution image signal having edges sharper than ever.

  Subsequently, a configuration example of the image processing apparatus 200 according to the second embodiment will be described. FIG. 15 is a block diagram illustrating a configuration example of the image processing apparatus 200 according to the second embodiment. An image processing apparatus 200 shown in FIG. 15 performs class classification adaptive processing using an ADRC class and a differential value class. In addition, the same code | symbol is attached | subjected to the same component as the image processing apparatus 100 of 1st Embodiment mentioned above, and the detailed description is abbreviate | omitted.

  An image processing apparatus 200 shown in FIG. 15 includes an interpolation curved surface generation circuit 1, a feature amount calculation determination circuit 2, a class detection circuit 3, a class synthesis circuit 4B, a coefficient memory 5B, a prediction tap selection circuit 6, an estimated prediction calculation circuit 7, and an input. A terminal 8, an output terminal 9, a class tap selection circuit 10, and a class detection circuit 11 are provided. In this example, components different from the image processing apparatus 100 of the first embodiment described above are a class synthesis circuit 4B, a coefficient memory 5B, a class tap selection circuit 10, and a class detection circuit 11.

  The SD image data Din is input to the input terminal 8. The interpolation curved surface generation circuit 1 generates a spline interpolation curved surface from the SD image data Din and outputs the information to the feature amount calculation determination circuit 2. The feature amount calculation determination circuit 2 calculates the feature amount of the edge phase and the feature amount of the edge direction from the spline interpolation curved surface. For example, the feature amount calculation determination circuit 2 obtains a primary differential value and a secondary differential value from a spline interpolation curved surface at the target pixel position in order to determine an edge phase and an edge direction. The class detection circuit 3 detects the differential value class to which the pixel of interest belongs from the primary differential value and the secondary differential value of the spline interpolation curved surface (see FIGS. 4 and 6).

  On the other hand, the class tap selection circuit 10 is an example of a second pixel data selection unit, and inputs SD image data Din, and as a class tap corresponding to the target pixel, for example, has a predetermined positional relationship with the target pixel. A plurality of SD pixels are selected. The class detection circuit 11 is an example of a second class detection unit, detects a level distribution pattern of SD pixels constituting a class tap, and detects a class to which the target pixel belongs based on the level distribution pattern. For example, the class detection circuit 11 obtains the feature amount of the SD pixel constituting the class tap using ADRC and classifies it into a predetermined class.

  The class synthesis circuit 4B generates a class code based on the ADRC class obtained by the class detection circuit 11 and the differential value class obtained by the class detection circuit 3, and outputs it to the coefficient memory 5B. The coefficient memory 5B outputs coefficient data to the estimated prediction calculation circuit 7 based on this class code.

  The prediction tap selection circuit 6 selects a prediction tap corresponding to the target pixel from the SD image data Din and outputs the prediction tap to the estimated prediction calculation circuit 7. The estimated prediction calculation circuit 7 performs the calculation shown in Expression (1) using the coefficient data and the SD pixels constituting the prediction tap from the prediction tap selection circuit 6. Thereby, the predicted value E [y] of the pixel value y of the target pixel (HD pixel) is obtained, and this is output from the output terminal 9 as HD image data Dout composed of the pixel value of the HD pixel.

  As described above, the image processing apparatus 200 illustrated in FIG. 15 classifies the class of the pixel of interest by combining the ADRC class and the differential value class. As a result, when the SD pixel is converted to the HD pixel, the position of the edge portion of the HD pixel can be clearly grasped by the differential value class, and highly accurate conversion can be realized at the pixel level by the ADRC class. become.

  Next, a configuration example and an operation example of the coefficient data generation device 300 will be described with reference to FIGS. 16 and 17. FIG. 16 is a block diagram illustrating a configuration example of the coefficient data generation device 300. FIG. 17 is a flowchart illustrating an operation example of the coefficient data generation device 300.

  A coefficient data generation apparatus 300 illustrated in FIG. 16 is an apparatus that generates coefficient data by learning. The coefficient data generation device 300 includes a student image generation circuit 21, an interpolation curved surface generation circuit 1, a feature amount calculation determination circuit 2, a class detection circuit 3, a class code generation circuit 4A, a prediction tap selection circuit 6, a teacher tap selection circuit 29, a regularity A chemical equation adding circuit 30, a coefficient data generating circuit 31, an input terminal 32, and an output terminal 33 are provided. In addition, the same code | symbol is attached | subjected to the same component as the image processing apparatus 100 of 1st Embodiment mentioned above, and the detailed description is abbreviate | omitted.

  In step ST21 shown in FIG. 17, the teacher image (HD) data as a sample is input to the input terminal 32, and the process proceeds to step ST22. In step ST22, the student image generation circuit 21 is an example of a student signal generation unit, and downconverts this teacher image data to generate student image (SD) data. The student image generation circuit 21 is configured by, for example, an LPF (Low Pass Filter). Subsequently, the process proceeds to step ST23.

  In step ST23, the interpolation curved surface generation circuit 1 generates a spline interpolation curved surface from the student image data, outputs the information to the feature amount calculation determination circuit 2, and proceeds to step ST24. In step ST24, the feature amount calculation determination circuit 2 calculates the feature amount of the edge phase and the feature amount of the edge direction from the spline interpolation curved surface. For example, the feature amount calculation determination circuit 2 obtains a primary differential value and a secondary differential value from a spline interpolation curved surface at the target pixel position in order to determine an edge phase and an edge direction. Subsequently, the process proceeds to step ST25.

  In step ST25, the class detection circuit 3 generates a differential value class to which the target pixel belongs from the primary differential value and the secondary differential value of the spline interpolation curved surface (see FIGS. 4 and 6). Subsequently, the process proceeds to step ST26.

  In step ST26, the class code generation circuit 4A generates a class code based on the differential value class obtained by the class detection circuit 3, outputs the class code to the normalized equation addition circuit 30, and proceeds to step ST27. In step ST27, the prediction tap selection circuit 6 selects a prediction tap (see FIG. 10) corresponding to the target pixel from the student image data, and outputs the prediction tap to the normalization equation addition circuit 30. Subsequently, the process proceeds to step ST28. In step ST28, the teacher tap selection circuit 29 selects a teacher tap corresponding to the target pixel from the teacher image data, outputs the selected teacher tap to the normalization equation addition circuit 30, and proceeds to step 9.

  In step ST29, the normalization equation adding circuit 30 obtains the normal equation shown in the above-described equation (10) for each class code based on the class code, the prediction tap data, and the teacher tap data. Subsequently, the process proceeds to step ST30. In step ST30, it is determined whether or not the teacher image data as a sample has been completed. If the teacher image data as a sample has not been completed, the process returns to step ST21. When the teacher image data as a sample is completed, the process proceeds to step ST31.

  In step ST31, the coefficient data generation circuit 31 calculates coefficient equations by solving the normal equations after calculating the number of normal equations for the number of coefficient data to be obtained. In solving the equation (10), for example, a sweeping method (Gauss-Jordan elimination method) or the like can be applied. The obtained coefficient data is output to the output terminal 33. For example, a non-volatile memory (not shown) is provided at the subsequent stage of the output terminal 33, and coefficient data is stored in the non-volatile memory for each class code. Note that the normalized equation addition circuit 30 and the coefficient data generation circuit 31 are examples of a calculation unit.

  As described above, the coefficient data generation device 300 shown in FIG. 16 generates coefficient data based on the class code classified by the differential value class. Thereby, when converting from the SD pixel to the HD pixel using the coefficient data, the position of the edge portion of the HD pixel can be clearly grasped.

  Note that coefficient data may be generated using both the ADRC class and the differential value class. That is, student image data is input, a class tap is selected, and a feature amount of a pixel is obtained by ADRC using this class tap. Then, the ADRC feature quantity is classified into a predetermined class, and the class code is output to the class code generation circuit 4A in FIG. As a result, the class code of the pixel of interest combining the ADRC class and the differential value class can be generated. The coefficient data generated based on this class code is applied to the image processing apparatus 200 shown in FIG.

  The present invention is suitable for use in processing for converting a standard resolution image into a high resolution image.

1 is a block diagram illustrating a configuration example of an image processing apparatus 100 according to a first embodiment. It is a conceptual diagram which shows an example of an edge phase and an edge direction. It is a conceptual diagram which shows the example of a production | generation of a spline interpolation curved surface. It is a conceptual diagram which shows the example of calculation of the phase class of an edge. It is explanatory drawing which shows the structural example of a tan < ^-1 > -class code conversion table (edge phase class). It is a conceptual diagram which shows the example of calculation of the direction class of edge. It is explanatory drawing which shows the structural example of a tan < ^-1 > -class code conversion table (edge direction class). A and B are schematic diagrams illustrating an example of an edge of a target (teacher) image. A and B are schematic diagrams illustrating an example of an edge of an input (student) image. It is the schematic which shows the structural example of a prediction tap. It is the schematic which shows the structural example of a prediction tap and a class tap. A and B are explanatory diagrams illustrating an example of a result of HD conversion processing in which the number of pixels only in the horizontal direction is increased by eight times. A and B are explanatory diagrams illustrating an example of qualitative evaluation of the image processing apparatuses 100 and 400. 4 is a flowchart illustrating an operation example of the image processing apparatus 100 according to the present invention. It is a block diagram which shows the structural example of the image processing apparatus 200 which concerns on 2nd Embodiment. It is a block diagram which shows the structural example of the coefficient data generation apparatus 300 which concerns on this invention. 5 is a flowchart illustrating an operation example of a coefficient data generation device 300. It is a block diagram which shows the structural example of the image processing apparatus 400 which concerns on a prior art example.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Interpolation curved surface production | generation circuit (interpolation curved surface production | generation part), 2 ... Feature quantity calculation determination circuit (feature quantity calculation part), 3 ... Class detection circuit (1st class detection part), 5A, 5B ... Coefficient memory (coefficient data selection unit), 6 ... Prediction tap selection circuit (first pixel data selection unit), 7 ... Estimated prediction calculation circuit (estimation calculation unit), 10 ... Class tap Selection circuit (second pixel data selection unit), 11... Class detection circuit (second class detection unit), 21... Student image generation circuit (student signal generation unit), 30. Adder circuit (arithmetic unit), 31... Coefficient data generation circuit (arithmetic unit), 100, 200... Image processing device, 300.

Claims (11)

Interpolation curved surface generation for generating an interpolation curved surface by selecting a plurality of first pixel data located around the target position in the second image signal having a higher resolution than the first image signal from the first image signal And
A feature amount calculation unit that calculates a feature amount indicating a phase of an edge in the second image signal from the interpolation curved surface generated by the interpolation curved surface generation unit;
A first class detection unit that detects a class to which the pixel data of the target position belongs from a feature amount indicating a phase of an edge calculated by the feature amount calculation unit;
A coefficient data selection unit that selects coefficient data with reference to the class detected by the first class detection unit;
A first pixel data selection unit that selects, from the first image signal, a plurality of second pixel data located around a target position in the second image signal;
An estimation calculation unit that calculates a plurality of second pixel data selected by the first pixel data selection unit and coefficient data selected by the coefficient data selection unit to obtain pixel data at the target position; An image processing apparatus. The feature amount calculation unit includes:
A feature amount indicating the direction of an edge in the second image signal is calculated from the interpolation curved surface generated by the interpolation curved surface generation unit,
The first class detection unit includes:
The image processing apparatus according to claim 1, wherein a class to which pixel data of the target position belongs is detected from a feature quantity indicating the phase of the edge and a feature quantity indicating the direction of the edge. A second pixel data selection unit that selects, from the first image signal, a plurality of third pixel data located around a target position in the second image signal;
A second level detection unit that detects a level distribution pattern of the plurality of third pixel data selected by the second pixel data selection unit, and detects a class to which the pixel data of the target position belongs based on the level distribution pattern; The image processing apparatus according to claim 1, further comprising a class detection unit. The feature amount calculation unit includes:
Differentiating the interpolated curved surface based on a predetermined component to obtain a primary differential value and a secondary differential value as a feature amount indicating the phase of the edge,
The first class detection unit includes:
Class division is performed in a circumferential direction on a plane composed of the primary differential value and the secondary differential value, and the obtained primary differential value and secondary differential value are developed on the plane, The image processing apparatus according to claim 2, wherein the phase class to which the pixel data belongs is detected. The feature amount calculation unit includes:
Differentiating the interpolated curved surface based on a predetermined component to obtain a first derivative value as a feature amount indicating the direction of the edge,
The first class detection unit includes:
Class division is performed in the circumferential direction on a plane constituted by each of the primary differential values, and the obtained primary differential value is developed on the plane to detect the class in the direction to which the pixel data of the target position belongs. The image processing apparatus according to claim 4. A first interpolation signal is generated by selecting a plurality of first pixel data located around a target position in a second image signal having a higher resolution than the first image signal from the first image signal. Steps,
A second step of calculating a feature amount indicating a phase of an edge in the second image signal from the interpolation curved surface generated in the first step;
A third step of detecting a class to which the pixel data of the target position belongs from the feature amount indicating the phase of the edge calculated in the second step;
A fourth step of selecting coefficient data with reference to the class detected in the third step;
A fifth step of selecting, from the first image signal, a plurality of second pixel data located around a target position in the second image signal;
A sixth step of calculating a plurality of second pixel data selected in the fifth step and coefficient data selected in the fourth step to obtain pixel data of the target position; Processing method. In the second step,
A feature amount indicating the direction of an edge in the second image signal is calculated from the interpolation curved surface generated in the first step,
In the third step,
The image processing method according to claim 6, wherein a class to which pixel data of the target position belongs is detected from a feature amount indicating the phase of the edge and a feature amount indicating the direction of the edge. A student signal generator for generating a student signal corresponding to the first image signal from a teacher signal corresponding to the second image signal having a higher resolution than the first image signal;
An interpolation curved surface generation unit that generates an interpolation curved surface by selecting a plurality of first pixel data located around a target position in the teacher signal from the student signal generated by the student signal generation unit;
A feature amount calculation unit that calculates a feature amount indicating a phase of an edge in the teacher signal from the interpolation curved surface generated by the interpolation curved surface generation unit;
A first class detection unit that detects a class to which the pixel data of the target position belongs from a feature amount indicating a phase of an edge calculated by the feature amount calculation unit;
A first pixel data selection unit that selects a plurality of second pixel data located around a target position in the teacher signal from the student signal generated by the student signal generation unit;
The calculation is performed using the class detected by the first class detection unit, the plurality of second pixel data selected by the first pixel data selection unit, and the pixel data of the target position in the teacher signal. A coefficient data generation apparatus comprising: an arithmetic unit that obtains coefficient data used for converting one image signal into the second image signal for each class. The feature amount calculation unit includes:
From the interpolated curved surface generated by the interpolated curved surface generating unit, a feature amount indicating the direction of the edge in the teacher signal is calculated,
The first class detection unit includes:
9. The coefficient data generation apparatus according to claim 8, wherein a class to which pixel data of the target position belongs is detected from a feature amount indicating the phase of the edge and a feature amount indicating the direction of the edge. A first step of generating a student signal corresponding to the first image signal from a teacher signal corresponding to the second image signal having a higher resolution than the first image signal;
A second step of generating an interpolated curved surface by selecting a plurality of first pixel data located around a target position in the teacher signal from the student signal generated in the first step;
A third step of calculating a feature amount indicating a phase of an edge in the teacher signal from the interpolation curved surface generated in the second step;
A fourth step of detecting a class to which the pixel data of the target position belongs from the feature amount indicating the phase of the edge calculated in the third step;
A fifth step of selecting, from the student signal generated in the first step, a plurality of third pixel data located around a target position in the teacher signal;
The first image signal is calculated by using the class detected in the fourth step, the plurality of third pixel data selected in the fifth step, and the pixel data of the target position in the teacher signal. And a sixth step of obtaining, for each class, coefficient data used when converting to the second image signal. In the third step,
From the interpolation curved surface generated in the second step, a feature amount indicating the direction of the edge in the teacher signal is calculated,
In the fourth step,
The coefficient data generation method according to claim 10, wherein a class to which pixel data of the target position belongs is detected from a feature quantity indicating the phase of the edge and a feature quantity indicating the direction of the edge.