JP3862531B2 - Image interpolation device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image interpolating apparatus using line interpolation when generating a frame image from a field image when performing up-conversion of a television system having a different number of scanning lines or converting an interlaced scanning video signal into a sequentially scanned video signal.
[0002]
[Prior art]
Conventionally, as a scanning line interpolation method for generating progressive video from interlaced video for up-conversion of television systems with different numbers of scanning lines and high-resolution images, intra-frame interpolation is performed for still images. Is called. However, in a moving image, inter-field interpolation is performed because the frame correlation is lost, and in-frame interpolation causes a malfunction such as erroneously converting a moving vertical line to jagged.
[0003]
As a method of intra-field interpolation, as a simple method, there are a line interpolation method using the previous line as it is as a line to be interpolated, and a line interpolation method using the average value of the upper and lower lines. In the former interpolation method, an image having no correlation in the vertical direction such as a slanted line is accompanied by image deterioration such as a jagged edge in the image outline, and the latter interpolation method is blurred in the image.
[0004]
As a technique for solving these drawbacks, there is an interpolation method described in Japanese Patent Laid-Open No. 63-187785. In this interpolation method, each pixel of the line to be interpolated is interpolated using pixel information in the direction having the strongest correlation around the pixel. First, in order to know in which direction the pixel signal extending radially from the interpolated pixel has the strongest correlation, the absolute value of the difference between adjacent pixels in the vertical direction, right diagonal direction, and left diagonal direction is obtained. It is determined that the direction in which the absolute value of the difference is minimum is the direction having the strongest correlation, and an average value of each pixel in the direction is obtained and used as the value of the interpolation pixel.
[0005]
However, in this method, for each pixel of the line to be interpolated, that is, for each pixel, at least the above-described three-direction absolute difference values are calculated, and further, the minimum of the absolute difference values is determined. A process of obtaining the value of the interpolation pixel is required. For this reason, the above-described series of processing is performed even when the interpolation processing speed is slow and there is no difference in the strength of correlation in any direction, such as a region other than the edge portion in the image (a region where the pixel value does not change). Therefore, useless processing time is consumed, the interpolation processing speed is slow, and it is difficult to expand the range of pixels for which correlation is obtained.
Therefore, when interpolating an image having a correlation at a considerable distance, such as a diagonal line with a small inclination angle, interpolation using poorly correlated pixels cannot be performed, resulting in poor interpolation accuracy and still a diagonal line. The edge of the part was supposed to be rattled.
[0006]
In Japanese Patent Laid-Open No. 5-30487, in order to improve the drawbacks of the above-mentioned publication, the interpolation processing speed is improved and the range for obtaining the correlation is increased with the improvement of the processing speed to improve the interpolation accuracy. A method is disclosed. The image interpolation method disclosed in this publication will be described below with reference to FIGS.
[0007]
First, in this interpolation method, as shown in FIG. 13, the adjacent n-line and n + 1-line pixels of the two-dimensional image are compared, and edge portions a and b of the two-dimensional image on the respective lines are detected. . In the interpolation line to be interpolated, interpolation is performed using pixels of any adjacent line except between a and b where the edge is detected, and an edge portion of the interpolation line is obtained.
[0008]
Next, as shown in FIG. 13, at the edge locations a and b, the “number of pixels” centered on the pixel of interest (here, A) on one of the adjacent lines (n lines) and “from the interpolation position” Is set as a neighboring pixel row (3, 0) consisting of the “shift amount”, and the target pixel row (3, 1) correlated therewith is selected from the other adjacent lines (n + 1 line).
Here, as a general expression for obtaining the correlation between the neighboring pixel column and the target pixel column, the neighboring pixel column = (2m + 1, 0) and the target pixel column = (2m + 1, ± Y) are used, and m = Y = 1. , 2, 3, 4 and 5 are changed in order.
[0009]
First, as shown in FIG. 14, an operation is performed on a pixel pair as indicated by an arrow between the neighboring pixel column (3, 0) and the target pixel column (3, 1), and a difference in pixel level is predetermined. The correlation is determined based on whether the value is within the reference value. In the case shown in FIG. 15, since there is no correlation in the center pixel pair, m = Y = 2, the neighboring pixel column is (5, 0), the target pixel column is (5, 2), and the arrows in the figure. The operation is performed on the correct pixel pair. If it is determined that there is a correlation by calculation with a pixel pair as indicated by an arrow in the figure, the shift amount from the interpolation position at that time is 2 pixels. As shown in FIG. 16, the a part and the b part are shifted by two pixels. A pixel column obtained by shifting the selected pixel column or the neighboring pixel column in a direction opposite to the position shift direction by half of the obtained number of pixels, that is, a pixel column centered at a point c shifted from A to the right by one pixel. Interpolate using.
[0010]
[Problems to be solved by the invention]
  The interpolation method disclosed in Japanese Patent Application Laid-Open No. 5-30487 is an invention for improving the processing speed by incorporating interpolation by a pixel row as compared with a commonly performed interpolation method by a pixel unit. Therefore, in order to determine the edge portion of the pixel row, the inter-line calculation is performed while performing the pixel shift from at least about 3 pixels to about 11 pixels in the vicinity of the edge portion. For this reason, this method is still problematic in that it takes a long calculation time. Furthermore, this interpolation method is a method of smoothing the outline of the image,For exampleOf a thin line image close to the horizontalCase,If the field image of the interlaced video signal is interrupted, there will be a discontinuous part, and if there is no correlation between the pixel of the fine line image and the pixel of the image line immediately above it, it will not be recognized as the contour of the image. Generate interpolated image to fill partIt also has the problem that it cannot be done.
[0011]
An object of the present invention is to provide an image interpolating apparatus that can greatly reduce the processing speed and can generate a smooth frame image without interruption.
[0012]
[Means for Solving the Problems]
  The present inventionInterlaced scanning video signalinputfieldInterpolate between image lines for an imageTo generate an image of a progressively scanned video signalIn the image interpolation device,
  inputfieldTarget pixel of image and target pixelDirectly belowWith adjacent pixelsLuminanceA difference characteristic is obtained, and an image line pattern that is a continuous pixel array having a common difference characteristic in the line direction is extracted, and the image line patternPosition and luminance difference characteristicsPattern information generating means for generating pattern information on theAnd saidTake out image line patterns that match the pattern information related to luminance difference characteristics, and interpolate between linesUsed forInterpolation pattern determining means for determining an image line pattern;Were determinedImage line patternmake use ofInterpolation execution means for setting the interpolation pixel string and executing interpolation,
The pattern information generating means includes a twice-difference value calculating means for obtaining a twice-difference value obtained by adding each difference of luminance values between a target pixel of the input field image and a pixel of an adjacent line directly opposite immediately above the target pixel; , For each image line, pattern extraction means for recognizing and extracting a continuous pixel sequence having a luminance difference characteristic based on the two-time difference value in the line direction as an image line pattern, and the extracted image line pattern An information collecting means for collecting pattern information,
The interpolation pattern determining means is a pattern for determining whether or not the pattern information of the searched image line patterns matches with the matching pattern search means for searching for an image line pattern that may be matched from the pattern information. A match determination means, and a vector determination means for performing a vector determination for determining whether the image line pattern should be continuously used for interpolation from the start position and length of the image line pattern determined to match,
The interpolation execution means includes an interpolation execution section determination means for determining a position and a length of a section for performing interpolation based on pattern information of the image line pattern determined by the interpolation pattern determination section, and the determined section. Interpolating pixel creating means for creating the pixel row to be interpolated based on the image line pattern determined by the interpolation pattern determining means.An image interpolating apparatus characterized by the above.
[0013]
  The present invention also provides:further,Interpolation based on the pattern informationUsed forInterpolation to generate prediction information about the location of possible image line patternspatternPredictive meansComprising
  The interpolation pattern prediction means includes an interpolation direction prediction means for predicting an interpolable direction / non-interpolable direction from the pattern information, and a search distance prediction for predicting a distance to a position where an image line pattern matching the pattern information exists. And an interpolation execution distance prediction means for predicting a distance that does not cause image corruption even if interpolation is performed with a matching image line pattern.Equipped withDoAn image interpolating apparatus characterized by the above.
[0014]
  In the present invention, the pattern information generating meansFurthermore, normalization means for normalizing the vertical difference value twice for each pixel and classifying it into a type indicating a plurality of luminance difference characteristics,
The pattern extraction unit extracts an image line pattern based on the classification indicating the luminance difference characteristic obtained by the normalization unit.An image interpolating apparatus characterized by the above.
[0015]
  Further, the present invention provides the aboveThe pattern information generating means further comprises coring means for performing coring processing to remove the image line pattern when the image line pattern extracted by the pattern output means is image distortion or noise,
The coring means distorts an image having a plurality of maximum / minimum points in the luminance waveform in the image line pattern and an acute change point in the luminance waveform in the pattern, and a pixel column that does not reach the reference pixel number as noise. Detection means for detecting, editing determination means for determining whether editing for deleting the image line pattern is necessary based on detection data of the detecting means, and an image line pattern determined to require editing Editing execution means for executing deletion editing andAn image interpolating apparatus characterized by comprising:
[0016]
  Further, the present invention provides the aboveAn information collecting means; means for collecting a maximum luminance value in the image line pattern; means for collecting a maximum luminance increase amount from a start position of the image line pattern to a maximum luminance position; and Means for collecting the maximum amount of decrease in brightness up to the pattern end position.An image interpolating apparatus characterized by the above.
[0017]
  Further, the present invention provides the aboveAn information collecting means; means for collecting a minimum luminance value in the image line pattern; means for collecting a maximum luminance reduction amount from the image line pattern start position to a minimum luminance position; and Means for collecting the maximum amount of increase in luminance up to the pattern end position;An image interpolating apparatus characterized by the above.
[0023]
DETAILED DESCRIPTION OF THE INVENTION
  Hereinafter, embodiments of the present invention will be described with reference to FIGS.
  FIG. 1 is a block diagram showing a configuration of an embodiment of an image interpolation apparatus according to the present invention. Figure 2 shows an example of an input image with a black thin line image on a white background 1In interlaced video signalThe monitor screen on which the field image 2 is projected is shown. FIG. 3 shows an input video line and an interpolation line at the enlargement point 3 shown in FIG. In the figure, A, C, E, G, I, and K are input video lines, and B, D, F, H, and J are interpolation lines.
[0024]
The image interpolation apparatus shown in FIG. 1 includes a pattern information generation unit 100, an interpolation pattern prediction unit 120, an image database 130, an interpolation pattern determination unit 140, and an interpolation execution unit 150. Then, a first step of generating a database of image line patterns by calculating a difference between brightness values twice (A−2 × C + E) between opposing pixels in three adjacent lines of a field image, coring processing, and information collecting processing; A second step of predicting an interpolation direction and an interpolation distance of each image line pattern from the pattern information data, a third step of determining a position to be interpolated based on a database of image line patterns, and a fourth step of executing interpolation. Interpolate with.
[0025]
Below, each part of the image interpolation apparatus of FIG. 1 is explained in full detail.
[0026]
<Pattern information generation unit>
First, generation of an image line pattern in the pattern information generation unit 100 will be described.
The pattern information generation unit 100 includes a vertical luminance value double difference calculation unit 101, a normalization processing unit 102, a pattern extraction unit 103, a vertical direction correlation detection unit 104, an information collection unit 105, and a coring processing unit 110. The Further, the coring processing unit 110 includes an intra-pattern image distortion / noise detection unit 111, a pattern edit determination unit 112, and a pattern edit execution unit 113.
[0027]
The field image input to the pattern information generation unit 100 is input to the vertical difference luminance value twice difference calculation unit 101. The vertical luminance value twice difference calculation unit 101 calculates the vertical difference value of the vertical luminance value of each pixel twice. Here, the vertical difference value of the vertical direction luminance value is a difference value between the luminance value of the target pixel and the luminance value of the pixel on the line adjacent to the target pixel and facing the target pixel in the vertical direction. It is the added value.
For example, the difference value twice is within a range of ± 512, and in FIG. 3, each of the input images of A line C line E line, C line E line G line, E line G line I line, G line I line K line A difference value is calculated twice between three lines. That is, using the pixels of the C line, E line, G line, and I line as the target pixels, the difference value of the luminance value with the opposite pixel immediately above the target pixel is obtained twice. Taking the three image lines of A line, C line, and E line as an example, the calculation of the difference between the vertical luminance values twice is as follows.
[0028]
Two-time difference value of the vertical luminance value of the target pixel on the C line = luminance value of the A line opposing pixel−2 × luminance value of the C line target pixel + luminance value of the E line opposing pixel
[0029]
  The vertical correlation detection unit 104 detects whether or not there is a correlation in the vertical direction among the three image lines used by the vertical luminance value double difference calculation unit 101. For example, for each pixel of the three image lines shown in FIG.First,The horizontal difference increase / decrease direction of the luminance value, the horizontal increase / decrease amount of the luminance value, and the twice-difference value of the horizontal luminance value are obtained.Based on the calculated value, paying attention to one pixel,In the vertical direction of adjacent image linesThe pixel (opposite pixel directly below)the sameWhen taking a value (brightness value horizontal increase / decrease direction, luminance value horizontal increase / decrease amount, horizontal difference value twice)And there is a vertical correlationdetectionTo do.
[0030]
  Next, the normalization processing unit 102 performs normalization processing of the difference value twice of the luminance value in the vertical direction of each pixel on the image line based on the set threshold value, and each pixel on the image lineTwice difference value of vertical luminance value ofAre classified into three types, +,-, and 0. The pattern extraction unit 103 extracts, as an image line pattern, a section in which the same type of + or − is continuous, and is detected by the vertical direction correlation detection unit 104 as having no vertical direction correlation.
  The extracted image line pattern is the target pixelOf luminance value in the vertical directionBased on the twice-difference value, a luminance difference characteristic (a classification of +, 0, and − in the twice-difference value corresponds) is obtained, and is a continuous pixel row in the line direction having a common luminance difference characteristic.
[0031]
  At this time, in the information collection unit 105, the image line pattern extracted by the pattern extraction unit 103 is displayed.Position information (data) and image features in the image line patterninformation(data)To collect. collectImage linepatternInsideofInformation about image features (data)For example,Image lineMaximum / minimum brightness value in the pattern,Image lineThe width and direction of increase / decrease of the luminance value in the horizontal direction (line direction) in the pattern,Image lineBrightness values at both ends of the pattern,Image lineThere is an increase / decrease amount of luminance values at both ends of the pattern. If the data collected by the information collection unit 105 is performed for all three input image lines used by the vertical difference value twice-difference calculation unit 101, the accuracy of interpolation is improved.
[0032]
  In addition, extractedImage lineThe pattern of the coring processing unit 110Image lineIn the noise distortion / noise detection unit 111 of the image in the pattern,Image lineDetects the presence of distortion and noise in the image inside the pattern. Image distortion is, for example,Image lineMultiple luminance maxima / minimal points in the patternSharpWhen there is a corner change pointTo.Therefore, an image without distortion means an image having one maximum point or minimum point in the luminance waveform in the image line pattern, or an image having no maximum point or minimum point.Noise is the standardNumber of pixelsNot reachPixel rowIt is.
  Next, in the pattern editing determination unit 112, when the above distortion or noise is detected,ExtractedIt is determined whether or not editing for deleting the image line pattern is performed by comparing with an editing reference value. This edit reference value is set so that it can be comprehensively determined according to the degree of each distortion and noise. Then, the image line pattern determined to be deleted is deleted by the pattern editing execution unit 113.
[0033]
  FIG. 4 is a diagram illustrating an image line pattern obtained through the above processing in the pattern information generation unit 100. In the figure, 40 to 43 and 62 to 65 are image line patterns classified as-, 51 to 54 are image line patterns classified as +, and the other are image line patterns classified as 0. In the pattern information generating unit 100, each image line pattern that has been segmented for each image line as shown in FIG.Image lineStart position, length, sign for each patternAlso, Created as data collected by the information collection unit 105, and the interpolation pattern prediction unit 120Sent toThe
[0034]
<Interpolation pattern prediction unit>
The prediction of the interpolation direction, search distance, and interpolation execution distance in the interpolation pattern prediction unit 120 will be described.
The interpolation pattern prediction unit 120 includes an interpolation direction prediction unit 121, a search distance prediction unit 122, and an interpolation execution distance prediction unit 123.
The interpolation pattern prediction unit 120 uses the image line pattern data generated by the pattern information generation unit 100 to predict the possibility of the presence of an interpolable image line pattern and the distance to the interpolable image line pattern. .
[0035]
The interpolation direction prediction unit 121 of the interpolation pattern prediction unit 120 compares information on both left and right ends of the image line pattern.
For example, in FIG. 5, the left end position of the image line pattern 52 on the E line generated from the three image lines of the C line, the E line, and the G line, the luminance increase / decrease amount of the image lines E-23 and 24, and the E line The amount of increase / decrease in brightness of the image lines G-30 and 31 is compared with the position on the G line immediately below the right end of the image line pattern 52. When the increase / decrease is in the reverse direction or the increase / decrease amount is greatly different, the interpolation direction predicting unit 121 predicts that the connection of the image line pattern 52 in the lower right direction is invalid (hereinafter, “interpolation impossible direction”). Further, the interpolation possible direction / non-interpolable direction are predicted in the same way for the upper right, upper left, and lower left.
[0036]
  Next, the search distance predicting unit 122 of the interpolation pattern predicting unit 120 compares the luminance values at the left and right ends of the input image line pattern and matches the values.Image linePredict the distance to where the pattern will be.
  Already, the difference value of the luminance value in the horizontal direction, the horizontal direction increase / decrease amount of the luminance value, and the twice-difference value of the luminance value in the horizontal direction are already obtained.For example, in FIG. 5, the position of the left end of the image line pattern 52 on the E line generated from the three image lines of the C line, the E line, and the G line.It is inThe luminance value of the image line E-24 and the position on the G line immediately below the right end of the image line pattern 52 on the E line.It is inThe difference between the luminance value of the image line G-30 and the position on the G line immediately below the right end of the image line pattern 52 on the E line.It is inIncrease / decrease amount of brightness of image lines G-30 and 31(Luminance increase / decrease amount per pixel)Divide byThatresultOn the basis of theContinue to the lower rightImage linePattern (interpolableImage lineDistance to pattern(Pixel count)And Similarly, the distance is also calculated in the lower left direction.Thus, if the luminance of the pixels immediately below the pixels at both ends of the image line pattern is increased or decreased, the distance of the image line pattern to be interpolated can be predicted from the increase / decrease amount per pixel.
[0037]
  Then, the interpolation execution distance predicting unit 123 of the interpolation pattern predicting unit 120 calculates the change amount and length of the luminance in the input image line pattern.(Pixel count)From etc., matchImage lineLimit distance that image does not break even if interpolated with pattern(Pixel count)Predict.
  For example, in FIG. 5, the amount of change in the luminance of the image line pattern 52 on the E line generated from the three image lines of the C line, the E line, and the G line.Thus, when each image line pattern is shortened, a limit distance at which an image that should not be originally formed in the interpolated image is not formed is predicted.
[0038]
The interpolation pattern prediction unit 120 adds the above various prediction data to the data generated by the pattern information generation unit 100 and sends the data to the image database 130.
[0039]
<Image pattern database>
  In the image database 130, the pattern information generation unit 100Data obtained as a result of processingThe data (prediction information) obtained as a result of the processing of the interpolation pattern prediction unit 120 is an image.linepatternConcerningdata(Pattern data)Stored as
  The amount of data to be stored may be all in the field, or may be stored for a plurality of lines required by the interpolation pattern determination unit 140 and the interpolation execution unit 150.
[0040]
<Interpolation pattern determination unit>
The interpolation pattern determination unit 140 includes a pattern search unit 141, a pattern match determination unit 142, a pattern match condition setting unit 143, an interpolation direction determination unit 144, an interpolation execution distance determination unit 145, a vector determination unit 146, and a vector determination condition setting unit 147. It is the composition which becomes.
[0041]
  In the pattern search unit 141, an image input from the image database 130linepatternPatternFor other data, images of other lines stored in the image database 130linepatternPatternPattern data that may match from the dataExtractionTo do.
  For example, based on the search distance predicted by the search distance prediction unit 122 of the interpolation pattern prediction unit 120,It becomes a target to be judgedimagelineputterNCentered on axisOther line2 points on each side, 4 points in totalaboutimagelineCheck for the existence of the pattern. Here are the images found at the two points on the leftlineThe pattern isDifferent image line patternsIf it wasIt becomes a target to be judgedimagelineputterNThe side near the centerAnd the pattern data of the image line pattern to be determined and the found image line patternPattern data having the same sign information of the pattern data is adopted. Repeat for the two points on the right. Therefore, at most the imagelinePattern dataIsLeft and rightFromOneExtractionIs issued.
[0042]
  In FIG. 6, based on the search distance predicted by the search distance prediction unit 122, images in the section of the input image lines E-17, 18, 19, 20, 21, 21, 22 and 23.linepattern42data(Pattern data42)Patterns that might matchDataSearchDoIt shows a state. With the center of the image pattern data 42 of the input image line E as the axis, on the left side, the two searched points are images in the section of the input image lines G-11, 12, 13, 14, 15, 16, and 17.linepatternIs detected, data related to the image line pattern 43 (patternData 43)Is extracted. On the right side, the image of the searched input image line Elineputter42 is an image in the section of input image lines G-18, 19, 20, 21, 22, 23, 24 close to the center of 2.linepatternIs detected, data related to the image line pattern 53 (patternData 53)ButExtractedThe However, PaTurn data 42Code and pattern data 53 codeWhenHaichiSince it does not, finally the image in the section of the input image line G-25, 26, 27, 28, 29, 30, 31linepattern63 data (patternData 63)Is extracted.
[0043]
  The interpolation pattern determination unit 140 is now registered in the image database 130.linepatternPatternWhen processing data, it is assumed that processing is sequentially advanced line by line from the upper line to the lower line for each line of the field image. Then, it is a target to be currently determinedThe baseimagelinepatternPatterndata(Base pattern data)IsunderImage in directionlinepatternPatternIt will be compared with the data.
  Therefore, the pattern match determination unit 142 extracts the result of the search in the pattern search unit 141 based on the conditions set in the pattern match condition setting unit 143.1 line downimagelinepatternPatternIt is determined whether the data and the contents of the base pattern data match.
[0044]
  The contents determined by the pattern match determination unit 142 includeImage lineMaximum / minimum brightness value in the pattern,Image lineWidth and direction of luminance increase / decrease in the pattern,Image lineBrightness values at both ends of the pattern,Image lineThis is the detailed contents of data collected by the information collection unit 105 of the pattern information generation unit 100, such as the amount of increase or decrease in luminance at both ends of the pattern. thisRapIf the difference in the contents of the turn data is within the threshold set in the pattern matching condition setting unit 143, the result of the search in the base pattern data and the pattern search unit 141 is extracted.TapaIt is determined that the turn data match. Is determined to match the base pattern data.TapaThe turn data is hereinafter referred to as candidate pattern data.
[0045]
  For example, as shown in FIG. 4, an image in the section of input image lines E-24, 25, 26, 27, 28, 29, 30linepattern52 patternsImages in the section of data 52 and input image lines G-18, 19, 20, 21, 22, 23, 24linepattern53 patternsWhat is data 53?Image lineMaximum / minimum brightness value in the pattern,Image lineWidth and direction of luminance increase / decrease in the pattern,Image lineBrightness values at both ends of the pattern,Image lineSince the difference in data such as the amount of increase / decrease in luminance at both ends of the pattern is within the conditions set in the pattern matching condition setting unit 143, the image on the input image line Elinepattern52 patternsData 52 and image on input image line Glinepattern53 patternsIt is determined that the data 53 match.
[0046]
  Also, images in the section of input image lines E-17, 18, 19, 20, 21, 22, 23linepattern42 patternsImages in the section of data 42 and input image lines G-25, 26, 27, 28, 29, 30, 31linepattern63 patternsWith data 63,Image lineSince the increase / decrease direction of the luminance in the pattern is reverse, the pattern on the input image line E is not matched with the condition set in the pattern matching condition setting unit 143.linepattern42 patternsData 42 and image on input image line Glinepattern63 patternsIt is determined that the data 63 does not match.
[0047]
  At this time,Base pattern data andTwo images on the left and rightlinepatternPatternThis match determination is performed on the data, but both the left and right patternsBoth of the basic pattern data andIf it is determined that they match, since there is a high possibility of erroneous extraction, both the left and right match determinations are discarded and both are handled as not matching.
  As a result, as shown in FIG.ToAn imagelinepatternPatterndataInIt is specified that candidate pattern data exists only in one direction.
[0048]
Next, the interpolation direction determination unit 144 connects the candidate pattern data determined to match with the base pattern data, the candidate pattern data input from the interpolation direction prediction unit 121 of the interpolation pattern prediction unit 120, and the base pattern data. Judgment is made between the interpolable direction / non-interpolable direction of each pattern data.
[0049]
  Using the position information of the base pattern data and candidate pattern data,First, base pattern dataFrom image line dataCandidate pattern dataImage line dataDirection in whichThis direction isIt is determined whether the direction is interpolable, and then the candidate pattern dataImage line dataTo base pattern dataImage line dataDirection in whichDerive this directionIt is determined whether or not is in the interpolation possible direction. If both the determinations are valid, it is determined that the candidate pattern data is in a direction in which interpolation is possible.
  For example, the image of the image line Elinepattern52 patternsImage of data 52 and image line Glinepattern53 patternsIn the determination between the data 53, first, the image of the image line E which is the basic pattern datalinepattern52 patternsIt is determined whether the lower left direction of the data 52 is an interpolable direction, and then the image of the image line G that is candidate pattern datalinepattern53 patternsIt is determined whether the upper right direction of the data 53 is an interpolation possible direction.
[0050]
Next, in the interpolation execution distance determination unit 145, based on the interpolation execution distance input from the execution distance prediction unit 123 between the interpolation pattern prediction units 120, between the candidate pattern data extracted as matching and the base pattern data Make a determination about the distance. If the distance between the candidate pattern data and the base pattern data is smaller than the interpolation execution distance, it is determined that the candidate pattern data exists at an interpolable distance.
[0051]
  Furthermore, there are cases where there are fine lines or edge portions that are interrupted in an actual image, and vector determination is performed to prevent erroneous extraction of matching patterns.
  Now, the interpolation pattern determination unit 140 is registered in the image database 130.RuWhen processing turn data, it is assumed that the processing is sequentially advanced line by line from the upper line to the lower line for each line of the field image. Then, the base pattern data that is the target of the current determination andOne line above the line where the image line data of the base pattern data existsOn the lineOf existing image line patternsWhether or not there is a coincidence between pattern data has already been determined. Therefore, the vector determination unit 146 uses the base pattern data as candidate pattern data for the base pattern data currently being determined.One line above the line where the image line data of the base pattern data existsOn the lineOf existing image line patternsAn inquiry is made to the image database 130 for pattern data.
[0052]
  Use base pattern data as candidate pattern dataIf the pattern data does not exist, the current basic pattern data and the candidate pattern data are registered in the image database 130, and the process ends.
  Use base pattern data as candidate pattern dataWhen pattern data exists, vector determination is performed for three patterns including the current base pattern data and candidate pattern data. This is based on the setting of the vector determination condition setting unit 147.The paPerforms judgment (vector judgment) on the start position, length, overlap amount, etc. of the turn data.The paTurn data should be continuously interpolatedImage lineIt is determined whether it is a pattern.
[0053]
  For example, an image on the input image line ElinePattern data 42Pattern data 42When is base pattern data, PaThe turn data 42 includes an image on the input image line G as candidate pattern data.linepattern43 patternsData 43 exists. At this time, the image on the input image line C on one line has already beenlinepattern41 patternsData 41 and imagelinepattern42 patternsSince the matching relationship of the data 42 is determined, threeThe paTurn data 41, 42, 43Respective image line patterns 41, 42, 43The start position / length / overlap amount is determined.
  In the present embodiment, as shown in FIG. 7, three image pattern data 41, 42, 43Respective image line patterns 41, 42, 43Both are the same length and threeThe paTurn dataEach image line patternBoth of them are on a straight line, so they should be interpolated continuously.Image lineIt is determined to be a pattern.
[0054]
<Interpolation execution unit>
  Next, the interpolation execution process in the interpolation execution unit 150 will be described.
  The interpolation execution unit 150 includes an interpolation execution section determination unit 151, a correction unit 153, an interpolation pixel string calculation unit 152, and a motion information processing unit 154.
  In the interpolation execution section determination unit 151, the coincidence relationship information input from the interpolation pattern determination unit 140, the base pattern data obtained from the image database 130, and candidate pattern dataEach image line patternInterpolation from the start position and length dataTheExecutionDosection(Position and length)To decide.
[0055]
  interpolationTheExecutionDoThe length of the section is the base pattern dataImage line pattern corresponding toLength and candidate pattern dataImage line pattern corresponding toAverage length ofTossThe Also interpolationTheExecutionDoThe position of the section is the base pattern dataImage line pattern corresponding toUpper image line and candidate pattern dataImage line pattern corresponding toThe base pattern data on the interpolation line between the lower image line withImage line pattern corresponding toCenter point and candidate pattern dataImage line pattern corresponding toCenter pointStraight line connectingThe center of is interpolatedTheExecutionDoIt is determined to be the center point of the section.
  However, interpolationTheExecutionDoThe method for determining the position and length of the section is not limited to the above method.
[0056]
  The interpolation determined by the interpolation execution interval determination unit 151 in FIGS. 8 and 9TheExecutionDoIndicates the section. In this embodiment, the basic pattern data of the image lineImage line pattern corresponding toAnd candidate pattern dataImage line pattern corresponding toSince the length of is expressed the same, interpolationTheExecutionDoThe section is the same length.
  Image line E base pattern data 42The image line pattern 42 corresponding toAnd candidate pattern data 43 of the image line GImage line pattern 43 corresponding toInterpolationTheExecutionDoThe section is a section 72 on the interpolation line F, and the basic pattern data 52 of the image line EThe image line pattern 52 corresponding toAnd candidate pattern data 53 of the image line GImage line pattern 53 corresponding toInterpolationTheExecutionDoThe section is the section 82 on the interpolation line F.
[0057]
  Next, in the interpolation pixel string calculation unit 152, the interpolation determined by the interpolation execution section determination unit 151 is performed.TheExecutionDoOn the intervalOverwrite pixel string (Interpolated pixel row)Create
  Base pattern dataImage line pattern length corresponding toAnd candidate pattern dataImage line pattern corresponding toIf the lengths are the same, the base pattern dataImage line pattern corresponding toAnd candidate pattern dataImage line pattern corresponding toEach pixelEveryThe average is the interpolated pixel row.
  Base pattern dataImage line pattern length corresponding toAnd candidate pattern dataImage line pattern corresponding toIf the lengths are different, the base pattern dataImage line pattern corresponding toAnd candidate pattern dataThe image line pattern corresponding tointerpolationTheExecutionDoWardAnd betweenAn interpolated pixel array is obtained by averaging each pixel after enlarging or reducing the same length. Base pattern data without using enlargement / reductionImage line pattern corresponding toAnd candidate pattern dataImage line pattern corresponding toThere is also a method of adjusting the start position / length of the, and is not limited to the above method.
[0058]
  The image data created by the interpolation pixel string calculation unit 152 in FIGS. 10 and 11 is replaced with the interpolation determined by the interpolation execution interval determination unit 151.TheExecutionDoShown in the state of overwriting the section. In this embodiment, on the image lineIt is inMatchAnd the length of the image line pattern corresponding toInterpolation lineIt's aboveinterpolationTheExecutionDoSince the lengths of the sections are expressed in the same way, processing for adjusting the length such as enlargement / reduction is not performed. For example, the base pattern data 42 of the image line EThe image line pattern 42 corresponding toAnd candidate pattern data 43 of the image line GImage line pattern 43 corresponding toSince the image pixel columns are the same, interpolationTheExecutionDosectionIsThe pixel column overwritten in the section 72 on the interpolation line F is the basic pattern data 42.Image line pattern corresponding to42 and candidate pattern data 43Image line pattern 43 corresponding toIs exactly the same.
[0059]
  Through the above processing, an interpolation pixel sequence for each interpolation execution section is set, and the motion information processing unit 154 uses the input motion information to harmonize with another field, and the correction unit 153 determines the pixels around the interpolation line. Interpolation with interpolated pixel array while maintaining harmonyTheExecutionDosectionTheOverwrite or existing interpolationTheExecutionDoInterval pixelsColumnLine interpolation is executed by blending with. However, the motion information processing unit 154 and the correction unit 153 are not necessarily necessary units.
[0060]
  Interpolation on the interpolation lineTheExecutionDoPixels other than the interval are interpolated by a conventional interpolation method. The interpolation method of the prior art is, for example, a method of supplementing an interpolation line with the average value of the image data of the upper and lower input image lines, a method of supplementing the interpolation line with the image data of the upper input line, or an interpolation line with the next field image There are methods.
[0061]
【The invention's effect】
  The following effects are brought about by the present invention described above.
  According to the present invention,Extract pixel rows that have common luminance difference characteristics and continue in the line direction, generate information about the pixel rows, determine pixel rows that can be interpolated between lines from the generated information data, and perform interpolation Between pixel columnsSince interpolation is performed by setting an interpolation pixel row, the processing speed can be greatly reduced. Furthermore, even when there is an image break between lines in a field image, interpolation can be performed with a matching pixel row on an adjacent line, and a smooth frame image without break can be generated.
[0062]
According to the present invention, the pattern information generating means collects information on the pixel columns of the image line pattern and eliminates the calculation in units of pixels, thereby improving the processing speed. In particular, the information collection means collects information of continuous pixel columns as a pattern from the input image, so that the search for matching patterns can be performed without using pixel column calculation, and the interpolation processing speed can be greatly improved.
[0063]
According to the present invention, by performing coring processing, distortion and noise are detected, and inconspicuous parts that do not affect image display and parts that are difficult to judge are deleted, so that the interpolation processing speed is greatly increased while maintaining image quality. Can be improved.
[0064]
According to the present invention, since the position where an interpolable image line pattern exists is predicted, line interpolation without error is possible.
[0065]
According to the present invention, it is determined whether or not the pattern information of the image line pattern matches, and the image line pattern that can be interpolated is determined. Therefore, the arithmetic processing can be executed speedily. In particular, since vector determination is performed on image line patterns determined to match, line interpolation without misperception is possible.
[0066]
According to the present invention, since a pixel row between image line patterns having the same pattern information is set and interpolated, a smooth frame image that is not interrupted even when the image is interrupted between lines, such as a horizontally thin line image. Can be generated.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an embodiment of an image interpolation apparatus according to the present invention.
FIG. 2 is an explanatory diagram of a monitor screen in which a field image of a black thin line image is projected on a white background.
FIG. 3 is an enlarged view of the field image of FIG. 2;
FIG. 4 is an explanatory diagram showing an image line pattern at an image line position.
FIG. 5 is an explanatory diagram for performing interpolation pattern prediction from both ends of an image line pattern.
FIG. 6 is an explanatory diagram for searching for a matching image line pattern.
FIG. 7 is an explanatory diagram showing a vector direction at the end of matching pattern search.
FIG. 8 is a relationship diagram between an image line pattern and an interpolation execution position value when one line interpolation is performed.
FIG. 9 is a relationship diagram between an enlarged frame after one line interpolation and an interpolation execution position.
FIG. 10 is a relationship diagram between an image line pattern over a plurality of lines and an interpolation execution position.
FIG. 11 is a relationship diagram between an enlarged frame after interpolation of a plurality of lines and an interpolation execution position.
FIG. 12 is an enlarged view of a frame after multiple line interpolation.
FIG. 13 is an explanatory diagram showing a positional relationship between an image line and each pixel column on the interpolation line in the image interpolation method according to the related art.
FIG. 14 is an explanatory diagram illustrating processing for obtaining a correlation between a neighboring pixel column and a target pixel column in an image interpolation method according to a conventional technique.
FIG. 15 is an explanatory diagram illustrating processing for obtaining a correlation between another neighboring pixel column and a target pixel column in an image interpolation method according to a conventional technique.
FIG. 16 is an explanatory diagram illustrating interpolation at an edge portion in an image interpolation method according to a conventional technique.
[Explanation of symbols]
100 pattern information generator
101 Two-time difference calculation unit for luminance values in the vertical direction
102 Normalization processing unit
103 Pattern extraction unit
104 Longitudinal correlation detector
105 Information gathering department
110 Coring processor
111 Image distortion / noise detector in pattern
112 Pattern edit determination unit
113 Pattern editing execution unit
120 Interpolation pattern prediction unit
121 Interpolation direction prediction unit
122 Search distance prediction unit
123 Interpolation execution distance prediction unit
130 Image Database
140 Interpolation pattern determination unit
141 Pattern search part
142 Pattern match determination unit
143 Pattern matching condition setting part
144 Interpolation direction determination unit
145 Interpolation execution distance determination unit
146 Vector judgment part
147 Vector judgment condition setting part
150 Interpolation execution unit
151 Interpolation execution section determination unit
152 Interpolated pixel string calculation unit
153 Correction means
154 Motion Information Processing Unit

Claims (6)

In an image interpolating apparatus for sequentially generating an image of a scanning video signal by performing interpolation between image lines for an input field image of an interlace scanning video signal ,
The luminance difference characteristic between the target pixel of the input field image and a pixel adjacent immediately below the target pixel is obtained, and an image line pattern that is a continuous pixel row having a common differential characteristic in the line direction is extracted, and the Pattern information generating means for generating pattern information related to the position of the image line pattern and the luminance difference characteristic ;
And have you between each line, the pattern information relating to the luminance difference characteristic of the image line pattern is taken out for a match, the interpolation pattern determining means for determining an image line pattern used for interpolation between lines,
Interpolation execution means for executing interpolation by setting an interpolation pixel row using the determined image line pattern ,
The pattern information generating means
A twice-difference value calculating means for obtaining a twice-difference value obtained by adding each difference of luminance values between a target pixel of the input field image and a pixel on an adjacent line directly above and below the target pixel;
For each image line, pattern extraction means for recognizing and extracting a continuous pixel sequence having a luminance difference characteristic based on the twice difference value in the line direction as an image line pattern;
Comprising information collecting means for collecting the pattern information of the extracted image line pattern,
The interpolation pattern determining means includes
Matching pattern search means for searching for an image line pattern that may match from the pattern information;
Pattern matching determination means for determining whether the pattern information of the searched image line patterns matches;
Vector determining means for performing a vector determination for determining whether the image line pattern is to be used for interpolation continuously from the start position and length of the image line pattern determined to match,
The interpolation execution means includes
Interpolation execution interval determination means for determining the position and length of the interval for executing interpolation based on the pattern information of the image line pattern determined by the interpolation pattern determination means;
An image interpolation apparatus comprising: interpolation pixel generation means for generating the pixel row to be interpolated based on the image line pattern determined by the interpolation pattern determination means in the determined section . Furthermore, based on the pattern information , comprising an interpolation prediction means for generating prediction information related to the position of the image line pattern that can be used for interpolation ,
The interpolation pattern prediction means includes:
Interpolation direction prediction means for predicting an interpolable direction / non-interpolable direction from the pattern information;
A search distance prediction means for predicting a distance to a position where an image line pattern matching the pattern information exists;
The image interpolation apparatus according to claim 1, further comprising an interpolation execution distance predicting unit that predicts a distance at which the image does not break even when interpolated with the matching image line pattern . The pattern information generation means further comprises normalization means for normalizing the vertical difference value twice for each pixel and classifying the difference values into types indicating a plurality of luminance difference characteristics,
The pattern extraction unit, the image interpolation apparatus according to claim 1 or 2, characterized in that to extract an image line pattern based on the classification indicating the luminance difference characteristic obtained by the normalization means. The pattern information generating means further comprises coring means for performing coring processing to remove the image line pattern when the image line pattern extracted by the pattern extracting means is image distortion or noise,
The coring means is
Detecting means for detecting, as distortion, an image having a plurality of maximum / minimum points in the luminance waveform in the image line pattern and an acute change point in the luminance waveform in the pattern, and detecting a pixel row not reaching the reference pixel number as noise;
Editing determination means for determining whether or not editing for deleting the image line pattern is necessary based on detection data of the detection means;
Image interpolation apparatus according to any one of claims 1 to 3, characterized in that it comprises the editing execution means for executing a deletion editing the image line pattern is determined to require editing. The information collecting means includes
Means for collecting a maximum luminance value in the image line pattern;
Means for collecting a maximum luminance increase amount from the image line pattern start position to a maximum luminance position;
Image interpolation device according to the maximum luminance position to any one of claims 1 to 4, characterized in that it comprises a means for collecting the maximum luminance reduction amount to the image line pattern end position. The information collecting means includes
Means for collecting a minimum luminance value in the image line pattern;
Means for collecting a maximum luminance decrease amount from the image line pattern start position to a minimum luminance position;
Image interpolation apparatus according to any one of claims 1 to 4, characterized in that it comprises a means for collecting the maximum brightness increment until the image line pattern end position from the minimum luminance position.

Images