JP2006308665A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus, capable of attaining excellent image quality by improving image blurring generated when magnifying and pixel disappearance generated when contracting, regarding the image processing apparatus in which an image is displayed in an arbitrary resolution of a user. <P>SOLUTION: The apparatus comprises; an arithmetic device for calculating a pixel value which is interpolated by a three dimensional convolution interpolation method; a line buffer controller for outputting pixel data used for the calculation in a certain pixel group; a sorting device for sequentially sorting the pixel group; an edge detector for detecting an edge of an image from the pixel group; an inverse arithmetic circuit which is characterized by applying compensation to arguments of a coefficient table based on information from the edge detector; and a coefficient table for supplying an arithmetic coefficient from the arguments outputted by the inverse arithmetic circuit. When the edge is detected, compensation is applied to the arguments. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, and more particularly to a device such as a computer or a television that needs to display an image.

  In general, a computer can display a plurality of types of resolutions in accordance with the resolution of a display device. For example, if the maximum resolution of the display device is XGA, the computer is often required to display images of XGA and lower resolutions (VGA, SVGA). This is because it is possible to select whether to display more information on the screen or to display the information larger for easier viewing according to the user's preference.

  At this time, a resolution conversion technique is required for the computer main body to provide a desired resolution to the display device.

  In Japan, the NTSC system has been adopted in Japan. This is 483 to 485 scanning lines, the number of pixels per scanning line is 720 to 760, and the total number of pixels is about 350,000 pixels. However, since it is normally displayed in an interlaced manner, the resolution is actually about 150,000 pixels.

  On the other hand, in recent years, the high-definition method has been widely used. The high-definition method is also called HD and has about 750,000 pixels. Corresponding to this, the conventional NTSC broadcast is called SD.

  Furthermore, there are image formats such as 480i, 720p, and 1080i in digital broadcasting. 480i represents a 640 × 480 interlace, 720p represents a 1024 × 720 progressive, and 1080i represents a 1280 × 1080 interlace.

  There are other uses such as PC image display, game display, digital camera video display, and multi-screen display for displaying multiple screens, and the opportunity to display images in many formats on the TV has increased.

  Here, in the case of analog data such as conventional NTSC broadcasting, it is possible to convert the image format relatively easily by changing the sampling rate. However, in order to convert the format of digital data such as digital broadcasting and digital camera images, it is necessary to perform interpolation and thinning of the image data.

  In addition, in order to provide a display effect such as multi-screen display, it is necessary to perform resolution conversion for enlarging / reducing an image as well as image format conversion.

  The resolution conversion techniques are roughly classified into frequency modulation and time modulation.

  In frequency modulation, image data is once converted into a frequency domain by DCT or the like, and then signal processing for resolution conversion is performed. In order to display an image on the display device, since it cannot be displayed with frequency domain data, it is necessary to reconvert from frequency domain to time domain data at the time of output. For this reason, the circuit scale tends to increase.

  On the other hand, in time modulation, input data is processed in time order. In this method, the pixel value to be interpolated is estimated from the pixel values before and after the pixel value.

  In general, the time modulation can be configured with a simpler circuit than the frequency modulation.

  Typical examples of time modulation include nearest neighbor method, linear interpolation method, and convolutional interpolation method. FIG. 10 is a diagram illustrating a pixel interpolation method of each interpolation method. In each figure, the vertical axis represents pixel values, and the horizontal axis represents original coordinates before resolution conversion. FIG. 10A shows an original image before conversion, and shows an image in which the pixel values at coordinates 0, 1, 3, 4, and 5 are the same and only the pixel value at coordinates 2 is different. FIGS. 10B, 10C, and 10D show how to obtain the pixel value of the interpolation pixel when the resolution of the original coordinates is converted to three times. In this figure, the reference coordinate on the horizontal axis represents the original coordinate so that it can be understood which coordinate value of the original image is referred to. (B) represents the nearest neighbor method, (c) represents a linear interpolation method, and (d) represents a convolutional interpolation method.

  The nearest neighbor method is a method of determining the position of the original pixel closest to the position of the interpolation pixel and taking the value of the closest original pixel. This method can simplify the circuit most among the above three methods. However, since the reference pixel is one pixel, a jagged feeling of the image is likely to occur.

  The linear interpolation method is a method of referring to two pixels before and after an interpolation pixel (four pixels in length and width), connecting the pixel values with a straight line, and setting the value on the straight line as the pixel value of the interpolation pixel. As a result, it is possible to eliminate the jagged feeling of the image that occurs in the nearest neighbor method. However, since only a change between two pixels is observed in this method, it cannot be determined whether the pixel change in that portion is a sudden change or a smooth change. In the linear interpolation method, since the pixel values are interpolated on the assumption that they change smoothly, there is a problem that the image becomes blurred.

  The convolutional interpolation method is a method in which four pixels before and after the interpolation pixel (16 pixels in length and width) are referred to, the pixel values are connected by a curve, and the value on the curve is used as the pixel value of the interpolation pixel. In this method, the pixel value change is reflected in the weighting and the edge can be emphasized, so that the blur of the image can be improved as compared with the linear interpolation method.

  Further, an edge enhancement circuit as shown in FIG. 11 is generally used as a method for improving image blur. In this method, an edge component is detected by adding (−½) of the pixel value before and after the pixel value of the target pixel, and an arbitrary coefficient is applied to the edge component as the target pixel. By adding, the strength of the edge is controlled.

  Further, Patent Document 1 discloses a method for improving image blur by linear interpolation without using convolution interpolation. In this method, before performing the interpolation pixel calculation, the directionality of the edge is detected from four adjacent pixels, and the calculation coefficient is changed to improve the blur of the image.

  Here, the convolution interpolation method will be briefly described.

  The convolution operation is calculated using 16 pixels of the original image and 8 coefficients for each Output 1 pixel.

here,
Pre-conversion coordinates: x, y
Coordinates after conversion: X, Y
Conversion formula: X = A * x, Y = B * y (A: x-direction magnification, B: y-direction magnification)
Then, since the coordinates before conversion (x, y) are actually non-continuous and the coordinates after conversion (X, Y) are continuous,
Inverse conversion formula: x = X / A
y = Y / B (Formula 1)
Is used to obtain the coordinates before conversion.

  The integer part of this inverse transformation is used to extract the pixel value of the original image as the pre-conversion coordinates. The decimal part (remainder) is used for coefficient creation.

For each pixel (X, Y), 16 original image pixel values G are acquired from the original image coordinates (x, y).
y-1 line G ₁₁ G ₂₁ G ₃₁ G ₄₁
y lines G ₁₂ G ₂₂ G ₃₂ G ₄₂
y + 1 line G ₁₃ G ₂₃ G ₃₃ G ₄₃
y + 2 line G ₁₄ G ₂₄ G ₃₄ G ₄₄
x-1 column x column x + 1 column x + 2 column

Further, when the decimal part of the value obtained by the inverse transformation is expressed as x- [x], y- [y], the coefficient f (t) is obtained by the following calculation.

  From the 16 pixel values of these original images and the x, y8 coefficients, the pixel value P of the converted image is obtained by the following calculation.

P: Output pixel value G: Original image pixel value f (t): Coefficient (x, y): Inverse transformation coordinates However, when the convolution interpolation method is used, the circuit scale and the amount of memory used are larger than those of the linear interpolation method. It gets bigger.
JP 2001-8037 A

  However, the above-described method shown in FIG. 11 has a problem in that an emphasis process is applied regardless of the nature of the image, so that an unnatural picture tends to occur in the natural image portion.

  In the method of Patent Document 1, the determination of whether the pixel change, which is a drawback of the linear interpolation method, is abrupt or smoothly changing is made by performing image determination at four neighboring points. This improves the blurring of the image of the vertical edge, the horizontal edge, and the oblique edge with an angle of 45 degrees. However, it is not enough to deal with oblique edges other than 45 degrees, and there is a risk of erroneous detection as a vertical edge or a horizontal edge.

  Also in the convolutional interpolation method, when the pixel change is large, the blurring of the image may be noticeable. 12 is an interpolation pixel when the image of FIG. 12A is enlarged three times by (b) nearest neighbor method, (c) linear interpolation method, and (d) convolution interpolation method, respectively, as in FIG. It is the figure which showed how to take. FIG. 12A is a diagram in the case where the pixel values of the coordinates 0, 1, 2 and the coordinates 3, 4, 5 are the same. In FIG. 12D, pixel values that emphasize edges are taken for interpolated pixels between the reference coordinates 1-2 and 3-4. However, the interpolated pixel between the reference coordinates 2-3 is determined to be a smooth change, and has a value that is not different from the linear interpolation method.

  The above content will be described below using mathematical formulas.

  Substituting the coefficients (f (y1), f (y2), f (y3), f (y4)) = (0, 1, 0, 0) when the magnification in the Y direction is 1 in the above (Equation 3). And expands to

P = G12 * f (x1) + G22 * f (x2) + G32 * f (x3) + G42 * f (x4) (Formula 4)
Here, if G12 = G22 and G32 = G42,
P = G22 × (f (x1) + f (x2)) + G32 × (f (x3) + f (x4)) (Formula 5)
Further, the coefficients when the fraction of the reference coordinate is 0.3 are (f (x1), f (x2), f (x3), f (x4)) = (− 0.1, 0.8, 0.4). , −0.1), P = G22 × 0.7 + G32 × 0.3, and the coefficients when 0.6 are (f (x1), f (x2), f (x3), f (x4)) = (− 0.1, 0.5, 0.7, −0.1), P = G22 × 0.4 + G32 × 0.6. This is almost the same as the calculation when the linear interpolation method is performed. The same result is obtained when G12 = G32 and G22 = G42.

  As a technique for improving the blurring of the image in the convolution interpolation method, a technique of determining whether the pixel block is a natural image or a character image and switching the coefficient table for the convolution calculation can be considered.

  However, in this method, it is necessary to have a plurality of coefficient tables or to perform coefficient calculation each time, which may increase the circuit scale. In the case of G12 = G22 and G32 = G42 in the above example, even if the weights of G12 and G42 are changed, the coefficients multiplied by G22 and G32 are collected. Eventually, there was a case where the result of convolution calculation would not change even if the coefficient table was changed.

  As described above, there are insufficient measures for blurring when a conventional image is enlarged.

  In addition to image blurring during enlargement, there is also a problem of pixel loss during reduction. FIG. 15 is a diagram illustrating an example of pixel disappearance.

  The upper part of FIG. 15A represents the arrangement of pixels before resolution conversion. White circles and black circles represent pixels, and the numbers above the pixels represent addresses. The lower part represents a pixel arrangement when the upper pixel data group is reduced to ½ in the convolution interpolation method and the linear interpolation method. Here, an arrow extending from the upper part to the lower part indicates which original pixel is referenced to derive the converted pixel.

  In both the convolutional interpolation method and the linear interpolation method, first, a reference pixel is derived by inverse operation. That is, when the magnification is 1/2, the inverse calculation is x = X × 2 (x: coordinates before conversion, X: coordinates after conversion), and the reference pixel coordinates are only even values.

  Here, as described above, the convolutional interpolation method refers to the four pixels before and after the reference pixel coordinates. However, since no remainder is generated in the inverse operation, the coefficients (f (x1), f (x2), f (x3), f (x4)) are always (f (x1), f (x2), f ( x3), f (x4)) = (0, 1, 0, 0). When applied to the (Equation 4), the pixel value P after conversion is always P = G22. That is, the same value as that of the linear interpolation method is derived.

  Therefore, the disappearance of the odd-numbered pixels of the original coordinates occurs. FIG. 15A shows that the pixel value of the original coordinate 5 is not reflected in the converted pixel.

  As a means for preventing such pixel disappearance, an average operation method shown in FIG. 15B is often used. The average operation method is a method using an average value of pixels within a certain range. For example, in FIG. 15B, 1/2 conversion of resolution is performed by obtaining an average value of adjacent pixel values such as the original coordinate 1 and the original coordinate 2.

  When the average operation method is used, the disappearance of the pixel value is eliminated. However, there is a problem that the original pixel value is not stored and the image is blurred.

  An object of the present invention is to solve such problems and to provide an image processing apparatus capable of obtaining a high-quality image in enlargement or reduction processing.

  In the present invention, an image processing apparatus that performs image enlargement or reduction processing, an input unit that inputs pixel data, an output unit that outputs a part of the pixel data in a certain pixel group, Rearrangement means for rearranging the pixel group in time series, an edge detector for detecting an edge of an image from the pixel group, a correction circuit for correcting an argument of a coefficient table based on information from the edge detector; Coefficient means for determining a calculation coefficient from an argument output from the correction circuit, and calculation for obtaining interpolated pixel data by calculating pixel data output from the rearrangement device based on a coefficient output from the coefficient means Means.

  According to the present invention, it is possible to solve the problem of blurring of an image at the time of enlargement and obtain an enlarged image with good image quality. It is also effective for the problem of pixel loss that occurs during reduction. That is, it is possible to obtain a reduced image with good image quality with reduced pixel loss and less blur.

  Embodiments of the present invention will be described below.

  First, the concept of interpolation processing in this embodiment will be described.

  In the cubic convolution interpolation, the conversion magnification is inversely converted, the reference coordinates of the original image are calculated from the quotient, and the coefficient is calculated from the remainder. In this case, a method of extracting a coefficient from a lookup table (LUT) using the remainder as an argument is common.

  In this proposal, the edge is detected from the pixel data used for the calculation of the cubic convolution interpolation method, and the value of the argument is changed according to the detected data. FIG. 13 is a diagram showing how to take pixel values when the present method is applied to the interpolation method of FIG. A hatched portion in FIG. 13 represents a range in which correction is performed. In FIG. 12D, the coefficient is determined between the reference coordinates 2-3 by the arguments 0.3 and 0.6, and the result of performing the convolution operation using the coefficient is adopted as the interpolated pixel value. On the other hand, FIG. 13 shows how to take a pixel value when an edge is detected between the reference coordinates 2-3. For example, the correction is performed such that the argument 0.3 is set to 0.2, the argument 0.6 is set to 0.7, and the value of the original pixel is closer. The degree of correction is proportional to the size of the edge. That is, the greater the difference between the original pixel values of the reference coordinates 2 and 3 in FIG. Here, when no edge is detected, such correction is not performed. That is, the correction by edge detection is not performed between the reference coordinates 1-2 and 3-4 in FIG. 13, and the calculation result by the cubic convolution interpolation method is adopted as it is.

  Next, an edge detection method will be described.

  For edge detection, pixel data of 4 × 4 pixels used for the cubic convolution interpolation method is used. A vertical edge and a horizontal edge are detected by judging the arrangement of 4 × 4 pixels, and the edge portion is corrected as described above. For diagonal edges, the edge enhancement effect by the cubic convolution interpolation method is used.

  FIG. 14 is a diagram illustrating a state in which 4 × 4 pixel data is doubled in the X direction and converted into 4 × 8 pixel data. 14A shows a black vertical line on a white background, and FIG. 14B shows a black diagonal line on a white background. 14A and 14B, the upper arrow represents interpolation by the nearest neighbor method, and the lower arrow represents interpolation by linear interpolation or convolution interpolation. As described above, the convolutional interpolation method provides an emphasis effect, so the linear interpolation method and the convolutional interpolation method are not exactly the same, but the same here in the sense that a halftone that does not exist in the original pixel occurs. Treat it as a thing.

  In the nearest neighbor method of FIG. 14A, the black line represented by one pixel is changed to a black line represented by two pixels by double enlargement. On the other hand, in the linear interpolation method and the convolutional interpolation method, a black line of one pixel and a halftone gray line of one pixel sandwiching it are expressed. In the case of such a line, the former is displayed more clearly and clearly, and the latter is displayed more clearly.

  Similarly, when looking at FIG. 14B, the nearest neighbor method gives a jagged feeling, but the linear interpolation and the convolutional interpolation method can smoothly express a portion that cannot be expressed in binary by using halftones. It is possible to display. At this time, as described above, the convolutional interpolation method can produce the edge effect more than the linear interpolation method.

  Therefore, interpolation is performed using a cubic convolution interpolation method at an oblique edge, and correction is performed according to the intensity of the edge at a vertical edge or a horizontal edge. By these operations, higher-quality interpolation processing can be performed.

  Furthermore, it is possible to deal with the problem of pixel disappearance at the time of reduction by a similar method.

  FIG. 16 shows how to take pixel values when 1/2 conversion is performed in each conversion method. 16A shows an original image before conversion, FIG. 16B shows conversion by convolution interpolation method and linear interpolation method, FIG. 16C shows conversion by average operation method, and FIG. 16D shows conversion by the present invention. . The horizontal axis of (b), (c), and (d) represents the original coordinate so that it can be understood which coordinate value of the original image is referenced. Also, the reason for connecting the pixels with straight lines is to make it easy to understand the change of the pixels for the sake of convenience.

  In the conversion of FIG. 16B, only the even coordinates are referred to, so the value of coordinate 5 is not reflected.

  In the conversion of FIG. 16C, the average of the pixel value of the previous coordinate with respect to the reference coordinate is obtained. For example, the value at the reference coordinate 2 is obtained by (original coordinate 1 value + original coordinate 2 value) / 2. For this reason, an intermediate value is output regardless of the coordinate position.

  On the other hand, in the conversion of FIG. 16D according to the present invention, the influence of the coordinate position is also given to the output while leaving the influence of all the pixel values. In this method, in addition to the conversion of (b), a portion where the pixel value change is large is detected, and the portion where the pixel value change is large is corrected according to the amount of change. The correction method at this time is different from that at the time of enlargement. That is, correction is performed so as to be close to the original pixel at the time of enlargement, but correction is performed in a direction close to a pixel that is not normally referred to at the time of reduction, that is, a direction in which an intermediate value is generated.

  An actual conversion state is shown in FIG.

  FIG. 17 is a diagram illustrating how to obtain pixel values in the case where an 8 × 8 pixel data group is halved in the horizontal direction in each method. FIG. 17A shows an original image, which represents a diagonal black line on a white background with 8 × 8 pixel data. FIG. 17 (b) shows the conversion by the convolution interpolation method and the linear interpolation method, FIG. 17 (c) shows the conversion by the average operation method, and FIG.

  In FIG. 17B, since the coordinates referred to are skipped as described above, the black line of the pixel data is also skipped in the converted data. In FIG. 17C, the black lines of the pixel data are all gray. On the other hand, in FIG. 17D, all the black lines are gray, but the change due to the pixel position is represented by shading.

  However, in the case of reduction, there is a case where it cannot be expressed by the above method depending on the reduction ratio.

  FIG. 18 is a diagram illustrating an example of reduction. FIG. 18 shows how to take pixel values when 1/2 conversion is performed in each conversion method. FIG. 18A is an original image before conversion, and shows a state in which the pixel value changes greatly for each pixel. (B) shows the conversion by the convolution interpolation method and the linear interpolation method, and (c) shows the conversion by the average operation method. The horizontal axis of (b) and (c) represents the original coordinate so that it can be understood which coordinate value of the original image is referenced.

  As shown in FIG. 18, when conversion is performed by the convolution interpolation method and the linear interpolation method, an output as shown in (b) is obtained. Here, even if the method of the present invention is used as in the case of FIG. 16, the pixel value is corrected and becomes a little larger, so that it does not change much in appearance. Also, even if another method is used to express an image that changes for each pixel as in the original image, it cannot be expressed accurately because the number of pixels is small in 1/2 reduction.

  In such a case, the average operation method is used as shown in (c). In the present invention, conversion by the average operation method can be performed only by changing the coefficient argument in such a case.

  FIG. 19 is a table showing a part of the coefficient table used in the present invention. This coefficient table is symmetrical with respect to the argument “0.5” except for the arguments “0” and “1.1”.

  When the magnitude of the pixel change exceeds a threshold set to an arbitrary value of the user, the argument is corrected. In the table of FIG. 19, in the case of enlargement, the argument is corrected in a direction away from “0.5”, and in the case of reduction, the argument is corrected in a direction closer to “0.5”. The correction amount at this time is proportional to the amount exceeding the threshold. Also, the argument “1.1” in the table is an argument that does not normally occur in the inverse operation. This argument is an exceptional argument and is not applied by the above correction. This argument is selected in the case as shown in FIG. 18, and a fixed coefficient is selected. In this example, the coefficients are selected so that the influence of the four pixels is almost uniform. However, the balance of the coefficients may be changed according to the intended use. It is also possible to prepare a plurality of arguments for such a special case and select a fixed coefficient corresponding thereto.

[First embodiment]
FIG. 1 is a diagram illustrating the overall configuration of a resolution conversion apparatus according to the present invention.

  Reference numeral 101 denotes an input range setting device, 102 denotes a line buffer control device, 103 denotes an edge detector, 104 denotes an arithmetic device, and 105 denotes an output control device.

  S01 is an input control signal (IHSYNC is an input horizontal sync signal, IVSYNC is an input vertical sync signal, ISX is a horizontal start address, IEX is a horizontal end address, ISY is a vertical start address, and IEY is a vertical end address). , S02 is input image data, S03 is arithmetic device input data, S04 is edge detection data, S05 is arithmetic device output data, S06 is output image data, S07 is an IACT signal designating an effective input range, and S08 is horizontal (X). / Vertical (Y) magnification setting data, S09 is an address designation signal (XQUOT, YQUOT) for designating an address of desired data from the arithmetic unit, S10 is an output range designation signal (OSX is a horizontal start address, OEX is a horizontal direction) End address, OSY is vertical start address , OEy the vertical end address), S11 is an output control signal (OHSYNC output horizontal sync signal, OVSYNC output vertical sync signal, OACT represents the output scope specification signal).

  The input control signal S01 is input to the input range setting device 101, and the input range of the image is set as desired by the user.

  Based on the input control signal S01, the input range setting device 101 generates an IACT signal S07 that goes high in the effective range of the input image.

  The IACT signal S07 and the input image data S02 are input to the line buffer controller 102. Here, in FIG. 1, the input image data S02 is represented by YCbCr, but RGB or YUV is not affected in the present invention.

The line buffer control device 102 has a plurality of line buffers and holds a plurality of lines of input image data S02. The line buffer control device 102 performs (X _n−1 , Y _n−1 ) for the specified address (X _n , Y _n ) in accordance with the instruction of the address specification signal (XQUOT, YQUOT) S09 generated by the arithmetic unit 104. _{), (X n-1,} Y n), (X n-1, Y n + 1), (X n-1, Y n + 2), (X n, Y n-1), (X n, Y n), _{(X n, Y n + 1} ), (X n, Y n + 2), (X n + 1, Y n-1), (X n + 1, Y n), (X n + 1, Y n + 1), (X n + 1, Y n + 2), _{(X n + 2, Y n} -1), (X n + 2, Y n), (X n + 2, Y n + 1), as _{(X n + 2, Y n} + 2), the arithmetic unit input data S03 a total of 16 pixels of data in the address Send it out.

  The edge detector 103 extracts edge information from the arithmetic device input data S03 and sends the edge detection data S04 to the arithmetic device 104.

  The arithmetic unit 104 performs interpolation data calculation in resolution conversion from the edge detection data S04 from the edge detector 103, the arithmetic unit input data S03 from the line buffer control unit 102, and the horizontal and vertical magnification setting data S08. The computed data is sent to the output control device 105 as computing device output data S05. Further, the address designation signal (XQUOT, YQUOT) S09 is generated and sent to the line buffer controller 102.

  The output control device 105 performs timing adjustment so that image data is generated within the effective output range in accordance with the output range designation signal (OSX, OEX, OSY, OEY) S10. The timing-adjusted image data is output as output image data S06. In addition, the time required for the calculation is adjusted, and an output control signal (OVSYNC, OHSYNC, OACT) S11 is also generated and output so that the input / output timing matches.

  FIG. 2 is a diagram illustrating an internal configuration example of the line buffer control device 102.

  201 is a line controller that controls input / output line data of the line buffer controller 102, 202 is an address controller that controls the address of each line in the X direction, and 203 is an enable signal control that controls writing / reading to the line buffer. Reference numeral 204 denotes a line buffer block including five line buffers (LINE_BUF1, LINE_BUF2, LINE_BUF3, LINE_BUF4, and LINE_BUF5), and 205 denotes a rearrangement device for rearranging data.

  The line control device 201 counts effective input lines based on unillustrated input control signals (IVSYNC, IHSYNC) S01 and IACT signal S07. Further, a comparison is made with the designated address in the Y direction required by the YQUOT signal, and if the condition is met, a line enable signal is sent to the enable signal control device 203.

  The address control device 202 counts the number of effective bits in the X direction based on an input control signal (IVSYNC, IHSYNC) S01 (not shown) and the IACT signal S07. Further, comparison is made with a designated address in the X direction required by the XQUOT signal, and if the condition is met, a bit enable signal is sent to the enable signal controller 203.

  The enable signal control device 203 arbitrates the line enable signal sent from the line control device 201 and the bit enable signal sent from the address control device 202, and includes five line buffers included in the line buffer block 204. A write enable signal for instructing which line buffer to write to and a read enable signal for instructing from which line buffer to read out are transmitted.

  The line buffer block 204 includes five line buffers, and writes / reads image data in accordance with instructions from the enable signal control device 203. For example, if the write enable signal and the read enable signal are expressed in binary form of 5 bits, LINE_BUF1 is selected for 00001b, LINE_BUF2 for 00001b, LINE_BUF3 for 0100b, LINE_BUF4 for 01000b, and LINE_BUF5 for 10000b. Shall. The write enable signal changes for each IHSYNC in the order of 00001b → 00010b → 00100b → 01000b → 10000b → 00001b →..., And input image data for each line is stored in each line buffer. On the other hand, the read enable signal changes from 11110b → 11101b → 11011b → 10111b → 01111b → 11110b →... So that the data in the address in the specified X direction is simultaneously read from the four lines where writing is not performed. It has become.

  The rearrangement device 205 rearranges the image data sent from the line buffer block 204 in order of old data for each line. In addition, data in the X direction is always held for four pixels, and image data for 4 × 4 = 16 pixels is collectively transmitted for each clock.

  FIG. 3 is a diagram illustrating an example of the internal configuration of the arithmetic device 104.

  301 is a reverse operation circuit, 302 is a correction circuit, 303 is a coefficient table, 304 is a timing adjustment circuit, 305 is a first Y direction convolution circuit, 306 is a second Y direction convolution circuit, and 307 is a third Y direction convolution. The circuit, 308 represents a fourth Y-direction convolution circuit, and 309 represents an X-direction convolution circuit.

  The inverse calculation circuit 301 receives the X-direction and Y-direction magnification setting data S08 and performs reverse calculation of the resolution conversion magnification. As a result, the quotient of the inverse operation is sent to the line buffer controller 102 as an address designation signal (XQUOT, YQUOT) S09. The remainder of the inverse operation is sent to the coefficient table 303 as a coefficient argument (XREMD, YREMD).

  The correction circuit 302 receives edge detection data S04 generated by the edge detector 103. The correction circuit 302 generates correction data based on the strength of the edge and sends it to the inverse operation circuit 301. The inverse operation circuit 301 changes coefficient arguments (XREMD, YREMD) according to the correction data.

  The coefficient table 303 sends coefficient data corresponding to the value from the argument (XREMD, YREMD) of the coefficient sent from the inverse operation circuit 301.

  The timing adjustment circuit 304 adjusts the calculation timing for the coefficient in the X direction.

The first Y-direction convolution circuit 305 by using the Y coefficients sent from the coefficient table _{303, (XQUOT, YQUOT) =} (X n, Y n) _{to, (X n-1, Y} n-1) , (X _n−1 , Y _n ), (X _n−1 , Y _{n + 1} ), and (X _n−1 , Y _{n + 2} ) are subjected to a convolution operation in the Y direction.

Similarly, the second Y-direction convolution circuit 306 includes (X _n , Y _n−1 ), (X _n , Y _n ), (X _n , Y _{n + 1} ), and (X _n , Y _{n + 2} ) four pixels. The third Y-direction convolution circuit 307 includes four pixels (X _{n + 1} , Y _n−1 ), (X _{n + 1} , Y _n ), (X _{n + 1} , Y _{n + 1} ), and (X _{n + 1} , Y _{n + 2} ). , fourth Y-direction convolution circuit _308, the four pixels _{(X n + 2, Y n} -1), (X n + 2, Y n), (X n + 2, Y n + 1), (X n + 2, Y n + 2), respectively Perform a convolution operation in the Y direction.

  The X direction convolution circuit 309 performs a convolution operation in the X direction on the output data of the first to fourth Y direction convolution circuits (305 to 308) using the X coefficient whose timing is adjusted by the timing adjustment circuit 304. The output of the X direction convolution circuit 309 is output as the arithmetic unit output data S05.

  FIG. 4 is a diagram showing an example of the internal configuration of the output control apparatus.

  Reference numeral 401 denotes a VSYNC adjustment circuit, 402 denotes an HSYNC adjustment circuit, 403 denotes an output ACT generation circuit, 404 denotes an output control circuit, 405 denotes a line buffer block, and 406 denotes a switching circuit.

  The VSYNC adjustment circuit 401 adjusts the timing of the output vertical synchronization signal (OVSYNC). In this circuit configuration, since a plurality of lines are held in a line buffer and an operation is performed, an operation delay occurs between input and output data. The VSYNC adjustment circuit 401 adjusts the calculation delay.

  FIG. 5 is a diagram for explaining the adjustment state of the operation delay.

  Input data is input in synchronization with the input vertical synchronization signal (IVSYNC). Here, the blanking period of the input data is ISY.

  Next, the output data can be output at a time delayed from the input data by an operation delay. Assuming that the blanking period of the output data is OSY, OVSYNC is a signal having the same period and delayed by V_delay compared to IVSYNC. At this time, V_delay can be expressed as “V_delay = operation delay + ISY−OSY”.

  The HSYNC adjustment circuit 402 adjusts an output horizontal synchronization signal (OHSYNC).

  FIG. 6 is a diagram showing an example of output timing when 1/2 reduction is performed in the Y direction.

  In 1/2 operation in the Y direction, output data for one line is obtained with two input lines. However, if the input horizontal synchronization signal (IHSYNC) is used as it is for output, a line without image data is generated. There are two possible methods for dealing with this. One is a method of changing the cycle of the output horizontal synchronization signal (OHSYNC) in the HSYNC adjustment circuit 402. The other is a method in which an output ACT signal (OACT) that goes high is simultaneously output to the effective data portion, and the receiver of the image data after resolution conversion determines the presence or absence of the image data and performs processing. OHSYNC in FIG. 6 represents an output example when the former method is adopted.

  The output ACT generation circuit 403 is a circuit that generates the OACT signal shown in FIG. The output ACT generation circuit 403 generates an OACT signal from the above-described OVSYNC, OHSYNC, and output range designation signal S10.

  The output control circuit 404 generates a line buffer read / write signal from the OACT signal.

  The line buffer block 405 includes two line buffers (LINE_BUF1, LINE_BUF2), and image data is read and written alternately. For example, when the arithmetic device output data S05 is written in LINE_BUF1, reading is performed from LINE_BUF2. The line buffer read / write is switched for each OHSYNC, and the next OHSYNC is written to LINE_BUF2, and the image data written at the previous OHSYNC is read from LINE_BUF2.

  The switching circuit 406 determines from which line buffer the data is read, and outputs the read data as output image data S06.

  Next, the operation of the edge detector 103 will be described.

  The edge detector 103 determines the edge component of the interpolation pixel from the 4 × 4 pixel data used for the calculation of the convolutional interpolation method.

  FIG. 7 is a diagram showing reference pixels used for the cubic convolution operation. (1), (2), (3), and (4) represent the arrangement of pixels in the X direction, and I, II, III, and IV represent the arrangement of pixels in the Y direction. These pixels are represented as I (1), I (2),..., IV (4), the coefficients in the X direction are (AX, BX, CX, DX), and the coefficients in the Y direction are (AY, BY, When expressed as (CY, DY), the output pixel value P obtained by the cubic convolution operation is expressed by the following equation.

P = (I (1) × AY + II (1) × BY + III (1) × CY + IV (1) × DY) × AX
+ (I (2) × AY + II (2) × BY + III (2) × CY + IV (2) × DY) × BX
+ (I (3) × AY + II (3) × BY + III (3) × CY + IV (3) × DY) × CX
+ (I (4) × AY + II (4) × BY + III (4) × CY + IV (4) × DY) × DX (formula 6)
Here, the X-direction coefficients (AX, BX, CX, DX) and the Y-direction coefficients (AY, BY, CY, DY) are extracted from the LUT by arguments XREMD and YREMD, respectively.

  For example, the edge detector 103 generates a detection signal when the following conditions are detected.

Vertical edge condition | II (2) -II (3) |> TX and II (2) = III (2) and II (3) = III (3)
-Lateral edge condition | II (2) -III (2) |> TY and II (2) = II (3) and III (2) = III (3)
Here, TX and TY are thresholds that can be set arbitrarily by the user. In addition, the accuracy of the edge can be increased by setting the determination conditions as follows.

・ Vertical edge condition | II (2) -II (3) |> TX
and I (2) = II (2) = III (2) = IV (2) and I (3) = II (3) = III (3) = IV (3)
・ Horizontal edge condition | II (2) -III (2) |> TY
and II (1) = II (2) = II (3) = II (4) and III (1) = III (2) = III (3) = III (4)
The correction circuit 302 in FIG. 3 described above determines the argument XREMD according to the presence / absence of the detected edge and the intensity of the edge represented by | II (2) -II (3) | or | II (2) -III (2) | , YREMD correction is applied.

[Second Embodiment]
FIG. 8 is a diagram showing the configuration of the second embodiment of the present plan. The second embodiment is a modification of the configuration of the first embodiment. As described below, it is possible to reduce the circuit scale by combining the convolution operation circuits in the Y direction.

  Reference numeral 801 denotes an input range setting device, 802 denotes an arithmetic control device, and 803 denotes an output control device. As signal lines, S81 is an input control signal (IHSYNC is an input horizontal sync signal, IVSYNC is an input vertical sync signal, ISX is a horizontal start address, IEX is a horizontal end address, ISY is a vertical start address, and IEY is vertical. Direction end address), S82 is an IACT signal designating an input effective range, S83 is horizontal (X) / vertical (Y) magnification setting data, S84 is input image data, S85 is arithmetic device output data, and S86 is an output range designation. Signal (OSX is horizontal start address, OEX is horizontal end address, OSY is vertical start address, OEY is vertical end address), S87 is output image data, S88 is output control signal (OHSYNC is output horizontal sync signal, OVSYNC is the output vertical sync signal, OACT is the output Scope specification signal) represents.

  The input control signal S81 is input to the input range setting device 801, and the input range of the image is set as desired by the user.

  The input range setting device 801 generates an IACT signal S82 that goes high in the effective range of the input image based on the input control signal S81.

  The IACT signal S82, the input image data S84, and the magnification setting data S83 are input to the arithmetic control device 802.

  The arithmetic control device 802 performs resolution conversion calculation based on the magnification setting data S83. As a result, the input signal S84 is interpolated and output as arithmetic device output data S85.

  The arithmetic device output data S85 is input to the output control device 803. The output control device 803 performs timing adjustment so that image data is generated within the effective output range in accordance with the output range designation signal (OSX, OEX, OSY, OEY) S86. The timing-adjusted image data is output as output image data S87. In addition, the time required for the calculation is adjusted, and an output control signal (OVSYNC, OHSYNC, OACT) S88 is also generated and output so that the input / output timing matches.

  FIG. 9 is a block diagram illustrating the inside of the arithmetic and control unit 802 in FIG.

  901 is an inverse operation circuit, 902 is an enable signal generator, 903 is a line buffer block, 904 is a rearranger, 905 is a Y-direction edge detector, 906 is a Y-direction coefficient table, 907 is a Y-direction convolution circuit, and 908 is a register Block 909 represents an X direction edge detector, 910 represents an X direction coefficient table, 911 represents a timing adjustment circuit, and 912 represents an X direction convolution circuit.

  The inverse operation circuit 901 performs an inverse operation from the magnification setting data described above and the count value of an internal counter (not shown) to generate an address designation signal (XQUOT, YQOT) and a coefficient table argument (XREMD, YREMD). XQUOT and YQUOT are sent to the enable signal generator 902, XREMD is sent to the X direction coefficient table 910, and YREMD is sent to the Y direction coefficient table 906, respectively.

  The enable signal generation device 902 includes an address control device, a line control device, and an enable signal control device. XQUOT is input to the address controller and generates an address enable signal. YQUOT is input to the line control device to generate a line enable signal. The address enable signal and the line enable signal are input to the enable signal control device to output a write enable signal / read enable signal.

  The write enable signal / read enable signal is input to the line buffer block 903. The line buffer block 903 includes five line buffers. The write enable signal controls which line buffer the image data is written to, and the read enable signal controls from which line buffer the image data is read. In this way, after the input image data is once written in the line buffer block 903, necessary data is read as appropriate according to the read timing of the read enable signal.

  Image data read from the line buffer block 903 is input to the rearrangement circuit 904. The rearrangement circuit 904 rearranges and outputs a plurality of data simultaneously read from the line buffer block 903 in time series.

  The data output from the rearrangement circuit 904 is input to the Y direction edge detector 905 and the Y direction convolution circuit 907.

  The Y-direction edge detector 905 detects the presence or absence of an edge and the intensity of the edge from the input data for four pixels. As an edge detection method, for example, detection is performed by the following method. The difference between the second and third data in time series order is compared, and it is determined whether the difference is larger or smaller than the first threshold set by the user. Further, the difference between the first and second data is detected, and the difference between the second and fourth data is detected and compared with a second threshold value set arbitrarily by the user. Information on the difference between the first threshold value and the second threshold value generated in this way is sent to the Y-direction coefficient table 906.

  A Y-direction coefficient table 906 selects a Y-direction coefficient based on YREMD sent from the inverse operation circuit 901. At this time, if there is information on the difference from the first threshold value from the Y-direction edge detector 905, the coefficient is corrected according to the value. In the coefficient correction, the YREMD value is corrected in a direction away from “0.5” in the enlargement mode, and the YREMD value is corrected in a direction closer to “0.5” in the reduction mode. If there is information on the difference from the second threshold, a fixed coefficient for averaging four points is selected. The coefficient thus determined is sent to the Y-direction convolution circuit 907.

  A Y-direction convolution circuit 907 performs a convolution operation using image data and a coefficient in the Y direction. The image data that has been convolved in the Y direction is sent to the register block 908.

  Register block 908 consists of four registers. The image data sent from the Y-direction convolution circuit 907 is first input to REG1. Next, when new data is input to REG1, the data held in REG1 until then is input to REG2. Thereafter, input to REG3 and REG4 is performed in order. The data sending timing at this time is controlled by the timing of the convolution calculation in the X direction. Further, the image data held in each register is read at the timing of the convolution operation.

  The read image data is input to the X direction edge detector 909 and the X direction convolution circuit 912.

  The X-direction edge detector 909 sends information on the difference from the first threshold and the difference from the second threshold to the X-direction coefficient table 910 as in the case of the Y direction.

  The X direction coefficient table 910 selects a coefficient in the X direction based on XREMD sent from the inverse operation circuit 901. At this time, as in the Y direction, a coefficient is selected from the X direction edge detector 909 based on the difference between the first threshold value and the second threshold value. The coefficient thus determined is sent to the X-direction convolution circuit 912.

  The X direction convolution circuit 912 performs a convolution operation using the image data and the X direction coefficient. The data calculated in this way is output as arithmetic device output data.

[Third embodiment]
FIG. 20 is a diagram showing how to take pixel values when this method is applied by the linear interpolation method.

  As shown in FIG. 20, the effect of the edge portion can be similarly expressed in the linear interpolation method. In the case of linear interpolation, since the coefficients are simpler than those of the cubic convolution interpolation method, it is possible to calculate the coefficients by calculation without using the coefficient table. In that case, the reciprocal of the magnification that is the basis of the calculation is corrected.

The block diagram of an example of the whole structure in 1st Example of this invention Internal block diagram of line buffer controller Internal block diagram of computing device Output control device internal block diagram Diagram explaining the calculation delay adjustment status The figure explaining the output timing when 1/2 reduction is performed in the Y direction A diagram showing reference pixels used for the cubic convolution operation The block diagram of an example of the whole structure in 2nd Example of this invention Internal block diagram of arithmetic control unit Diagram explaining how to take pixel values in each conversion method Diagram illustrating an example of a general edge enhancement circuit Diagram explaining how to take pixel values in each conversion method The figure explaining how to take the pixel value at the time of expansion in the present invention A diagram explaining how to take pixel values at the time of enlargement A diagram explaining an example of pixel loss The figure explaining how to take the pixel value at the time of reduction in each conversion method Diagram explaining how to take pixel values at the time of reduction The figure explaining how to take the pixel value at the time of reduction in each conversion method Table showing a part of the coefficient table used in the present invention Diagram showing how to take pixel values when this method is applied by the linear interpolation method

Explanation of symbols

101 Input Range Setting Device 102 Line Buffer Control Device 103 Edge Detector 104 Arithmetic Device 105 Output Control Device S01 Input Control Signal S02 Input Image Data S03 Arithmetic Device Input Data S04 Edge Detection Data S05 Arithmetic Device Output Data S06 Output Image Data S07 Input Valid IATC signal S08 for designating a range Horizontal (X) / vertical (Y) magnification setting data S09 Address designating signal S10 for designating the address of desired data from the arithmetic unit Output range designating signal S11 Output control signal

Claims (6)

An image processing apparatus that executes image enlargement or reduction processing,
An input means for inputting pixel data;
Output means for collectively outputting a part of the pixel data to a certain pixel group;
Rearrangement means for rearranging the pixel groups in time series;
An edge detector for detecting an edge of an image from the pixel group;
A correction circuit for correcting an argument of a coefficient table based on information from the edge detector;
Coefficient means for determining a calculation coefficient from an argument output from the correction circuit;
An image processing apparatus comprising: an operation unit that obtains interpolated pixel data by calculating pixel data output from the rearrangement device based on a coefficient output from the coefficient unit.   The image processing apparatus according to claim 1, wherein the calculation unit performs a calculation by a cubic convolution interpolation method.   The image processing apparatus according to claim 1, wherein the correction circuit performs different corrections for enlargement and reduction.   The image processing apparatus according to claim 1, wherein the correction circuit refers to a difference amount of edge detection and changes the correction amount according to the difference amount.   The edge detector detects a vertical edge or a horizontal edge from difference information between a pixel data value of a pixel of interest which is an integer value component and a pixel adjacent to the pixel of interest, and comparison information between pixel values of the next line. The image processing apparatus according to claim 4, wherein correction information that changes in accordance with difference information between adjacent pixels is output.   In the case of the reduction process, the edge detector performs, for each pixel, the difference information between the pixel data value of the pixel of interest that is an integer value component and the pixel adjacent to the pixel of interest, and the comparison information between the pixel values on both sides thereof. The high-frequency component of a changing image is detected and special correction information is output, the correction circuit outputs a special argument from the special correction information, and the coefficient means outputs a fixed coefficient from the special argument. Item 5. The image processing apparatus according to Item 4.