JP2009157449A - Image processing system, image processing method, and program for image processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To clear an outline of an enlarged image to improve the quality of an image even when the enlarged image is deteriorated due to a block noise and a color mixture. <P>SOLUTION: A representative color prediction means 802 extracts a block of pixels centered by the pixels of an interpolation image, performs clustering of the colors of the pixels in the bloc to a first class and to a second class, and derives a probability that the color of each pixel in the block belongs to each class, and a representative color of each class. When a distance between two representative colors is longer than a threshold, a color correction means 803 replaces the color of the pixel to the representative color of the first class if a condition that the probability that the color of the pixel belongs to the first class is stronger than the probability that the color belongs to the second class is satisfied, and replaces the color of the pixel to the representative color of the second class if the condition is not satisfied for each pixel in the block extracted by the prediction means 802. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing system, an image processing method, an image processing program, an image compression device, an image expansion device, and an image transmission system, and more particularly to an image processing system, an image processing method, and an image processing for improving the quality of an enlarged image. And an image compression apparatus, an image expansion apparatus and an image transmission system to which the image processing system is applied.

  Various methods for enlarging images have been proposed. A particularly common method is a method of performing image filtering after image upsampling (or simultaneously with uplink). Bilinear interpolation and bicubic interpolation are known as methods for enlarging an image by obtaining the luminance of the pixel to be increased by interpolation (see, for example, Non-Patent Document 1). FIG. 27 is an example of an image obtained by up-sampling a low-resolution image at a magnification of 2 times in the vertical and horizontal directions. FIG. 28 is an example of an image obtained by interpolating the luminance of the pixels in the enlarged image by bicubic interpolation.

  These methods tend to blur the enlarged image. This is because high-frequency components of the image are lost as the image is enlarged. Therefore, it is common to apply contour enhancement processing to compensate for the lost high-frequency component and restore the original contour.

  An example of an image processing apparatus for contour enhancement is described in Patent Document 1. FIG. 29 is a block diagram illustrating an image processing apparatus described in Patent Document 1. As shown in FIG. 29, the image processing apparatus described in Patent Document 1 includes an analysis circuit 920 including a contour component extraction circuit 901, a signal level detection circuit 902, and a contour detection circuit 904, a parameter calculation circuit 903, a contour And a correction circuit 905.

  The analysis circuit 920 receives the video signal Si and the input source determination signal Ssel. The input source determination signal Ssel is a signal indicating the type of the input video signal Si. For example, when the input video signal Si is a video signal based on the HDTV (High Definition Television) system, the input source determination signal Ssel is H level (high level). In the case of a video signal based on the NTSC (National Television System Committee) system, it becomes L level (low level).

  In the image processing apparatus, the contour component extraction circuit 901 is a component corresponding to the contour in the image indicated by the input video signal Si (“edge component” or “contour signal”) in accordance with the input source determination signal Ssel. Sa) is extracted from the input video signal Si. The contour component extraction circuit 901 is composed of a high-pass filter, and extracts, for example, a secondary differential component of the input video signal Si using a secondary differential filter and outputs it as a contour component Sa.

  The signal level detection circuit 902 detects the signal level Lb of the input video signal Si and outputs a signal having the signal level Lb as a signal value.

  The parameter calculation circuit 903 receives the first parameter αi, the second parameter βi, and the third parameter Di from the outside, and suppresses over-enhancement of the contour based on these external parameters αi, βi, Di. An overemphasis correction threshold value α as a first threshold value, a fine component enhancement threshold value β as a second threshold value for emphasizing the contour of a fine image, and a contour detection threshold value D as a third threshold value for contour detection Is calculated.

  The contour detection circuit 904 compares the value obtained by calculating the change amount of the input video signal Si with the contour detection threshold D, and outputs a contour detection signal Sc indicating the presence or absence of the contour based on the comparison result.

  The contour correction circuit 905 includes a contour component Sa output from the contour component extraction circuit 901, a contour detection signal Sc output from the contour detection circuit 904, an overemphasis correction threshold α and a fine component output from the parameter calculation circuit 903. Using the emphasis threshold β and a coefficient γ given as an external gain adjustment parameter, signal processing for contour emphasis is performed on the input video signal Si, and an output video signal So is generated as a result of this signal processing.

  Non-Patent Document 1 describes an EM algorithm (Expectation Maximization algorithm).

JP 2002-290773 A (paragraphs 0041-0043, FIG. 1) Supervised by Mikio Takagi and Yoshihisa Shimoda, “New Edition Image Analysis Handbook”, first edition, The University of Tokyo Press, September 10, 2004, p. 728-732, p. 1362-1367

  There is a problem that even if contour enhancement is applied to an enlarged image, the target contour cannot be restored. This is because the shape of the contour changes around the contour due to image quality degradation such as block noise accompanying image compression and a mixture of colors adjacent to the contour. FIG. 30 is an image showing an example of a result of applying contour enhancement to an enlarged image by bicubic interpolation. As illustrated in FIG. 30, even when the contour emphasis process is performed, the image does not have a sufficiently clear contour.

  The present invention provides an image processing system, an image processing method, and an image processing method for clarifying an outline in an enlarged image and improving the quality of the image even when the enlarged image is deteriorated due to block noise or color mixing. It is an object of the present invention to provide a program and an image compression apparatus, an image expansion apparatus, and an image transmission system to which the image processing system is applied.

  The image processing system of the present invention selects a pixel in an interpolated image that is an enlarged image including pixels interpolated between pixels of an image before enlargement, and extracts a block of pixels centered on the selected pixel And clustering the colors of the pixels in the block into a first class and a second class, and the probability that the color of each pixel in the block belongs to the first class and the probability of belonging to the second class, Representative color estimation means for deriving the representative colors of the first class and the second class, and when the distance between the representative colors of the first class and the second class is greater than or equal to a threshold, When the condition that the probability that the color of the pixel belongs to the first class is higher than the probability of the second class is satisfied for each individual pixel in the block extracted by the estimation unit, the color of the pixel is 1 class representative If the condition is not satisfied, the color of the pixel is replaced with the representative color of the second class, and when one pixel belongs to a plurality of blocks, the pixel is derived for each block. And color correction means for setting the average of the replaced colors to the color of the pixel.

  The image compression apparatus of the present invention includes a downsampling unit that generates a low resolution image having a lower resolution than the input image by downsampling the input image, and an enlarged image generation unit that expands the low resolution image. Image subtracting means for calculating difference data that is a difference between pixel values of corresponding pixels of the input image and the low-resolution image, and the enlarged image generating means inserts pixels between pixels of the enlargement target image. Interpolating image generating means for generating an interpolated image obtained by enlarging the image to be enlarged by inserting, selecting a pixel in the interpolated image, extracting a block of pixels centered on the selected pixel, Are clustered into a first class and a second class, and the probability that the color of each pixel in the block belongs to the first class and the second class. The representative color estimation means for deriving the probability of belonging to the first class and the representative colors of the first class and the second class, and the distance between the representative colors of the first class and the second class is greater than or equal to a threshold value When the condition that the probability that the color of the pixel belongs to the first class is higher than the probability of the second class is satisfied for each individual pixel in the block extracted by the representative color estimation unit, the pixel If the color is replaced with the representative color of the first class and the condition is not satisfied, the color of the pixel is replaced with the representative color of the second class, and one pixel belongs to a plurality of blocks And color correction means for setting the average color after replacement derived for the pixel for each block to the color of the pixel.

  The image expansion device of the present invention receives a low-resolution image obtained by reducing the resolution of the original image, and difference data that is a difference between pixel values of corresponding pixels of the original image and the low-resolution image. An image expansion device that expands a low-resolution image, comprising: an enlarged image generating unit that expands the low-resolution image; and a sum of pixel values of pixels corresponding to the image enlarged by the enlarged image generating unit and the difference data Interpolating image generating means for calculating, interpolating image generating means for generating an interpolated image obtained by enlarging the enlarging target image by interpolating pixels between pixels of the enlarging target image; and Select a pixel in the interpolated image, extract a block of pixels centered on the selected pixel, cluster the colors of the pixels in the block into a first class and a second class, Representative color estimation means for deriving the probability that the color of each pixel belongs to the first class, the probability of belonging to the second class, and the representative colors of the first class and the second class; When the distance between the representative colors of the first class and the second class is equal to or greater than the threshold, the probability that the color of the pixel belongs to the first class is the first for each individual pixel in the block extracted by the representative color estimation unit. If the condition of higher than the probability of belonging to the second class is satisfied, the color of the pixel is replaced with the representative color of the first class, and if the condition is not satisfied, the color of the pixel is changed to the second class. And a color correction unit that uses the average of the replaced colors derived for each pixel for each block when the pixel belongs to a plurality of blocks. It is characterized by that.

  The image transmission system of the present invention further includes an image compression device and an image expansion device, and the image compression device generates a low-resolution image having a lower resolution than the input image by down-sampling the input image. Downsampling means for transmitting the low resolution image to the image expansion device, first enlarged image generating means for enlarging the low resolution image, and pixel values of corresponding pixels of the input image and the low resolution image Second subtracting the low-resolution image received from the image compressing device, the image subtracting means for calculating the difference data as a difference between the image compressing device and transmitting the difference data to the image decompressing device. Of the enlarged image generating means, the image enlarged by the second enlarged image generating means, and the sum of the pixel values of corresponding pixels of the difference data received from the image compression apparatus An image adding means for calculating, and the first enlarged image generating means and the second enlarged image generating means enlarge the enlargement target image by interpolating pixels between pixels of the enlargement target image. An interpolated image generating means for generating the interpolated image, a pixel in the interpolated image is selected, a block of pixels centered on the selected pixel is extracted, and the color of the pixel in the block is set to the first class and the first Clustering into two classes, the probability that the color of each pixel in the block belongs to the first class, the probability of belonging to the second class, the representative colors of the first class and the second class, When the distance between the representative color estimation means for deriving and the representative colors of the first class and the second class is greater than or equal to a threshold value, for each individual pixel in the block extracted by the representative color estimation means, The color is first If the condition that the probability of belonging to the class is higher than the probability of belonging to the second class is satisfied, the color of the pixel is replaced with a representative color of the first class, and if the condition is not satisfied, the color of the pixel Is replaced with the representative color of the second class, and when one pixel belongs to a plurality of blocks, the average of the replaced colors derived for the pixel for each block is used as the color of the pixel. Color correction means.

  The image processing method of the present invention selects a pixel in an interpolated image, which is an enlarged image including pixels interpolated between pixels of the image before enlargement, and a block of pixels centered on the selected pixel. Are extracted, and the colors of pixels in the block are clustered into a first class and a second class, and the probability that each pixel color in the block belongs to the first class and the second class. A representative color estimating step for deriving a probability and representative colors of the first class and the second class, and a distance between the representative colors of the first class and the second class is equal to or greater than a threshold value, When the condition that the probability that the color of the pixel belongs to the first class is higher than the probability of the second class is satisfied for each individual pixel in the block extracted in the representative color estimation step, the color of the pixel is The first If the condition is not satisfied, the pixel color is replaced with the second class representative color, and when one pixel belongs to a plurality of blocks, the pixel is replaced for each block. And a color correction step using an average of the colors after replacement derived for the pixel as the color of the pixel.

  Also, the image processing program of the present invention selects a pixel in an interpolated image, which is an enlarged image including pixels interpolated between pixels of the image before enlargement, on the computer, and the selected pixel is the center. A block of pixels is extracted, and the colors of the pixels in the block are clustered into a first class and a second class, and the probability that the color of each pixel in the block belongs to the first class and the second class A representative color estimation process for deriving the probability of belonging to a class and the representative colors of the first class and the second class, and the distance between the representative colors of the first class and the second class is greater than or equal to a threshold value When the condition that the probability that the color of the pixel belongs to the first class is higher than the probability of the second class is satisfied for each individual pixel in the block extracted by the representative color estimation process, the pixel When a color is replaced with a representative color of the first class and the condition is not satisfied, a color of the pixel is replaced with a representative color of the second class, and one pixel belongs to a plurality of blocks, It is characterized in that a color correction process is performed in which the average of the color after replacement derived for the pixel for each block is the color of the pixel.

  According to the present invention, even if the enlarged image is deteriorated due to block noise or color mixing, the contour in the enlarged image can be clarified and the quality of the image can be improved.

  Embodiments of the present invention will be described below with reference to the drawings.

Embodiment 1. FIG.
FIG. 1 is a block diagram showing an example of an image processing system according to the first embodiment of the present invention. The image processing system according to the first embodiment is realized by, for example, a computer 100 that operates under program control. The computer 100 is a computer including a central processing unit (a processor or a data processing device may be used).

  A computer 100 serving as an image processing system includes a brightness gradient calculating unit 102, a contour extracting unit 103, and a contour correcting unit 104.

  An image (hereinafter referred to as an interpolated image) Sa enlarged by interpolation processing (for example, bilinear interpolation or bicubic interpolation) Sa is input to the luminance gradient calculating unit 102. The interpolated image Sa is an enlarged image including pixels interpolated between pixels of the image before enlargement. The luminance gradient calculation means 102 calculates the intensity of the luminance gradient (hereinafter referred to as luminance gradient intensity) and the direction of the luminance gradient (hereinafter referred to as luminance gradient direction) for each pixel of the interpolation image Sa. Here, the case where the enlargement ratio from the low resolution image (image before enlargement) to the image after enlargement is constant will be described as an example.

  The luminance gradient is a value representing how much the luminance of a pixel changes with respect to a pixel near the pixel. For example, it is the amount of change in luminance of the pixel when the coordinate changes by one. The larger the luminance gradient, the larger the amount of change in luminance. The luminance gradient is calculated for each of the x-axis direction (horizontal direction) and the y-axis direction (vertical direction). FIG. 2 is an explanatory diagram showing luminance gradient strength and luminance gradient direction. In FIG. 2, the luminance gradient intensity and the luminance gradient direction will be described by focusing on one pixel 71 in the image, but the luminance gradient intensity and the luminance gradient direction are calculated in the same manner for the other pixels. A value representing the amount of change in luminance in the horizontal direction in one pixel 71 is the luminance gradient in the x-axis direction in the pixel 71. A vector 72 shown in FIG. 2 is a luminance gradient in the x-axis direction in the pixel 71 and is a vector facing the x-axis direction. Hereinafter, the vector 72 is referred to as an x-axis direction luminance gradient vector. A value representing the amount of change in luminance in the vertical direction in the pixel 71 is the luminance gradient in the y-axis direction in the pixel 71. A vector 73 illustrated in FIG. 2 is a luminance gradient in the y-axis direction in the pixel 71 and is a vector facing the y-axis direction. Hereinafter, this vector 73 is referred to as a y-axis direction luminance gradient vector. The luminance gradient strength in the pixel 71 is the magnitude of the combined vector 74 (see FIG. 2) of the x-axis direction luminance gradient vector 72 and the y-axis direction luminance gradient vector 73. In addition, the luminance gradient direction in the pixel 71 is an angle formed by a reference direction (specifically, the x-axis direction) and the direction of the combined vector 74 described above. In FIG. 2, the luminance gradient direction is represented as “θ”. The luminance gradient calculating means 102 calculates the luminance gradient intensity (the magnitude of the combined vector 74) and the luminance gradient direction (θ) for each pixel of the interpolation image Sa.

  The contour extraction unit 103 determines a pixel representing the original contour before deterioration using the luminance gradient strength and luminance gradient direction in each pixel. For example, the contour extracting unit 103 uses the luminance of the adjacent pixels in the luminance gradient direction and the luminance of the adjacent pixels in the opposite direction around the pixel (hereinafter referred to as the contour determination target pixel) for determining whether or not the pixel represents the contour. Compare gradient strength. Then, the contour extraction unit 103 sets the contour determination target pixel on condition that the luminance gradient strength of the contour determination target pixel is larger than a threshold value (contour extraction threshold value) and larger than the luminance gradient strength of the two adjacent pixels. Are determined to be pixels on the contour (pixels representing the contour).

  The contour correcting unit 104 estimates representative colors representing the two regions in contact with the contour, and corrects the color around the contour based on the representative color. The contour correcting unit 104 identifies a pixel (denoted as a first isolated pixel) that is a fixed distance away from a pixel representing the contour (denoted as a reference pixel) in the luminance gradient direction of the reference pixel. The contour correcting unit 104 sets the color of the first isolated pixel as the first representative color, and corrects each pixel existing on the straight line connecting the reference pixel and the first isolated pixel to the first representative color. Also, a pixel (referred to as a second isolated pixel) that is separated from the pixel representing the contour (reference pixel) by a certain distance in the direction in which the luminance gradient direction of the reference pixel is rotated by 180 ° is specified. The contour correcting unit 104 sets the color of the second isolated pixel as the second representative color, and the first representative among the pixels existing on the straight line connecting the first isolated pixel and the second isolated pixel. A pixel that has not been corrected to a color is corrected to a second representative color. The contour correcting unit 104 performs this process for each pixel representing the contour. Here, it is assumed that the predetermined distance is determined in advance.

  The contour correction unit 104 corrects the color of the pixels in the vicinity of the pixels on the contour, thereby obtaining a target high-quality enlarged image.

  The brightness gradient calculating unit 102, the contour extracting unit 103, and the contour correcting unit 104 are realized by, for example, a central processing unit (CPU) that operates according to a program. That is, the central processing unit reads an image processing program from a storage device (not shown) included in the computer 100 and operates as a luminance gradient calculating unit 102, contour extracting unit 103, and contour correcting unit 104 in accordance with the image processing program. May be. Further, the brightness gradient calculating unit 102, the contour extracting unit 103, and the contour correcting unit 104 may be realized as separate circuits.

Next, the operation will be described in detail.
FIG. 3 is a flowchart illustrating an example of processing progress of the first embodiment. An interpolation image Sa (for example, an image illustrated in FIG. 28) enlarged by the interpolation processing is input to the image processing system. The image processing system corrects the color (specifically, pixel value) of pixels around the contour portion in the interpolation image Sa, and generates a high-quality image with a clear contour.

  In the present embodiment, the interpolated image Sa is input to the brightness gradient calculating unit 102 and the contour correcting unit 104. When the interpolation image Sa is input, the luminance gradient calculating unit 102 calculates the luminance gradient intensity and the luminance gradient direction for each pixel of the interpolated image Sa, and the contour extracting unit 103 calculates the luminance gradient intensity and the luminance gradient direction of each pixel. (Step S11). At this time, the luminance gradient calculating unit 102 also outputs the luminance gradient direction of each pixel to the contour correcting unit 104. Hereinafter, the luminance gradient strength is denoted as Sb, and the luminance gradient direction is denoted as Sc. In addition, the coordinates may be designated to represent the luminance gradient strength Sb and the luminance gradient direction Sc. That is, the luminance gradient intensity at the coordinates (x, y) may be denoted as Sb (x, y), and the luminance gradient direction Sc at the coordinates (x, y) may be denoted as Sc (x, y).

  In step S11, first, the luminance gradient calculating unit 102 specifies a luminance value from the pixel value for each individual pixel. The pixel value is a value set for each pixel of the input image. For example, when the pixel is expressed in RGB format, the pixel value of the pixel is a value representing the density of each of R, G, and B. It is. The luminance value is a value that represents the magnitude of the luminance. For example, assume that pixels are represented in RGB format. In this case, the luminance gradient calculating unit 102 may calculate the luminance value of each pixel by calculating the following formula (1) for each pixel.

Y (x, y) = 0.299.R (x, y) + 0.587.G (x, y) + 0.114.B (x, y)
Formula (1)

  In equation (1), Y (x, y) is the luminance value of the pixel at coordinates (x, y). Further, R (x, y), G (x, y), and B (x, y) are R, G, and B pixel values in the pixel at coordinates (x, y).

  In step S11, the luminance gradient calculation unit 102 calculates the luminance gradient intensity and the luminance gradient direction from the luminance value of each pixel. An example of processing in which the luminance gradient calculating unit 102 calculates the luminance gradient intensity and the luminance gradient direction will be described. As a calculation process of the luminance gradient intensity and the luminance gradient direction, for example, there is a method using an edge detection operator. In the method using the edge detection operator, a predetermined value (coefficient) arranged in n rows and n columns is associated with pixels in the n rows and n columns included in the image, and the corresponding coefficients and luminance values of the pixels are obtained. The luminance gradient is calculated by calculating the product of and. The coefficient used for this calculation is called an edge detection operator coefficient. Examples of edge detection operators include a Sobel filter, a Robinson operator, a Prewitt operator, and the like. In the following, the case of calculating the brightness gradient using the Sobel filter coefficient is exemplified, but the processing in the case of using other edge detection operator coefficients (such as the Robinson operator coefficient and the Prewitt operator coefficient) is also the same. is there.

  The Sobel filter coefficients are predetermined as an array of n rows and n columns in which a predetermined number is arranged, such as 3 rows, 3 columns, 5 rows, 5 columns, 7 rows, 7 columns, and the like. Here, n is an odd number. In addition, two types of coefficients are used as coefficients for the Sobel filter: coefficients used for calculating the luminance gradient in the horizontal direction and coefficients used for calculating the luminance gradient in the vertical direction. FIG. 4 is an explanatory diagram illustrating an example of coefficients of the Sobel filter. Hereinafter, a case where the coefficients of the Sobel filter are arranged in a number of 3 rows and 3 columns will be described as an example. FIG. 4A shows coefficients of a Sobel filter (hereinafter referred to as a horizontal Sobel filter) used for calculating a luminance gradient in the horizontal direction (x-axis direction). FIG. 4B shows coefficients of a Sobel filter (hereinafter referred to as a vertical Sobel filter) used for calculating a luminance gradient in the vertical direction (y-axis direction).

  The luminance gradient calculating unit 102 convolves a horizontal direction Sobel filter with each luminance value of the interpolated image Sa to generate a horizontal luminance gradient image (denoted as Dh). In addition, the luminance gradient calculating unit 102 convolves the vertical direction Sobel filter with the interpolation image Sa to generate a vertical luminance gradient image (denoted as Dv). The luminance gradient image Dh in the horizontal direction is an image in which the luminance gradient in the horizontal (x-axis) direction of the corresponding pixel in the interpolation image Sa is assigned to each pixel. The luminance gradient image Dv in the vertical direction is an image in which the luminance gradient in the vertical (y-axis) direction of the corresponding pixel in the interpolation image Sa is assigned to each pixel.

  Further, the convolution of the Sobel filter is to identify a plurality of pixels having the same arrangement as the Sobel filter coefficient centered on the pixel for which the luminance gradient is to be calculated, and for each pixel, the corresponding Sobel filter coefficient and Is a process of calculating the product of and calculating the sum of the products. In this example, since the Sobel filter coefficients are arranged in a number of 3 rows and 3 columns, the luminance gradient calculation unit 102 calculates pixels of 3 rows and 3 columns centering on the luminance gradient calculation target pixel. Identify. FIG. 5 is an image diagram showing the luminance value of each pixel in n rows and n columns (3 rows and 3 columns) with the luminance gradient calculation target pixel as the center. When the luminance gradient calculating unit 102 specifies nine pixels having the luminance values 1 to t illustrated in FIG. 5, the luminance gradient calculating unit 102 calculates the product of the luminance value and the corresponding Sobel filter coefficient for each of the plurality of pixels. . The calculation target pixel located at the center of the identified pixel of n rows and n columns corresponds to the coefficient arranged at the center among the coefficients of the Sobel filter of n rows and n columns. In addition, when the position of the pixel based on the calculation target pixel is the same position as the coefficient position based on the coefficient arranged at the center among the coefficients of the Sobel filter, the pixel and the coefficient correspond to each other. It shall be. For example, the pixel having the luminance value q illustrated in FIG. 5 corresponds to the coefficient “2” of the Sobel filter illustrated in FIG. The luminance gradient calculation means 102 calculates the product of the coefficient of the corresponding Sobel filter and the luminance value of the pixel, and obtains the sum.

  As shown in FIGS. 4A and 4B, since there are two types of coefficients of the Sobel filter in the horizontal direction and the vertical direction, the luminance gradient calculating means 102 uses the horizontal direction Sobel filter and the vertical direction Sobel filter, respectively. Perform the above calculation. It is assumed that the individual coefficients included in the horizontal Sobel filter coefficient are represented as Sobelh (i, j). i represents a column in the coefficient sequence, and takes a value of -1, 0, 1 here. Further, j represents a row in the coefficient sequence, and takes values of -1, 0, 1 here. The luminance value of the pixel at the coordinates (x, y) in the interpolated image Sa is represented as Sa (x, y). At this time, the luminance gradient calculating means 102 performs the calculation of the following equation (2) as the above-described product and sum calculation, thereby obtaining the horizontal luminance gradient (Dh ( x, y))).

  In addition, it is assumed that individual coefficients included in the vertical Sobel filter coefficients are represented as Sobelv (i, j). At this time, the luminance gradient calculating means 102 performs the following equation (3) as the above-mentioned product and sum calculation, thereby obtaining the vertical luminance gradient (Dv (Dv ()) in the pixel at the coordinate (x, y). x, y))).

  The luminance gradient calculating unit 102 calculates the luminance gradient in the horizontal direction and the vertical direction for each pixel in the interpolation image Sa. Further, the luminance gradient calculating unit 102 calculates the luminance gradient intensity and the luminance gradient direction for each pixel using the luminance gradient in the horizontal direction and the vertical direction.

  The luminance gradient strength Sb (x, y) at the coordinates (x, y) is the magnitude of the combined vector of the x-axis direction luminance gradient vector 72 and the y-axis direction luminance gradient vector 73 (see FIG. 2). The luminance gradient calculating means 102 calculates the magnitude of this vector (luminance gradient strength Sb (x, y)). The luminance gradient calculating unit 102 may calculate the luminance gradient strength Sb (x, y) by calculating the following equation (4).

  Sb (x, y) = abs (Dh (x, y)) + abs (Dv (x, y)) Equation (4)

  Expression (4) is an approximate expression for obtaining the magnitude of the combined vector. In addition, the luminance gradient calculating unit 102 may calculate the luminance gradient intensity Sb (x, y) by calculation of the following equation (5) instead of the approximate equation of equation (4).

Sb (x, y) = √ (Dh (x, y) ² + Dv (x, y) ² ) (5)

  Moreover, the brightness | luminance gradient calculation means 102 should just obtain | require the brightness | luminance gradient direction Sc (x, y) of a coordinate (x, y) by calculation of the following formula | equation (6).

Sc (x, y) = tan ⁻¹ (Dv (x, y) / Dh (x, y)) Equation (6)

  Note that the luminance gradient calculating unit 102 may quantize the calculated luminance gradient strength Sb and luminance gradient direction Sc in an appropriate step. In other words, the luminance gradient calculating unit 102 may convert the calculated luminance gradient intensity and luminance gradient direction so as to represent them with a smaller number of bits. For example, when the calculated luminance gradient strength is represented by 8 bits, it may be converted so as to be represented by 3 bits. When converting 8 bits into 3 bits, the luminance gradient intensity represented by 8 bits is divided by 16 and rounded off after the decimal point.

  Next, the contour extracting unit 103 represents the contour in the interpolation image Sa using the luminance gradient strength Sb, luminance gradient direction Sc of each pixel calculated by the luminance gradient calculating unit 102, and the contour extraction threshold value Pc. A pixel is determined (step S12). Here, it is assumed that the contour extraction threshold value Pc is predetermined. FIG. 6 is an explanatory diagram illustrating an example of a process for determining a pixel representing an outline. A pixel 300 illustrated in FIG. 6 is a contour determination target pixel. A direction 301 illustrated in FIG. 6 indicates a luminance gradient direction (Sc (x, y)) of the contour determination target pixel. A direction 302 illustrated in FIG. 6 indicates a direction (Sc (x, y) + π) obtained by rotating the luminance gradient direction 302 of the contour determination target pixel by π radians (180 °). The contour extraction unit 103 determines the brightness gradient strength of the neighboring pixels 303 in the brightness gradient direction 301 of the contour determination target pixel and the brightness gradient direction 301 as the brightness gradient strength of the contour determination target pixel 300 is larger than the contour extraction threshold value Pc. When it is larger than any of the luminance gradient intensities of the adjacent pixels 304 in the direction 302 rotated by 180 °, it is determined that the contour determination target pixel 300 is a pixel representing a contour. On the other hand, when the brightness gradient intensity of the contour determination target pixel 300 is equal to or less than the contour extraction threshold value Pc, it is determined that the contour determination target pixel 300 is not a pixel representing a contour. Further, the brightness gradient strength of the contour determination target pixel 300 is the brightness gradient strength of the adjacent pixel 303 in the brightness gradient direction 301 of the contour determination target pixel and the brightness gradient of the adjacent pixel 304 in the direction 302 obtained by rotating the brightness gradient direction 301 by 180 °. Even when the condition of greater than any of the intensities is not satisfied, the contour determination target pixel 300 is determined not to be a pixel representing a contour. The contour extraction unit 103 performs this determination for each pixel. Note that the pixel determination processing method for the contour is not limited to the above-described method, and the contour extraction unit 103 may determine the pixel on the contour by another method.

  The contour extraction unit 103 stores a pixel value other than 0 for the pixel determined to represent the contour, and stores an image (denoted as a contour image Sd) in which the pixel value “0” is stored for the other pixels. Output to the contour correction means 104. Based on the contour image Sd, the contour correcting unit 104 can distinguish between pixels determined to represent the contour and other pixels. The contour extracting unit 103 may output the coordinate value of the pixel determined to represent the contour to the contour correcting unit 104. The contour extracting unit 103 may notify the contour extracting unit 103 of the pixel determined to represent the contour so that the pixel determined to represent the contour can be distinguished from other pixels.

  After step S12, the contour correcting unit 104 determines a representative color that represents each of the two regions in contact with the contour, and corrects pixel values around the contour in the interpolated image Sa (step S13). FIG. 7 is an explanatory diagram illustrating an example of correction processing for pixels around the contour. FIG. 7A shows an interpolated image before correction, and FIG. 7B shows an interpolated image after correction. Pixels indicated by diagonal lines in FIG. 7A are pixels that are determined to represent an outline, and a series of these pixels is the outline of the image. In step S <b> 13, the contour correcting unit 104 selects one pixel (reference pixel) determined to represent the contour by the contour extracting unit 103. A pixel 400 in FIG. 7 represents a reference pixel. A direction 401 illustrated in FIG. 7 indicates a luminance gradient direction (Sc (x, y)) of the reference pixel 400. Reference numeral 403 shown in FIG. 7 indicates a direction (Sc (x, y) + π) obtained by rotating the luminance gradient direction 401 of the reference pixel by π radians (180 °).

  The contour correcting unit 104 identifies the first isolated pixel 402 that is separated from the reference pixel 400 by a certain distance in the luminance gradient direction of the reference pixel 400. The contour correcting unit 104 sets the color of the first isolated pixel 402 in the interpolation image Sa as the first representative color. The contour correcting unit 104 corrects the color of each pixel existing on the straight line connecting the reference pixel 400 and the first isolated pixel 402 to the same color as the first representative color. That is, the pixel value of each pixel existing on the straight line connecting the reference pixel 400 and the first isolation pixel 402 is replaced with the pixel value of the first isolation pixel 402. For example, when each pixel of the interpolated image Sa is represented in RGB format, it may be replaced with the R, G, and B values of the first isolated pixel 402. The contour correction unit 104 also replaces the pixel value of the reference pixel 400 with the pixel value of the first isolated pixel 402.

  Similarly, the contour correcting unit 104 identifies a second isolated pixel 404 that is a fixed distance away from the reference pixel 400 in a direction 403 (see FIG. 7A, a direction obtained by inverting the luminance gradient direction 401 by 180 °). The contour correcting unit 104 sets the color of the second isolated pixel 404 in the interpolation image Sa as the second representative color. Then, the contour correcting unit 104 is a pixel that is not replaced with the pixel value of the first isolation pixel 402 among the pixels existing on the straight line connecting the second isolation pixel 404 and the first isolation pixel 402. Replace the value with the pixel value of the second isolated pixel 404.

  As a result of this processing, as shown in FIG. 7B, the reference pixel 400 and the pixel 405 are corrected to the first representative color, and the pixel 407 is corrected to the second representative color. Here, the case where the pixel value of the reference pixel 400 is replaced with the pixel value of the first isolated pixel 402 is shown, but the pixel value of the reference pixel 400 may be replaced with the pixel value of the second isolated pixel 404. .

  The contour correcting unit 104 sequentially selects all the pixels determined to represent the contour by the contour extracting unit 103, repeats the above processing, and corrects pixel values around the contour. Then, the corrected image is output. An example of an enlarged image generated by the correction processing by the contour correcting unit 104 is shown in FIG. As shown in FIG. 8, a high-quality image with a clear outline is obtained.

Next, the effect of this embodiment will be described.
In the present embodiment, the contour extracting unit 103 identifies a pixel on the contour of the image, and the contour correcting unit 104 determines two pixels between the first isolated pixel and the second isolated pixel sandwiching the pixel therebetween. The pixels are corrected so as to be divided into two representative colors. Therefore, a high-quality enlarged image with a clear outline can be generated. Further, since the pixels separated from the reference pixel by a certain distance are specified as the first isolated pixel and the second isolated pixel, a color that is not affected by the color mixture can be used as a representative color, and a high-quality enlarged image Can be generated.

  Next, a modification of the first embodiment will be described. In the first embodiment, the case where the Sobel filter coefficient is fixed in n rows and n columns (for example, 3 rows and 3 columns) and the contour extraction threshold value Pc is a predetermined constant has been described as an example. In addition, the fixed distance used when specifying the first isolation pixel 402 and the second isolation pixel 404 (see FIG. 7A) in step S13 has been described as being predetermined. In the modification of the first embodiment described below, these values are variable. FIG. 9 is a block diagram illustrating a modification of the first embodiment. Constituent elements similar to those already described are denoted by the same reference numerals as in FIG. As shown in FIG. 9, the image processing system may include a parameter setting unit 110 in addition to the luminance gradient calculation unit 102, the contour extraction unit 103, and the contour correction unit 104. Although FIG. 9 shows a case where the parameter setting unit 110 is provided separately from the computer 100, the computer 100 may include the parameter setting unit 110. The parameter setting unit 110 may be realized by a central processing unit that operates according to a program. That is, the central processing unit may read the image processing program and operate as the parameter setting unit 110, the brightness gradient calculating unit 102, the contour extracting unit 103, and the contour correcting unit 104 in accordance with the image processing program.

  In the modification of the first embodiment, the number of coefficients arranged in n rows and n columns as the Sobel filter coefficients is variable. That is, “n” in n rows and n columns is variable, and Sobel filter coefficients such as 3 rows, 3 columns, and 5 rows and 5 columns can be selected. The parameter setting unit 110 calculates a parameter for determining how many rows and columns of coefficients to select as the Sobel filter coefficient by the luminance gradient calculating unit 102 and outputs the parameter to the luminance gradient calculating unit 102. Hereinafter, this parameter is referred to as a luminance gradient calculation parameter and is represented as “Pb”. The brightness gradient calculation means 102 holds in advance the coefficients of the Sobel filter corresponding to the number of rows such as 3 rows, 3 columns, 5 rows, 5 columns, 7 rows and 7 columns, and the brightness gradient calculation parameter Pb. The Sobel filter coefficients for the number of rows and the number of columns are selected, and step S11 is performed.

  Further, the parameter setting unit 110 calculates a contour extraction threshold value Pc and outputs it to the contour extraction unit 103. The contour extraction unit 103 performs step S12 using the contour extraction threshold value Pc calculated by the parameter setting unit 110.

  In step S13, the parameter setting unit 110 calculates a distance for determining the first and second isolated pixels from the reference pixel, and outputs the distance to the contour correcting unit 104. This distance is referred to as a contour correction parameter and is represented as “Pd”. When the first and second isolated pixels are determined in step S13, the contour correcting unit 104 sets the first isolated pixel and the second isolated pixel as pixels separated from the reference pixel by the distance indicated by the contour correction parameter Pd. Determine.

  Here, the meanings of the values of “n (n indicating the number of columns and rows)”, the contour extraction threshold value Pc, and the contour correction parameter Pd, which represent the size of the Sobel filter coefficients, will be described.

  The coefficients of the Sobel filter are arranged in a number of n rows and n columns. When n is small, a large luminance gradient is calculated with respect to the undulation of luminance within a narrow range. In other words, when n is reduced, even when there is a slight luminance undulation within a narrow range, it becomes easy to determine a pixel at a location where the luminance is maximum within that range as a pixel representing an outline. On the other hand, when n is large, a large luminance gradient is not calculated for the undulation of the luminance within a narrow range. That is, when n is increased, a slight luminance undulation within a narrow range is difficult to detect as a contour, and a pixel at a point where the luminance becomes maximum when there is a luminance undulation within a wide range is defined as a pixel representing the contour. It becomes easy to judge. If n is too small, noise is easily detected as a contour.

  The contour extraction threshold value Pc represents the lower limit of the luminance gradient strength to be determined as the contour. If the contour extraction threshold value Pc is too small, it is easy to erroneously determine that the pixel represents a contour even if the luminance gradient strength is increased due to noise. On the other hand, if the contour extraction threshold value Pc is too large, erroneous determination with noise as a contour is eliminated, but it is easy to erroneously determine that a pixel representing a contour does not represent a contour.

  FIG. 10 is an image diagram showing the undulation of the luminance gradient for each pixel. A pixel corresponding to the maximum point of the luminance gradient shown in FIG. 10 is a pixel representing an outline. “N” determined in accordance with the luminance gradient calculation parameter Pb is a value that determines how much the undulation of the luminance gradient intensity is to be emphasized within a wide range. The contour extraction threshold value Pc is a value that determines how large the luminance gradient intensity is to represent a pixel representing the contour.

  FIG. 11 is an image diagram showing changes in pixel values in the vicinity of the contour before and after image enlargement. FIG. 11A shows changes in pixel values near the contour before image enlargement, and FIG. 11B shows changes in pixel values near the contour after image enlargement. As shown in FIG. 11B, by enlarging the image, the change in the pixel value around the contour becomes moderate. From the state shown in FIG. 11B, the contour correcting unit 104 corrects pixels within a certain distance in the luminance gradient direction from the reference pixel to the first representative color, and converts pixels within a certain distance in the opposite direction to the second pixel. The representative color is corrected (see FIG. 11C). As a result, as shown in FIG. 11C, the first representative color and the second representative color are divided from the contour as a boundary, the luminance change becomes steep, and the contour is clarified. The contour correction parameter Pd represents the distance from the reference pixel.

  Further, as described in the first embodiment, the interpolation image Sa is input to the luminance gradient calculating unit 102 and the contour correcting unit 104. In the present modification, an enlargement ratio (referred to as Pa) from the low resolution image (image before enlargement) to the interpolated image Sa is input to the parameter setting means 110. The parameter setting unit 110 calculates the brightness gradient calculation parameter Pb, the contour extraction threshold value Pc, and the contour correction parameter Pd from the input enlargement factor Pa.

  When the enlargement factor Pa from the low-resolution image to the interpolated image Sa is unchanged, the parameter setting unit 110 stores the enlargement factor Pa as a constant, and the brightness gradient calculation parameter Pb, The contour extraction threshold value Pc and the contour correction parameter Pd may be calculated.

  Note that Pa is a value represented by Wh / Wl, where W1 is the horizontal width (number of pixels in the horizontal direction) of the low-resolution image (image before enlargement) and Wh is the horizontal width of the interpolated image Sa. Further, a value obtained by dividing the vertical width of the interpolation image Sa by the vertical width of the low-resolution image may be used as the enlargement ratio Pa.

  FIG. 12 is a flowchart illustrating the processing progress of the modification of the first embodiment. The same processes as those described in the first embodiment are denoted by the same reference numerals as those in FIG. When the enlargement factor Pa is input, the parameter setting unit 110 calculates Pb, Pc, and Pd using the enlargement factor Pa (step S1).

  In step S <b> 1, the parameter setting unit 110 calculates the following equation (7), calculates a luminance gradient calculation parameter Pb, and outputs the Pb to the luminance gradient calculation unit 102.

  Pb = α · Pa + β Formula (7)

  α and β are constants. In addition, α is a value larger than 0, and as Pa increases, Pb also increases. Β is a value representing the lower limit of Pb. By defining β as the lower limit value of Pb, it is guaranteed that the value of Pb is equal to or higher than the lower limit value even if the value of the enlargement factor Pa is small. α and β are set in advance by, for example, an administrator of the image processing system. An administrator of the image processing system may determine α and β so that a Sobel filter coefficient corresponding to the enlargement ratio is selected.

  Further, the parameter setting unit 110 calculates the expression (8) in step S <b> 1, calculates the contour extraction threshold value Pc, and outputs the Pc to the contour extraction unit 103.

  Pc = γ · Pa Formula (8)

  γ is a constant and is set in advance by, for example, an administrator of the image processing system. An administrator of the image processing system may determine γ so that the relationship between the enlargement ratio and the contour extraction threshold value Pc is a desired proportional relationship.

  In step S1, the parameter setting unit 110 calculates Equation (9), calculates a contour correction parameter Pd, and outputs the Pd to the contour correction unit 104.

  Pd = δ · Pa Formula (9)

  δ is a constant and is set in advance by an administrator of the image processing system, for example. An administrator of the image processing system may determine δ so that the relationship between the enlargement ratio and the contour correction parameter Pd is a desired proportional relationship.

  Pb, Pc, and Pd are calculated as linear functions with the enlargement factor Pa as a variable, for example.

  Examples of the above α, β, γ, δ include values of α = 2.0, β = 1.0, γ = 10.0, δ = 1.5, etc., but are not limited to these values. I don't mean.

  Note that a part of Pb, Pc, and Pd may be determined in advance as a fixed value, and the parameter setting unit 110 may calculate a parameter that is not determined as a fixed value among Pb, Pc, and Pd in step S1. .

  After the parameter setting unit 110 obtains Pb, Pc, and Pd in step S1, the image processing system performs steps S11 to S13.

  The brightness gradient calculating means 102 holds in advance a rule such as “If Pb is greater than or equal to x1 and less than x2, the coefficient of the Sobel filter of 3 rows and 3 columns is selected”, and the parameter setting means 110 follows the rule. A Sobel filter coefficient corresponding to the calculated luminance gradient calculation parameter Pb is selected. Then, using the coefficient of the selected Sobel filter, the luminance gradient strength and the luminance gradient direction are calculated for each pixel of the interpolation image Sa, and the luminance gradient strength and the luminance gradient direction of each pixel are output to the contour extracting means 103 ( Step S11). Other than the selection of the Sobel filter coefficient, it is the same as step S11 already described.

  The contour extracting unit 103 determines a pixel representing the contour in the interpolation image Sa using the contour extraction threshold value Pc calculated by the parameter setting unit 110 (step S12). Except for the point that the parameter setting means 110 determines the contour extraction threshold value Pc, it is the same as step S12 already described.

  The contour correcting unit 104 uses the contour correction parameter Pd calculated by the parameter setting unit 110 as a distance for determining the first and second isolated pixels from the reference pixel, and corrects the pixel value around the contour (step S13). ). The parameter setting unit 110 is the same as step S13 described above except that the parameter setting unit 110 determines the distance for determining the first and second isolated pixels from the reference pixel.

  According to the modification of the first embodiment, even when the enlargement ratio from the low resolution image to the interpolated image Sa is not constant, the contour can be clarified with appropriate parameters according to the enlargement ratio. it can.

  Next, another modification of the first embodiment will be described. FIG. 13 is a block diagram illustrating another modification of the first embodiment. Constituent elements similar to those already described are denoted by the same reference numerals as those in FIGS. As shown in FIG. 13, the image processing system may include an interpolated image generating unit 101 in addition to the luminance gradient calculating unit 102, the contour extracting unit 103, the contour correcting unit 104, and the parameter setting unit 110. FIG. 13 shows a case where the computer 100 includes the interpolation image generation unit 101. The interpolated image generating means 101 may be realized by a central processing unit that operates according to a program. That is, the central processing unit reads an image processing program and operates as an interpolated image generating unit 101, a luminance gradient calculating unit 102, a contour extracting unit 103, a contour correcting unit 104, and a parameter setting unit 110 in accordance with the image processing program. Also good.

  A low resolution image (denoted as Si) is input to the interpolated image generating means 101, and the interpolated image generating means 101 enlarges the low resolution image Si by interpolation to generate an interpolated image Sa, and the interpolated image Sa is converted into a luminance gradient. It outputs to the calculation means 102 and the outline correction means 104. That is, the interpolated image generating means 101 generates an interpolated image Sa that is an enlarged image by interpolating pixels between the pixels of the low resolution image Si. The interpolation image generation unit 101 enlarges the low resolution image Si by, for example, bilinear interpolation or bicubic interpolation.

  In addition, the resolution of the enlarged image to be generated is input to the parameter setting unit 110. For example, the width of the enlarged interpolation image Sa (the number of pixels in the horizontal direction) may be input. In addition, information about the low-resolution image Si before enlargement (for example, the width of the low-resolution image Si) is also input to the parameter setting unit 110. The parameter setting unit 110 calculates the enlargement factor Pa using the input information and outputs it to the interpolation image generation unit 101.

  It is assumed that information input to the parameter setting unit 110 is the lateral width (number of pixels in the horizontal direction) Wh of the enlarged interpolation image Sa and the lateral width Wl of the low-resolution image Si. In this case, the parameter setting unit 110 may calculate Pa = Wh / Wl to obtain the enlargement factor Pa. The information input to the parameter setting unit 110 may be the vertical width (vertical pixel nest) Vh of the interpolated image Sa and the vertical width Vl of the low-resolution image Si. In this case, the parameter setting unit 110 may calculate Pa = Vh / Vl to obtain the enlargement factor Pa.

  The parameter setting means 110 may calculate each parameter of Pb, Pc, and Pd from the enlargement factor Pa as described above after calculating the enlargement factor Pa.

  The interpolated image generating means 101 generates an interpolated image Sa by interpolating the low resolution image Si with the enlargement factor Pa calculated by the parameter setting means 110. The interpolated image generating means 101 interpolates pixels between adjacent pixels in the low resolution image Si according to the enlargement factor Pa. The interpolated image Sa is an image enlarged from the low-resolution image Si by including the interpolated pixels. The interpolated image generating unit 101 may increase the number of pixels to be interpolated as the enlargement ratio Pa is larger.

  FIG. 14 is a flowchart showing the processing progress of this modification. The same processes as those described in the first embodiment are denoted by the same reference numerals as those in FIG. For example, the horizontal width Wl of the low-resolution image Si and the horizontal width Wh of the desired interpolation image Sa of the user are input to the parameter setting unit 110. When Wl and Wh are input, parameter setting means 110 calculates enlargement factor Pa by calculating Wh / Wl. However, the calculation method of the enlargement factor Pa shown here is an example, and the enlargement factor Pa may be calculated by other methods. The parameter setting unit 110 outputs the calculated enlargement ratio Pa to the interpolation image generation unit 101. The parameter setting means 110 calculates the brightness gradient calculation parameter Pb, the contour extraction threshold value Pc, and the contour correction parameter Pd using the enlargement factor Pa (step S2). The parameter setting unit 110 outputs Pb to the luminance gradient calculation unit 102, outputs Pc to the contour extraction unit 103, and outputs Pd to the contour correction unit 104. The operation of calculating Pb, Pc, and Pd from the enlargement factor Pa is the same as that in step S1 (see FIG. 12) already described.

  The interpolated image generating means 101 receives the low resolution image Si (image before enlargement). After step S2, the interpolated image generating means 101 interpolates the low resolution image Si with the enlargement factor Pa, and generates an interpolated image Sa having the user's desired resolution (step S10). The interpolation image generation unit 101 may enlarge the image by interpolating a number of pixels corresponding to the enlargement factor Pa between adjacent pixels in the low resolution image Si. The interpolated image generating unit 101 may perform bilinear interpolation or bicubic interpolation to interpolate pixels and enlarge the image, for example.

  The operation when the interpolated image generating means 101 performs bilinear interpolation will be described. FIG. 15 is an image diagram illustrating an example of bilinear interpolation. The vertical axis shown in FIG. 15 represents pixel values, and the other two axes represent coordinates in the x-axis direction and the y-axis direction, respectively. The coordinates (u, v) shown in FIG. 15 are the coordinates of the pixel to be interpolated. However, (u, v) is a coordinate in the low-resolution image Si and is represented by a decimal. The x-coordinate and y-coordinate of the pixel included in the low-resolution image Si are represented by integers such as 1, 2, 3,..., And the pixel is interpolated between the coordinates. It is represented by The coordinates (u, v) shown in FIG. 15 are coordinate values in such a low resolution image Si. Also, the coordinates (k, l), (k + 1, l), (k, l + 1), and (k + 1, l + 1) shown in FIG. 15 are the coordinate values of the pixels existing in the low-resolution image Si, and these pixels The pixel values of are known. These four pixels are pixels surrounding the pixel (u, v) to be interpolated, k is a value obtained by rounding down the decimal point of u, and l is a value obtained by rounding down the decimal point of v. The interpolated image generation means 101 calculates a pixel value at the coordinates (u, v) to be interpolated.

  When the pixel value at the coordinates (u, v) is obtained by bilinear interpolation, the pixel values at the coordinates (u, l) are obtained by using the pixel values at two coordinates (k, l) and (k + 1, l) having the same y coordinate. Is linearly interpolated. Similarly, the pixel values at the coordinates (u, l + 1) are linearly interpolated using the pixel values at (k, l + 1) and (k + 1, l + 1). Further, the pixel value at the coordinates (u, v) is linearly interpolated using the pixel values at two coordinates (u, l) and (u, l + 1) having the same x coordinate.

  The pixel value P at the coordinates (u, v) can be obtained by the following equation (10).

  The interpolated image generating means 101 may calculate the pixel value P in the pixel to be interpolated by calculating the equation shown in equation (10). Note that P1, P2, P3, and P4 in Expression (10) are pixel values at coordinates (k, l), (k + 1, l), (k, l + 1), and (k + 1, l + 1), respectively. A pixel having coordinates (u, v) at which the pixel value P is determined in this way is a pixel to be interpolated.

  An operation when the interpolated image generating unit 101 performs bicubic interpolation will be described. FIG. 16 is an image diagram illustrating an example of bicubic interpolation. The vertical axis shown in FIG. 16 represents the pixel value, and the other two axes represent the coordinates in the x-axis direction and the y-axis direction, respectively. The coordinates (u, v) shown in FIG. 16 are the coordinates of the pixel to be interpolated. As in the case shown in FIG. 15, (u, v) are coordinates in the low resolution image Si and are represented by decimal numbers. In bicubic interpolation, 16 pixels (k−1, l−1), (k, l−1), (k + 1, l−1), (k + 2, l−1) surrounding (u, v), (K-1, l), (k, l), (k + 1, l), (k + 2, l), (k-1, l + 1), (k, l + 1), (k + 1, l + 1), (k + 2, l + 1) ), (K-1, l + 2), (k, l + 2), (k + 1, l + 2), and (k + 2, l + 2) pixel values at (u, v) are interpolated. k is a value obtained by truncating the decimal part of u, and l is a value obtained by truncating the decimal part of v.

  When obtaining pixel values at coordinates (u, v) by bicubic interpolation, four coordinates (k−1, l−1), (k, l−1), (k + 1, l−1), with the same y coordinate, The pixel value at (u, l−1) is interpolated using the pixel value at (k + 2, l−1). Similarly, pixel values at (u, l), (u, l + 1), and (u, l + 2) are interpolated using four coordinates having the same y coordinate. Further, the pixel value at the coordinates (u, v) is obtained by using the pixel values at four coordinates (u, l−1), (u, l), (u, l + 1), (u, l + 2) having the same x coordinate. Is interpolated.

  The luminance P at the coordinates (u, v) can be obtained by the following equation (11). A pixel having coordinates (u, v) at which the pixel value P is determined in this way is a pixel to be interpolated.

  The interpolated image generating means 101 may calculate the pixel value P in the pixel to be interpolated by calculating the equation shown in equation (11). P1 to P16 in Equation 9 are (k-1, l-1), (k, l-1), (k + 1, l-1), (k + 2, l-1), (k-1, l), (K, l), (k + 1, l), (k + 2, l), (k-1, l + 1), (k, l + 1), (k + 1, l + 1), (k + 2, l + 1), (k-1, l + 2) ), (K, l + 2), (k + 1, l + 2), and (k + 2, l + 2).

In the equation (11), f ₋₁ (t), f ₀ (t), f ₁ (t), and f ₂ (t) are represented by the following equations, respectively.

f ₋₁ (t) = (− t ³ + 2t ² −t) / 2 Formula (12)

f ₀ (t) = (3t ³ -5t ² +2) / 2 Formula (13)

f ₁ (t) = (− 3t ³ + 4t ² + t) / 2 Formula (14)

f ₂ (t) = (t ³ −t ² ) / 2 Formula (15)

  Further, u ′ in the equation (11) is a value calculated as u ′ = u−k. In the formula (11), v ′ is a value calculated as v ′ = v−1.

  In addition, when a plurality of types of pixel values (for example, three types of R, G, and B) are set as in the pixel expressed in the RGB format, each of the pixel values such as R, G, and B is different. It is sufficient to interpolate pixel values.

  Here, bilinear interpolation and bicubic interpolation are exemplified, but the interpolated image generating means 101 may enlarge the low-resolution image Si by performing interpolation using other methods.

  The interpolated image generating unit 101 outputs an interpolated image Sa generated by interpolating pixels with respect to the low resolution image Si to the luminance gradient calculating unit 102 and the contour correcting unit 140. The luminance gradient calculation unit 102 selects a Sobel filter coefficient corresponding to the luminance gradient calculation parameter Pb calculated by the parameter setting unit 110. The luminance gradient calculating means 102 calculates the luminance gradient intensity and the luminance gradient direction for each pixel of the interpolation image Sa using the Sobel filter coefficient, and sends the luminance gradient intensity and the luminance gradient direction of each pixel to the contour extracting means 103. Output (step S11). The contour extracting unit 103 determines a pixel representing the contour in the interpolation image Sa using the contour extraction threshold value Pc calculated by the parameter setting unit 110 (step S12). The contour correcting unit 104 uses the contour correction parameter Pd calculated by the parameter setting unit 110 as a distance for determining the first and second isolated pixels from the reference pixel, and corrects the pixel value around the contour (step S13). ). The operations in steps S11 to S13 are the same as those in steps S11 to S13 in the modification already described.

  According to this modification, the interpolation image generation means 101 interpolates the low resolution image Si to generate the interpolation image Sa. Therefore, if the low resolution image Si is input, a high quality enlarged image can be obtained.

  In the configuration including the interpolation image generation unit 101, the parameter setting unit 110 may not be provided. In this case, the luminance gradient calculation parameter Pb may be set in advance in the luminance gradient calculation means 102. Alternatively, the Sobel filter coefficient may be set in advance. The contour extraction unit 103 may set a contour extraction threshold value Pc in advance, and the contour correction unit 104 may set a contour correction parameter Pd in advance.

Embodiment 2. FIG.
FIG. 17 is a block diagram illustrating an example of an image processing system according to the second embodiment of this invention. Similar to the first embodiment, the image processing system of the present embodiment is realized by a computer 800 that operates under program control, for example. The computer 800 includes representative color estimation means 802 and color correction means 803. Here, the case where the enlargement ratio from the low resolution image (image before enlargement) to the image after enlargement is constant will be described as an example. Further, a case where the pixels are expressed in the RGB format will be described as an example, but the representation of the pixels is not limited to the RGB format.

  The representative color estimation unit 802 receives an interpolation image Sa enlarged by interpolation processing (for example, bilinear interpolation or bicubic interpolation). The representative color estimation unit 802 selects individual pixels included in the interpolation image Sa. The representative color estimation unit 802 extracts a block of pixels having a fixed size centered on the selected pixel (for example, a block of 5 rows and 5 columns), and clusters the colors of the pixels in the block into two classes. The representative color estimation unit 802 specifies the representative colors M1 and M2 representing the class for each of the two classes. Further, the probability that the color of each pixel in the block belongs to the first class and the probability that the color belongs to the second class is obtained. The probability and the representative color may be obtained by, for example, an EM algorithm (Expectation Maximization algorithm). The representative color estimation unit 802 selects individual pixels in the interpolated image Sa, extracts a block for each pixel, and derives a representative color and the above probability.

  FIG. 18 is an image diagram for explaining clustering and representative colors. It is assumed that a 5 × 5 range centered on the selected pixel is extracted as a block. It is assumed that a red pixel exists in the upper left portion of the block and a green pixel exists in the lower right portion (see FIG. 18A). The pixel values (R, G, B) of red pixels are represented by, for example, (255, 0, 0), (240, 0, 0), (250, 1, 0), etc. It exists in the near position (refer FIG.18 (b)). Similarly, the pixel values of green pixels are represented by, for example, (0, 255, 0), (0, 255, 3), (1, 240, 5), etc., and exist at positions close to each other in the RGB space. (See FIG. 18B). The representative color estimation means 802 determines a representative color representing such a pixel color class, and obtains the probability that the color of each pixel belongs to that class for each class. FIG. 18 illustrates a case where the pixel is clearly divided into a red color and a green color. However, even if the pixels in the block are of the same color, for example, the representative color or the above-described color may be obtained by an EM algorithm. Probability can be obtained.

  When the distance in the color space of the representative colors M1 and M2 is equal to or greater than the threshold value, the color correction unit 803 determines the probability of belonging to the first class and the second probability for each pixel in the block extracted by the representative color estimation unit 802. The pixel color is corrected to one of the representative colors M1 and M2 according to the probability of belonging to the class. The color correction unit 803 performs this process for each block extracted by the representative color estimation unit 802.

  Since a block is a set of pixels having a certain size centered on each pixel of the interpolation image Sa, one pixel may belong to a plurality of blocks. In this case, the color correction unit 803 sets the average of the correction results in each block for that pixel.

  The representative color estimation unit 802 and the color correction unit 803 are realized by a central processing unit (CPU) that operates according to a program, for example. That is, the central processing unit may read an image processing program from a storage device (not shown) included in the computer 800 and operate as the representative color estimation unit 802 and the color correction unit 803 according to the image processing program. Further, the representative color estimation unit 802 and the color correction unit 803 may be realized as separate circuits.

Next, the operation will be described in detail.
FIG. 19 is a flowchart illustrating an example of processing progress of the second embodiment. In this embodiment, the interpolation image Sa is input to the representative color estimation unit 802 and the color correction unit 803. When the interpolated image Sa is input, the representative color estimating unit 802 selects one pixel in the interpolated image Sa and extracts a block in a certain range centered on the selected pixel (step S51). It is assumed that the range of this block is predetermined such as 5 rows and 5 columns.

  Next, the representative color estimation unit 802 clusters (classifies) the colors of each pixel in the extracted block into two classes C1 and C2, and specifies the representative colors M1 and M2 of each class (step S52). In step S52, the representative color estimation unit 802 obtains a probability belonging to the first class C1 and a probability belonging to the second class C2 for each pixel in the extracted block. Hereinafter, the process of step S52 will be described using the case of using the EM algorithm as an example.

In the following description, it is assumed that μ is a vector whose elements are R, G, and B values of the center color of the class (a representative color). Further, ω is a weighting factor called a mixing parameter. σ is a value representing how much the pixel values of the clustered pixels are spread in the color space. J is a variable for identifying two classes. For example, j = 0 means the first class, and j = 1 means the second class. This j may be used as a subscript of a symbol. The representative color estimation unit 802 generates two sets of arbitrarily defined initial values of ω, μ, and σ. The two sets correspond to the first class and the second class. The representative color estimation unit 802 repeats recalculation of ω, μ, and σ for each group. By this recalculation, μ (here, R, G, B values) is converged and a representative color is determined. Θ is a symbol indicating a set of ω, μ, and σ. When (t) is added to each symbol, the t represents the number of recalculations. In addition, a vector having the pixel value (R, G, B) value of each pixel in the block extracted in step S51 as an element is denoted by x, and the vector x for the nth pixel in the block is a subscript. Add _n and write _xn .

When the representative color estimation unit 802 generates two sets of arbitrarily defined initial values of ω, μ, and σ, for each set, the representative color estimation unit 802 calculates the probability that the pixel in the block belongs to the class of the set. This probability is expressed as P (j | x _n , θ ^(t) ). The representative color estimation unit 802 obtains P (j | x _n , θ ^(t) ) according to the following equation (16).

The representative color estimation unit 802 calculates P (x _n | j) on the right side by calculating the following equation (17) when calculating the equation (16).

The representative color estimation unit 802 calculates p (x _n ) on the right side by calculating the following equation (18) when calculating the equation (16).

ｗ_ｊは、混合パラメータと呼ばれる重み係数であり、以下の条件を満たす。

w _j is a weighting factor called a mixing parameter and satisfies the following condition.

The representative color estimation means 802 calculates P (j | x _n , θ ^(t) ) for each j (that is, for each class) using equation (16), and calculates ω _j , μ _j , σ _j for each j. Recalculate.

ω _j corresponds to the left side of the following equation (19), and the representative color estimation unit 802 recalculates ω _j by the calculation of equation (19).

μ _j corresponds to the left side of the following equation (20), and the representative color estimation unit 802 recalculates μ _j by the calculation of equation (20).

σ _j corresponds to the left side of the following equation (21), and the representative color estimation means 802 recalculates σ _j by the calculation of equation (21).

  N in the equations (19) to (20) is the total number of pixels belonging to the block extracted in step S51. M is the number of dimensions of data (pixel value). In this example, since the pixel value is three-dimensional data of R, G, and B, M = 3.

Representative color estimating unit 802, upon re-calculate ω _j, μ _j, σ _j, its ω _j, μ _j, σ _j
To P (j | x _n , θ ^(t) ) according to equation (16), and ω _j , μ _j , and σ _{j according} to equations (19) to (21).

The representative color estimation unit 802 determines that μ _j has converged when the amount of change in μ _j before and after the calculation is equal to or less than the threshold, and stops recalculation. The μ _j obtained at that time is the value of each component of the representative colors R, G, and B. The representative color estimation unit 802 further calculates P (j | x _n , θ ^(t) ) from ω _j , μ _j , and σ _j when converged. This P (j | x _n , θ ^(t) ) is the probability that the color of the pixel x _n belongs to the class j.

  Through the processing in step S52 described above, the probability that each pixel in the block belongs to the first class, the probability that it belongs to the second class, and the representative colors M1 and M2 are obtained. Hereinafter, the probability that the color of the pixel (x, y) in the block belongs to the first class is referred to as P1 (x, y), and the probability that the color belongs to the second class is referred to as P2 (x, y).

  Next, the color correction unit 803 calculates an index according to the distance D between the representative colors M1 and M2. The distance between two colors is a value obtained as a square root of the sum of the calculation results of calculating the square of the difference for each component representing the color. When the color is represented by R, G, and B components as in this example, the square of the difference of the R component, the square of the difference of the B component, and the square of the difference of the B component are calculated, respectively. The square root is the distance. That is, assuming that the R, G, B components of the representative color M1 are (R1, G1, B1) and the R, G, B components of the representative color M2 are (R2, G2, B2), the distance D is It is represented by the formula (22) shown.

D = √ {(R1-R2) ² + (G1-G2) ² + (B1-B2) ² } Formula (22)

The color correction unit 803 may calculate the distance D itself, or may calculate an index corresponding to the distance D. Here, (R1−R2) ² + (G1−G2) ² + (B1−B2) ² is calculated as an index corresponding to the distance D. The color correction unit 803 determines whether the distance between the representative colors is equal to or greater than a threshold value based on whether or not the index corresponding to the distance D is equal to or greater than a predetermined value (step S53).

When the calculation result of (R1−R2) ² + (G1−G2) ² + (B1−B2) ² is equal to or greater than a predetermined value, the color correction unit 803 determines that the distance D is equal to or greater than the threshold. In this case (Yes in step S53), the color correction unit 803 determines the pixel correction according to the probability belonging to the first class and the probability belonging to the second class for each pixel in the block extracted in step S51. The color is corrected (step S54).

  In step S54, the color correction unit 803 selects each pixel in the block, the probability P1 (x, y) that the color of the pixel belongs to the first class (class of the representative color M1), and the second class. The probability P2 (x, y) belonging to (class of representative color M2) is compared. If P1 (x, y)> P2 (x, y), the color correction unit 803 replaces the color of the selected pixel (x, y) with the representative color M1. That is, the pixel values (R, G, and B values) of the selected pixel are replaced with the R, G, and B values of the representative color M1. On the other hand, if P1 (x, y)> P2 (x, y) does not hold, the color correcting unit 803 replaces the color of the selected pixel (x, y) with the representative color M2. That is, the pixel values (R, G, and B values) of the selected pixel are replaced with the R, G, and B values of the representative color M2. The color correction unit 803 performs this replacement for each pixel in the block. When the replacement is completed for each pixel in the block, the process proceeds to step S55.

When the calculation result of (R1−R2) ² + (G1−G2) ² + (B1−B2) ² is less than a predetermined value, the color correction unit 803 determines that the distance D is less than the threshold value. In this case (No in step S53), the process proceeds to step S55 without performing step S54.

  In step S55, the color correction unit 803 determines whether or not the processing from step S51 onward is completed for each block centered on each pixel in the interpolation image Sa. If there is a block that has not been processed after step S51, the process after step S51 is performed on the unprocessed block. If the processing after step S51 is completed for all blocks, the processing is terminated.

  However, when one pixel belongs to a plurality of blocks and a color (pixel value) after replacement of the pixel is derived for each block, the color correction unit 803 applies the one pixel to each block. The average value of the derived pixel values after replacement is set as the pixel value of the pixel.

  According to the present embodiment, when the distance between two representative colors in a block is greater than or equal to a threshold, the pixels are corrected by replacing the pixels in the block with the representative colors, so that the outline is clear and the quality is high. An enlarged image can be generated. Further, since the statistical processing by clustering is used when specifying the representative colors M1 and M2, the representative colors can be stably acquired without being affected by noise or the like.

  Next, a modification of the second embodiment will be described. In the second embodiment described above, the individual pixels belonging to the interpolated image Sa are selected, the block centered on the pixel is extracted, and the processing after step S51 is performed for each block. In the modification shown below, the selection target pixel in the interpolated image Sa is set as a pixel on the contour, and a block centering on each pixel on the contour is extracted, and the processing from step S51 is performed. FIG. 20 is a block diagram illustrating a modification of the second embodiment. Constituent elements similar to those already described are given the same reference numerals as in FIG. As shown in FIG. 20, the image processing system may include a luminance gradient calculation unit 102 and a contour extraction unit 103 in addition to the representative color estimation unit 802 and the color correction unit 803. FIG. 20 shows a case where the computer 800 includes a brightness gradient calculating unit 102 and a contour extracting unit 103. The brightness gradient calculating unit 102 and the contour extracting unit 103 may be realized by a central processing unit that operates according to a program. That is, the central processing unit may read the image processing program and operate as the luminance gradient calculation unit 102, the contour extraction unit 103, the representative color estimation unit 802, and the color correction unit 803 according to the image processing program.

  The luminance gradient calculation unit 102 and the contour extraction unit 103 in this modification perform the same processing as the luminance gradient calculation unit 102 and the contour extraction unit 103 included in the image processing system of the first embodiment. The brightness gradient calculating unit 102 and the contour extracting unit 103 of this modification will be described with the same reference numerals as those in the first embodiment.

  The luminance gradient calculation unit 102 receives the interpolation image Sa, and the luminance gradient calculation unit 102 calculates the luminance gradient intensity and the luminance gradient direction for each pixel of the interpolation image Sa. In addition, the contour extracting unit 103 determines a pixel representing an original contour before deterioration using the luminance gradient intensity and luminance gradient direction in each pixel.

  FIG. 21 is a flowchart showing the processing progress of a modification of the second embodiment. The same processes as those described in the second embodiment are denoted by the same reference numerals as those in FIG.

  When the interpolation image Sa is input, the luminance gradient calculating unit 102 specifies the luminance value from the pixel value for each individual pixel in the application image Sa, as in the first embodiment. For example, the luminance value is calculated for each pixel by the calculation of Expression (1). Then, the luminance gradient calculation unit 102 performs convolution of coefficients of a predetermined Sobel filter, and calculates a horizontal luminance gradient and a vertical luminance gradient for each pixel of the interpolation image Sa. Further, the luminance gradient calculating means 102 calculates the luminance gradient intensity Sb (x, y) and the luminance gradient direction Sc (x, y) for each pixel (step S49). The brightness gradient strength Sb (x, y) may be calculated by, for example, the formula (4) or the formula (5). The luminance gradient direction Sc (x, y) may be calculated by, for example, the equation (6). Here, the case where the Sobel filter coefficient is used is illustrated, but the processing in the case of using other edge detection operator coefficients (such as Robinson operator coefficients and Prewitt operator coefficients) is also the same. is there.

  After step S49, the contour extracting unit 103 uses the luminance gradient intensity and luminance gradient direction of each pixel calculated by the luminance gradient calculating unit 102 and a predetermined contour extraction threshold value Pc in the interpolation image Sa. Is determined (step S50). The contour extracting unit 103 selects each pixel as a contour determination target pixel. Then, the luminance gradient strength of the contour determination target pixel is larger than the contour extraction threshold value Pc, and the luminance gradient strength of the adjacent pixel in the luminance gradient direction of the contour determination target pixel is adjacent to the direction in which the luminance gradient direction is rotated by 180 °. When it is larger than any of the brightness gradient intensities of the pixel, it is determined that the contour determination target pixel is a pixel on the contour. The contour extracting unit 103 notifies the representative color estimating unit 802 of the pixel determined as the pixel representing the contour.

  Steps S49 and S50 are the same processes as steps S11 and S12 in the first embodiment.

  After step S50, the representative color estimation unit 802 and the color correction unit 803 perform the processing of steps S51 to S55 described in the second embodiment. However, the representative color estimation unit 802 selects only pixels representing the contour notified from the contour extraction unit 103 in step S51. In step S55, the color correction unit 803 determines whether or not the processing after step S51 has been completed for each pixel representing the contour in the interpolation image Sa. If there is any pixel that has not been subjected to the processing after step S51 among the pixels representing the contour, the processing after step S51 is repeated. If the processing from step S51 onward is completed for each pixel representing the contour, the processing ends. The other points are the same as in the second embodiment.

  As a result, since the processing after step S51 is omitted for pixels that do not represent the contour, the processing can be simplified and the processing time can be shortened.

  Next, another modification of the second embodiment will be described. In the following modification, the range of blocks extracted in step S51 is variable. FIG. 22 is a block diagram illustrating another modification of the second embodiment. Components that are the same as those already described are given the same reference numerals as in FIG. As shown in FIG. 22, the image processing system may include a parameter setting unit 810 in addition to the luminance gradient calculation unit 102, the contour extraction unit 103, the representative color estimation unit 802, and the color correction unit 803. Although FIG. 22 shows a case where the parameter setting unit 810 is provided separately from the computer 800, the computer 800 may include the parameter setting unit 810. The parameter setting unit 810 may be realized by a central processing unit that operates according to a program. That is, the central processing unit reads the image processing program and operates as the luminance gradient calculation unit 102, the contour extraction unit 103, the representative color estimation unit 802, the color correction unit 803, and the parameter setting unit 810 according to the image processing program. Also good.

  The parameter setting unit 810 calculates a parameter that determines how many rows and columns are extracted as the pixel block extracted in step S51, and outputs the calculated parameter to the representative color estimation unit 802 and the color correction unit 803. Hereinafter, this parameter is referred to as a block determination parameter and is represented as “Pe”. In step S51, the representative color estimation unit 802 extracts a block having a size corresponding to the block determination parameter Pe.

  The parameter setting means 810 receives an enlargement factor Pa from the low resolution image to the interpolated image Sa. The parameter setting means 810 calculates a block determination parameter Pe from the input enlargement factor Pa. When the enlargement rate Pa is unchanged, the parameter setting unit 810 may store the enlargement rate Pa as a constant and calculate the block determination parameter Pe from the enlargement rate Pa.

  The parameter setting unit 810 may calculate the block determination parameter Pe by calculating the following equation (23).

  Pe = ε · Pa + ζ Equation (23)

  ε and ζ are constants. Further, ε is a value larger than 0, and Pe increases as Pa increases. Ζ is a value representing the lower limit of Pe. By defining ζ as the lower limit value of Pe, it is ensured that the value of Pb is not less than the lower limit value even if the value of the enlargement factor Pe is small. For example, ε and ζ are set in advance by an administrator of the image processing system. An administrator of the image processing system may determine ε and ζ so that the number of blocks having a size corresponding to the enlargement ratio is selected. Pe is calculated, for example, as a linear function with the enlargement ratio Pa as a variable. Examples of ε and ζ include values such as ε = 2.0 and ζ = −1.0, but are not limited to these values.

  Also, the parameter setting unit 810 calculates the luminance gradient calculation parameter Pb and the contour extraction threshold value Pc, and calculates the luminance gradient calculation parameter Pb as the luminance gradient, similarly to the parameter setting unit 110 in the modification of the first embodiment. The contour extraction threshold value Pc is output to the contour extraction means 103. The luminance gradient calculation means 102 selects a Sobel filter coefficient corresponding to the luminance gradient calculation parameter Pb, and calculates the luminance gradient intensity and luminance gradient direction for each pixel of the interpolation image Sa. In addition, the contour extraction unit 103 determines a pixel representing the contour in the interpolation image Sa using the contour extraction threshold value Pc.

  FIG. 23 is a flowchart showing the processing progress of another modification of the second embodiment. The same processes as those already described are denoted by the same reference numerals as those in FIGS. When the enlargement factor Pa is input, the parameter setting unit 810 calculates the brightness gradient calculation parameter Pb, the contour extraction threshold value Pc, and the block determination parameter Pe using the enlargement factor Pa (step S41). Note that a part of Pb, Pc, and Pe may be determined in advance as fixed values, and the parameter setting unit 810 may calculate a parameter that is not determined as a fixed value among Pb, Pc, and Pe in step S41. .

  When the interpolation image Sa is input, the brightness gradient calculating unit 102 specifies a brightness value from the pixel value for each pixel in the application image Sa. In addition, a rule such as “if the Pb is greater than or equal to x1 and less than x2 selects a coefficient of a 3 × 3 Sobel filter” is stored in advance, and the Sobel corresponding to the luminance gradient calculation parameter Pb according to the rule. Select filter coefficients. The luminance gradient calculating means 102 calculates the luminance gradient intensity and the luminance gradient direction for each pixel of the interpolation image Sa using the selected Sobel filter coefficient (step S49). Other than the selection of the Sobel filter coefficient, this is the same as step S49 already described.

  The contour extraction unit 103 determines a pixel representing the contour in the interpolation image Sa using the contour extraction threshold value Pc calculated by the parameter setting unit 810 (step S50). Except for the point that the parameter setting means 810 determines the contour extraction threshold value Pc, it is the same as step S50 already described.

  After step S50, the representative color estimation unit 802 and the color correction unit 803 perform the processes of steps S51 to S55, as in the modified example of the base embodiment already described. The representative color estimation unit 802 selects only pixels representing the contour notified from the contour extraction unit 103 in step S51. In step S55, the color correction unit 803 determines whether or not the processing after step S51 has been completed for each pixel representing the contour in the interpolation image Sa. If there is any pixel that has not been subjected to the processing after step S51 among the pixels representing the contour, the processing after step S51 is repeated. If the processing from step S51 onward is completed for each pixel representing the contour, the processing ends.

  Also in the present modification, the processing after step S51 is omitted for pixels that do not represent an outline, so that the processing can be simplified and the processing time can be shortened. Even when the enlargement ratio from the low resolution image to the interpolated image Sa is not constant, the contour can be clarified with an appropriate parameter corresponding to the enlargement ratio.

  Next, another modification of the second embodiment will be described. FIG. 24 is a block diagram illustrating another modification of the second embodiment. Constituent elements similar to those already described are denoted by the same reference numerals as in FIGS. 17, 20, and 22. As shown in FIG. 24, the image processing system includes an interpolation image generation unit 101 in addition to the luminance gradient calculation unit 102, the contour extraction unit 103, the representative color estimation unit 802, the color correction unit 803, and the parameter setting unit 810. May be. The interpolated image generating unit 101 performs the same processing as the interpolated image generating unit 101 (see FIG. 13) shown in the modification of the first embodiment. FIG. 24 shows a case where the computer 800 includes the interpolated image generating means 101. The interpolated image generating means 101 may be realized by a central processing unit that operates according to a program. That is, the central processing unit reads an image processing program, and in accordance with the image processing program, the interpolated image generation means 101, the luminance gradient calculation means 102, the contour extraction means 103, the representative color estimation means 802, the color correction means 803, the parameter setting It may operate as the means 810.

  The low-resolution image Si is input to the interpolated image generating means 101, and the interpolated image generating means 101 enlarges the low-resolution image Si by interpolation to generate an interpolated image Sa, and the interpolated image Sa is used as the luminance gradient calculating means 102 and the representative. The data is output to the color estimation unit 802 and the color correction unit 803. That is, the interpolated image generating means 101 generates an interpolated image Sa that is an enlarged image by interpolating pixels between the pixels of the low resolution image Si. The interpolation image generation unit 101 enlarges the low resolution image Si by, for example, bilinear interpolation or bicubic interpolation.

  In addition, the resolution of the enlarged image to be generated is input to the parameter setting unit 810. For example, the width of the enlarged interpolation image Sa (the number of pixels in the horizontal direction) may be input. In addition, information about the low-resolution image Si before enlargement (for example, the horizontal width of the low-resolution image Si) is also input to the parameter setting unit 810. The parameter setting unit 810 calculates the enlargement factor Pa using the input information and outputs it to the interpolated image generation unit 101. The parameter setting means 810 may calculate each parameter of Pb, Pc, and Pd from the enlargement factor Pa as described above after calculating the enlargement factor Pa. This operation is the same as that of the parameter setting unit 110 (see FIG. 13) shown in the modification of the first embodiment.

  The interpolated image generation unit 101 generates an interpolated image Sa by interpolating the low resolution image Si with the enlargement factor Pa calculated by the parameter setting unit 810.

  FIG. 25 is a flowchart showing the processing progress of this modification. The same processes as those already described are denoted by the same reference numerals as those in FIGS. For example, the horizontal width Wl of the low resolution image Si and the horizontal width Wh of the desired interpolation image Sa of the user are input to the parameter setting means 810. When Wl and Wh are input, the parameter setting unit 810 calculates the enlargement factor Pa by calculating Wh / Wl. However, the calculation method of the enlargement factor Pa shown here is an example, and the enlargement factor Pa may be calculated by other methods. The parameter setting unit 810 outputs the calculated enlargement factor Pa to the interpolation image generation unit 101. Also, the parameter setting means 810 calculates the brightness gradient calculation parameter Pb, the contour extraction threshold value Pc, and the block determination parameter Pe using the enlargement factor Pa (step S47). The parameter setting unit 810 outputs Pb to the luminance gradient calculation unit 102, outputs Pc to the contour extraction unit 103, and outputs the block determination parameter Pe to the representative color estimation unit 802 and the color correction unit 803. The operation for calculating Pb, Pc, Pd from the enlargement factor Pa is the same as the operation already described.

  The low resolution image Si is input to the interpolated image generating means 101. After step S47, the interpolated image generating means 101 interpolates the low resolution image Si with the enlargement factor Pa to generate an interpolated image Sa having the user's desired resolution (step S48). This process is the same as step S10 (see FIG. 14) described in the modification of the first embodiment.

  The interpolated image generating unit 101 outputs an interpolated image Sa generated by interpolating pixels from the low resolution image Si to the luminance gradient calculating unit 102, the representative color estimating unit 802, and the color correcting unit 803. The luminance gradient calculation unit 102 selects a Sobel filter coefficient corresponding to the luminance gradient calculation parameter Pb calculated by the parameter setting unit 810. The luminance gradient calculation means 102 calculates the luminance gradient intensity and the luminance gradient direction for each pixel of the interpolation image Sa using the Sobel filter coefficient, and sends the luminance gradient intensity and the luminance gradient direction of each pixel to the contour extraction means 103. Output (step S49). The contour extraction unit 103 determines a pixel representing the contour in the interpolation image Sa using the contour extraction threshold value Pc calculated by the parameter setting unit 810 (step S50).

  After step S50, the representative color estimation unit 802 and the color correction unit 803 perform the processes of steps S51 to S55, as in the modified example of the base embodiment already described. The representative color estimation unit 802 selects only pixels representing the contour notified from the contour extraction unit 103 in step S51. In step S55, the color correction unit 803 determines whether or not the processing after step S51 has been completed for each pixel representing the contour in the interpolation image Sa. If there is any pixel that has not been subjected to the processing after step S51 among the pixels representing the contour, the processing after step S51 is repeated. If the processing from step S51 onward is completed for each pixel representing the contour, the processing ends.

  According to this modification, the interpolation image generation means 101 interpolates the low resolution image Si to generate the interpolation image Sa. Therefore, if the low resolution image Si is input, a high quality enlarged image can be obtained.

  In the configuration including the parameter setting unit 810 (for example, the configuration illustrated in FIGS. 22 and 24), the image processing system may not include the luminance gradient calculation unit 102 and the contour extraction unit 103. In this case, the parameter setting unit 810 does not have to calculate Pb and Pc.

Embodiment 3. FIG.
FIG. 26 is a block diagram showing a third embodiment of the present invention. The third embodiment of the present invention is an image transmission system including an image compression apparatus 600 that compresses and transmits an image, and an image expansion apparatus 610 that receives and expands the image. The image compression device 600 and the image decompression device 610 transmit and receive images via the transmission path 620.

  A high-resolution input image 640 is input to the image compression device 600. The image compression apparatus 600 sends the low-resolution image 605 obtained by down-sampling the input image 640 and the residual 606 between the down-sampled image and the input image as compressed data 607 to the transmission path 620. Also, the image decompression device 610 enlarges the low resolution image 605 included in the received compressed data 607 and adds it to the residual 606 included in the compressed data 607 to generate a decompressed image 615. Hereinafter, configurations of the image compression device 600 and the image expansion device 610 will be described.

  The image compression apparatus 600 includes a low-pass downsampling unit 601, an enlarged image generation unit 602, and an image subtraction unit 603.

  An input image (high resolution image) 604 is input to the low-pass / down-sampling means 601. Note that the input image 604 is also input to the image subtracting means 603. The low-pass / down-sampling means 601 applies a low-pass filter to the input image 604 to down-sample the input image 604 to a predetermined resolution. Downsampling refers to periodically removing pixels arranged in the horizontal direction and the vertical direction to reduce the resolution of an image. The low-pass / down-sampling unit 601 transmits the generated low-resolution image 605 to the transmission path 620 and outputs it to the enlarged image generation unit 603.

  The enlarged image generating unit 602 receives the low resolution image 605 generated by the low pass / down sampling unit 601 and enlarges the low resolution image 605 to an image having the same resolution as the input image 604. The enlarged image generating unit 602 performs image processing for clarifying the outline in the enlarged image, and outputs the resulting image to the image subtracting unit 603. This processing is executed by the image processing system including the interpolated image generation unit 101 described as a modification of the first embodiment, or the image processing system including the interpolated image generation unit 101 described as a modification of the second embodiment. This is the same processing as that to be performed.

  The enlarged image generating unit 602 is realized by, for example, an image processing system (see FIG. 13) including the interpolated image generating unit 101 described as a modification of the first embodiment. The parameter setting unit 110 may set the enlargement factor Pa, the luminance gradient calculation parameter Pb, the contour extraction threshold value Pc, and the contour correction parameter Pd. Information (Wh, Wl) for the parameter setting means 110 to calculate the enlargement ratio Pa may be determined in advance, for example.

  The enlarged image generating unit 602 may be realized by an image processing system (see FIG. 24) including the interpolated image generating unit 101 described as a modification of the second embodiment. Further, the parameter setting means 110 may set the enlargement factor Pa, the luminance gradient calculation parameter Pb, the contour extraction threshold value Pc, and the block determination parameter Pe. Information (Wh, Wl) for the parameter setting means 810 to calculate the enlargement factor Pa may be determined in advance, for example.

  Further, without including the parameter setting means 110 and 810 shown in FIGS. 13 and 24, the enlargement ratio Pa, the luminance gradient calculation parameter Pb, the contour extraction threshold value Pc, the contour correction parameter Pd, and the block determination parameter Pe are determined in advance. It may be done. Further, instead of the luminance gradient calculation parameter Pb, a Sobel filter coefficient may be determined in advance.

  The image subtracting unit 603 calculates a residual 606 between the input image 604 and the high-resolution image output by the enlarged image generating unit 602 (an image having the same resolution as the input image 604). A pixel value of a pixel at coordinates (x, y) in the input image 604 is assumed to be I1 (x, y). Further, the pixel value of the pixel at the coordinate (x, y) in the high-resolution image generated by the enlarged image generation unit 602 is assumed to be I2 (x, y). The image subtracting unit 603 obtains I2 (x, y) −I1 (x, y) for each corresponding pixel of the high resolution image output from the input image 604 and the enlarged image generating unit 602 (that is, for each pixel having the same coordinate). ). This calculation result is a residual 606. The residual is data representing a difference in pixel value between corresponding pixels of the high resolution image and the input image 604.

  Further, the image subtracting means 603 also transmits the residual 606 when the low-pass / down-sampling means 601 transmits the low-resolution image 605. Data obtained by combining the low-resolution image 605 and the residual 606 is compressed data 607.

  The image compression apparatus 600 is configured as described above. When the input image 604 is input to the low-pass down-sampling unit 601 and the image subtracting unit 603, the low-pass down-sampling unit 601 down-samples the input image 604. A low resolution image 605 is generated. Then, the enlarged image generating unit 602 enlarges the low resolution image 605, and the image subtracting unit 603 calculates a residual 606 between the enlarged image and the dormitory image obtained as a result. The low-pass down-sampling unit 601 transmits the low-resolution image 605 to the image expansion device 610, and the image subtraction unit 603 transmits the residual 606 together with the low-resolution image 605 to the image expansion device 610.

  The image expansion device 610 includes an enlarged image generation unit 611 and an image addition unit 612.

  The image expansion device 610 extracts the low resolution image 605 and the residual 614 from the received compressed data 607.

  The enlarged image generating unit 611 receives the low resolution image 605 in the received compressed data 607. The enlarged image generating means 611 receives the low resolution image 605 as an input, and enlarges the low resolution image 605 to an image having the same resolution as the input image 604. Further, the enlarged image generating unit 602 performs image processing for clarifying the outline in the enlarged image, and outputs an image obtained as a result to the image adding unit 612. This processing is executed by the image processing system including the interpolated image generation unit 101 described as a modification of the first embodiment, or the image processing system including the interpolated image generation unit 101 described as a modification of the second embodiment. This is the same processing as that to be performed.

  An enlarged image generation unit 611 included in the image expansion device 610 is the same as the enlarged image generation unit 602 included in the image compression device 600. For example, an image including the interpolated image generation unit 101 described as a modification of the first embodiment. This is realized by a processing system (see FIG. 13). Further, for example, it may be realized by an image processing system (see FIG. 24) including the interpolated image generation unit 101 described as a modification of the second embodiment.

  The residual 606 in the received compressed data 607 is input to the image adding means 612. The image adding unit 612 adds the enlarged image output from the enlarged image generating unit 611 and the residual 606. The pixel value of the pixel at coordinates (x, y) in the enlarged image is I3 (x, y). Further, the pixel value of the pixel at the coordinate (x, y) in the residual 606 is assumed to be I4 (x, y). The image adding unit 612 calculates I3 (x, y) + I4 (x, y) for each pixel corresponding to the enlarged image generated by the enlarged image generating unit 611 and the residual 606 (that is, for each pixel having the same coordinates). As a result, a target high-resolution decompressed image 615 is generated.

  The image expansion device 610 is configured as described above. When the low resolution image 605 is input to the enlarged image generation unit 611, the enlarged image generation unit 611 enlarges the low resolution image 605. The image adding unit 612 adds the enlarged image obtained as a result and the residual 606 to generate an expanded image 615.

  Note that when the image compression apparatus 600 transmits the low resolution image 605 and the residual 606 to the image expansion apparatus 610 via the transmission path 620, the low resolution image 605 and the residual 606 (compressed data 605) are converted into existing data. Further compression may be performed using a compression method. In this case, the image decompression device 610 may perform the above-described processing after decompressing data by a data decompression method corresponding to the existing data compression method.

  According to the third embodiment, the image expansion device 610 performs an image enlargement process described in the first embodiment or the second embodiment, thereby obtaining a high-quality enlarged image from the received low-resolution image 605. Can be generated. Therefore, the image compression apparatus 601 can reduce the amount of residual data transmitted via the transmission path 620.

  In the embodiment described above, the representative color estimation means uses the EM algorithm to determine the probability that the color of each pixel in the block belongs to the first class, the probability that the pixel belongs to the second class, the first class, and the first class. A configuration for deriving two classes of representative colors is described.

  In addition, a configuration is described in which the representative color estimation unit selects each pixel in the interpolated image and extracts a block of pixels centered on the selected pixel.

  In addition, the luminance gradient calculating means for calculating the luminance gradient intensity and the luminance gradient direction in each pixel of the interpolation image, and the luminance gradient intensity and the luminance gradient direction are used to determine the pixel representing the contour in the interpolation image. A configuration in which the representative color estimation unit selects each pixel representing the contour in the interpolation image and extracts a block of pixels centered on the selected pixel.

  In addition, the luminance gradient calculating unit specifies a plurality of pixels having the same arrangement as the coefficient of the edge detection operator around the calculation target pixel in the intensity of the luminance gradient and the direction of the luminance gradient, and corresponds to each of the specified pixels. By calculating the product of the coefficients of the edge detection operator and calculating the sum thereof, the luminance gradient in the horizontal direction and the vertical direction is calculated, and the vector whose magnitude is the luminance gradient in the horizontal direction and the magnitude in the vertical direction are calculated. Contour extraction with the magnitude of the combined vector with the brightness gradient vector as the intensity of the intensity gradient of the pixel to be calculated, and the arc tangent obtained by dividing the vertical intensity gradient by the horizontal intensity gradient as the intensity gradient direction The means selects a determination target pixel whether or not it is a pixel representing an outline, and the intensity of the luminance gradient of the determination target pixel is larger than the contour extraction threshold, and the determination target image A configuration is described in which a determination target pixel is a pixel representing an outline when the adjacent pixel in the direction of the luminance gradient is larger than the intensity of the luminance gradient of each adjacent pixel in the direction rotated by 180 °. ing.

  Further, the image processing apparatus includes parameter setting means for determining a block determination parameter for determining the size of a block to be extracted based on an enlargement ratio from an image before enlargement to an interpolated image, and the representative color estimation means is centered on the selected pixel. As a pixel block, a configuration is described in which a block having a size corresponding to a block determination parameter determined by a parameter setting unit is extracted.

  In addition, a configuration is described in which the parameter setting means determines the block determination parameter from the enlargement ratio input to the parameter setting means.

  In addition, a configuration is described in which the parameter setting means determines block determination parameters from a predetermined enlargement ratio.

  Also, for determining the brightness gradient calculation parameter for selecting the coefficient of the edge detection operator from the enlargement ratio from the image before enlargement to the interpolated image, the contour extraction threshold, and the block determination for determining the size of the block to be extracted Parameter setting means for determining parameters is provided, and the luminance gradient calculating means holds the coefficients of a plurality of types of edge detection operators in advance, and uses the edge detection operator coefficients corresponding to the luminance gradient calculation parameters to determine the intensity and luminance of the luminance gradient. The direction of the gradient is calculated, the contour extracting unit specifies the pixel representing the contour using the contour extraction threshold determined by the parameter setting unit, and the representative color estimating unit determines the pixel block centered on the selected pixel. A configuration is described in which a block having a size corresponding to a block determining parameter determined by the parameter setting means is extracted.

  In addition, a configuration is described in which the parameter setting unit determines a luminance gradient calculation parameter, a contour extraction threshold value, and a block determination parameter from an enlargement ratio input to the parameter setting unit.

  In addition, a configuration is described in which the parameter setting means determines a luminance gradient calculation parameter, a contour extraction threshold value, and a block determination parameter from a predetermined magnification.

  Further, there is described a configuration including an interpolation image generation unit that receives an enlargement target image and generates an interpolation image obtained by enlarging the enlargement target image by interpolating pixels between pixels of the enlargement target image.

  The present invention is suitably applied to an image processing system that performs image processing for clarifying a contour in an image enlarged from a low-resolution still image or moving image, and also compresses and transmits a high-resolution still image or moving image. It can also be applied to systems that store and store data.

1 is a block diagram illustrating an example of an image processing system according to a first embodiment of the present invention. It is explanatory drawing which shows a luminance gradient intensity | strength and a luminance gradient direction. It is a flowchart which shows the example of the process progress of 1st Embodiment. It is explanatory drawing which shows the example of the coefficient of a Sobel filter. It is an image figure which shows the luminance value of each pixel of n row n column (3 rows 3 columns) centering on the pixel for calculation of a luminance gradient. It is explanatory drawing which shows the example of the determination process of the pixel showing an outline. It is explanatory drawing which shows the example of the correction process of the pixel around an outline. It is an image which shows the example of the correction process result by an outline correction means. It is a block diagram which shows the modification of 1st Embodiment. It is an image figure which shows the undulation of the brightness | luminance gradient for every pixel. It is an image figure which shows the change of the pixel value in the vicinity of the outline before and behind the expansion of an image. It is a flowchart which shows the process progress of the modification of 1st Embodiment. It is a block diagram which shows the other modification of 1st Embodiment. It is a flowchart which shows process progress of the other modification of 1st Embodiment. It is an image figure which shows the example of bilinear interpolation. It is an image figure which shows the example of bicubic interpolation. It is a block diagram which shows the example of the image processing system of the 2nd Embodiment of this invention. It is an image figure explaining clustering and a representative color. It is a flowchart which shows the example of the process progress of 2nd Embodiment. It is a block diagram which shows the modification of 2nd Embodiment. It is a flowchart which shows the process progress of the modification of 2nd Embodiment. It is a block diagram which shows the other modification of 2nd Embodiment. It is a flowchart which shows process progress of the other modification of 2nd Embodiment. It is a block diagram which shows the other modification of 2nd Embodiment. It is a flowchart which shows process progress of the other modification of 2nd Embodiment. It is a block diagram which shows the 3rd Embodiment of this invention. It is an image which shows the example of the upsampling result of a low-resolution image. It is an image which shows the example of the result of bicubic interpolation. 1 is a block diagram illustrating an image processing apparatus described in Patent Literature 1. FIG. It is an image which shows the example of the result of applying outline emphasis to the expansion picture by bicubic interpolation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Interpolation image generation means 102 Luminance gradient calculation means 103 Contour extraction means 104 Contour correction means 110 Parameter setting means 802 Representative color estimation means 803 Color correction means 810 Parameter setting means

Claims (21)

Select a pixel in an interpolated image that is an enlarged image including pixels interpolated between pixels of the image before enlargement, extract a block of pixels centered on the selected pixel, and extract the pixels in the block Clustering colors into a first class and a second class, the probability that the color of each pixel in the block belongs to the first class, the probability that the pixel belongs to the second class, the first class and the Representative color estimation means for deriving a second class of representative colors;
Probability that the color of the pixel belongs to the first class for each individual pixel in the block extracted by the representative color estimation means when the distance between the representative colors of the first class and the second class is greater than or equal to a threshold value If the condition is higher than the probability of belonging to the second class, the color of the pixel is replaced with a representative color of the first class, and if the condition is not satisfied, the color of the pixel is changed to the second class. A color correcting unit that substitutes the average color of the replaced color derived for each block for each block when one pixel belongs to a plurality of blocks. An image processing system comprising: The representative color estimation means uses the EM algorithm to determine the probability that the color of each pixel in the block belongs to the first class, the probability that the color belongs to the second class, the representative of the first class and the second class. The image processing system according to claim 1, wherein a color is derived. The image processing system according to claim 1, wherein the representative color estimation unit selects each pixel in the interpolated image and extracts a block of pixels centered on the selected pixel. A luminance gradient calculating means for calculating the intensity of the luminance gradient and the direction of the luminance gradient in each pixel of the interpolation image;
Contour extraction means for determining pixels representing the contour in the interpolated image using the intensity of the luminance gradient and the direction of the luminance gradient;
The image processing system according to claim 1, wherein the representative color estimation unit selects each pixel representing an outline in the interpolated image, and extracts a block of pixels centered on the selected pixel. The luminance gradient calculation means identifies a plurality of pixels having the same arrangement as the coefficient of the edge detection operator around the pixel to be calculated in the intensity of the luminance gradient and the direction of the luminance gradient, and detects a corresponding edge for each of the identified pixels. By calculating the product of the operator's coefficient and calculating the sum, the luminance gradient in the horizontal and vertical directions is calculated, and the vector whose magnitude is the horizontal luminance gradient and the luminance in the vertical direction are calculated. The magnitude of the combined vector with the vector that is the gradient is the intensity of the luminance gradient of the pixel to be calculated, and the arc tangent as a result of dividing the vertical luminance gradient by the horizontal luminance gradient is the luminance gradient direction,
The contour extraction unit selects a pixel to be determined as to whether or not the pixel represents a contour, the intensity of the luminance gradient of the determination target pixel is greater than a contour extraction threshold, and adjacent pixels in the direction of the luminance gradient of the determination target pixel 5. The image processing system according to claim 4, wherein the determination target pixel is determined to be a pixel representing an outline when the intensity gradient is greater than an intensity of a luminance gradient of each of adjacent pixels in a direction rotated by 180 °. Parameter setting means for determining a parameter for determining a block for determining the size of a block to be extracted from an enlargement ratio from the image before enlargement to the interpolated image,
The representative color estimation unit extracts a block having a size corresponding to the block determination parameter determined by the parameter setting unit as a block of pixels centered on the selected pixel. The image processing system according to item 1. The image processing system according to claim 6, wherein the parameter setting unit determines a block determination parameter from an enlargement ratio input to the parameter setting unit. The image processing system according to claim 6, wherein the parameter setting means determines a block determination parameter from a predetermined enlargement ratio. For determining the brightness gradient calculation parameter for selecting the coefficient of the edge detection operator, the contour extraction threshold, and the block size for determining the size of the block to be extracted from the enlargement ratio from the image before enlargement to the interpolated image Equipped with parameter setting means for determining parameters,
The brightness gradient calculating means holds coefficients of a plurality of types of edge detection operators in advance, calculates the intensity of the brightness gradient and the direction of the brightness gradient using the edge detection operator coefficients according to the brightness gradient calculation parameters,
The contour extracting means identifies the pixel representing the contour using the contour extraction threshold determined by the parameter setting means,
The image processing system according to claim 5, wherein the representative color estimation unit extracts a block having a size corresponding to a block determination parameter determined by the parameter setting unit, as a block of pixels centered on the selected pixel. The image processing system according to claim 9, wherein the parameter setting unit determines a luminance gradient calculation parameter, a contour extraction threshold value, and a block determination parameter from an enlargement ratio input to the parameter setting unit. The image processing system according to claim 9, wherein the parameter setting means determines a luminance gradient calculation parameter, a contour extraction threshold value, and a block determination parameter from a predetermined magnification. An interpolation image generating unit that generates an interpolation image in which the enlargement target image is enlarged by interpolating pixels between pixels of the enlargement target image is input. The image processing system according to any one of the above. Downsampling means for generating a low resolution image having a lower resolution than the input image by downsampling the input image;
An enlarged image generating means for enlarging the low resolution image;
Image subtracting means for calculating difference data which is a difference between pixel values of corresponding pixels of the input image and the low-resolution image;
The enlarged image generating means includes
Interpolation image generation means for generating an interpolation image obtained by enlarging the enlargement target image by interpolating pixels between pixels of the enlargement target image;
Selecting a pixel in the interpolated image, extracting a block of pixels centered on the selected pixel, clustering the color of the pixels in the block into a first class and a second class, Representative color estimation means for deriving a probability that a pixel color belongs to the first class, a probability that the color of the pixel belongs to the second class, and a representative color of the first class and the second class;
Probability that the color of the pixel belongs to the first class for each individual pixel in the block extracted by the representative color estimation means when the distance between the representative colors of the first class and the second class is greater than or equal to a threshold value If the condition is higher than the probability of belonging to the second class, the color of the pixel is replaced with a representative color of the first class, and if the condition is not satisfied, the color of the pixel is changed to the second class. A color correcting unit that substitutes the average color of the replaced color derived for each block for each block when one pixel belongs to a plurality of blocks. An image compression apparatus comprising: The representative color estimation means uses the EM algorithm to determine the probability that the color of each pixel in the block belongs to the first class, the probability that the color belongs to the second class, the representative of the first class and the second class. The image compression apparatus according to claim 13, wherein a color is derived. An image expansion device that receives a low-resolution image obtained by reducing the resolution of an original image and difference data that is a difference between pixel values of corresponding pixels of the original image and the low-resolution image, and expands the low-resolution image. There,
An enlarged image generating means for enlarging the low resolution image;
The enlarged image generating means includes an enlarged image and an image adding means for calculating a sum of pixel values of corresponding pixels of the difference data,
The enlarged image generating means includes
Interpolation image generation means for generating an interpolation image obtained by enlarging the enlargement target image by interpolating pixels between pixels of the enlargement target image;
Selecting a pixel in the interpolated image, extracting a block of pixels centered on the selected pixel, clustering the color of the pixels in the block into a first class and a second class, Representative color estimation means for deriving a probability that a pixel color belongs to the first class, a probability that the color of the pixel belongs to the second class, and a representative color of the first class and the second class;
Probability that the color of the pixel belongs to the first class for each individual pixel in the block extracted by the representative color estimation means when the distance between the representative colors of the first class and the second class is greater than or equal to a threshold value If the condition is higher than the probability of belonging to the second class, the color of the pixel is replaced with a representative color of the first class, and if the condition is not satisfied, the color of the pixel is changed to the second class. A color correcting unit that substitutes the average color of the replaced color derived for each block for each block when one pixel belongs to a plurality of blocks. An image expansion apparatus characterized by comprising: The representative color estimation means uses the EM algorithm to determine the probability that the color of each pixel in the block belongs to the first class, the probability that the color belongs to the second class, the representative of the first class and the second class. The image expansion device according to claim 15, wherein a color is derived. An image compression device and an image expansion device;
The image compression device includes:
Down-sampling means for generating a low-resolution image having a lower resolution than the input image by down-sampling the input image, and transmitting the low-resolution image to the image expansion device;
First enlarged image generating means for enlarging the low resolution image;
Image subtraction means for calculating difference data that is a difference between pixel values of corresponding pixels of the input image and the low-resolution image, and transmitting the difference data to the image expansion device;
The image expansion device includes:
Second enlarged image generating means for enlarging the low resolution image received from the image compression device;
An image enlarged by the second enlarged image generating means, and an image adding means for calculating a sum of pixel values of corresponding pixels of the difference data received from the image compression device;
The first enlarged image generating means and the second enlarged image generating means are:
Interpolation image generation means for generating an interpolation image obtained by enlarging the enlargement target image by interpolating pixels between pixels of the enlargement target image;
Selecting a pixel in the interpolated image, extracting a block of pixels centered on the selected pixel, clustering the color of the pixels in the block into a first class and a second class, Representative color estimation means for deriving a probability that a pixel color belongs to the first class, a probability that the color of the pixel belongs to the second class, and a representative color of the first class and the second class;
Probability that the color of the pixel belongs to the first class for each individual pixel in the block extracted by the representative color estimation means when the distance between the representative colors of the first class and the second class is greater than or equal to a threshold value If the condition is higher than the probability of belonging to the second class, the color of the pixel is replaced with a representative color of the first class, and if the condition is not satisfied, the color of the pixel is changed to the second class. A color correcting unit that substitutes the average color of the replaced color derived for each block for each block when one pixel belongs to a plurality of blocks. An image transmission system comprising: Select a pixel in an interpolated image that is an enlarged image including pixels interpolated between pixels of the image before enlargement, extract a block of pixels centered on the selected pixel, and extract the pixels in the block Clustering colors into a first class and a second class, the probability that the color of each pixel in the block belongs to the first class, the probability that the pixel belongs to the second class, the first class and the A representative color estimation step for deriving a second class of representative colors;
Probability that the color of the pixel belongs to the first class for each individual pixel in the block extracted in the representative color estimation step when the distance between the representative colors of the first class and the second class is greater than or equal to a threshold value If the condition is higher than the probability of belonging to the second class, the color of the pixel is replaced with a representative color of the first class, and if the condition is not satisfied, the color of the pixel is changed to the second class. A color correction step in which, when one pixel belongs to a plurality of blocks, the average color of the replaced color derived for each block is used as the color of the pixel. An image processing method characterized by comprising: In the representative color estimation step, the probability that the color of each pixel in the block belongs to the first class, the probability that the color of each pixel in the block belongs to the second class, the representative of the first class and the second class, by the EM algorithm The image processing method according to claim 18, wherein a color is derived. On the computer,
Select a pixel in an interpolated image that is an enlarged image including pixels interpolated between pixels of the image before enlargement, extract a block of pixels centered on the selected pixel, and extract the pixels in the block Clustering colors into a first class and a second class, the probability that the color of each pixel in the block belongs to the first class, the probability that the pixel belongs to the second class, the first class and the A representative color estimation process for deriving a representative color of the second class; and a distance in the block extracted by the representative color estimation process when a distance between the representative colors of the first class and the second class is greater than or equal to a threshold value If the condition that the probability that the color of the pixel belongs to the first class is higher than the probability that the pixel belongs to the second class is satisfied for each pixel, the color of the pixel is replaced with the representative color of the first class And the above conditions are met If not, the color of the pixel is replaced with the representative color of the second class, and when one pixel belongs to a plurality of blocks, the color of the replaced color derived for the pixel for each block An image processing program for executing color correction processing using an average as the color of the pixel. In the computer,
In the representative color estimation process, the probability that the color of each pixel in the block belongs to the first class, the probability that the color of each pixel in the block belongs to the second class, the representative of the first class and the second class, by the EM algorithm. The image processing program according to claim 20, wherein a color is derived.

Images