JP4517288B2 - Image processing apparatus, image processing method, and program thereof - Google Patents

Description

  The present invention relates to an image processing apparatus that enlarges an image.

The binary image enlargement process is one of the basic processes for a system that performs editing, filing, display, printing, and the like of a digital copying machine, a multifunction machine, a printer, a FAX, a scanner, and the like. In recent years, with the widespread use of high-resolution digital copiers, multifunction peripherals, and printers, it has been desired to obtain high-quality output results at the time of print output, and the importance of high-quality enlargement processing has increased.
Typical existing methods for enlarging a binary image include a nearest neighbor method and a linear interpolation method. The nearest neighbor method is a method of using the pixel value of the pixel having the closest distance when the pixel is inversely mapped on the original image as the pixel value after enlargement. Since this method has a small amount of calculation, it can be processed at high speed. However, since one pixel of the original image is directly expanded into a rectangular shape, in a binary image, jaggy in the hatched portion and the edge portion is conspicuous, and the degree of image quality deterioration is very large when the magnification is large.
As an attempt to solve these problems, new methods such as Patent Document 1 and Patent Document 2 have been proposed.
In the technique described in Patent Document 1, matching is performed with a template prepared in advance for each predetermined region centered on the pixel of interest, and enlargement is performed using a smoothing enlargement pattern corresponding to the matching pattern. Accordingly, the pixel is divided into one or more pixels, and a smoothing process is performed for each pixel to enlarge and multivalue the pixel of interest. In addition, patterns are divided into a plurality of pattern groups in advance, and the priority of matching is determined in advance to prevent erroneous pattern detection. However, the present technology basically enlarges a binary image only by template matching and smoothing processing, requires a very large number of templates, and cannot perform smoothing enlargement processing on edges of patterns that are not prepared. .
In the technique described in Patent Document 2, the original image is first binarized, and the direction of the diagonal component included in the original image from the binary image matches the two-dimensional pattern (matrix data) prepared in advance for each direction. Interpolation processing is performed by obtaining intersection points and pixel values at the intersection points when the diagonal lines and pixel positions in the obtained direction are regarded as grid points. However, although this technique is corrected according to the obtained direction, it is basically an interpolation process by a linear interpolation (interpolation) method, and cannot sufficiently reduce jaggy.
JP-A-7-221976 JP 2000-228723 A

  The present invention has been made from the above-described background, and an object thereof is to provide an image processing apparatus that enlarges an image with high image quality.

[Image processing device]
In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus that performs an enlargement process of an input image, and for a target pixel included in the input image, a predetermined size including the target pixel is provided. An area feature extracting unit for extracting an image area feature, a pixel value converting unit for converting at least a pixel value in the image area based on the feature extracted by the area feature extracting unit, and the area feature extracting unit An enlarged image corresponding to the target pixel included in the image area based on the image area in which the pixel value is converted by the pixel value conversion unit with respect to the target pixel of the image area in which the extracted feature satisfies a predetermined condition An enlarged image area generating means for generating an area, and an enlarged image generating means for generating an enlarged image of the input image by applying an enlargement processing method different from the enlarged image area generating means; And an image integration means for integrating the the enlarged image region generated by the enlarged image area generation unit, and an enlarged image generated by the enlarged image generating means.

  Preferably, the pixel value conversion means converts a pixel value in the image region and a pixel value in a neighborhood region of the image region.

  Preferably, the area feature extraction unit extracts a pixel value pattern of the image area as a feature of the image area when the pixel value of the image area satisfies a predetermined condition.

  Preferably, the area feature extraction unit corrects the pixel value pattern when the extracted pixel value pattern is a predetermined pattern.

  Preferably, the area feature extraction unit calculates a plurality of angles related to a gradation change based on a pixel value pattern in the image area, and the gradation change in the image area based on the calculated plurality of angles. The change direction is extracted as the feature, and the pixel value conversion unit converts the pixel value in the image region according to the change direction extracted by the region feature extraction unit.

  Preferably, the pixel value conversion unit performs a smoothing process with reference to a pixel group at a position corresponding to the change direction extracted by the region feature extraction unit.

  Preferably, the pixel value conversion means performs an edge emphasis process for emphasizing a gradation change by using the plurality of pixel values subjected to the smoothing process.

  Preferably, the pixel value conversion unit converts a pixel value in the image region and a pixel value in a neighborhood region of the image region based on the feature extracted by the region feature extraction unit, and The enlarged image area generation unit is configured to detect the feature of the image area and the image area converted by the pixel value conversion unit for the target pixel of the image area in which the feature extracted by the area feature extraction unit satisfies a predetermined condition. Based on the pixel value and the pixel value of the neighboring region converted by the pixel value conversion means, an enlarged image region corresponding to the target pixel is generated.

  Preferably, the enlarged image generation unit generates an enlarged image of the input image by applying an enlargement method that has a smaller processing load than the enlargement process performed by the enlarged image region generation unit.

  Preferably, the enlarged image generating unit generates an enlarged image for each input image or for each of a plurality of lines in the input image.

  Preferably, when the plurality of enlarged image regions generated by the enlarged image region generating unit overlap each other in the enlarged image, the image integration unit is configured to use the pixels of the overlapping portion based on the enlarged image regions. Determine the value.

  Preferably, when the plurality of enlarged image regions generated by the enlarged image region generating unit overlap each other in the enlarged image, the image integration unit corresponds to the same pixel with respect to an overlapping portion of the enlarged image regions. An average value of a plurality of pixel values is calculated.

  Preferably, the image integration unit replaces at least a part of the enlarged image generated by the enlarged image generation unit with the enlarged image region generated by the enlarged image region generation unit.

[Image processing method]
The image processing method according to the present invention is an image processing method for performing an enlargement process of an input image, and for a target pixel included in the input image, a feature of an image region having a predetermined size including the target pixel is extracted. Then, based on the extracted feature, at least the pixel value in the image region is converted, and the extracted feature is converted into the image region in which the pixel value is converted with respect to the target pixel of the image region that satisfies the predetermined condition. Based on this, an enlarged image region corresponding to the target pixel included in the image region is generated, and an enlarged image method of the input image is generated by applying an enlargement processing method different from the generation processing of the enlarged image region. The enlarged image area and the generated enlarged image are integrated.

[program]
In addition, the program according to the present invention includes a step of extracting a feature of an image area having a predetermined size including a target pixel for the target pixel included in the input image in an image processing apparatus that performs an enlargement process of the input image; Based on the extracted feature, at least a step of converting a pixel value in the image region, and for the target pixel of the image region in which the extracted feature satisfies a predetermined condition, the pixel value is converted into the image region. And generating a magnified image of the input image by applying a magnified image processing method different from the magnified image region generating process, and generating a magnified image region corresponding to the target pixel included in the image region. And causing the image processing apparatus to execute a step of integrating the generated enlarged image region and the generated enlarged image.

  According to the image processing apparatus of the present invention, a high-quality enlarged image can be obtained with a relatively light processing load.

[Hardware configuration]
First, the hardware configuration of the image processing apparatus 2 in the present embodiment will be described.
FIG. 1 is a diagram illustrating a hardware configuration of an image processing apparatus 2 to which an image processing method according to the present invention is applied, centering on a control apparatus 20.
As illustrated in FIG. 1, the image processing apparatus 2 includes a control device 20 including a CPU 202 and a memory 204, a communication device 22, a recording device 24 such as an HDD / CD device, an LCD display device or a CRT display device, and a keyboard. A user interface device (UI device) 26 including a touch panel and the like is included.
The image processing apparatus 2 is, for example, a general-purpose computer in which an image enlargement program 5 (described later) is installed. The image processing apparatus 2 acquires image data via the communication device 22 or the recording device 24 and converts the acquired image data to an output resolution. Enlarge accordingly. For example, the image processing apparatus 2 converts the resolution to 600 dpi or 2400 dpi when outputting image data to the printer apparatus 10, and 75 dpi or the like when outputting image data to the UI apparatus 26. Convert to the resolution.

[Image enlargement program]
FIG. 2 is a diagram illustrating a functional configuration of the image enlargement program 5 which is executed by the control device 20 (FIG. 1) and implements the image processing method according to the present invention.
As illustrated in FIG. 2, the image enlargement program 5 includes an enlargement processing unit 500, an image data storage unit 570, a multi-value conversion unit 580, and a binary conversion unit 590. The enlargement processing unit 500 includes an image block setting unit 510, an image block feature extraction unit 520, a pixel value conversion unit 530, an enlarged image block generation unit 540, a simple enlarged image generation unit 550, and an image block arrangement unit 560. The pixel value conversion unit 530 includes a smoothing unit 532 and an enhancement unit 534.
Note that all or part of the image enlargement program 5 may be realized by hardware such as an ASIC.

  The image data storage unit 570 controls the memory 204 (FIG. 1) or the recording device 24 (FIG. 1), temporarily stores the image data until it is enlarged by the enlargement processing unit 500, and resolution conversion. Alternatively, a function of temporarily storing the enlarged image data subjected to the enlargement process until it is output to an output device (not shown) is realized. The image data is data described in an image format that can be processed by the image processing apparatus 2 (for example, BMP, TIFF, PNG, etc.), and is captured by a FAX (not shown) or a scanner (not shown). Image data or image data created by an application program for processing creation or editing in a personal computer (image processing apparatus 2 or the like). Enlarged image data (image data after enlargement) is also data in the same image format.

The multi-value conversion unit 580 converts the image data expressed in binary into image data expressed in multi-value, and outputs the image data to the enlargement processing unit 500. For example, when the input binary image data is represented by 1 bit of 0 or 1, the multi-value conversion unit 580 converts the binary image data into an 8-bit representation of 0 or 255.
If the input binary image data is stored in the image data storage unit 570 in an 8-bit representation in advance, the conversion unit 580 and the other conversion unit 580 enlarge the image data (8-bit representation) as it is. To the unit 500.

  The binary conversion unit 590 binarizes the multi-value enlarged image data output from the enlargement processing unit 500 and outputs the binarized image data to the image data storage unit 570. The binary conversion unit 590 performs binarization processing by threshold processing, for example. The binary conversion unit 590 of this example sets the threshold t to t = 128 in advance, outputs 0 when the pixel value of the enlarged image data is less than t, and 1 when the pixel value is greater than or equal to t. Output. Note that the threshold t may be set in advance as described above, or may be dynamically set based on, for example, the density distribution of input image data.

  The enlargement processing unit 500 includes an image block setting unit 510, an image block feature extraction unit 520, a pixel value conversion unit 530, an enlarged image block generation unit 540 (enlarged image region generation unit), and a simple enlarged image generation unit 550 (enlarged image generation unit). ), And an image block arrangement unit 560 (image integration unit). The image data to be enlarged is acquired from the multi-value conversion unit 580, and the enlarged image data generated by the enlargement processing is obtained. The data is output to the binary conversion unit 590.

  The image block setting unit 510 sets predetermined block sizes required for the processing in the image block feature extraction unit 520 and the enlarged image block generation unit 540, respectively, and the input image data (multi-value conversion unit 580) The image blocks having the respective block sizes including the target pixel are sequentially cut out from the value expression and output to the image block feature extraction unit 520.

The image block feature extraction unit 520 sets at least a part of the image blocks sequentially input from the image block setting unit 510 (for example, a rectangular portion near the center) as the attention area, and determines the image feature amount in the attention area as the attention area or attention. Extraction or calculation is performed with reference to an image block including a peripheral portion of the region. The extracted or calculated image feature amount is, for example, an edge pattern, an edge direction, the number of white (or black) pixels in the image block, or the ratio of white (or black) pixels in the image block.
Further, the image block feature extraction unit 520 further performs the region-of-interest segmentation based on the extracted or calculated image feature amount. Here, the attention area is divided, for example, by dividing an area where the attention area includes an edge (hereinafter referred to as an edge area) and an area including only white (or black) (hereinafter referred to as a flat area).

The pixel value conversion unit 530 converts a pixel value in an image block including an attention region determined to be an edge region based on the result of segmentation of the attention region by the image block feature extraction unit 520 and its peripheral portion. The pixel value in the image block converted by the pixel value conversion unit 530 is used for enlarged image block generation in the enlarged image block generation unit 540 described later.
More specifically, the pixel value conversion unit 530 includes a smoothing unit 532 and an enhancement unit 534, and the smoothing unit 532 is based on the edge direction in the attention area calculated by the image block feature extraction unit 520. Smooth the pixel values in the image block. The enhancement unit 534 enhances the contrast of the pixel values in the image block that has been smoothed based on the edge direction by the smoothing unit 532.

  The enlarged image block generation unit 540 generates an enlarged image block corresponding to the attention area determined as the edge area according to the image feature amount extracted or calculated by the image block feature extraction unit 520. The enlarged image block generation processing in the enlarged image block generation unit 540 will be described in detail later, but processing with higher priority on image quality is performed while suppressing the occurrence of jaggies for the region of interest.

  The simple enlarged image generation unit 550 enlarges the input image data stored in the image data storage unit 570 and multi-valued by the multi-value conversion unit 580, and outputs this enlarged image to the image block arrangement portion 560. The enlargement process in the simple enlargement image generation unit 550 is different from the enlargement process in the enlargement image block generation unit 540, and performs an enlargement process with a smaller processing load than the enlargement process by the enlargement image block generation unit 540, such as nearest neighbor interpolation enlargement. . Furthermore, the enlargement process by the simple enlarged image generation unit 550 can be performed not in units of image blocks but in units of input images or units of lines of input images.

The image block arrangement unit 560 is an enlarged image block generated by the enlarged image block generation unit 540 or enlarged by the simple enlarged image generation unit 550 in accordance with the image feature amount extracted or calculated by the image block feature extraction unit 520. The enlarged image data is generated by applying the processed image data. More specifically, the image block arrangement unit 560 uses the enlarged image block generation unit 540 to perform the attention region determined as the edge region based on the image feature amount extracted or calculated by the image block feature extraction unit 520. The generated enlarged image blocks are sequentially arranged at corresponding positions on the enlarged image output from the simple enlarged image generation unit 550 to generate resolution-converted or enlarged enlarged image data, and the generated enlarged image data Is output to the binary conversion unit 590.
The image block arrangement unit 560 may be arranged to replace the enlarged image block with each pixel value of the enlarged image block when sequentially arranging the enlarged image block at the corresponding position on the enlarged image output from the simple enlarged image generation unit 550. Alternatively, it may be arranged so as to be superimposed on each pixel value at a corresponding position of the enlarged image output from the simple enlarged image generation unit 550. The image block arrangement portion 560 is arranged so that a plurality of adjacent enlarged image blocks overlap each other, calculates the sum of overlapping pixels, and divides the sum of pixel values by the number of overlaps. The pixel value of the pixel is calculated.

Next, the image block feature extraction unit 520, the pixel value conversion unit 530, and the enlarged image block generation unit 540 will be described in more detail.
First, details of the image block feature extraction unit 520 will be described. Note that the region of interest is a 2 × 2 pixel size block (a square region in which the direction orthogonal to each other is 2 pixels), and the peripheral region including the region of interest is a 4 × 4 pixel size block (the direction orthogonal to each other is 4 pixels). A case of a square area) will be described as a specific example.

FIG. 3 is a block diagram illustrating a functional configuration of the image block feature extraction unit 520 in the present embodiment.
As shown in FIG. 3, the image block feature extraction unit 520 includes an edge pattern selection unit 522 and an edge direction estimation unit 524.
The edge pattern selection unit 522 selects the edge pattern of the 2 × 2 image block that is the attention area, and based on the selected edge pattern, the attention area is the edge area or is a flat area that is only white (or black). Determine if there is.
In addition, when the attention area is an edge area, the edge pattern selection unit 522 determines an edge angle corresponding to the edge pattern as an initial edge angle of the attention area.

FIG. 4 is a diagram illustrating patterns and edge angles in a 2 × 2 image block.
As illustrated in FIG. 4, corresponding edge angles are set for the respective edge patterns, and the edge pattern selection unit 522 determines the initial edge angle depending on which edge pattern is the region of interest.
When the attention area is a flat pattern, the attention area is cut as a flat area.
However, when the attention area is an edge pattern (checkered edge pattern) indicated by a thick frame in FIG. 4, the initial angle cannot be uniquely determined only by the edge pattern of the attention area. That is, it cannot be determined whether the initial edge angle is 45 ° or 135 °. The reason for this is that taking the upper checkered edge pattern a in the thick frame illustrated in FIG. 4 as an example, in a binary image, when the region of interest is this checkered edge pattern a, black pixels appear in the 45 ° direction. Both the case of a part of the continuing area and the case of a part of the area where white pixels continue in the 135 ° direction are conceivable, and it cannot be determined only by the attention area alone of the 2 × 2 image block. It is.

Therefore, the edge pattern selection unit 522 in the present embodiment, as shown in FIG. 5, includes a part of the attention area (checkered edge pattern area in FIG. 5) and the 2 × 2 image immediately before in the raster scan order. Based on the edge pattern of the block area (area surrounded by a thick frame in FIG. 5), the initial edge angle of the attention area which is a checkered edge pattern is determined.
FIG. 6 shows an example of all possible combinations of the checkered edge pattern and the edge pattern of the previous 2 × 2 image block area (upstream in the scanning direction) and an initial edge angle determined by the combination. FIG.
As shown in FIG. 6, when the region of interest is a checkered edge pattern, the edge pattern selection unit 522 refers to the edge pattern of the previous image block and determines either 45 ° or 135 °. Decide whether to use the initial edge angle.
In this example, the initial edge angle is determined based on the combination of the two blocks of the checkered edge pattern and the edge pattern of the previous image block. However, the present invention is not limited to this. For example, the edge pattern selection unit 522 may determine the initial edge angle based on the combination of the edge pattern of the region of interest and the edge pattern of the next image block, and the previous edge and the next edge. The initial edge angle in the checkered edge pattern may be determined by a combination of edge patterns of three blocks, and the number and position of image blocks to be combined for determining the initial edge angle of the checkered edge pattern depends on the required processing speed and image quality. It is variable according to.

Further, the edge pattern selection unit 522 extracts a specific pattern and corrects the initial edge angle and the edge pattern of the related image block in order to forcibly correct the extracted specific pattern to a predetermined pattern.
FIG. 7 illustrates a part of an image including a specific pattern to be corrected as an example of extraction of the specific pattern.
As illustrated in FIG. 7, the uneven pattern of the image surrounded by the dotted line in the figure is a pattern that protrudes (or is recessed) by one pixel. For example, the resolution is obtained when a document is captured as an electronic document by a FAX or a scanner. This is a pattern generated due to the above, and is not a meaningful feature pattern constituting an original image. Such a concavo-convex pattern causes jaggies after enlarging processing and greatly affects image quality.

FIG. 8 is a diagram specifically illustrating the extraction and correction of the concave / convex pattern (projecting or recessed by one pixel). In this figure, a region surrounded by a thick frame is a region of interest.
As shown in FIG. 8 (A), a thick frame region of interest (fourth region shown in FIG. 8 (A)) and a plurality of peripheral regions including a part thereof (the first region shown in FIG. 8 (A)). 1 to 3), the first area is “1110”, the second area is “1000”, and the third area is a bit pattern. Is “1011”, and the fourth area (ie, the attention area) is “0010”. Accordingly, as shown in FIG. 8A, the edge pattern selection unit 522 determines that the edge pattern sequences in the first to fourth areas are “1110”, “1000”, “1011”, and “0010”. In this case, it is determined that there is a concavo-convex pattern (in this example, a convex pattern protruding to the right), and this concavo-convex pattern is extracted.

The edge pattern selection unit 522 corrects the concavo-convex pattern extracted as described above (in this example, a pattern convex to the right), and the edge pattern and initial edge angle of each of the first to fourth regions are corrected as shown in FIG. Change as shown in B). That is, the edge pattern selection unit 522 changes the first area to the edge pattern “1010” and the initial edge angle of 0 °, changes the second area to the flat pattern (bit pattern “0000”), The region 3 is changed to the edge pattern “1010” and the initial edge angle 0 °, and the fourth region (region of interest) is changed to the flat pattern “0000”.
As described above, the edge pattern selection unit 522 can correct the uneven pattern (in this example, the convex pattern to the right) by changing the edge pattern and the initial edge angle of the attention area and the peripheral area.

  FIG. 9 is a diagram showing a specific example of the uneven pattern correction in a case different from the example shown in FIG. By correcting the uneven pattern that protrudes or is recessed by one pixel in this way, the uneven pattern is corrected in the image illustrated in FIG. 7 as shown in FIG. In the present embodiment, as described above, the description is limited to the extraction and correction of the concavo-convex pattern. However, the present invention is not limited to this, and depends on the type of input image data, the required image quality, or the processing speed. Thus, extraction and correction of a specific pattern different from the concave / convex pattern (for example, an isolated point pattern isolated by one pixel) may be performed.

Next, details of the processing of the edge direction estimation unit 524 will be described.
FIG. 11 is an explanatory diagram of a specific example of the attention area and the peripheral area and an example of the edge direction of the attention area. FIG. 11A shows an example of the attention area (the area within the thick frame in FIG. 11) and the peripheral area. The flow of edge direction estimation processing by the edge direction estimation unit 524 will be described using the example shown in FIG.
FIG. 12 is a flowchart illustrating an example of edge direction estimation processing (S10) by the edge direction estimation unit 22.

In step 110 (S110), the edge direction estimation unit 524 sets the initial edge of the region of interest that has already been set by the edge pattern selection unit 522 as a variable (total edge angle) that represents the sum of the edge angles of the region of interest and the reference region. Substitute the angle.
The edge direction estimation unit 524 initializes the angle reference number as a variable for counting the reference number of the edge angle to “1”.

In step 120 (S120), the edge direction estimation unit 524 determines whether the edge direction estimation unit 524 is in the peripheral region (region outside the bold frame) shown in FIG. 11A according to the initial edge angle of the region of interest set by the edge pattern selection unit 522. To select a reference region to be used for estimating the edge direction.
FIG. 13 is a diagram illustrating a reference region used for estimating the edge direction. A 2 × 2 pixel indicated by a bold frame in the drawing is a reference area. FIG. 13A illustrates a reference region when the initial edge angle set by the edge pattern selection unit 522 is “angle 0 °”, and FIG. 13B illustrates that the initial edge angle is “angle 90 °”. ”Is illustrated, and FIG. 13C illustrates the reference region when the initial edge angle is an angle other than“ angle 0 ° or 90 ° ”. Note that the number of reference regions in the case of the angle 0 ° and the angle 90 ° illustrated in FIGS. 13A and 13B is 2, and the angle other than the angles 0 ° and 90 ° illustrated in FIG. The number of reference areas is four.
In the specific example shown in FIG. 11, since the initial edge angle of the attention area is “angle 45 °”, the four reference areas illustrated in FIG. 14C are candidates for selection. The selection of the reference area is not limited to that shown in FIG. 13. For example, in the case of FIG. 13C, the number of reference areas is set to 8, or the reference area corresponding to each angle is selected. It may be set.

In step 130 (S130), the edge direction estimation unit 524 determines whether one of the reference regions selected in S120 is an edge pattern suitable for edge direction estimation. For example, the edge direction estimation unit 524 takes three types of edges as shown in FIG. 14B when the reference area on the right side is taken as an example of the reference area with respect to the attention area illustrated in FIG. A pattern is a candidate. Then, in S130, the edge direction estimation unit 524 generates a third edge pattern (initial edge angle 135 °) that is significantly different from the initial edge angle (45 ° in this example) in the region of interest as shown in FIG. Is determined to be an edge pattern that is not suitable for the edge direction estimation of the region of interest shown in FIG.
In the case of the specific example shown in FIG. 11, the edge pattern of the reference area on the right side with respect to the attention area is the first edge pattern shown in FIG. It is determined that the edge pattern is suitable for estimation.
As a result of this determination, the edge direction estimation unit 524 proceeds to the process of S140 when the reference area is suitable for the edge direction estimation in the target area of interest, and proceeds to the process of S150 otherwise. To do.

  In step 140 (S140), the edge direction estimation unit 524 adds the initial edge angle of the reference region to the total edge angle, and increments the angle reference number.

  In step 150 (S150), the edge direction estimation unit 524 determines whether or not all selectable reference areas have been processed. If it is determined that the processing has been completed, the process proceeds to S160 and ends. If it is determined that it is not, the process returns to S130 and the next reference area is processed.

  In step 160 (S160), the edge direction estimation unit 524 refers to the sum of the initial edge angle of the region of interest and the initial edge angle of the reference region determined to be suitable for the edge direction estimation in S130 (edge angle sum). The average edge angle divided by the number is set as the estimated edge angle of the region of interest, and the process ends.

In the specific example shown in FIG. 11, as shown in FIG. 11B, the initial edge angle of the upper reference area is 0 °, the initial edge angle of the left reference area is 0 °, and the initial edge angle of the lower reference area. 45 °, the initial edge angle of the right reference region is 45 °, and the sum of the initial edge angle of the attention region of 45 ° is 135 °. The edge direction estimation unit 524 obtains an average edge angle of 27 ° by dividing the total 135 ° by the angle reference number of 5. This average edge angle is the estimated edge angle with respect to the region of interest shown in FIG.
Further, the edge direction estimation unit 524 classifies the estimated edge angle Θ with respect to the obtained attention area into angle ranges (directions 0 to 7) in directions (8 directions) divided every 22.5 °. In this example, an angle range centered on 0 ° or ± 180 ° is “direction 0”, and an angle range centered on 22.5 ° or −157.5 ° is “direction 1”, 45 ° or −135. The angle range centered on ° is “direction 2”, the angle range centered on 67.5 ° or −112.5 ° is “direction 3”, and the angle range centered on 90 ° or −90 ° is “direction”. 4 ”, an angle range centered on 112.5 ° or −67.5 ° is“ direction 5 ”, and an angle range centered on 135 ° or −45 ° is“ direction 6 ”, 157.5 ° or −22. An angle range centered at .5 ° is defined as “direction 7”, and the edge direction estimation unit 524 classifies the estimated edge angle Θ into any angle range (range of ± 11.25 ° from each of the above center angles). . Since the estimated edge angle Θ (= 27 °) in the specific example of FIG. 11 described above is included in the range of 22.5 ° ± 11.25 °, the edge direction estimation unit 524 performs the processing for the attention area shown in FIG. The final estimated edge direction is determined to be “direction 1” (the direction of the arrow line in FIG. 11B).
In this example, the estimated edge angle Θ is calculated and then normalized (classified) in any of the eight directions. However, the present invention is not limited to this. If a more accurate edge direction is required, twelve directions ( You may normalize in more directions, such as 1 direction (every 15.0 degrees) and 16 directions (every 12.25 degrees).

Next, the pixel value conversion unit 530 will be described.
As illustrated in FIG. 2, the pixel value conversion unit 530 includes a smoothing unit 532 and an enhancement unit 534.
The smoothing unit 532 performs pixel value smoothing processing corresponding to the estimated edge direction of the attention area estimated by the image block feature extraction unit 520 on the pixels in the attention area and the peripheral area including the attention area.
FIG. 15 is a diagram illustrating a smoothing kernel corresponding to the estimated edge direction. It should be noted that the numbers in the smoothing kernel shown in the figure are weighting coefficients by which the pixel values are multiplied.
For example, when the estimated edge direction of the attention area is “direction 3”, the smoothing unit 532 performs a smoothing process using the third smoothing kernel in FIG. 15 according to this “direction 3”. . In this case, as shown in FIG. 16, for example, when the smoothing process of the center pixel P is performed, the smoothing unit 532 uses the pixel a and the pixel b to calculate the smoothed pixel value P ′ as follows: Calculate according to (1).

  Pixel value of interest P ′ = (2.0 × P + 1.0 × (a + b)) / 4.0 (1)

  Note that the smoothing kernel used by the smoothing unit 532 is not limited to that shown in FIG. 15, and is different depending on the type of input image data or the required degree of smoothing. Also good.

The emphasizing unit 534 performs processing on the attention area that has been subjected to the smoothing process (smoothing processing using the smoothing kernel corresponding to the estimated edge direction of the attention area) by the smoothing unit 532 and the pixels in the peripheral area including the attention area. Then, contrast enhancement processing is performed.
FIG. 17 is a diagram illustrating a specific example of the enhancement kernel applied by the enhancement unit 534.
The value of “1.60” shown in FIG. 17 represents a weighting factor corresponding to the pixel value at the center of the enhancement kernel shown in FIG. 17A, and the value of “0.15” is above, below, left, and right of the center pixel. The weighting coefficient corresponding to the arranged pixel value is represented.
FIG. 17B is an explanatory diagram when the pixel P is emphasized by the enhancement kernel shown in FIG. As shown in FIG. 17B, the emphasis unit 534 refers to the pixels a, b, c, and d that exist at the top, bottom, left, and right, for example, when performing the emphasis processing of the center reference pixel P, and also performs weighting. For example, “1.60” is used for the pixel of interest at the center, and “0.15” is used for the peripheral reference pixels. Thus, the enhancement unit 534 calculates the pixel value P ′ of the pixel P according to the following equation (2).

    Pixel value of interest P ′ = 1.60 × P−0.15 × (a + b + c + d) (2)

  As described above, through the processing of the smoothing unit 532 and the emphasizing unit 534, the pixel value conversion unit 530 converts the pixel values of the attention area and the surrounding area for use in the enlarged image block generation unit 540 described later.

Next, the enlarged image block generation unit 540 will be described in detail.
The enlarged image block generation unit 540 uses the edge pattern and the estimated edge direction for the region of interest obtained by the image block feature extraction unit 520, and the pixel values of the region of interest and its surrounding region converted by the pixel value conversion unit 530. Thus, an enlarged image block for the attention area is generated.

FIG. 18 is a diagram for explaining enlarged image block generation processing by the enlarged image block generation unit 540.
First, the enlarged image block generation unit 540 calculates a pixel value corresponding to a 3 × 3 image block using the pixel value converted by the pixel value conversion unit 530 based on the edge pattern and the estimated edge direction of the region of interest.
FIG. 18A shows the attention area and the peripheral area exemplified in FIG. As for the region of interest, the edge pattern “1110” and the estimated edge direction “direction 1” are obtained by the image block feature extraction unit 520 as described above.
When the combination of feature amounts is (edge pattern 1110) and (estimated edge direction “direction 1”), the enlarged image block generation unit 540 corresponds to a 3 × 3 image block as shown in FIG. Assuming that the respective pixels are p0 to p8, the pixel values of p0 to p8 are calculated by the following formula based on the pixel values {a, b, c, d} of the attention area shown in FIG. .

p0 = a
p1 = (a + b) / 2
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = d
p6 = c
p7 = (p4 + b) / 2
p8 = d

  The enlarged image block generation unit 540 uniquely determines these calculation formulas based on a combination of (edge pattern) and (estimated edge direction) (a combination of feature amounts), and 3 × using the determined calculation formula Pixel values corresponding to three image blocks are calculated.

FIG. 19 is a diagram illustrating calculation formulas corresponding to other combinations of feature quantities (combination of edge patterns and estimated edge directions).
FIG. 19A illustrates a calculation formula corresponding to the combination of the edge pattern “1000” and the estimated edge direction “direction 1”.
p0 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p1 = (p4 + c) / 2
The enlarged image block generation unit 540 calculates a pixel value corresponding to a 3 × 3 image block using the above formula when the combination of feature amounts is the edge pattern “1000” and the estimated edge direction “direction 1”. .

FIG. 19B illustrates a calculation formula corresponding to the combination of the edge pattern “1100” and the estimated edge direction “direction 5”.
p0 = a
p1 = (a + b) / 2
p2 = b
p4 = (a + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p3 = (p4 + c) / 2
p5 = (p4 + b) / 2
The enlarged image block generation unit 540 calculates a pixel value corresponding to a 3 × 3 image block using the above formula when the combination of feature amounts is the edge pattern “1100” and the estimated edge direction “direction 5”. .

FIG. 19C illustrates a calculation formula corresponding to the combination of the edge pattern “1100” and the estimated edge direction “direction 2”.
p0 = a
p1 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
When the combination of feature amounts is the edge pattern “1100” and the estimated edge direction “direction 2”, the enlarged image block generation unit 540 calculates a pixel value corresponding to a 3 × 3 image block using the above calculation formula. .

FIG. 19D illustrates a calculation formula corresponding to the combination of the edge pattern “0101” and the estimated edge direction “direction 7”.
p0 = a
p2 = b
p3 = (a + c) / 2
p4 = (a + d) / 2
p5 = (b + d) / 2
p6 = c
p8 = d
p1 = (p4 + b) / 2
p7 = (p4 + c) / 2
The enlarged image block generation unit 540 calculates a pixel value corresponding to a 3 × 3 image block using the above formula when the combination of feature amounts is the edge pattern “0101” and the estimated edge direction “direction 7”. .

  In the case of other combinations (edge pattern and estimated edge direction), the enlarged image block generation unit 540 similarly performs calculation according to the calculation formula corresponding to each edge pattern and estimated edge direction to obtain 3 × Pixel values corresponding to three image blocks are calculated.

  Next, the enlarged image block generation unit 540 calculates the pixel value corresponding to the 3 × 3 image block calculated as described above, and a plurality of reference pixels in the peripheral area selected based on the estimated edge direction of the attention area. Is used to generate a 4 × 4 image block.

FIG. 20 is a diagram illustrating a method of selecting reference pixels r0 to r13 based on the estimated edge direction.
When the estimated edge direction of the attention area is from direction 1 (22.5 °) to direction 3 (67.5 °), the enlarged image block generation unit 540 is surrounded by a thick line frame in FIG. The reference pixels r0 to r5 are selected so as to be 3 pixels from the upper left to the lower and 3 pixels from the lower right to the upper.
The enlarged image block generation unit 540 is indicated by a thick frame in FIG. 20B when the estimated edge direction of the attention area is from the direction 5 (112.5 °) to the direction 7 (157.5 °). In this way, the reference pixels r0 to r5 are selected so that there are 3 pixels from the lower left to the upper and 3 pixels from the upper right to the lower.
In addition, the enlarged image block generation unit 540 selects four upper and lower pixels as the reference pixels r6 to r13, as shown in FIGS. 20A and 20B, regardless of the estimated edge direction.
As described above, the enlarged image block generation unit 540 selects the reference pixel based on the estimated edge direction in the attention area. Note that the selection of reference pixels is not limited to the selection from the two patterns illustrated in FIG. 20, and more reference pixel selection patterns may be prepared according to the estimated edge direction. Also, the reference pixel to be selected may be changed according to the estimated edge direction.

FIG. 21 is a diagram illustrating a process of generating a 4 × 4 pixel enlarged image block.
As illustrated in FIG. 21, the enlarged image block generation unit 540 includes the calculated pixel values p0 to p8 corresponding to 3 × 3 image blocks, and reference pixels r0 to r13 selected based on the estimated edge direction in the attention area. Are used to calculate pixel values (s0 to s15) corresponding to the enlarged image block of 4 × 4 pixels according to the following calculation formula to generate a 4 × 4 enlarged image block.

s0 = 0.2 × r6 + 0.16 × r0 + 0.64 × p0
s1 = 0.2 × r7 + 0.32 × p0 + 0.48 × p1
s2 = 0.2 × r8 + 0.48 × p1 + 0.32 × p2
s3 = 0.2 × r9 + 0.64 × p2 + 0.16 × r1
s4 = 0.08 * r0 + 0.32 * p0 + 0.12 * r2 + 0.48 * p3
s5 = 0.16 × p0 + 0.24 × p1 + 0.24 × p3 + 0.36 × p4
s6 = 0.24 × p1 + 0.16 × p2 + 0.36 × p4 + 0.24 × p5
s7 = 0.32 × p2 + 0.08 × r1 + 0.48 × p5 + 0.12 × r3
s8 = 0.12 * r2 + 0.48 * p3 + 0.08 * r4 + 0.32 * p6
s9 = 0.24 × p3 + 0.36 × p4 + 0.16 × p6 + 0.24 × p7
s10 = 0.36 × p4 + 0.24 × p5 + 0.24 × p7 + 0.16 × p8
s11 = 0.48 × p5 + 0.12 × r3 + 0.32 × p8 + 0.08 × r5
s12 = 0.16 × r4 + 0.64 × p6 + 0.2 × r10
s13 = 0.32 × p6 + 0.48 × p7 + 0.2 × r11
s14 = 0.48 × p7 + 0.32 × p8 + 0.2 × r12
s15 = 0.64 × p8 + 0.16 × r5 + 0.2 × r13

  By performing the processing as described above, 4 × 4 pixel enlarged image blocks (s0 to s15) corresponding to the attention area determined as the edge area by the image block feature extraction unit 520 are generated.

  As described above, the enlarged image block generation unit 540 performs the enlarged block generation process on the image block determined as the edge region according to the image feature amount extracted or calculated by the image block feature extraction unit 540 as described above. I do. As a result, it is possible to generate an enlarged image block having smooth edges with reduced jaggy.

Next, the image block arrangement unit 560 will be described.
The image block placement unit 560 sequentially places the enlarged image block corresponding to the attention area generated by the enlarged image block generation unit 540 at a corresponding position on the enlarged image output from the simple enlarged image generation unit 550 by a predetermined method. Deploy.

FIG. 22 is a diagram illustrating a specific example in which the 4 × 4 pixel enlarged image block generated by the enlarged image block generation unit 540 is arranged on the enlarged image output from the simple enlarged image generation unit 550.
As shown in FIG. 22, the enlarged image block bk0 and the enlarged image block bk1 sequentially generated by the enlarged image block generation unit 540 are sequentially arranged at corresponding positions on the enlarged image output from the simple enlarged image generation unit 550. The At this time, the image block arrangement unit 560 may arrange the pixel values on the enlarged image so as to replace the pixel values (S0 to S15) of the enlarged block, or each pixel value on the enlarged image and the enlarged block. These pixel values (S0 to S15) may be superposed.
In addition, as illustrated in FIG. 22, the enlarged image block generation unit 540 overlaps the overlapping positions of the enlarged image blocks (the enlarged image block bk0 and the enlarged image block bk1 in the figure) on the enlarged image. Calculates the average value of the overlapping pixels of each enlarged image block. Alternatively, the image block arrangement unit 560 may calculate each pixel value by calculating the sum of the overlapping pixels and dividing the sum of the pixel values by the number of overlaps. Note that the enlarged image block of 4 × 4 pixels is generated by the enlarged image block generation unit 540 with respect to the attention area determined as the edge area by the image block feature extraction unit 520, and therefore, an edge portion in the input image data Is generated in a concentrated manner. For this reason, the overlap of the above-mentioned enlarged image blocks frequently occurs at the edge portion. The averaging process of the pixel values of the enlarged image block by the overlap has an effect of further smoothing the edge, and can suppress the occurrence of jaggies.

Next, the overall operation of the enlargement processing unit 500 will be described.
FIG. 23 is a flowchart illustrating the image enlargement process (S20) by the enlargement processing unit 500.
In step 210 (S210), the image block setting unit 510 sets predetermined image block sizes required for processing in the image block feature extraction unit 520 and the enlarged image block generation unit 540, and the set block size. Are sequentially cut out (for example, in the raster scan order), and image blocks of each block size are output to the image block feature extraction unit 520.

In step 220 (S220), the image block feature extraction unit 520 determines whether the attention area in the input image block is an edge area or a flat area.
When the attention area is an edge area, the image block feature extraction unit 520 sets an edge pattern of the attention area and an edge angle corresponding to the edge pattern as an initial edge angle, and performs extraction and correction of a specific edge pattern. In order to perform an enlarged image block generation process that reduces jaggy, the process proceeds to S230.
Further, when the attention area is a flat area, the image block feature extraction unit 520 returns to the process of S210 without performing the processes of S230 to S280, and proceeds to the process of the next image block.

  In step 230 (S230), the image block feature extraction unit 520 determines the edge direction of the attention area from the initial edge angle Θ of the attention area determined as the edge area and the reference area in one or more peripheral areas including the attention area. Estimate θ. For example, the image block feature extraction unit 520 calculates an average value of the obtained plurality of edge angles Θ, and sets the average value as the edge direction θ. The edge direction is not limited to the average edge angle.

In step 240 (S240), the pixel value conversion unit 530 (smoothing unit 532) uses the smoothing kernel corresponding to the estimated edge direction θ obtained by the image block feature extraction unit 520, and uses the smoothed kernel and its surrounding region. The inner pixel is smoothed.
Further, the pixel value conversion unit 530 (enhancement unit 534) performs contrast enhancement processing on the attention area and the pixels in the surrounding area on which the smoothing process has been performed.

In step 250 (S250), the enlarged image block generation unit 540 uses the edge pattern of the attention area obtained by the image block feature extraction section 520 and the estimated edge direction θ of the attention area, and the pixel value conversion section 530 performs pixel detection in S240. An enlarged image block for the region of interest is generated using the region of interest subjected to the value conversion process and the pixel values in the peripheral region.
As described above, when it is determined in S220 that the region is an edge region, an enlarged image block that suppresses jaggies of such an edge portion is generated. Thereby, it is possible to generate a high-quality enlarged image block with less jaggy for the edge portion and the like.

In step 260 (S260), the simple enlarged image generation unit 550 converts the input image data multi-valued by the multi-value conversion unit 580 (input image data stored in the image data storage unit 570) for each image or several lines. Each image is acquired, subjected to nearest neighbor enlargement processing, and output to the image block placement unit 560.
That is, the simple enlarged image generation unit 550 generates an enlarged image by performing nearest neighbor interpolation enlargement processing or the like in parallel with or before or after the processing from S210 to S250.

In step 270 (S270), the image block placement unit 560 sequentially places the enlarged image blocks for the region of interest generated by the enlarged image block generation unit 540 at corresponding positions on the enlarged image generated by the simple enlarged image generation unit 550. Deploy.
Further, the image block arrangement unit 560 performs the averaging process as described above when the enlarged image blocks are arranged in an overlapping manner.
The enlarged image block or the enlarged image in units of lines after all these processes are output to the binarization processing unit 590, and after the binarization processing, are output to the image data storage unit 570 and stored therein.

  In step 280 (S280), the enlargement processing unit 500 checks whether or not the processing has been completed for all the input image data. If not, the processing returns to the processing of S210 to continue the processing. In this case, the enlarged image data is output to a predetermined output destination, and the enlargement process is terminated.

  As described above, the image processing apparatus 2 in the present embodiment extracts features of an image region having a predetermined size including the pixel of interest, and based on the extracted features, the image processing device 2 extracts the features of the image region. The pixel values in the neighboring area are converted, an enlarged image area is generated for the image area in which the extracted features satisfy a predetermined condition, and the input image is enlarged by an enlargement method different from the enlargement method of the image area. An enlarged image is generated, and the enlarged image region is arranged at a corresponding position on the enlarged image to generate an enlarged output image. By such enlargement processing, for example, with respect to the edge portion in the input image, an enlargement method for smoothing the edge is applied, and such an enlargement method for each image area with a heavy processing load is characterized as an edge portion. It can be limited to only a specific part. That is, it is possible to perform high-quality enlargement processing that suppresses jaggies in the edge portion, and to perform enlargement processing at high speed while suppressing the processing load.

It is a figure which illustrates the hardware constitutions of the image processing apparatus 2 with which the image processing method concerning this invention is applied centering on the control apparatus 20. FIG. It is a figure which illustrates the functional structure of the image expansion program 5 which is performed by the control apparatus 20 (FIG. 1) and implement | achieves the image processing method concerning this invention. It is a block diagram which shows the function structure of the image block feature extraction part 520. FIG. It is a figure which illustrates the pattern and its edge angle in a 2x2 image block. It is a figure which illustrates the field referred in order to determine the initial edge angle of the attention field, when the attention field is a checkered edge pattern. It is a figure which illustrates the initial edge angle determined by the combination with a checkered edge pattern. It is a figure showing a part of image containing the specific pattern corrected. It is a figure explaining extraction and correction | amendment of an uneven | corrugated pattern. It is a figure explaining correction | amendment of the uneven | corrugated pattern different from FIG. It is a figure which illustrates after correction | amendment of the image (a specific pattern is included) shown in FIG. It is a figure which illustrates the specific example of an attention area and a peripheral area, and the edge direction of an attention area. It is a flowchart of the edge direction estimation process (S10) by the edge direction estimation part 534. FIG. It is a figure which illustrates the reference field used for estimation of an edge direction. It is a figure explaining appropriate edge pattern judgment. It is a figure which illustrates the smoothing kernel corresponding to an estimation edge direction. It is a figure explaining the smoothing process by the smoothing kernel. It is a figure which illustrates an emphasis kernel and explains emphasis processing by this emphasis kernel. It is a figure explaining the production | generation process of the enlarged image block by the enlarged image block production | generation part 540. FIG. It is a figure which shows the other example of the calculation formula corresponding to the combination of an edge pattern and an estimated edge direction. It is a figure explaining the selection method of the reference pixels r0-r13 based on the estimated edge direction of an attention area. It is a figure explaining the production | generation process of a 4x4 pixel enlarged image block. It is a figure which illustrates the arrangement method of the enlarged image block produced | generated by the enlarged image block production | generation part 540. FIG. It is a flowchart explaining the image expansion process (S20) by the expansion process part 500. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 2 ... Image processing apparatus 5 ... Image enlargement program 500 ... Enlargement processing part 510 ... Image block setting part 520 ... Image block feature extraction part 530 ... Pixel value conversion part 532 ... Smoothing unit 534 ... enhancement unit 540 ... enlarged image block generation unit 550 ... simple enlarged image generation unit 560 ... image block arrangement unit

Claims (12)

An image processing apparatus for enlarging an input image,
For the target pixel included in the input image, based on the pixel value pattern of the target region of a predetermined size including the target pixel, determine whether the target region is an edge region,
The initial edge angle is calculated based on the pixel value pattern of the edge area, and when the initial edge angle cannot be uniquely calculated only by the pixel value pattern of the edge area, the previous edge scan is performed in the raster scan order. And / or calculate the initial edge angle with reference to the pixel value pattern of the next image area,
In accordance with the initial edge angle, set a reference region to be referred to in order to extract the gradation change direction,
Area feature extraction means for extracting a change direction of gradation as a feature of the attention area based on the pixel value pattern of the reference area and the pixel value pattern of the attention area;
A pixel value conversion unit that converts at least a pixel value in the region of interest according to the change direction of the gradation extracted by the region feature extraction unit;
Based on the attention area in which the pixel value is converted by the pixel value conversion means for the attention pixel of the attention area whose gradation change direction extracted by the area feature extraction means satisfies a predetermined condition. An enlarged image area generating means for generating an enlarged image area corresponding to a target pixel included in the area;
Applying an enlargement processing method different from the enlarged image region generating means, an enlarged image generating means for generating an enlarged image of the input image,
An image processing apparatus comprising: an enlarged image region generated by the enlarged image region generating unit; and an image integrating unit that integrates the enlarged image generated by the enlarged image generating unit. The image processing apparatus according to claim 1, wherein the pixel value conversion unit converts a pixel value in the attention area and a pixel value in a vicinity area of the attention area. The image processing apparatus according to claim 1, wherein the region feature extraction unit corrects the pixel value pattern when the extracted pixel value pattern is a predetermined pattern. The image processing apparatus according to claim 1, wherein the pixel value conversion unit performs a smoothing process with reference to a pixel group at a position corresponding to a change direction extracted by the region feature extraction unit. The image processing apparatus according to claim 4, wherein the pixel value conversion unit performs an edge enhancement process for enhancing a gradation change, using the plurality of pixel values subjected to the smoothing process. The pixel value conversion means converts a pixel value in the attention area and a pixel value in a vicinity area of the attention area based on the change direction of the gradation extracted by the area feature extraction means,
The enlarged image area generation unit is configured to detect the change direction of the gradation of the attention area, the pixel value for the attention pixel of the attention area in which the change direction of the gradation extracted by the area feature extraction unit satisfies a predetermined condition. The image according to claim 2, wherein an enlarged image area corresponding to the target pixel is generated based on the pixel value of the attention area converted by the conversion means and the pixel value of the neighboring area converted by the pixel value conversion means. Processing equipment. The image processing apparatus according to claim 1, wherein the enlarged image generation unit generates an enlarged image of the input image by applying an enlargement method that has a smaller processing load than the enlargement process performed by the enlarged image region generation unit. The image processing apparatus according to claim 1, wherein the enlarged image generation unit generates an enlarged image for each input image or for each of a plurality of lines in the input image. The image integration unit determines a pixel value of the overlapping portion based on the enlarged image regions when a plurality of enlarged image regions generated by the enlarged image region generation unit overlap each other in the enlarged image. The image processing apparatus according to claim 1. In the case where a plurality of enlarged image areas generated by the enlarged image area generating means overlap each other in the enlarged image, the image integrating means has a plurality of pixel values corresponding to the same pixel with respect to overlapping portions of the enlarged image areas. The image processing apparatus according to claim 9, wherein an average value of the image processing apparatus is calculated. The image processing device according to claim 1, wherein the image integration unit replaces at least a part of the enlarged image generated by the enlarged image generation unit with the enlarged image region generated by the enlarged image region generation unit. . In an image processing apparatus that performs an enlargement process of an input image,
For a target pixel included in the input image, based on a pixel value pattern of a target region of a predetermined size including the target pixel, determining whether the target region is an edge region;
The initial edge angle is calculated based on the pixel value pattern of the edge area, and when the initial edge angle cannot be uniquely calculated only by the pixel value pattern of the edge area, the previous edge scan is performed in the raster scan order. And / or calculating an initial edge angle with reference to a pixel value pattern of the next image area;
Setting a reference region to be referred to in order to extract a change direction of gradation according to the initial edge angle;
Extracting a change direction of gradation as a feature of the region of interest based on the pixel value pattern of the reference region and the pixel value pattern of the region of interest;
Depending on the changing direction of the extracted tone by said step of extracting, and converting at least a pixel value of the attention region,
For the attention pixel of the attention area in which the gradation change direction extracted by the extraction step satisfies a predetermined condition, the attention value is converted into the attention area based on the attention area whose pixel value is converted by the conversion step. Generating an enlarged image region corresponding to the pixel of interest included;
Applying an enlargement processing method different from the step of generating the enlarged image region to generate an enlarged image of the input image;
A program for causing the image processing apparatus to execute the step of integrating the enlarged image region generated by the step of generating the enlarged image region and the enlarged image generated by the step of generating the enlarged image.

Images