JP4517287B2 - Image processing apparatus, image processing method, and program thereof - Google Patents

Description

  The present invention relates to an image processing apparatus that enlarges an image.

The binary image enlargement process is one of the basic processes for a system that performs editing, filing, display, printing, and the like of a digital copying machine, a multifunction machine, a printer, a FAX, a scanner, and the like. In recent years, with the widespread use of high-resolution digital copiers, multifunction peripherals, and printers, it has been desired to obtain high-quality output results at the time of print output, and the importance of high-quality enlargement processing has increased.
Typical existing methods for enlarging a binary image include a nearest neighbor method and a linear interpolation method. The nearest neighbor method is a method of using the pixel value of the pixel having the closest distance when the pixel is inversely mapped on the original image as the pixel value after enlargement. Since this method has a small amount of calculation, it can be processed at high speed. However, since one pixel of the original image is directly expanded into a rectangular shape, in a binary image, jaggy in the hatched portion and the edge portion is conspicuous, and the degree of image quality deterioration is very large when the magnification is large.
As an attempt to solve these problems, new methods such as Patent Document 1 and Patent Document 2 have been proposed.
In the technique described in Patent Document 1, matching is performed with a template prepared in advance for each predetermined region centered on the pixel of interest, and enlargement is performed using a smoothing enlargement pattern corresponding to the matching pattern. Accordingly, the pixel is divided into one or more pixels, and a smoothing process is performed for each pixel to enlarge and multivalue the pixel of interest. In addition, patterns are divided into a plurality of pattern groups in advance, and the priority of matching is determined in advance to prevent erroneous pattern detection. However, the present technology basically enlarges a binary image only by template matching and smoothing processing, requires a very large number of templates, and cannot perform smoothing enlargement processing on edges of patterns that are not prepared. .
In the technique described in Patent Document 2, the original image is first binarized, and the direction of the diagonal component included in the original image from the binary image matches the two-dimensional pattern (matrix data) prepared in advance for each direction. Interpolation processing is performed by obtaining intersection points and pixel values at the intersection points when the diagonal lines and pixel positions in the obtained direction are regarded as grid points. However, although this technique is corrected according to the obtained direction, it is basically an interpolation process by a linear interpolation (interpolation) method, and cannot sufficiently reduce jaggy.
JP-A-7-221976 JP 2000-228723 A

  The present invention has been made from the above-described background, and an object thereof is to provide an image processing apparatus that enlarges an image with high image quality.

[Image processing device]
In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus that performs an enlargement process of an input image, and for a target pixel included in the input image, a predetermined size including the target pixel is provided. An area feature extracting unit for extracting features of an image area, a pixel value converting unit for converting at least a pixel value in the image area based on the feature extracted by the area feature extracting unit, and the pixel value converting unit Enlarged image area generation including one generating means for generating an enlarged image area based on the pixel value of the converted image area, and another generating means for generating the enlarged image area by a process different from the one generating means And an image arrangement unit that arranges the enlarged image region generated by the enlarged image region generation unit, and the enlarged image region generation unit includes the region feature extraction unit. Applying said generating means in response to the extracted feature Ri.

  Preferably, the pixel value conversion means calculates the pixel value in the image area and the pixel value in the vicinity area of the image area only when the pixel value of the image area satisfies a predetermined condition. Convert.

  Preferably, the area feature extraction unit extracts a pixel value pattern of the image area as a feature of the image area when the pixel value of the image area satisfies a predetermined condition.

  Preferably, the area feature extraction unit corrects the pixel value pattern when the extracted pixel value pattern is a predetermined pattern.

  Preferably, the area feature extraction unit calculates a plurality of angles related to a gradation change based on a pixel value pattern in the image area, and the gradation change in the image area based on the calculated plurality of angles. The change direction is extracted as the feature, and the pixel value conversion unit converts the pixel value in the image region according to the change direction extracted by the region feature extraction unit.

  Preferably, the pixel value conversion unit performs a smoothing process with reference to a pixel group at a position corresponding to the change direction extracted by the region feature extraction unit.

  Preferably, the pixel value conversion means performs an edge emphasis process for emphasizing a gradation change by using the plurality of pixel values subjected to the smoothing process.

  Preferably, the pixel value conversion unit converts a pixel value in the image region and a pixel value in a neighborhood region of the image region based on the feature extracted by the region feature extraction unit, and One generating unit is configured to detect the feature of the image region and the pixel of the image region converted by the pixel value converting unit with respect to the target pixel of the image region in which the feature extracted by the region feature extracting unit satisfies a predetermined condition. An enlarged image area corresponding to the target pixel is generated based on the value and the pixel value of the neighboring area converted by the pixel value conversion means.

  Preferably, the area feature extraction unit extracts a gradation change degree indicating a degree of gradation change in the image area as the feature, and the one generation unit has a processing load higher than that of the other generation unit. The enlarged image area generation means applies the one generation means when the gradation change degree of the image area is equal to or greater than a predetermined reference, and the gradation change degree of the image area If it is less than this, the other generation means is applied.

  Preferably, when the plurality of enlarged image regions generated by the enlarged image region generating unit overlap with each other in the enlarged image, the image arrangement unit is configured to use the pixels of the overlapping portion based on the enlarged image regions. Determine the value.

  Preferably, when the plurality of enlarged image regions generated by the enlarged image region generating unit overlap each other in the enlarged image, the image arranging unit corresponds to the same pixel with respect to the overlapping portion of the enlarged image regions. An average value of a plurality of pixel values is calculated.

[Image processing method]
The image processing method according to the present invention is an image processing method for performing an enlargement process of an input image, and for a target pixel included in the input image, a feature of an image region having a predetermined size including the target pixel is extracted. Then, according to the extracted feature, at least a pixel value in the image region is converted, and one generation for generating an enlarged image region based on the pixel value of the converted image region based on the extracted feature Select a method or another generation method for generating an enlarged image region by a process different from the one generation method, apply the selected generation method to generate an enlarged image region, and generate the enlarged image Arrange the area.

[program]
In addition, the program according to the present invention includes a step of extracting a feature of an image area having a predetermined size including a target pixel for the target pixel included in the input image in an image processing apparatus that performs an enlargement process of the input image; According to the extracted feature, at least a step of converting the pixel value in the image region, and one generation for generating an enlarged image region based on the pixel value of the converted image region based on the extracted feature Selecting a method or another generation method for generating an enlarged image region by processing different from the one generation method, generating an enlarged image region by applying the selected generation method, and generating And causing the image processing apparatus to execute a step of arranging the enlarged image area.

  According to the image processing apparatus of the present invention, a high-quality enlarged image can be obtained with a relatively light processing load.

[Hardware configuration]
First, the hardware configuration of the image processing apparatus 2 in the present embodiment will be described.
FIG. 1 is a diagram illustrating a hardware configuration of an image processing apparatus 2 to which an image processing method according to the present invention is applied, centering on a control apparatus 20.
As illustrated in FIG. 1, the image processing apparatus 2 includes a control device 20 including a CPU 202 and a memory 204, a communication device 22, a recording device 24 such as an HDD / CD device, an LCD display device or a CRT display device, and a keyboard. A user interface device (UI device) 26 including a touch panel and the like is included.
The image processing apparatus 2 is, for example, a general-purpose computer in which an image enlargement program 5 (described later) is installed. The image processing apparatus 2 acquires image data via the communication device 22 or the recording device 24 and converts the acquired image data to an output resolution. Enlarge accordingly. For example, the image processing apparatus 2 converts the resolution to 600 dpi or 2400 dpi when outputting image data to the printer apparatus 10, and 75 dpi or the like when outputting image data to the UI apparatus 26. Convert to the resolution.

[Image enlargement program]
FIG. 2 is a diagram illustrating a functional configuration of the image enlargement program 5 which is executed by the control device 20 (FIG. 1) and implements the image processing method according to the present invention.
As illustrated in FIG. 2, the image enlargement program 5 includes an enlargement processing unit 500, an image data storage unit 570, a multi-value conversion unit 580, and a binary conversion unit 590. The enlargement processing unit 500 includes an image block setting unit 510, an image block feature extraction unit 520, a pixel value conversion unit 530, an enlarged image block generation unit 540, and an image block arrangement unit 560.
Note that all or part of the image enlargement program 5 may be realized by hardware such as an ASIC.

  The image data storage unit 570 controls the memory 204 (FIG. 1) or the recording device 24 (FIG. 1), temporarily stores the image data until it is enlarged by the enlargement processing unit 500, and resolution conversion. Alternatively, a function of temporarily storing the enlarged image data subjected to the enlargement process until it is output to an output device (not shown) is realized. The image data is data described in an image format that can be processed by the image processing apparatus 2 (for example, BMP, TIFF, PNG, etc.), and is captured by a FAX (not shown) or a scanner (not shown). Image data or image data created by an application program for processing creation or editing in a personal computer (image processing apparatus 2 or the like). Enlarged image data (image data after enlargement) is also data in the same image format.

The multi-value conversion unit 580 converts the image data expressed in binary into image data expressed in multi-value, and outputs the image data to the enlargement processing unit 500. For example, when the input binary image data is represented by 1 bit of 0 or 1, the multi-value conversion unit 580 converts the binary image data into an 8-bit representation of 0 or 255.
If the input binary image data is stored in the image data storage unit 570 in an 8-bit representation in advance, the conversion unit 580 and the other conversion unit 580 enlarge the image data (8-bit representation) as it is. To the unit 500.

  The binary conversion unit 590 binarizes the multi-value enlarged image data output from the enlargement processing unit 500 and outputs the binarized image data to the image data storage unit 570. The binary conversion unit 590 performs binarization processing by threshold processing, for example. The binary conversion unit 590 of this example sets the threshold t to t = 128 in advance, outputs 0 when the pixel value of the enlarged image data is less than t, and 1 when the pixel value is greater than or equal to t. Output. Note that the threshold t may be set in advance as described above, or may be dynamically set based on, for example, the density distribution of input image data.

  The enlargement processing unit 500 includes an image block setting unit 510, an image block feature extraction unit 520, a pixel value conversion unit 530, an enlarged image block generation unit 540, and an image block arrangement unit 560. A multi-value conversion unit Image data to be subjected to enlargement processing is acquired from 580, and enlarged image data generated by the enlargement processing is output to the binary conversion unit 590.

  The image block setting unit 510 sets predetermined block sizes required for the processing in the image block feature extraction unit 520 and the enlarged image block generation unit 540, respectively, and the input image data (multi-value conversion unit 580) The image blocks having the respective block sizes including the target pixel are sequentially cut out from the value expression and output to the image block feature extraction unit 520.

The image block feature extraction unit 520 sets at least a part of the image blocks sequentially input from the image block setting unit 510 (for example, a rectangular portion near the center) as the attention area, and determines the image feature amount in the attention area as the attention area or attention. Extraction or calculation is performed with reference to an image block including a peripheral portion of the region. The extracted or calculated image feature amount is, for example, an edge pattern, an edge direction, the number of white (or black) pixels in the image block, or the ratio of white (or black) pixels in the image block.
Further, the image block feature extraction unit 520 further performs the region-of-interest segmentation based on the extracted or calculated image feature amount. Here, the attention area separation means classifying the attention area according to the attribute of the attention area. For example, the attention area includes only an area including an edge (hereinafter referred to as an edge area) and white (or black). It is to separate a region (hereinafter referred to as a flat region).

The pixel value conversion unit 530 converts a pixel value in an image block including an attention region determined to be an edge region based on the result of segmentation of the attention region by the image block feature extraction unit 520 and its peripheral portion. The pixel value in the image block converted by the pixel value conversion unit 530 is used for generating an enlarged image block in the first generation unit 542 of the enlarged image block generation unit 540 described later.
More specifically, the pixel value conversion unit 530 includes a smoothing unit 532 and an enhancement unit 534, and the smoothing unit 532 is based on the edge direction in the attention area calculated by the image block feature extraction unit 520. Smooth the pixel values in the image block. The enhancement unit 534 enhances the contrast of the pixel values in the image block that has been smoothed based on the edge direction by the smoothing unit 532.

The enlarged image block generation unit 540 includes a plurality of enlargement units that generate enlarged image blocks by different enlargement processes. The enlarged image block generation unit 540 of this example includes a first generation unit 542 and a second generation unit 544.
The enlarged image block generation unit 540 switches a plurality of enlargement means (in this example, the first generation unit 542 and the second generation unit 544) based on the image feature amount extracted or calculated by the image block feature extraction unit 520. The enlarged image block corresponding to the attention area is generated.
For example, the enlarged image block generation unit 540 determines whether the region of interest is the first generation unit 542 or the second generation unit 544 depending on the edge region and the flat region cut by the image block feature extraction unit 520. Switch to generate an enlarged image block. In this example, an enlarged image block is generated by the first generation unit 543 in the case of an attention area determined to be an edge area, and a second generation is generated in the case of an attention area determined to be a flat area. A case where an enlarged image block is generated by the unit 544 is taken as a specific example.
The enlargement method in the first generation unit 542 and the second generation unit 544 is arbitrary, but the first generation unit 542 that operates when the attention area is determined to be an edge area as in the above example is Compared to the generation unit 544, it is desirable to apply an enlargement method that suppresses the occurrence of jaggies and prioritizes image quality. In this case, it is desirable for the second generation unit 544 to apply an enlargement method capable of high-speed processing than the first generation unit 542, thereby reducing the overall processing time.

  The image block arrangement unit 560 sequentially arranges the enlarged image blocks generated by the enlarged image block generation unit 540, and outputs the resolution-converted or enlarged enlarged image data to the binary conversion unit 590. The arrangement method of the enlarged image blocks will be described later. For example, in addition to the method of sequentially arranging the enlarged image blocks, the enlarged image blocks are sequentially arranged and overlapped so as to overlap, and the sum of the overlapping pixel values is divided by the number of overlapped pixels. It can also be configured to calculate a value.

Next, the image block feature extraction unit 520, the pixel value conversion unit 530, and the enlarged image block generation unit 540 will be described in more detail.
First, details of the image block feature extraction unit 520 will be described. Note that the region of interest is a 2 × 2 pixel size block (a square region in which the direction orthogonal to each other is 2 pixels), and the peripheral region including the region of interest is a 4 × 4 pixel size block (the direction orthogonal to each other is 4 pixels). A case of a square area) will be described as a specific example.

FIG. 3 is a block diagram illustrating a functional configuration of the image block feature extraction unit 520 in the present embodiment.
As shown in FIG. 3, the image block feature extraction unit 520 includes an edge pattern selection unit 522 and an edge direction estimation unit 524.
The edge pattern selection unit 522 selects the edge pattern of the 2 × 2 image block that is the attention area, and based on the selected edge pattern, the attention area is the edge area or is a flat area that is only white (or black). Determine if there is.
In addition, when the attention area is an edge area, the edge pattern selection unit 522 determines an edge angle corresponding to the edge pattern as an initial edge angle of the attention area.

FIG. 4 is a diagram illustrating patterns and edge angles in a 2 × 2 image block.
As illustrated in FIG. 4, corresponding edge angles are set for the respective edge patterns, and the edge pattern selection unit 522 determines the initial edge angle depending on which edge pattern is the region of interest.
When the attention area is a flat pattern, the attention area is cut as a flat area.
However, when the attention area is an edge pattern (checkered edge pattern) indicated by a thick frame in FIG. 4, the initial angle cannot be uniquely determined only by the edge pattern of the attention area. That is, it cannot be determined whether the initial edge angle is 45 ° or 135 °. The reason for this is that taking the upper checkered edge pattern a in the thick frame illustrated in FIG. 4 as an example, in a binary image, when the region of interest is this checkered edge pattern a, black pixels appear in the 45 ° direction. Both the case of a part of the continuing area and the case of a part of the area where white pixels continue in the 135 ° direction are conceivable, and it cannot be determined only by the attention area alone of the 2 × 2 image block. It is.

Therefore, the edge pattern selection unit 522 in the present embodiment, as shown in FIG. 5, the 2 × 2 image immediately before in the raster scan order partially including the attention area (checkered edge pattern area in FIG. 5). Based on the edge pattern of the block area (area surrounded by a thick frame in FIG. 5), the initial edge angle of the attention area which is a checkered edge pattern is determined.
FIG. 6 shows an example of all possible combinations of the checkered edge pattern and the edge pattern of the previous 2 × 2 image block area (upstream in the scanning direction) and an initial edge angle determined by the combination. FIG.
As shown in FIG. 6, when the region of interest is a checkered edge pattern, the edge pattern selection unit 522 refers to the edge pattern of the previous image block and determines either 45 ° or 135 °. Decide whether to use the initial edge angle.
In this example, the initial edge angle is determined based on the combination of the two blocks of the checkered edge pattern and the edge pattern of the previous image block. However, the present invention is not limited to this. For example, the edge pattern selection unit 522 may determine the initial edge angle based on the combination of the edge pattern of the region of interest and the edge pattern of the next image block, and the previous edge and the next edge. The initial edge angle in the checkered edge pattern may be determined by a combination of edge patterns of three blocks, and the number and position of image blocks to be combined for determining the initial edge angle of the checkered edge pattern depends on the required processing speed and image quality. It is variable according to.

Further, the edge pattern selection unit 522 extracts a specific pattern and corrects the initial edge angle and the edge pattern of the related image block in order to forcibly correct the extracted specific pattern to a predetermined pattern.
FIG. 7 illustrates a part of an image including a specific pattern to be corrected as an example of extraction of the specific pattern.
As illustrated in FIG. 7, the uneven pattern of the image surrounded by the dotted line in the figure is a pattern that protrudes (or is recessed) by one pixel. For example, the resolution is obtained when a document is captured as an electronic document by a FAX or a scanner. This is a pattern generated due to the above, and is not a meaningful feature pattern constituting an original image. Such a concavo-convex pattern causes jaggies after enlarging processing and greatly affects image quality.

FIG. 8 is a diagram specifically illustrating the extraction and correction of the concave / convex pattern (projecting or recessed by one pixel). In this figure, a region surrounded by a thick frame is a region of interest.
As shown in FIG. 8 (A), a thick frame region of interest (fourth region shown in FIG. 8 (A)) and a plurality of peripheral regions including a part thereof (the first region shown in FIG. 8 (A)). 1 to 3), the first area is “1110”, the second area is “1000”, and the third area is a bit pattern. Is “1011”, and the fourth area (ie, the attention area) is “0010”. Accordingly, as shown in FIG. 8A, the edge pattern selection unit 522 determines that the edge pattern sequences in the first to fourth areas are “1110”, “1000”, “1011”, and “0010”. In this case, it is determined that there is a concavo-convex pattern (in this example, a convex pattern on the right), and this concavo-convex pattern is extracted.

The edge pattern selection unit 522 corrects the concavo-convex pattern extracted as described above (in this example, a pattern convex to the right), and the edge pattern and initial edge angle of each of the first to fourth regions are corrected as shown in FIG. Change as shown in B). That is, the edge pattern selection unit 522 changes the first area to the edge pattern “1010” and the initial edge angle of 0 °, changes the second area to the flat pattern (bit pattern “0000”), The region 3 is changed to the edge pattern “1010” and the initial edge angle 0 °, and the fourth region (region of interest) is changed to the flat pattern “0000”.
As described above, the edge pattern selection unit 522 can correct the uneven pattern (the protruded pattern protruding to the right in this example) by changing the edge pattern and the initial edge angle of the attention area and the peripheral area. Become.

  FIG. 9 is a diagram showing a specific example of the uneven pattern correction in a case different from the example shown in FIG. By correcting the uneven pattern that protrudes or is recessed by one pixel in this way, the uneven pattern is corrected in the image illustrated in FIG. 7 as shown in FIG. In the present embodiment, as described above, the description is limited to the extraction and correction of the concavo-convex pattern. However, the present invention is not limited to this, and depends on the type of input image data, the required image quality, or the processing speed. Thus, extraction and correction of a specific pattern different from the concave / convex pattern (for example, an isolated point pattern isolated by one pixel) may be performed.

Next, details of the processing of the edge direction estimation unit 524 will be described.
FIG. 11 is an explanatory diagram of a specific example of the attention area and the peripheral area and an example of the edge direction of the attention area. FIG. 11A shows an example of the attention area (the area within the thick frame in FIG. 11) and the peripheral area. The flow of edge direction estimation processing by the edge direction estimation unit 524 will be described using the example shown in FIG.
FIG. 12 is a flowchart illustrating an example of edge direction estimation processing (S10) by the edge direction estimation unit 22.

In step 110 (S110), the edge direction estimation unit 524 sets the initial edge of the region of interest that has already been set by the edge pattern selection unit 522 as a variable (total edge angle) that represents the sum of the edge angles of the region of interest and the reference region. Substitute the angle.
The edge direction estimation unit 524 initializes the angle reference number as a variable for counting the reference number of the edge angle to “1”.

In step 120 (S120), the edge direction estimation unit 524 determines whether the edge direction estimation unit 524 is in the peripheral region (region outside the bold frame) shown in FIG. 11A according to the initial edge angle of the region of interest set by the edge pattern selection unit 522. To select a reference region to be used for estimating the edge direction.
FIG. 13 is a diagram illustrating a reference region used for estimating the edge direction. A 2 × 2 pixel indicated by a bold frame in the drawing is a reference area. FIG. 13A illustrates a reference region when the initial edge angle set by the edge pattern selection unit 522 is “angle 0 °”, and FIG. 13B illustrates that the initial edge angle is “angle 90 °”. ”Is illustrated, and FIG. 13C illustrates the reference region when the initial edge angle is an angle other than“ angle 0 ° or 90 ° ”. Note that the number of reference regions in the case of the angle 0 ° and the angle 90 ° illustrated in FIGS. 13A and 13B is 2, and the angle other than the angles 0 ° and 90 ° illustrated in FIG. The number of reference areas is four.
In the specific example shown in FIG. 11, since the initial edge angle of the attention area is “angle 45 °”, the four reference areas illustrated in FIG. 14C are candidates for selection. The selection of the reference area is not limited to that shown in FIG. 13. For example, in the case of FIG. 13C, the number of reference areas is set to 8, or the reference area corresponding to each angle is selected. It may be set.

In step 130 (S130), the edge direction estimation unit 524 determines whether one of the reference regions selected in S120 is an edge pattern suitable for edge direction estimation. For example, the edge direction estimation unit 524 takes three types of edges as shown in FIG. 14B when the reference area on the right side is taken as an example of the reference area with respect to the attention area illustrated in FIG. A pattern is a candidate. Then, in S130, the edge direction estimation unit 524 generates a third edge pattern (initial edge angle 135 °) that is significantly different from the initial edge angle (45 ° in this example) in the region of interest as shown in FIG. Is determined to be an edge pattern that is not suitable for the edge direction estimation of the region of interest shown in FIG.
In the case of the specific example shown in FIG. 11, the edge pattern of the reference area on the right side with respect to the attention area is the first edge pattern shown in FIG. It is determined that the edge pattern is suitable for estimation.
As a result of this determination, the edge direction estimation unit 524 proceeds to the process of S140 when the reference area is suitable for the edge direction estimation in the target area of interest, and proceeds to the process of S150 otherwise. To do.

  In step 140 (S140), the edge direction estimation unit 524 adds the initial edge angle of the reference region to the total edge angle, and increments the angle reference number.

  In step 150 (S150), the edge direction estimation unit 524 determines whether or not all selectable reference areas have been processed. If it is determined that the processing has been completed, the process proceeds to S160 and ends. If it is determined that it is not, the process returns to S130 and the next reference area is processed.

  In step 160 (S160), the edge direction estimation unit 524 refers to the sum of the initial edge angle of the region of interest and the initial edge angle of the reference region determined to be suitable for the edge direction estimation in S130 (edge angle sum). The average edge angle divided by the number is set as the estimated edge angle of the region of interest, and the process ends.

In the specific example shown in FIG. 11, as shown in FIG. 11B, the initial edge angle of the upper reference area is 0 °, the initial edge angle of the left reference area is 0 °, and the initial edge angle of the lower reference area. 45 °, the initial edge angle of the right reference region is 45 °, and the sum of the initial edge angle of the attention region of 45 ° is 135 °. The edge direction estimation unit 524 obtains an average edge angle of 27 ° by dividing the total 135 ° by the angle reference number of 5. This average edge angle is the estimated edge angle with respect to the region of interest shown in FIG.
Further, the edge direction estimation unit 524 classifies the estimated edge angle Θ with respect to the obtained attention area into angle ranges (directions 0 to 7) in directions (8 directions) divided every 22.5 °. In this example, an angle range centered on 0 ° or ± 180 ° is “direction 0”, and an angle range centered on 22.5 ° or −157.5 ° is “direction 1”, 45 ° or −135. The angle range centered on ° is “direction 2”, the angle range centered on 67.5 ° or −112.5 ° is “direction 3”, and the angle range centered on 90 ° or −90 ° is “direction”. 4 ”, an angle range centered on 112.5 ° or −67.5 ° is“ direction 5 ”, and an angle range centered on 135 ° or −45 ° is“ direction 6 ”, 157.5 ° or −22. An angle range centered at .5 ° is defined as “direction 7”, and the edge direction estimation unit 524 classifies the estimated edge angle Θ into any angle range (range of ± 11.25 ° from each of the above center angles). . Since the estimated edge angle Θ (= 27 °) in the specific example of FIG. 11 described above is included in the range of 22.5 ° ± 11.25 °, the edge direction estimation unit 524 performs the processing for the attention area shown in FIG. The final estimated edge direction is determined to be “direction 1” (the direction of the arrow line in FIG. 11B).
In this example, the estimated edge angle Θ is calculated and then normalized (classified) in any of the eight directions. However, the present invention is not limited to this. If a more accurate edge direction is required, twelve directions ( You may normalize in more directions, such as 1 direction (every 15.0 degrees) and 16 directions (every 12.25 degrees).

Next, the pixel value conversion unit 530 will be described.
As illustrated in FIG. 2, the pixel value conversion unit 530 includes a smoothing unit 532 and an enhancement unit 534.
The smoothing unit 532 performs pixel value smoothing processing corresponding to the estimated edge direction of the attention area estimated by the image block feature extraction unit 520 on the pixels in the attention area and the peripheral area including the attention area.
FIG. 15 is a diagram illustrating a smoothing kernel corresponding to the estimated edge direction. It should be noted that the numbers in the smoothing kernel shown in the figure are weighting coefficients by which the pixel values are multiplied.
For example, when the estimated edge direction of the attention area is “direction 3”, the smoothing unit 532 performs a smoothing process using the third smoothing kernel in FIG. 15 according to this “direction 3”. . In this case, as shown in FIG. 16, for example, when the smoothing process of the center pixel P is performed, the smoothing unit 532 uses the pixel a and the pixel b to calculate the smoothed pixel value P ′ as follows: Calculate according to (1).

  Pixel value of interest P ′ = (2.0 × P + 1.0 × (a + b)) / 4.0 (1)

  Note that the smoothing kernel used by the smoothing unit 532 is not limited to that shown in FIG. 15, and is different depending on the type of input image data or the required degree of smoothing. Also good.

The emphasizing unit 534 performs processing on the attention area that has been subjected to the smoothing process (smoothing processing using the smoothing kernel corresponding to the estimated edge direction of the attention area) by the smoothing unit 532 and the pixels in the peripheral area including the attention area. Then, contrast enhancement processing is performed.
FIG. 17 is a diagram illustrating a specific example of the enhancement kernel applied by the enhancement unit 534.
The value of “1.60” shown in FIG. 17 represents a weighting factor corresponding to the pixel value at the center of the enhancement kernel shown in FIG. 17A, and the value of “0.15” is above, below, left, and right of the center pixel. The weighting coefficient corresponding to the arranged pixel value is represented.
FIG. 17B is an explanatory diagram when the pixel P is emphasized by the enhancement kernel shown in FIG. As shown in FIG. 17B, the emphasis unit 534 refers to the pixels a, b, c, and d that exist at the top, bottom, left, and right, for example, when performing the emphasis processing of the center reference pixel P, and also performs weighting. For example, “1.60” is used for the pixel of interest at the center, and “0.15” is used for the peripheral reference pixels. Thus, the enhancement unit 534 calculates the pixel value P ′ of the pixel P according to the following equation (2).

    Pixel value of interest P ′ = 1.60 × P−0.15 × (a + b + c + d) (2)

  As described above, through the processing of the smoothing unit 532 and the emphasizing unit 534, the pixel value conversion unit 530 converts the pixel values of the attention area and the surrounding area for use in the enlarged image block generation unit 540 described later.

Next, the enlarged image block generation unit 540 will be described in detail.
First, the first generation unit 542 included in the enlarged image block generation unit 540 will be described. The first generation unit 542 performs an enlarged image block generation process on the region of interest that is determined to be an edge region by the image block feature extraction unit 520. More specifically, the first generation unit 542 includes the edge pattern and the estimated edge direction of the attention area obtained by the image block feature extraction section 520, the attention area converted by the pixel value conversion section 530, and the surrounding area. An enlarged image block for the region of interest is generated using the pixel value.

FIG. 18 is a diagram for explaining enlarged image block generation processing by the first generation unit 542.
First, the first generation unit 542 calculates a pixel value corresponding to a 3 × 3 image block using the pixel value converted by the pixel value conversion unit 530 based on the edge pattern and the estimated edge direction of the region of interest.
FIG. 18A shows the attention area and the peripheral area exemplified in FIG. As for the region of interest, the edge pattern “1110” and the estimated edge direction “direction 1” are obtained by the image block feature extraction unit 520 as described above.
When the combination of feature amounts is (edge pattern 1110) and (estimated edge direction “direction 1”), the first generation unit 542 corresponds to each 3 × 3 image block as illustrated in FIG. , P0 to p8, the pixel values of p0 to p8 are calculated by the following formula based on the pixel values {a, b, c, d} of the attention area shown in FIG.

p0 = a
p1 = (a + b) / 2
p2 = b
p3 = (a + c) / 2
p4 = (b + c) / 2
p5 = d
p6 = c
p7 = (p4 + b) / 2
p8 = d

  The first generation unit 542 uniquely determines these calculation formulas based on a combination (a combination of feature amounts) of (edge pattern) and (estimated edge direction), and uses the determined calculation formula to calculate 3 × 3. The pixel value corresponding to the image block is calculated.

FIG. 19 is a diagram illustrating calculation formulas corresponding to other combinations of feature quantities (combination of edge patterns and estimated edge directions).
FIG. 19A illustrates a calculation formula corresponding to the combination of the edge pattern “1000” and the estimated edge direction “direction 1”.
p0 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p1 = (p4 + c) / 2
When the combination of feature amounts is the edge pattern “1000” and the estimated edge direction “direction 1”, the first generation unit 542 calculates a pixel value corresponding to a 3 × 3 image block using the above calculation formula.

FIG. 19B illustrates a calculation formula corresponding to the combination of the edge pattern “1100” and the estimated edge direction “direction 5”.
p0 = a
p1 = (a + b) / 2
p2 = b
p4 = (a + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
p3 = (p4 + c) / 2
p5 = (p4 + b) / 2
When the combination of feature amounts is the edge pattern “1100” and the estimated edge direction “direction 5”, the first generation unit 542 calculates a pixel value corresponding to a 3 × 3 image block using the above calculation formula.

FIG. 19C illustrates a calculation formula corresponding to the combination of the edge pattern “1100” and the estimated edge direction “direction 2”.
p0 = a
p1 = a
p2 = b
p3 = a
p4 = (b + c) / 2
p5 = (b + d) / 2
p6 = c
p7 = (c + d) / 2
p8 = d
When the combination of feature amounts is the edge pattern “1100” and the estimated edge direction “direction 2”, the first generation unit 542 calculates a pixel value corresponding to a 3 × 3 image block using the above calculation formula.

FIG. 19D illustrates a calculation formula corresponding to the combination of the edge pattern “0101” and the estimated edge direction “direction 7”.
p0 = a
p2 = b
p3 = (a + c) / 2
p4 = (a + d) / 2
p5 = (b + d) / 2
p6 = c
p8 = d
p1 = (p4 + b) / 2
p7 = (p4 + c) / 2
When the combination of feature amounts is the edge pattern “0101” and the estimated edge direction “direction 7”, the first generation unit 542 calculates a pixel value corresponding to a 3 × 3 image block using the above calculation formula.

  In the case of other combinations (edge pattern and estimated edge direction), the first generation unit 542 similarly performs calculation according to a calculation formula corresponding to each edge pattern and estimated edge direction to obtain 3 × 3. The pixel value corresponding to the image block is calculated.

  Next, the first generation unit 542 calculates the pixel value corresponding to the 3 × 3 image block calculated as described above and a plurality of reference pixels in the peripheral area selected based on the estimated edge direction of the attention area. To generate a 4 × 4 image block.

FIG. 20 is a diagram illustrating a method of selecting reference pixels r0 to r13 based on the estimated edge direction.
When the estimated edge direction of the attention area is from direction 1 (22.5 °) to direction 3 (67.5 °), the first generation unit 542 is surrounded by a thick frame in FIG. The reference pixels r0 to r5 are selected to be 3 pixels from the upper left to the lower and 3 pixels from the lower right to the upper.
In addition, the first generation unit 542 is indicated by a thick line frame in FIG. 20B when the estimated edge direction of the attention area is from the direction 5 (112.5 °) to the direction 7 (157.5 °). The reference pixels r0 to r5 are selected so that there are 3 pixels from the lower left to the upper and 3 pixels from the upper right to the lower.
In addition, the first generation unit 542 selects four upper and lower pixels as the reference pixels r6 to r13, as shown in FIGS. 20A and 20B, regardless of the estimated edge direction.
As described above, the first generation unit 542 selects the reference pixel based on the estimated edge direction in the attention area. Note that the selection of reference pixels is not limited to the selection from the two patterns illustrated in FIG. 20, and more reference pixel selection patterns may be prepared according to the estimated edge direction. Also, the reference pixel to be selected may be changed according to the estimated edge direction.

FIG. 21 is a diagram illustrating a process of generating a 4 × 4 pixel enlarged image block.
As illustrated in FIG. 21, the first generation unit 542 uses the calculated pixel values p0 to p8 corresponding to 3 × 3 image blocks and reference pixels r0 to r13 selected based on the estimated edge direction in the attention area. The pixel value (s0 to s15) corresponding to the 4 × 4 pixel enlarged image block is calculated according to the following calculation formula to generate a 4 × 4 enlarged image block.

s0 = 0.2 × r6 + 0.16 × r0 + 0.64 × p0
s1 = 0.2 × r7 + 0.32 × p0 + 0.48 × p1
s2 = 0.2 × r8 + 0.48 × p1 + 0.32 × p2
s3 = 0.2 × r9 + 0.64 × p2 + 0.16 × r1
s4 = 0.08 * r0 + 0.32 * p0 + 0.12 * r2 + 0.48 * p3
s5 = 0.16 × p0 + 0.24 × p1 + 0.24 × p3 + 0.36 × p4
s6 = 0.24 × p1 + 0.16 × p2 + 0.36 × p4 + 0.24 × p5
s7 = 0.32 × p2 + 0.08 × r1 + 0.48 × p5 + 0.12 × r3
s8 = 0.12 * r2 + 0.48 * p3 + 0.08 * r4 + 0.32 * p6
s9 = 0.24 × p3 + 0.36 × p4 + 0.16 × p6 + 0.24 × p7
s10 = 0.36 × p4 + 0.24 × p5 + 0.24 × p7 + 0.16 × p8
s11 = 0.48 × p5 + 0.12 × r3 + 0.32 × p8 + 0.08 × r5
s12 = 0.16 × r4 + 0.64 × p6 + 0.2 × r10
s13 = 0.32 × p6 + 0.48 × p7 + 0.2 × r11
s14 = 0.48 × p7 + 0.32 × p8 + 0.2 × r12
s15 = 0.64 × p8 + 0.16 × r5 + 0.2 × r13

  By performing the processing as described above, 4 × 4 pixel enlarged image blocks (s0 to s15) corresponding to the attention area determined as the edge area by the image block feature extraction unit 520 are generated.

  Next, the second generation unit 544 included in the enlarged image block generation unit 540 will be described. The second generation unit 544 performs an enlargement process by the first generation unit 542 for smoothing the edge of the attention region as described above with respect to the attention region determined to be a flat region by the image block feature extraction unit 520. Different processing is applied. For example, the second generation unit 544 performs a simpler process for generating an enlarged image block with a reduced processing load compared to the first generation unit 542. The second generation unit 544 of this example generates a 4 × 4 pixel enlarged image block by nearest neighbor interpolation from a 2 × 2 pixel region of interest. Of course, the enlargement method applied to the second generation unit 544 is arbitrary, and any method may be used as long as the method has a lighter processing load than the first generation unit 542.

  As described above, the enlarged image block generation unit 540 selects the enlargement unit (the first generation unit 542 and the second generation unit 544) according to the image feature amount extracted or calculated by the image block feature extraction unit 520. By doing so, jaggies can be suppressed and the image quality can be increased, and the processing load can be reduced. Specifically, the second generation unit 544 uses an enlargement method with a lighter processing load than the first generation unit 542, thereby reducing the processing load of the entire enlargement process, shortening the processing time, and performing high-speed enlargement processing. The first generation unit 542 achieves higher image quality of the enlarged image by using an enlargement method that enlarges the edge region with higher image quality than the second generation unit 544. In this example, the enlarged image block generation unit 540 has two enlargement methods, and a mode in which any one of them is selectively used has been described as a specific example. However, a plurality of enlargement methods are further provided, Depending on various image feature amounts extracted by the image block feature extraction unit 520, any one of a plurality of enlargement methods may be selected to perform enlargement processing.

Next, the image block arrangement unit 560 will be described.
The image block arrangement unit 560 sequentially arranges the enlarged image blocks (enlarged image blocks corresponding to the region of interest) generated by the enlarged image block generation unit 540 by a predetermined method.

FIG. 22 is a diagram illustrating a specific example in which 4 × 4 pixel enlarged image blocks generated by the enlarged image block generation unit 540 are arranged.
As shown in FIG. 22, the enlarged image block bk0 and the enlarged image block bk1 sequentially generated by the enlarged image block generation unit 540 (the first generation unit 542 or the second generation unit 544) are arranged so as to overlap each other. The image block arrangement unit 560 arranges overlapping pixels (overlapping pixels) so as to average each of the previous pixel values. In addition, the image block arrangement unit 560 may calculate each pixel value by calculating the sum of the overlapping pixels and dividing the sum of the pixel values by the number of overlaps.
In this case, the image block arrangement unit 560 selects the pixel of interest while shifting the pixels one by one, and performs the enlargement process.
The image block arrangement unit 560 can also arrange the sequentially generated enlarged image blocks without overlapping. In this case, the image block arrangement unit 560 may select the attention area so as not to overlap.
In addition, since the 4 × 4 pixel enlarged image block is generated by either the first generation unit 542 or the second generation unit 544 in the enlarged image block generation unit 540, there is a concern about image quality deterioration due to a difference in the enlargement method. However, as described above, when the image block arrangement unit 560 overlaps and averages a plurality of enlarged image blocks, occurrence of such image quality deterioration can be suppressed.

Next, the overall operation of the enlargement processing unit 500 will be described.
FIG. 23 is a flowchart illustrating the image enlargement process (S20) by the enlargement processing unit 500.
In step 210 (S210), the image block setting unit 510 sets the sizes of predetermined image blocks required for processing by the image block feature extraction unit 520 and the enlarged image block generation unit 540, and the set block size. Are sequentially cut out (for example, in the raster scan order), and image blocks of each block size are output to the image block feature extraction unit 520.

In step 220 (S220), the image block feature extraction unit 520 determines whether the attention area in the input image block is an edge area or a flat area.
When the region of interest is an edge region, the enlargement processing unit 500 sets the edge pattern of the region of interest, the setting of the edge angle corresponding to the edge pattern as the initial edge angle, and the image block feature extraction unit 520 The process proceeds to S230 in order to perform extraction and correction of the specific edge pattern and to perform an enlarged image block generation process that reduces jaggies.
The enlargement processing unit 500 proceeds to the process of S260 when the attention area is a flat area.

  In step 230 (S230), the image block feature extracting unit 520 determines the edge direction of the attention area from the attention area determined as the edge area and the initial edge angle Θ of the reference area in one or more peripheral areas including the attention area. Estimate θ. For example, the image block feature extraction unit 520 calculates an average value of the obtained plurality of edge angles Θ, and sets the average value as the edge direction θ. The edge direction is not limited to the average edge angle.

In step 240 (S240), the pixel value conversion unit 530 (smoothing unit 532) uses the smoothing kernel corresponding to the estimated edge direction θ obtained by the image block feature extraction unit 520, and uses the smoothed kernel and its surrounding region. The inner pixel is smoothed.
Further, the pixel value conversion unit 530 (enhancement unit 534) performs contrast enhancement processing on the attention region that has been subjected to the smoothing processing and the pixels in the surrounding region.

In step 250 (S250), the first generation unit 542 of the enlarged image block generation unit 540 uses the edge pattern of the attention area and the estimated edge direction θ of the attention area obtained by the image block feature extraction section 520, and the pixel in S240. An enlarged image block for the attention area is generated using the attention area subjected to the pixel value conversion processing by the value conversion section 530 and the pixel values in the peripheral area.
As described above, the enlarged image block generation unit 540 causes the first generation unit 542 to generate an enlarged image block that suppresses the jaggy of such an edge portion when it is determined in S220 that the region is an edge region. Thereby, it is possible to generate a high-quality enlarged image block with less jaggy for the edge portion and the like.

In step 260 (S260), the second generation unit 544 of the enlarged image block generation unit 540 generates an enlarged image block for the region of interest determined to be a flat region by the image block feature extraction unit 530.
The enlarged image block generation processing performed by the second generation unit 544 is different from the enlarged image block generation processing (processing by the first generation unit 542) in S230 to S250, for example, enlargement processing with a small processing load such as nearest neighbor interpolation. It's okay. That is, when the region of interest is a flat region, it is not necessary to perform the complicated and time-consuming processing as described above, and an enlarged image block generation process with a smaller processing load is sufficient.
Therefore, when the attention area is a flat area, the enlarged image block generation unit 540 selects the second generation unit 544 and performs the generation process of the enlarged image block with a small processing load, thereby reducing the overall processing time. I am trying.

  In step 270 (S270), the image block arrangement unit 560 sequentially arranges the enlarged image blocks generated by the enlarged image block generation unit 540 (the first generation unit 542 or the second generation unit 544). As a method of arranging the enlarged image blocks, a method of sequentially arranging the enlarged image blocks generated by the enlarged image block generation unit 540 so as to overlap each other is conceivable. When overlapping a plurality of enlarged image blocks, the image block arrangement unit 560 sequentially averages the previous pixel values, or divides the sum of all overlapping pixel values by the number of overlaps. Apply the method. However, the image block placement unit 560 may apply any method as long as it uses the enlarged image block generated by the enlarged image block generation unit 540 in consideration of the processing speed and the image quality.

  In step 280 (S280), the enlargement processing unit 500 checks whether or not the processing has been completed for all the input image data. If not, the processing returns to the processing of S210 to continue the processing. In this case, the enlarged image data is output to a predetermined output destination, and the enlargement process is terminated.

As described above, the image processing apparatus 2 in the present embodiment extracts features of an image region having a predetermined size including the target pixel, and based on the extracted features, the image processing device 2 in the image region and its neighboring regions. Converting the pixel value, and generating an enlarged image region by switching a plurality of enlargement methods for enlarging an image region of a predetermined size including the target pixel according to the converted pixel value and the extracted feature, The obtained enlarged image area is arranged by a predetermined method to generate an enlarged output image.
As described above, the image processing apparatus 2 applies, for example, an enlargement method for smoothing an edge with respect to an edge portion in the input image, and a flat portion (a white or black image portion) in the input image. By applying an enlargement method with a small processing load, it is possible to perform an enlargement process with high image quality while suppressing jaggies at the edge portion, and to perform an enlargement process at a high speed while suppressing the processing load.

It is a figure which illustrates the hardware constitutions of the image processing apparatus 2 with which the image processing method concerning this invention is applied centering on the control apparatus 20. FIG. It is a figure which illustrates the functional structure of the image expansion program 5 which is performed by the control apparatus 20 (FIG. 1) and implement | achieves the image processing method concerning this invention. It is a block diagram which shows the function structure of the image block feature extraction part 520. FIG. It is a figure which illustrates the pattern and its edge angle in a 2x2 image block. It is a figure which illustrates the field referred in order to determine the initial edge angle of the attention field, when the attention field is a checkered edge pattern. It is a figure which illustrates the initial edge angle determined by the combination with a checkered edge pattern. It is a figure showing a part of image containing the specific pattern corrected. It is a figure explaining extraction and correction | amendment of an uneven | corrugated pattern. It is a figure explaining correction | amendment of the uneven | corrugated pattern different from FIG. It is a figure which illustrates after correction | amendment of the image (a specific pattern is included) shown in FIG. It is a figure which illustrates the specific example of an attention area and a peripheral area, and the edge direction of an attention area. It is a flowchart of the edge direction estimation process (S10) by the edge direction estimation part 534. FIG. It is a figure which illustrates the reference field used for estimation of an edge direction. It is a figure explaining appropriate edge pattern judgment. It is a figure which illustrates the smoothing kernel corresponding to an estimation edge direction. It is a figure explaining the smoothing process by the smoothing kernel. It is a figure which illustrates an emphasis kernel and explains emphasis processing by this emphasis kernel. FIG. 10 is a diagram for explaining enlarged image block generation processing by the first generation unit 542; It is a figure which shows the other example of the calculation formula corresponding to the combination of an edge pattern and an estimated edge direction. It is a figure explaining the selection method of the reference pixels r0-r13 based on the estimated edge direction of an attention area. It is a figure explaining the production | generation process of a 4x4 pixel enlarged image block. It is a figure which illustrates the arrangement method of the enlarged image block produced | generated by the enlarged image block production | generation part 540. FIG. It is a flowchart explaining the image expansion process (S20) by the expansion process part 500. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 2 ... Image processing apparatus 5 ... Image enlargement program 500 ... Enlargement processing part 510 ... Image block setting part 520 ... Image block feature extraction part 530 ... Pixel value conversion part 532 ... Smoothing unit 534 ... enhancement unit 540 ... enlarged image block generation unit 542 ... first generation unit 544 ... second generation unit 560 ... image block arrangement unit

Claims (9)

An image processing apparatus for enlarging an input image,
For the target pixel included in the input image, based on the pixel value pattern of the target region of a predetermined size including the target pixel, determine whether the target region is an edge region,
The initial edge angle is calculated based on the pixel value pattern of the edge area, and when the initial edge angle cannot be uniquely calculated only by the pixel value pattern of the edge area, the previous edge scan is performed in the raster scan order. And / or calculate the initial edge angle with reference to the pixel value pattern of the next image area,
In accordance with the initial edge angle, set a reference region to be referred to in order to extract the gradation change direction,
Area feature extraction means for extracting a change direction of gradation as a feature of the attention area based on the pixel value pattern of the reference area and the pixel value pattern of the attention area;
A pixel value conversion unit that converts at least a pixel value in the region of interest based on the change direction of the gradation extracted by the region feature extraction unit;
One generating means for generating an enlarged image area based on the pixel value of the attention area converted by the pixel value converting means, and another generating means for generating an enlarged image area by processing different from the one generating means An enlarged image region generating means including:
Image arrangement means for arranging the enlarged image area generated by the enlarged image area generation means,
The enlarged image region generation unit applies the generation unit according to the gradation change direction extracted by the region feature extraction unit. The pixel value conversion means converts the pixel value in the attention area and the pixel value in the vicinity area of the attention area only when the pixel value of the attention area satisfies a predetermined condition. The image processing apparatus according to 1. The image processing apparatus according to claim 1, wherein the region feature extraction unit corrects the pixel value pattern when the extracted pixel value pattern is a predetermined pattern. The image processing apparatus according to claim 3, wherein the pixel value conversion unit performs a smoothing process with reference to a pixel group at a position corresponding to the change direction extracted by the region feature extraction unit. The image processing apparatus according to claim 4, wherein the pixel value conversion unit performs an edge enhancement process for enhancing a gradation change, using the plurality of pixel values subjected to the smoothing process. The pixel value conversion means converts a pixel value in the attention area and a pixel value in a vicinity area of the attention area based on the change direction of the gradation extracted by the area feature extraction means,
The one generating unit is configured to change the gradation change direction of the attention region, the pixel value conversion, for the attention pixel of the attention region in which the gradation change direction extracted by the region feature extraction unit satisfies a predetermined condition. 2. The image processing according to claim 1, wherein an enlarged image area corresponding to the target pixel is generated based on the pixel value of the attention area converted by the means and the pixel value of the neighboring area converted by the pixel value conversion means. apparatus. When the plurality of enlarged image regions generated by the enlarged image region generating unit overlap each other in the enlarged image, the image arranging unit determines a pixel value of the overlapping portion based on the enlarged image regions. The image processing apparatus according to claim 1. When the plurality of enlarged image regions generated by the enlarged image region generating unit overlap each other in the enlarged image, the image arranging unit has a plurality of pixel values corresponding to the same pixel with respect to overlapping portions of the enlarged image regions. The image processing device according to claim 7, wherein an average value of the image processing device is calculated. In an image processing apparatus that performs an enlargement process of an input image,
For a target pixel included in the input image, based on a pixel value pattern of a target region of a predetermined size including the target pixel, determining whether the target region is an edge region;
The initial edge angle is calculated based on the pixel value pattern of the edge area, and when the initial edge angle cannot be uniquely calculated only by the pixel value pattern of the edge area, the previous edge scan is performed in the raster scan order. And / or calculating an initial edge angle with reference to a pixel value pattern of the next image area;
Setting a reference region to be referred to in order to extract a change direction of gradation according to the initial edge angle;
Extracting a change direction of gradation as a feature of the region of interest based on the pixel value pattern of the reference region and the pixel value pattern of the region of interest;
A method based on the change direction of the extracted grayscale by step, converting at least a pixel value of the target region for the extraction,
One generation step of generating an enlarged image area based on the pixel values of the conversion region of interest by said step of converting, or, the other generation step of generating an enlarged image area by a different process from that of the one generating step A step to choose;
Applying the selected generation method to generate an enlarged image region;
A program for causing the image processing apparatus to execute a step of arranging the generated enlarged image region.

Images