JPH07114638A - Image processor - Google Patents

Abstract

PURPOSE:To repair and estimate an image at a high speed with high quality by processing only pixels in a specific area in an image. CONSTITUTION:An image process part 2 while determining the value of a specific unknown pixel in an area to be processed by using the values of known pixels when the number of the known pixels having the known values among pixels nearby the specific unknown pixel in the area to be processed is larger than a specific area leave the unknown pixel as it is without determining its value when the number of the known pixels is smaller than the specific threshold value, regards the unknown pixel whose value is determined as a known pixel after processing the respective pixels in the area to be processed with the specific threshold value, and repeats similar processes for the remaining unknown pixels. When those processes are performed, a threshold value initial setting part 5 sets the initial value of the threshold value to an optimum value according to the size of the area to be processed and/or the shape of the area to be processed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus used for image restoration or the like.

[0002]

2. Description of the Related Art Conventionally, in the field of pattern recognition and the like, pre-processing is performed on the read image data in order to remove the influence of stains, stains, or damage on the original. As a representative of this type of pretreatment technology,
For example, the k-nearest neighbor method and the k-nearest neighbor method as shown in the document "Basics of Image Recognition [I]", Ohmsha, Ltd., PP24-35, 1986 by the author "Shunji Mori" are developed. Selective averaging, neighborhood weighted averaging, etc. For example,
In the k-nearest neighbor method, a mask with a predetermined size (for example, a mask with a size of 3 × 3) is set in the vicinity of the pixel of interest, and the neighboring points included in this mask (with a mask with a size of 3 × 3, 8
Select a fixed number k (k is fixed) of neighboring points from among the number of neighboring points), take the average of the k neighboring points, and replace the pixel of interest, that is, the value of the center of the neighborhood with the average value. As a result of which the isolated points on the image
(Effect of stains on the original) can be removed.

[0003]

However, in the prior art, in order to correct an image, for example, the above-mentioned mask is sequentially moved from the upper left corner to the lower right corner of the image, and the pixels in the image which are not defective are generated. As for the above, the averaging process was performed at a rate. For this reason, a considerable processing time is required, and since processing is performed even for pixels that need not be corrected, there is a problem that the overall image quality deteriorates.

Further, in the past, there was a drawback that defects of arbitrary shape and size that occurred in an image could not be repaired satisfactorily.

The present invention provides an image processing apparatus capable of performing processing such as restoration and estimation only on pixels within a predetermined area in an image and capable of restoration and estimation of an image at high speed and high quality. It is intended to be provided.

[0006]

In order to achieve the above object, the invention according to claim 1 to claim 7 is such that each pixel in the processing target area initially has a completely unknown value or an uncertain value. When an attention is paid to a specific unknown pixel in the processing target area, the number of known pixels having a known value among neighboring pixels of the unknown pixel is larger than a predetermined threshold value. , The value of the unknown pixel is determined using the value of the known pixel among the neighboring pixels, and when the number of the known pixels is smaller than the predetermined threshold, the value of the unknown pixel is not determined and the unknown pixel is left as it is. After each pixel in the processing target area is subjected to the above-mentioned processing with a predetermined threshold value, the unknown pixel whose value has been determined is set as a known pixel, and the threshold value is changed to the remaining unknown pixel. Repeat the process for In the image processing, an optimal threshold of the threshold, information on the size of the processing target area, and /
Alternatively, the determination is made based on information about the shape of the processing target area. As a result, it is possible to perform optimal processing according to the size and shape of the processing target area,
Images can be restored and estimated at high speed and with high quality.

[0007]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration example of an image processing apparatus according to the present invention. Referring to FIG. 1, as shown in FIG. 2, the image processing apparatus includes an area specifying unit 1 for specifying an area PCU to be processed (for example, a repair target) in an image PC.
And the processing target area PC specified by the area specifying unit 1.
An image processing unit 2 that performs predetermined image processing on pixels included in U is included.

Here, the image processing unit 2 is arranged such that each pixel included in the processing target area PCU specified by the area specifying unit 1 initially has a completely unknown value or a pixel which is not completely unknown but has an uncertain value. (Hereinafter, this is referred to as an unknown pixel), and the area PC of the image PC outside the processing target area PCU
If each pixel of K is a known pixel having a known value,
The value of the unknown pixel in the processing target area PCU is set to the known area PC.
The known pixel value of K is used to determine a known (certain) pixel. That is, the image processing unit 2 uses the image P
Instead of performing processing on all the pixels in C, focusing on only the pixels in the processing target area PCU in the image PC,
The processing is performed only on the pixels in the processing target area PCU.

Further, the image processing unit 2 has a processing target area PC
To perform processing on a specific unknown pixel in U, attention is paid to a pixel in the vicinity of the specific unknown pixel, and the value of the specific unknown pixel is determined by using a known pixel having a known value among the neighboring pixels. Is defined (assumed to be known). At this time, the image processing unit 2 counts the number of pixels having a known value among the neighboring pixels of the specific unknown pixel in the processing target area PCU, and the number of pixels having the known value is less than a predetermined threshold value. If the number of pixels having a known value is smaller than a predetermined threshold, the unknown pixel value is determined using the value of the known pixel having a known value among the neighboring pixels. An unknown pixel is left as it is without determining the pixel value. Then, after the above-mentioned processing is performed for each pixel in the processing target area PCU with a predetermined threshold value, the unknown pixel whose value has been determined is a known pixel, and the threshold value is changed stepwise from a large value to a small value, The same process is repeated for unknown pixels whose values have not yet been determined.

In order to realize such a processing function, FIG.
In the above configuration example, the image processing unit 2 includes a control unit 3 that controls the entire process, a neighboring pixel definition unit 4 that defines neighboring pixels, and a threshold initial setting unit that sets an initial value of a threshold. 5 and an operation type setting unit 6 for setting the operation type
And a calculation unit 7 that performs a calculation of the type set in the calculation type setting unit 6.

FIG. 3 is a flow chart showing a processing example of the image processing unit 2. Note that, in the following, for simplification of description, as an example, each pixel in the processing target area PCU is processed in a unit of one pixel, and the arithmetic unit 7 determines unknown pixels in the processing target area PCU. Assuming that the value is completely unknown (even if the unknown pixel has an uncertain value, it is not considered at all), and the value of the unknown pixel is determined based on the value of the known pixel among the neighboring pixels. Processing shall be done.

On the premise that the image processing unit 2 starts the actual processing, in the image, a processing target area PCU including unknown pixels and a known area PCK including known pixels other than the processing target area PCU are included in the area specifying unit. The position where each pixel exists (for example, the position in the memory) needs to be known. At this time, in order to identify each pixel in the processing target area PCU, it is assumed that each pixel is numbered j and the total number N of pixels in the processing target area PCU is known.

In addition, it is necessary that the neighboring pixel definition unit 4 defines a neighboring pixel for one unknown pixel in advance (for example, the shape and size of the neighborhood). Further, the type of processing calculation (for example, the type of calculation such as taking the average value of the values of known pixels among the neighboring pixels) needs to be set by the calculation type setting unit 6. Further, the threshold initial setting unit 5 needs to set the initial value of the threshold k.

That is, the definition of neighboring pixels, the kind of calculation,
The initial value of the threshold is the type of image to be processed (for example,
Whether the image is a binary image or a multi-valued image, or
Appropriate depending on whether the image is a document image, a photographic image, a pictorial image, etc., or depending on the type of processing target area (size, shape, etc. of the processing target area) Must be set to.

After such a premise is established (step S
1) and j are initialized to "1" (step S2). Next, the image processing unit 2 takes in the j-th pixel from the processing target area PCU (step S3), and counts the number of known pixels having a known value among the neighboring pixels of this pixel (step S4). Then, the count value CNT is compared with the threshold value k (step S5), and when the count value CNT is the same as the threshold value k or larger than the threshold value k, the calculation unit 7 knows a known pixel among the pixels adjacent to the j-th pixel. J based on known pixels with values
The value of the th pixel is determined (step S6). When the type of calculation is, for example, “average value”, the average value of the values of the known pixels is calculated, and this is determined as the value of the jth pixel. On the other hand, in step S5, when the count value CNT is smaller than the threshold value k, no calculation is performed and the value of the j-th pixel is left unknown (step S7).

After the processing of step S6 or step S7 is performed on the j-th pixel, j is incremented by "1" (step S8), and the process returns to step S3 again to the next pixel in the processing target area PCU. The same processing is repeatedly performed for. Such processing is sequentially repeated for each pixel in the processing target area PCU to obtain the processing target area PC.
When the processing for the last pixel in U is completed (step S9), the value of the unknown pixel in the processing target area PCU is rewritten according to the processing result for each pixel (step S10). As a result, regarding the unknown pixel whose value is determined by the process of step S6, the determined value is set as a known value, and this unknown pixel becomes the known pixel at this point. On the other hand, unknown pixels whose values have not been determined by the process of step S7 are not rewritten and remain as unknown pixels.

Next, in the rewriting process of step S10, it is determined whether or not there are no unknown pixels (or whether or not the number is equal to or less than a predetermined number), or whether or not the threshold value k is the minimum value k _0. (Step S11). As a result, when the threshold value k is not the minimum value k ₀ and the unknown pixels still remain, the threshold value k is reduced to determine the values for these unknown pixels (step S1).
2) Then, the process returns to step S3 again. At this time, a number j is newly added to the remaining unknown pixels. In this way, the value for the unknown pixel is determined stepwise, and when the unknown pixel is eliminated or the number of unknown pixels becomes equal to or less than the predetermined number in step S11, or the threshold value k is the minimum value. When k ₀ is reached, the process ends.

FIGS. 4A to 4D are views for explaining a concrete example of the image processing. In this specific example, a 3 × 3 size mask M _ij as shown in FIG. 4A is defined as the neighboring pixels. That is, the center of this mask M _ij is set to an unknown pixel, whereby the eight parts M ₁₁ , M ₁₂ , M ₁₃ , M ₂₁ , M ₂₃ , M ₃₁ , M ₃₂ , M ₃₃ of the mask are set.
Is a neighboring pixel of the unknown pixel set at the center M ₂₂ of the mask. Also, in this case, the type of calculation is, for example, “average value”.
And the initial value of the threshold value k is set to, for example, “7”. Now, when the processing target area PCU is specified as shown in FIG. 4B, each pixel # 1 to # 5 in this processing target area PCU has an unknown value at the beginning, The center M _{22 of the} mask M is sequentially set for each of the pixels # 1 to # 5, and the processing is performed. For example, when the center M ₂₂ of the mask M _ij is set to the pixel # 1, M ₁₁ , M ₁₂ , M ₁₃ ,
The seven pixels of M ₂₁ , M ₃₁ , M ₃₂ , and M ₃₃ are pixels in a known area and have a known value, and the pixel of the portion of M ₂₃ is an unknown pixel, so that there are 8 neighborhoods. Among the pixels, there are seven known pixels that have known values. Therefore, since it is the same as the threshold value k (= 7) and not smaller than the threshold value k (= 7), the value of the unknown pixel # 1 is determined based on the values of the seven known pixels. For example, this value is determined as the average value of the values of seven known pixels.

Similarly, when the center M ₂₂ of the mask is set to the pixel # 5, the value of the unknown pixel # 5 is determined since 7 of the 8 neighboring pixels also have known values in this case. It

On the other hand, the center M ₂₂ of the mask is set to the pixel #
When set to 2, # 3 or # 4, M ₁₁ , M ₁₂ ,
The pixels in the portions of M ₁₃ , M ₃₁ , M ₃₂ , and M ₃₃ have known values, and the pixels in the portions of M ₂₁ and M ₂₃ are unknown pixels, so that the known value among the 8 neighboring pixels is known. Has 6 pixels. Therefore, since it is smaller than the threshold value k (= 7), the values of the unknown pixels # 2, # 3, and # 4 are not determined at this stage.

In this way, the threshold value k is set to "7",
When the process is performed for all unknown pixels # 1 to # 5, the values are determined for the two pixels # 1 and # 5, and thus the determined values for the pixels # 1 and # 5 are known values. Is set as. That is, at this time, pixel # 1,
# 5 becomes a known pixel having a known value, and the three pixels # 2, # 3, and # 4 remain unknown as shown in FIG. 4 (c).

At the next stage, the threshold value k is changed from "7" to "6".
And the center M ₂₂ of the mask M _ij is changed to the unknown pixel # 2, #
3 and # 4 are sequentially set and processing is performed. In this case, if the center M ₂₂ of the mask is set to pixel # 2 or # 4,
Of the 8 neighboring pixels, 7 have known values,
Since it is larger than the threshold value k (= 6), the values of the pixels # 2 and # 4 are determined based on the values of the seven known pixels. Similarly, when the center M ₂₂ of the mask is set to the pixel # 3, 6 out of 8 neighboring pixels have known values, and the threshold k
Since it is the same as (= 6) and is not smaller than the threshold value k (= 6), the value of the pixel # 3 is determined based on the values of the six known pixels.

In this way, the threshold value k is set to "6",
When processing is performed for all unknown pixels # 2 to # 4, values are determined for all pixels # 2 to # 4 and set as known values. In this example, at this stage, as shown in FIG. 4D, all the pixels become known pixels having known values, and no unknown pixels remain, so the process ends.

First of the above-mentioned processing in the image processing unit 2
The feature (advantage) of is that only the value of the pixel in the processing target area PCU is determined. That is, since the processing is not performed for all the pixels in the image PC but only for the pixels in a part of the image PC, the image processing time can be shortened as compared with the conventional case. Since the pixels in the known area PCK that do not need to be processed are not processed, the overall image quality does not deteriorate.

The second characteristic (advantage) of the above-mentioned processing
Is to determine the value in order from the pixel whose value can be determined most reliably among the unknown pixels. That is, in the example of FIG. 4B, the pixels # 1 and # 5 have a larger number of neighboring pixels having known values than the pixels # 2 to # 4, and the values are most reliably determined. So, the value is determined first.

The third characteristic (advantage) of the above-mentioned processing
When the value of an unknown pixel is determined, and this unknown pixel becomes a known pixel having a known value, in the next step, this pixel is added to a group of pixels having a known value of neighboring pixels, and It is to determine the values of the remaining pixels. That is, in the example of FIG. 4C, for example, in determining the value of the pixel # 2, the value of the pixel # 1 that has become the known value in the previous stage is taken into consideration. Thereby, continuity of each pixel can be secured.

The fourth characteristic (advantage) of the above-mentioned processing
Is that all types of images can be processed regardless of the types of images. That is, even if the image is a binary image such as a document image, a multi-valued image such as a photographic image or a pictorial image can be obtained by appropriately setting the definition of neighboring pixels, the initial value of the threshold value, and the type of calculation. Even
This is to be able to favorably perform image restoration and estimation. In the specific example described above, no reference is made to the type of image, but when the image is a multi-valued image, the calculated value (for example, the average value) of the pixel values having known values in the neighboring images. That can be determined as the value of the unknown pixel. Further, when the image is a binary image, when the calculated value (for example, average value) of the pixels having known values in the neighboring images is “0.5” or less, “0” is set as the value for the unknown pixel. When the calculated value (for example, the average value) is "0.5" or more, "1" can be determined as the value for the unknown pixel.

By the way, as the initial value of the threshold value k set in the threshold value initial setting section 5, for example, the maximum value corresponding to the number of neighboring pixels can be set as described above. In the above example, when the number of neighboring pixels is “8”,
The initial value of the threshold value k can be set to "7". In this way, by setting the initial value of the threshold value k to the largest value according to the number of neighboring pixels and starting the processing from the position where the threshold value k is the largest, the value of each pixel in the processing target area PCU is the most reliable. It can be determined in order from the ones with nature.

However, depending on the size and shape of the region to be processed, even if the initial value of the threshold value k is not set to the maximum value, the image is processed to some extent with good reliability.
In some cases (eg repair) can be done. For example, when the size of the processing target area PCU is small, or when the shape of the processing target area PCU is not elongated, the threshold value k is larger than when the size of the area PCU is large or when the shape of the area PCU is elongated. Even if the processing for the pixels of this area PCU is not made most reliable by setting the initial value low, this area P for the quality of the entire image is
It is considered that the influence of pixels in the CU is small. Even in such a case, if the initial value of the threshold value k is always set to the largest value according to the number of neighboring pixels and the processing is started from the largest value of the threshold value k, the processing amount increases and the processing speed increases. On the contrary, the adverse effect of lowering occurs. In order to prevent such an adverse effect, the threshold initial setting unit 5 sets the processing target area PCU
It is preferable to set the initial value of the threshold value k to an optimum value in accordance with the size of (i.e., the number of pixels in the processing target area PCU) and / or the shape of the processing target area PCU.

FIG. 5 is a diagram showing a configuration example of the threshold value initial setting unit 5. Referring to FIG. 5, the threshold initial setting unit 5 holds a table T holding information for determining the initial value of the threshold k.
BL, area information detection unit 8 that detects information about the size of the processing target area PCU and / or information about the shape of the processing target area PCU as area information, and corresponds to the area information detected by the area information detection unit 8. It has an initial value determination unit 9 that automatically determines the initial value of the threshold value k with reference to the table TBL.

Here, the area information detection unit 8 uses the area P for the size of the entire image PC (that is, the number of pixels of the entire image PC) as information regarding the size of the processing target area PCU.
The ratio f (%) of the number N of pixels in the CU can be used. The number N of pixels in the processing target area PCU may be the number obtained when the processing target area PCU is specified by the area specifying unit 1 as described above.

As for the shape of the processing target area PCU, when the area specifying unit 1 specifies the processing target area PCU as shown in FIG. 6A, the area PCU is selected as shown in FIG. 6B. For example, the ratio of the length of the cut-out rectangle RC in the vertical direction i and the horizontal direction j, that is, the aspect ratio or the aspect ratio RT _ij (= dj / di or di / dj) is calculated, and the aspect ratio or the aspect ratio is calculated. It can be detected based on RT _ij . That is, it is possible to detect how thin the shape of the area PCU is.

The table TBL is shown in FIGS.
It can be configured as in (b) or FIG. 7 (c).
In the example of FIG. 7A, in the table TBL, the ratio f (%) of the number of pixels in the processing target area PCU to the number of pixels of the entire image PC and the initial value of the threshold value k corresponding to the ratio f are shown. Is set. Further, in the example of FIG. 7B, the table TB
L is the aspect ratio of the area PCU or the aspect ratio RT _ij , and R
The initial value of the threshold value k corresponding to T _ij is set. In FIG. 7B, RT _ij is the aspect ratio (dj / di).
Alternatively, it is assumed that the aspect ratio (di / dj) has a larger value, and thus RT _ij is “1” or more. In the example of FIG. 7C, the table PC has a region PC.
The ratio f of the number of pixels in U, the aspect ratio or aspect ratio RT _{ij of} the area PCU, and the initial value of the threshold value k corresponding to f and RT _ij are set.

Next, a specific example of the processing when the threshold initial setting unit 5 has the above-mentioned configuration will be described. Now, when the table TBL is as shown in FIG. 7C, for example, when the processing target area PCU in the image PC is specified as shown in FIG. Ratio f of the number of pixels in the target area PCU f
Since (%) is considerably large, the initial value determination unit 9 determines the initial value of the threshold value k to be “7”, for example. As a result, the pixel value in this large area PCU can be determined with extremely high reliability.

On the other hand, the processing target area P in the image PC
When the CU is specified as shown in FIG. 9, for example, the image P
Since the ratio f (%) of the number of pixels in the processing target area PCU to the total number of pixels of C is small and the aspect ratio RT _ij is also small, the initial value determination unit 9 sets the initial value of the threshold value k to, for example, “4”. "Decide." As a result, the calculation unit 7 starts the process for the pixels in the small area PCU from the small value “4” of the threshold value k. If the threshold value k starts from a small value “4”, the pixel values in the area PCU are not determined so reliably, but the size of the processing target area PCU is small. However, since the shape is not elongated, even if the pixel values in the area PCU cannot be determined with high reliability, the effect on the quality of the entire image PC is small, but the processing amount is reduced and the processing speed is high. Can be realized.

Further, when the processing target area PCU in the image PC is specified as shown in FIG. 10, for example, the ratio f (%) of the number of pixels in the processing target area PCU to the number of pixels in the entire image PC is shown in FIG. Similarly, although the aspect ratio RT _ij is large, the initial value determination unit 9 determines the initial value of the threshold value k to be “5”, for example. As a result, the area PC of FIG.
The process for U is started from a value "5", which is larger than the threshold k, as compared with the process for the region PCU in FIG. As a result, the pixel values in the elongated area PCU can be determined more reliably than in the case of FIG. 9, and the processing amount is reduced as compared with the case of FIG.
The processing speed can be increased.

As described above, in the present invention, the initial value of the threshold value k is automatically determined to be the optimum value according to the size and shape of the processing target area, so that the processing can be optimized.

In the above example, the case where one processing target area PCU exists in one image PC has been described, but as shown in FIG. 11, a plurality of (for example, two ) Even when the processing target areas PCU1 and PCU2 exist, for each area PCU1 and PCU2,
The present invention can be similarly applied to each of them. At this time, when the number of pixels of the entire image PC is N ₀ , the number of pixels in the area PCU ₁ is N _1, and the number of pixels in the area PCU ₂ is N ₂ , it is used when determining the initial value of the threshold value k. As a ratio f (%) to the number of pixels N ₀ of the entire image PC,
It is conceivable to use the ratio (N ₁ + N ₂ ) / N ₀ of the total number of processing target pixels (N ₁ + N ₂ ) to the total number of pixels N ₀ of the image PC. That is, the ratio f = (N ₁ + N) is set as the initial value of the threshold value k regardless of whether each pixel in the area PCU1 is processed or each pixel in the area PCU2 is processed. It is conceivable to use the same determined by ₂ ) / N ₀ . Area PCU1
And the area PCU2 have the same size, it is possible to use the same initial value of the threshold value k determined by the ratio f = (N ₁ + N ₂ ) / N ₀ . ,
When the area PCU1 and the area PCU2 have different sizes as in the example of FIG. 11, it is not preferable to uniformly use the same initial value of the threshold value k for them.

Therefore, when a plurality of (for example, two) processing target areas PCU1 and PCU2 exist in one image PC, it is preferable to determine the ratio f for each of the areas PCU1 and PCU2. That is, the threshold value k for the area PCU1
The initial value k ₁ of the ratio of the number of pixels N ₁ in this area PCU _{1 to} the number of pixels N ₀ of the entire image PC f ₁ = (N
₁ / N _0, and regarding the initial value k ₂ of the threshold value k for the area PCU2, the ratio f of the number of pixels N ₂ in this area PCU2 to the number of pixels N ₀ of the entire image PC f
_It is better to make the determination based on ₂ = N ₂ / N ₀ . Thus, the optimum threshold value k for each of the areas PCU1 and PCU2
By determining the initial values k ₁ and k ₂ of
Even if the size and shape of PCU2 are different from each other (more specifically, even when the distribution of defective pixels is not uniform in the image PC), the optimum processing is performed on each pixel of each area PCU1 and PCU2. Can be done.

In each of the above examples, as the information on the size of the area PCU (or PCU1, PCU2),
Area PCU (or P
CU1, the ratio of the number of pixels in PCU2) intragastrically f (%) (or f ₁ (%), was used f ₂ a (%)), region PCU (or P
The number of pixels in the area PCU (or PCU1, PCU2) itself may be used as the information regarding the size of CU1, PCU2). That is, if the entire image PC is very large, for example, the size (number of pixels) of the area PCU is considerably smaller than the entire image PC but is itself large, the value of the pixel in the area PCU is However, it is desirable to be determined with high reliability, but when the initial value of the threshold value k is determined by the ratio f of the number of pixels in the area PCU to the number of pixels in the entire image PC, the initial value of the area PCU is the same as that of the entire image PC. The size of the pixel in the area PCU is determined to be small and is set to be low, so that it becomes impossible to reliably determine the value of the pixel in the area PCU. In such a case, as the information regarding the size of the area PCU, the area P
It is better to use the number of pixels in the CU itself.

In the above example, the information about the shape of the area PCU is detected as the aspect ratio or the aspect ratio RT _ij of the cutout rectangle RC of the area PCU. As shown in FIG. 12 (a) or FIG. 12 (b), it is effective when the shape is elongated along the vertical direction i or along the horizontal direction j. However, the area P
In the case where the CU has a slender shape as shown in FIG. 12C, it cannot be detected as a slender shape by the above detection method. Therefore, in this case, as the information on the shape of the region PCU, a ratio RT ′ of the number N of pixels in the region PCU to the number N _R of pixels in the rectangle RC is obtained, and the region PCU has an elongated shape based on the ratio RT ′. Can be detected. That is, FIG.
In the case of (c), it is possible to detect that the region PCU has an elongated shape as the ratio RT ′ is smaller.

In each of FIGS. 12A, 12B, and 12C, since the area PCU can be detected as having an elongated shape, the rectangle R is used as information regarding the shape of the area PCU.
Along with the aspect ratio or aspect ratio RT _{ij of} C, the ratio RT ′ of the number N of pixels in the region PCU to the number N _R of pixels in the rectangle RC is obtained, and the region is calculated based on both the ratio RT _ij and the ratio RT ′. P
It is better to detect whether the CU has an elongated shape. According to this, for example, in the case of FIGS. 12 (a) and 12 (b), the ratio RT ′ is large, but the ratio RT _ij is large, so that it can be detected as an elongated shape. If the ratio RT
_ij is small, but the ratio RT 'is small, so the area PCU
Can be detected as having an elongated shape.

The image processing apparatus of the present invention as described above is
It can be applied to various purposes as long as the area to be processed can be specified. It should be noted that the area specifying unit 1 of the image processing apparatus needs to have a configuration suitable for the applied application, because the method of specifying the processing target area also differs depending on the applied application.

As an application example of the image processing apparatus of the present invention, it is conceivable to restore an image read by an image reading system such as a facsimile, a copying machine or an image scanner. FIG. 13 is a diagram showing a configuration example of an image reading system in which the image processing apparatus of the present invention is incorporated as an image restoration apparatus. Referring to FIG. 13, this image reading system includes a document table 11 on which a document 10 is placed, and a contact glass 1.
2, a light source 13 that irradiates the original 10 with light, and a CCD that detects reflected light from the original 10 and obtains a read image of the original 10.
An image reading device 14 such as an element or a line sensor, and an image memory 15 for storing an image (for example, a multi-valued image or a binary image) read by the image reading device 14.
And an image restoration device 16 for performing restoration processing on the image stored in the image memory 15.

In such an image reading system, a defect of an image to be processed, that is, an object to be repaired is, for example, a stain or a part of the contact glass 12 (or an optical component such as a filter or a lens (not shown)). This occurs when there is a scratch, when the image reading device 14 such as a CCD element or a line sensor has a failure, or when a part of the original 10 has stains, scratches, or wrinkles.

Here, when a part of the contact glass 12 (or an optical component such as a filter or a lens) has stains or scratches, the defects corresponding to these stains or scratches must be removed unless they are removed. It always appears in a certain area (same area) in the image memory 15. When the image restoration device 16 restores an image defect corresponding to stains or scratches present on a part of the contact glass 12 (or optical components such as a filter and a lens), first, the image memory 1
In FIG. 5, the defective area of the image, that is, the repair target area (processing target area) needs to be specified by the area specifying unit 1. This repair target area is always stored in the image memory 15 even if image reading is repeated. Since they are at the same position, for example, an image in a state where the original 10 is not placed on the original table 11 (for example, a blank sheet is placed) is read and stored in the image memory 15 as a prescan image, In order to identify a defective area, that is, an area to be repaired, from the prescan image stored in 15, the area specifying unit 1
Can be configured.

For example, the area specifying unit 1 uses the image memory 15
If the pre-scan image is stored in multiple values inside,
A region including pixels having a predetermined density value or more can be extracted and specified as a restoration target region. Further, when the prescan image is stored in the image memory 16 in binary, it is possible to extract and specify a region including pixels having a value of “1” as a restoration target region. FIG. 14 is a diagram showing a specific example of the area specifying unit 1. In the example of FIG. 14, the area specifying unit 1 stores the area extracting unit 17 that extracts an area and the position information of each pixel in the extracted area. The area memory 18 is provided.

When a prescan image such as that shown in FIG. 15A is obtained by performing prescan, the area extracting unit 17 of the area specifying unit 1 operates as shown in FIG.
Two restoration target areas PC from the prescan image of 5 (a)
U1 and PCU2 are extracted and specified, and each area PCU1 and PC
The position information of the pixel in U2 is stored in the area memory 18.
At this time, the threshold initial setting unit 5 of the image processing unit 2 refers to the position information stored in the area memory 18 and determines the optimum initial value k ₁ of the threshold value k for the repair target area PCU1 in the manner described above. The optimum initial value k ₂ of the threshold value k for the repair target area PCU2 is set. In this way, the area to be repaired is extracted and specified by the pre-scan, and after the optimum initial values k ₁ and k _{2 of} the threshold value k are determined, the actual document is read and its image is read in the image memory 1
Store in 5. FIG. 15B is an example of an actual original image. As can be seen by comparing FIG. 15B and FIG. 15A, FIG. 15B is almost the same as FIG. 15A. A defect corresponding to a stain or a scratch existing on a part of the contact glass 12 (or an optical component such as a filter or a lens) is generated at the position. Image processing unit 2 of the image restoration device
Are stored in the area memory 18 and stored in the area PCU1,
Referring to the position information of the PCU2, even if each pixel in the restoration target regions PCU1 and PCU2 has a value in the original image of FIG. 15B, the value is not taken into consideration (that is, the value of each pixel is Is assumed to be completely unknown), the above-described processing can be performed as the restoration processing on the document image stored in the image memory 15. For example, first, in the original image, the optimum initial value k of the threshold value k is set for each pixel of the restoration target area PCU1.
_The restoration process is performed from ₁ and then the optimum initial value k of the threshold value k is set for each pixel in the restoration target area PCU2.
Perform the repair process from point ₂ .

In this way, the contact glass 12
(Or, if there are defects in some areas of the image due to stains or scratches that exist on some parts (or optical parts such as filters and lenses), use the optimum method according to this area.
This area can be quickly restored to a high quality condition.

Further, even when a part of the image reading device 14 has a failure, as in the case where the contact glass 12 has a stain or a scratch, a defect corresponding to a part of the image reading device 14 has a defect. As shown in FIG. 16A or FIG. 16B, it always appears in a certain area (same area) in the image memory 15 unless the image reading device 14 is replaced. For example, it appears as a blank area as shown in FIG. 16 (a), or as a black line as shown in FIG. 16 (b). 16A and 16B are images when the image reading device 14 is a CCD.

Therefore, the area specifying unit 1 of the image restoration device 16
To have a structure similar to that shown in FIG. 14 for identifying a defective area corresponding to stains or scratches on the contact glass 12, so that a defect in an image when a part of the image reading device 14 has a failure. The area can be specified, and each pixel of the specified defective area can be repaired by the image processing unit 2. In other words, by configuring the image restoration device 16 as shown in FIG. 14, the defective area of the image corresponding to the defect of the entire image reading system including the contact glass 12, the image reading device 14 and the like is simultaneously identified, and these are identified as 1 It can be restored by one image restoration device 16.

Further, in the case of specifying only the defective area of the image which occurs when a part of the image reading device 14 is out of order, the area specifying unit 1 may be constructed as shown in FIG. it can. That is, in the configuration of FIG. 17, the area specifying unit 1 detects the signal level output from each element of the image reading device 14 such as the CCD or the line sensor and determines whether each element is normal or abnormal. An abnormality detection unit 21 for determination and an area memory 22 for storing position information of pixels on the image memory 15 corresponding to an element determined to be abnormal are included.

Generally, when a part of the elements of the image reading device 14 is electrically broken, the signal level output from this element is always an abnormal level, and the signal level shown in FIG.
The defects shown in (a) and FIG. 16 (b) occur. Therefore, in the case where the area specifying unit 1 has the configuration of FIG. 17, the area including the pixels corresponding to the element whose signal level is detected to be abnormal by the abnormality detecting unit 21 is specified as the repair target area, The position information is stored in the area memory 22. Then, the image processing unit 2 refers to the position information stored in the area memory 22, first determines the optimum initial value of the threshold value k, and then determines the optimum initial value of the threshold value k. Image restoration is performed from that point, and as shown in FIG.
An image having a defect such as 6 (b) can be satisfactorily restored by an optimum method according to this defect.

The image reading device 14 itself can be provided with the above-mentioned functions of the image restoration device 16. In this case, even if a part of the image reading device 14 is damaged, the image reading device itself can satisfactorily restore the image corresponding to the damaged part, and from the image reading device 14, a part of the image reading device 14 is lost. Since an image of good quality is output as when it is not damaged, the life of the device can be apparently extended.

On the other hand, when a part of the original 10 is dirty, scratched, wrinkled, or the like, the original 10 is not fixed to the image reading system like the contact glass 12 and the image reading device 13, so the original 10 is not fixed. When the document 10 is read, stains, scratches, wrinkles, etc. on the document 10 are not detected in the image memory 1.
5 does not always appear in the same region.

Therefore, when the image restoration device 16 restores an image defect due to stains, scratches, or wrinkles on the original 10, the image restoration device 16 is configured as shown in FIG. 18, for example. That is, in the image restoration apparatus 16 of FIG. 18, the area specifying unit 1 instructs the operator to display the display unit 25 that displays the image stored in the image memory 15 and a predetermined area of the image displayed on the display unit 25. It has a pointing device 26 such as a mouse for moving the mouse, and an area memory 27 for storing position information of each pixel in the area designated by the pointing device 26.

With such a configuration, when the document 10 is read, the document image stored in the image memory 15 is displayed on the display unit 25 as shown in FIG. 19A, for example. Looking at this display, the operator identifies the portion of the original 10 due to stains, scratches, wrinkles, etc., and the portion is indicated by PC in FIG.
As indicated by U, it is instructed (for example, enclosed) by a pointing device 26 such as a mouse. As a result, it is instructed (enclosed) by the pointing device 26.
The position information of each pixel in the area PCU is stored in the area memory 27. Then, the image processing unit 2 refers to the position information stored in the area memory 27, first determines the optimal initial value of the threshold value k, and then determines the optimal initial value of the threshold value. Thus, image restoration can be performed on each pixel in the area PCU in the above-described manner. Note that in the example of FIG. 18, the operator is instructed to specify the area to be processed. However, if the stain or scratch has a constant shape and size, the area to be processed also has a constant shape. In this case, the area can be specified without using the operator's instruction by using the pattern recognition technique.

As described above, the image processing apparatus of the present invention is capable of properly treating defects of any kind occurring in a read image in the image reading system in an optimum manner according to the size and shape of the defect. Images can be restored.

The above application example is for the case where an image is restored to a read image (two-dimensional image) of a document, but the present invention is a patent filed by the same applicant as the present application on the same date as the present application. As described in the application (reference number 9303862), not only the two-dimensional image but also the C
It can also be applied to restoration and estimation of three-dimensional images such as T images and nuclear magnetic resonance images or moving images, and higher-dimensional images higher than that.

The present invention can also be applied to, for example, repairing a defect in a facsimile received image that occurs when noise or the like is mixed into a line or the like during image transmission of a facsimile.

Further, in each of the above examples, in the arithmetic unit 7,
The unknown pixel in the processing target area PCU is set to a completely unknown value (even if the unknown pixel has an uncertain value, this value is not taken into consideration), and only the value of the known pixel among the neighboring pixels is set. Based on this, the value of the unknown pixel is determined, but when the unknown pixel is not completely unknown and has an uncertain value, the calculation processing that also takes this information into consideration is performed, resulting in higher quality. Sometimes it can be restored to an image.

Therefore, when the unknown pixel has an uncertain value, the arithmetic unit 7 assumes that the value of this unknown pixel is completely unknown as in the above-described processing example, and the value of the known pixel among the neighboring pixels. Alternatively, the value of the unknown pixel may be finally determined by adding the uncertain value originally possessed by the unknown pixel to the value thus determined.

That is, the uncertain value that the unknown pixel originally has is defined as p (x) (where x represents the position of the unknown pixel, and p represents the density value, for example), and this unknown pixel is When q (x) is a value determined only by the value of a known pixel among neighboring pixels on the assumption that the pixel is completely unknown, the result r (x) of the arithmetic operation of p (x) and q (x) Can be finally determined as the value of this unknown pixel.

As a specific example of the arithmetic operation of p (x) and q (x), for example, the average value of p (x) and q (x) <p
(x) + q (x)> can also be taken.

[0065]

## EQU1 ## r (x) = <p (x) + q (x)>

Alternatively, as shown in the following equation, the processing target area P
When the smallest one of the values p (x) obtained for each pixel of the CU is p '(x), the average value of p (x) and q (x) <p (x) + q (x) Value obtained by subtracting p ′ (x) from〉 {〈p
(x) + q (x)>-p '(x)} may be r (x).

[0067]

[Equation 2]

As a case where the unknown pixel in the processing target area PCU has an uncertain value, for example, there is a defect in the image read by the image reading system as shown in the first application example. That is, when a part of the contact glass 12 (or an optical component such as a filter or a lens) is dirty or scratched, or when the image reading device 1 is used.
When the sensitivity of some of the elements of 4 is low, or when a part of the document has stains, scratches, wrinkles, etc., the value of each pixel in the defective portion is not unknown at all, and its reliability is somewhat unknown. It may be low. In such a case, assuming that the value of the unknown pixel is completely unknown and the value r (x) of the unknown pixel is determined as p (x), the information originally possessed by the unknown pixel is obtained. Is not used at all. On the other hand, as described above, the value r (x) of the unknown pixel is set to p (x) and q
By using the arithmetic operation value with (x), the image can be restored with higher quality.

Further, as a modified example, it is judged whether the original value q (x) of the unknown pixel is highly reliable, and when it is highly reliable, the value r (x) of the unknown pixel is given by Alternatively, on the other hand, if the reliability is low, the original value of the unknown pixel is completely unknown, and the unknown pixel value r (x) is determined as p (x). good.

In each of the above examples, the processing target area PC
Although it is assumed that the pixels in U are processed in units of one pixel, in order to perform the processing at high speed, a group of a plurality of pixels is regarded as one pixel as shown in FIG. It is also possible to carry out processing. Note that, in the example of FIG. 20, a set of a plurality of pixels is made up of 2 × 2 pixels, and in this case, neighboring pixels can also be defined as being made up of a set of a plurality of pixels. For example,
Like one pixel in the area PCU, one of the neighboring pixels is
It can be defined as consisting of 2 × 2 pixels.

In this way, when a set of a plurality of pixels is regarded as one pixel and processing is performed in this pixel unit, 1 is set.
The number of pixels to be processed can be reduced as compared with the case where processing is performed in pixel units, and the processing can be performed at high speed. For example, when 2 × 2 pixels are regarded as one pixel and processing is performed in pixel units, the processing speed can be increased four times. However, the resolution of the image is deteriorated by performing such processing. In the above example, the resolution deteriorates to 1/2. Therefore, it can be applied to applications where the resolution may be deteriorated.

As an example of its application, for example, the facsimile machine on the transmission side has the ability to send an image signal at a resolution of 400 dpi, while the facsimile machine on the reception side has only the reproducibility of the resolution of 200 dpi. If not. In this case, the receiving facsimile machine
When the above-described processing is performed on the 400 dpi image from the facsimile apparatus on the transmission side, it is not necessary to maintain the resolution of 400 dpi on the transmission side.
A resolution of 0 dpi is sufficient. Therefore, the 2 × 2 pixels are regarded as one pixel, and the image restoration process is performed for each pixel to increase the processing speed four times, and
The resolution required by the receiving side can be satisfied.

[0073]

As described above, according to the first to seventh aspects of the invention, each pixel in the processing target area is initially an unknown pixel having a completely unknown value or an uncertain value. Given that, when focusing on a specific unknown pixel in the processing target region, only if the number of known pixels having a known value among the neighboring pixels of the unknown pixel is larger than a predetermined threshold value, among the neighboring pixels, While the value of the unknown pixel is determined using the value of the known pixel, the value of the unknown pixel is not determined and the unknown pixel is left as it is when the number of the known pixels is smaller than a predetermined threshold. After each pixel in the target area has been subjected to the above-described processing with a predetermined threshold value, the unknown pixel whose value has been determined is set as a known pixel, the threshold value is changed, and the processing is repeated for the remaining unknown pixels. In the image processing to be performed, The optimum threshold value is determined based on the information about the size of the processing target area and / or the information about the shape of the processing target area. Optimal processing can be performed, and images can be restored and estimated at high speed and with high quality.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration example of an image processing apparatus according to the present invention.

FIG. 2 is a diagram for explaining a processing target area and a known area in an image.

FIG. 3 is a flowchart for explaining the operation of the image processing apparatus in FIG.

4A to 4D are diagrams showing a specific example of processing of the image processing apparatus of FIG.

FIG. 5 is a diagram showing a configuration example of a threshold initial setting unit.

6A and 6B are diagrams for explaining a method of detecting the shape of a processing target area.

7A to 7C are diagrams showing a configuration example of a table.

FIG. 8 is a diagram illustrating a specific example of processing of a threshold initial setting unit.

FIG. 9 is a diagram showing a specific example of processing of a threshold initial setting unit.

FIG. 10 is a diagram illustrating a specific example of processing of a threshold initial setting unit.

FIG. 11 is a diagram showing a case where a plurality of processing target areas exist in one image.

12A to 12C are diagrams for explaining the shape of a region PCU.

FIG. 13 is a diagram showing a configuration example of an image reading system in which the image processing apparatus of the present invention is incorporated as an image restoration apparatus.

FIG. 14 is a diagram illustrating a specific configuration example of an image processing apparatus.

15A is a diagram showing an example of a prescan image, FIG.
(b) is a diagram showing an example of an actual document image.

16A and 16B are diagrams showing an example of an image when some elements of the image reading device have a defect.

FIG. 17 is a diagram illustrating a specific configuration example of an image processing apparatus.

FIG. 18 is a diagram illustrating a specific configuration example of an image processing apparatus.

19 (a) and 19 (b) are diagrams for explaining how to specify a processing target area on the display unit.

20 (a) and 20 (b) are diagrams showing an example of a case where a set of a plurality of pixels is regarded as one pixel and is processed with respect to a pixel in a processing target region.

[Explanation of symbols]

 1 area specifying section 2 image processing section 3 control section 4 neighborhood pixel defining section 5 threshold initial setting section 6 calculation type setting section 8 area information detecting section 9 initial value determining section 10 original document 11 original plate 12 contact glass 13 light source 14 image reading device 15 image memory 16 image processing device 17 area extraction unit 18, 22, 27 area memory 21 abnormality detection unit 25 display unit 26 pointing device PC image PCU processing target area PCK known area TBL table

Claims (7)

[Claims] 1. An image processing apparatus for performing a predetermined image processing on a pixel included in a processing target area in an image, wherein the image processing apparatus includes information on a size of the processing target area, and / or A threshold initial determination means for determining an optimum threshold initial value based on information about the shape of the processing target area, and each pixel in the processing target area initially has a completely unknown value or an unknown value. Given that it is a pixel, and paying attention to a specific unknown pixel in the processing target area,
Only when the number of known pixels having a known value among the neighboring pixels of the unknown pixel is larger than a predetermined threshold, while determining the value of the unknown pixel using the value of the known pixel of the neighboring pixels, When the number of known pixels is smaller than a predetermined threshold value, an arithmetic processing unit that performs arithmetic processing so as to leave the unknown pixel value as it is without determining the value of the unknown pixel, and the threshold value initial determination for each pixel in the processing target area. The arithmetic processing is performed by the arithmetic processing means from the optimum initial value of the threshold value determined by the means, and after the arithmetic processing is performed on each pixel in the processing target area, the unknown pixel of which the value is determined is a known pixel. In addition, the image processing apparatus further comprises: a control unit that changes the threshold value and repeats the calculation process for the remaining unknown pixels. 2. The image processing apparatus according to claim 1,
The image processing apparatus, wherein the threshold initial determination unit is configured to detect information regarding the size of the processing target area as the number of pixels in the processing target area. 3. The image processing apparatus according to claim 1,
The image processing apparatus, wherein the threshold initial determination means is configured to detect information regarding the size of the processing target area as a ratio of the number of pixels in the processing target area to the number of pixels of the entire image. 4. The image processing apparatus according to claim 1,
The threshold initial determination means is configured to detect the information on the shape of the processing target area based on the ratio of the length in the vertical direction and the horizontal direction of the cut-out rectangular area. An image processing device characterized by the above. 5. The image processing apparatus according to claim 1,
The threshold initial determination means detects information regarding the shape of the processing target area based on a ratio of the number of pixels in the cut-out rectangle and the number of pixels in the processing target area, which is obtained by cutting the processing target area into a rectangular shape. An image processing device characterized by: 6. The image processing apparatus according to claim 1,
The threshold initial determination means, information regarding the shape of the processing target area, the processing target area is cut out into a rectangular shape, the ratio of the length of the cut-out rectangle in the vertical direction and the horizontal direction, and the number of pixels in the cut-out rectangle. An image processing apparatus, which is configured to detect based on a ratio with the number of pixels in a processing target area. 7. The image processing apparatus according to claim 1,
The threshold initial determination means determines a larger initial value of the threshold as the size of the processing target area increases, and determines a larger initial value of the threshold as the shape of the processing target area becomes narrower. An image processing apparatus characterized in that.