JP2006279441A - Image processing method, apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method, apparatus, and program capable of automatically repairing a defect region caused by attachment of a very small substance onto a medium from a digital signal obtained by reading an image formed on the medium. <P>SOLUTION: The image processing apparatus uses a first morphology filter that detects a filamentous very small region with a strong contrast to a background from a digital image obtained by reading an image formed on the medium by photoelectric conversion and/or a second morphology filter that detects dotted regions with a strong contrast to the background and existing closely to each other to detect a defective region, and the image processing apparatus repairs the defective region by replacing the image in the detected defective region with an image in other regions than the defective region. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing method and apparatus used for image processing, specifically image restoration, and a program therefor.

  In recent years, with the cost reduction of large-capacity recording media, individuals can store a large amount of data, and it becomes possible for individuals to own a system for storing and managing a large amount of image data such as an electronic album. ing. In addition, a service for managing user photo image data like an electronic album instead of the user is also provided. The functions required of these management systems include digitizing existing photographic images formed on media such as photographic film, photographic prints, and paper, rather than managing only digital images captured by digital cameras. There is storage and management (photo mining).

  A system that provides a photomining service reads an image formed on a medium with a reading device such as a scanner and digitizes the image. However, depending on the storage age of the medium and the storage environment, lint may adhere to the medium. Therefore, there is a problem that in the image obtained by digitization, yarn waste, mold, and the like that are originally present appear as a defective area, and the image quality is adversely affected.

  In Patent Document 1, as a method of repairing a part of an originally defective area from a digital image obtained by reading a medium, a defect of a CCD of a reading device is compared by comparing digital images obtained by reading a plurality of media. A method is described in which a place or a place where dust adheres to the lens is found, and a portion corresponding to the image is repaired.

Non-Patent Document 1 describes a method in which a defect area is designated by an operator and the designated defect area is repaired.
Japanese Patent Laid-Open No. 7-114639 A. Crimini, P.A. Perez, and K.K. Toyama, Object Removable by Exemplar-based Imposing, Conf. on Computer Vision and Pattern Recognition, CVPR, Madison, Wisconsin, 2003

  However, the method described in Patent Document 1 can find and repair a defect area generated at a fixed position of an image obtained by the reading device due to a CCD defect of the reading device or dust attached to the lens. In addition, it cannot be applied to restoration of an image having a defective area caused by dust such as lint attached to an arbitrary place on the medium or mold.

  In addition, the method described in Non-Patent Document 1 has a problem in that it is impossible to specify the information accurately unless it is a skilled engineer in addition to placing a burden on the operator.

  In view of the above circumstances, the present invention provides a digital image obtained by reading an image formed on a medium by photoelectric conversion. An object of the present invention is to provide an image processing method and apparatus that can be efficiently restored, and a program therefor.

According to the image processing method of the present invention, a defective area caused by a minute substance that is not originally included in the image is attached to the medium with respect to a digital image obtained by photoelectrically converting an image formed on the medium. In the image processing method to be restored,
A first morphological filter for detecting a microscopic region that is filamentous and has a strong contrast with the background; and / or a second filter for detecting a point-like region that has a strong contrast with the background and that is close to the background. Detecting the defect area using a morphological filter,
The detected defect area is set as a repair target area, and the defect area is repaired by replacing an image of the repair target area with an image other than the repair target area.

  Here, when the defect area is repaired, it is preferable that an area obtained by expanding the detected defect area is the repair target area.

The image processing apparatus according to the present invention has a defect area caused by a minute substance that is not originally included in the image attached to the medium with respect to a digital image obtained by reading an image formed on the medium by photoelectric conversion. In the image processing apparatus to be restored,
A first morphological filter for detecting a microscopic region that is filamentous and has a strong contrast with the background; A defect area detecting means for detecting the defect area using a morphological filter;
And a repair execution means for repairing the defective area by replacing the detected defective area with a repair target area and replacing an image of the repair target area with an image other than the repair target area. Is.

The image processing apparatus of the present invention has expansion means for expanding the defective area detected by the defective area detection means,
It is preferable that the repair execution unit sets the defect area expanded by the expansion unit as the repair target area.

  The image processing apparatus of the present invention further includes a defect area designating unit for allowing a user to designate a defect area in an area other than the defect area detected by the defect area detection unit, and the repair executing unit includes the repair execution unit, More preferably, the defect area designated by the defect area designation means is also the repair target area.

  You may provide the image processing method of this invention as a program which makes a computer perform.

  According to the image processing method and apparatus of the present invention, attention is paid to the fact that most of dust attached to a print medium such as a photographic print or paper, a photographic film, or the like as a result of aging storage is yarn waste or mold. Depending on the characteristics of the first morphological filter for detecting a microscopic region having a filamentous shape and a strong contrast with the background and / or a strong contrast with the background from the image obtained by reading the medium. Since the defect area is detected and repaired by using the second morphological filter for detecting the point-like area that exists, the image is restored without imposing a burden on the operator. .

  In addition, by repairing a region obtained by expanding the detected defect region as a region to be repaired, it is possible to prevent a repair leak at the edge of the defect region.

  Further, a defect area designating unit may be provided so that a user can designate a defect area (for example, large mold or dust) that could not be detected by the above-described detection. In addition, most of the defect area has already been detected before the designation by the operator, so the burden on the operator is small.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. Note that the image processing apparatus according to the present embodiment performs a restoration process on an input image, and is realized by executing a processing program read into an auxiliary storage device on a computer (for example, a personal computer). Is done. Further, this processing program is stored in an information storage medium such as a CD-ROM, or distributed via a network such as the Internet and installed in a computer.

  The image data represents an image, and the following description will be given without particularly distinguishing the image from the image data.

  As shown in FIG. 1, the image processing apparatus according to the present embodiment includes an image input unit 1 that inputs an image, a repair region acquisition unit 10 that acquires a defect region (hereinafter referred to as a repair region) in the input image, and a repair. The repair unit 100 repairs the repair region by assigning a pixel value to the repair region acquired by the region acquisition unit 10.

  The image input means 1 is for inputting an image to be processed by the image processing apparatus of the present embodiment, and reads an image formed on a medium such as photographic film, paper, or print paper by photoelectric conversion. Thus, a scanner for obtaining a digital image can be used.

  The repair area acquisition means 10 acquires a repair area from a digital image (hereinafter simply referred to as an image) obtained by the image input means 1, and includes a detection means 10a and an expansion means 10b as shown in FIG. It becomes.

  The detection means 10a uses the first morphological filter to detect a minute region that is thread-like and has a strong contrast with the background, and uses the second morphological filter to have a strong and close contrast with the background. An existing point-like region is detected. Here, the process by the first morphological filter and the process by the second morphological filter will be specifically described as a first detection process and a second detection process, respectively.

  The first detection processing is a target area, that is, a fine area having a string shape and a strong contrast with the background. The brightness value is larger than 90 (the range of the brightness value of the image is 0 to 255), and the background and In this case, an area having a contrast greater than 30 and a width of less than 3 pixels is detected. In order to detect such an area, first, an image A is obtained by performing an opening process using a 3 × 3 square structure element on a pixel having a luminance value greater than 90 in the image obtained by the image input means 1. obtain. Then, the image B is obtained by subtracting the image A from the image obtained by the image input means 1. In the image B, binarization processing for binarizing with a threshold 30, that is, a binarized luminance value of a pixel having a luminance value greater than 30 is set to 1, and a binarized luminance value of a pixel having a luminance value of 30 or less is set to 0 Thus, a binarized image is obtained, and a bright area (that is, an area having a binarized luminance value of 1) in the binarized image is detected as a target area.

  The second detection process is a point-like region having a strong contrast with the target region, that is, the background, and a luminance value greater than 220 and a contrast with the background greater than 30, and a short point thereof. An area having a side length of less than 6 pixels is detected. In order to detect such a region, first, an image C is obtained by performing an opening process using a 6 × 6 square structure element on a pixel having a luminance value greater than 220 in the image obtained by the image input means 1. obtain. Then, the image D is obtained by subtracting the image C from the image obtained by the image input means 1. The image D is binarized by thresholding 30, that is, a binarized luminance value of a pixel having a luminance value greater than 30 is set to 1, and a binarized luminance value of a pixel having a luminance value of 30 or less is set to 0. Thus, a binarized image is obtained, and a bright area (that is, an area having a binarized luminance value of 1) in the binarized image is detected as a target area.

  The expansion means 10b expands each area detected by the detection means 10a. In this embodiment, the edge of each area is expanded by several pixels, for example, two pixels outward. Each area is acquired as a repair area. That is, the area detected by the detection means 10a and the area in the vicinity of the area are set as the repair area.

  The repair area acquisition means 10 acquires repair areas as many as dust and mold attached to the medium, and the repair means 100 repairs all of these repair areas. Since the repair method is the same, the case where there is only one repair region will be described below as an example. An image D0 shown in FIG. 3 is an example of an image from which a repair area has been acquired. The white space at the center of the image D0 indicates the repair area, and the shaded area is a known area composed of pixels with known pixel values.

  FIG. 2 is a block diagram showing a configuration of the repair unit 100.

  As shown in FIG. 2, the restoration unit 100 includes a multi-resolution image creation unit 105, a priority calculation unit 110, an embedding source image acquisition unit 115, an embedding unit 120, and an edge correction unit 130. It is. The details of each of these components will be described below.

The multi-resolution image creating means 105 creates a plurality of images Di having different resolutions from the image D0, and the resolution of each image Di is decreased in descending order of i. In the present embodiment, multi-resolution image creating device 105, first, an image D0 and _{D 1,} the image obtained by respectively reducing the image D0 in reduction ratio of _{1 / 2,1 / 4,1 / 8 D} 2 , D ₃ and D ₄ . FIG. 3 shows images D ₁ , D ₂ , D ₃ , and D ₄ obtained by the multi-resolution image creating means 105.

Priority calculation unit 110, image D0, that is, calculates a buried priority β position of each pixel on the edge of the repair area in the highest image D ₁ resolution. In the present embodiment, the priority calculation unit 110, calculates the four images Di (i = 1 to 4), the order from the low resolution image (i.e., the order from D ₄₎ the priority β respectively using To do. Here, first, a process for calculating the priority β using the image Di (i> 1) (that is, D ₄ , D ₃ , D ₂ ) will be described.

<Calculation of Priority β Using Image Di (i>1)>
In calculating the priority β using the image Di (i> 1), the priority calculation means 110 is a patch for calculating the priority (a patch of 9 pixels × 9 pixels in the present embodiment. As shown in FIG. 4, the calculation patch Fy is arranged so that its center is located at a pixel (pixel A in the figure) on the edge of the repair area in the image Di, and the embedding priority β at this position is obtained. .

  In obtaining the embedding priority β of the pixel A, the priority calculation unit 110 first checks the number of pixels in the known area in the priority calculation patch Fy. When the number of pixels in the known area in the priority calculation patch Fy is small as shown in FIG. 7 and is equal to or less than the predetermined threshold value K1, the priority of the pixel A is set to zero.

  On the other hand, when the number of pixels in the known area in the priority calculation patch Fy arranged in the pixel A is larger than the threshold value K1, the priority calculation unit 110 embeds a repair area in the pixel A (hereinafter referred to as an embedding direction). ). In the present embodiment, the four directions as shown in FIG. In FIG. 5, a black area and a white area indicate a known area and a repair area, respectively, and the embedding direction of the repair area is left to right (direction d1), top to bottom (direction d2), and right to left (direction d3). , From the bottom to the top (direction d4). The priority calculating means 110 determines the embedding direction based on the normal direction h from the known area to the repair area in the pixel A. Specifically, the normal direction h is the above four directions. , The normal direction h is determined as the embedding direction and the embedding priority β is obtained, while the normal direction h is equal to two of the four directions. In the case of being located in between, these two directions are set as the embedding direction candidates, the embedding priority is obtained for each of the two embedding direction candidates, and the direction with the higher embedding priority obtained is determined as the embedding direction, The embedding priority already obtained for this direction is set as the embedding priority β at this position.

  In the case of the position of the pixel A shown in FIG. 4, the normal direction h coincides with the direction from top to bottom as shown in FIG. 6A and the direction d2 shown in FIG. 110 determines the direction h (d2) as the embedding direction in the pixel A. On the other hand, in the case shown in FIG. 6B, since the normal direction h in the pixel B is between the embedding direction d1 and the embedding direction d4, the priority calculation unit 110 embeds the pixel B in the embedding direction. Using the direction d1 and the embedding direction d2 as candidates for embedding directions, embedding priorities are calculated for the two directions.

  After determining the embedding direction (or embedding direction candidate), the embedding priority calculating unit 110 detects a boundary line in a known area in the priority calculating patch Fy and calculates the priority.

  The priority calculation unit 110 first calculates a variance value of pixel values (for example, luminance values) of a known area in the priority calculation patch Fy, and if the variance value is equal to or less than a predetermined threshold K2, the priority calculation unit 110 calculates the priority. The priority is set to 0 on the assumption that the known area in the patch Fy is a uniform area and there is no boundary line between the areas.

Note that the variance value σ of the pixel values in the known area may be calculated according to the following equation (1), for example.

  On the other hand, when the variance value of the pixel values of the known area in the priority calculation patch Fy is larger than the threshold value K2, the priority calculation means 110 detects the boundary line of the known area in the priority calculation patch Fy. To set the priority.

  In the present embodiment, the priority calculation unit 110 detects a boundary line using a level set (Level Set) region division method. Specifically, first, a frame is arranged on a known area in the priority calculation patch Fy. Here, if a large frame is used for arranging the frames, the calculation time can be shortened. However, since the accuracy of area division is not good, the priority calculation unit 110 in FIG. As shown in b), a plurality (three in the illustrated example) of small frames W are arranged at positions separated from each other in the known area. Then, the priority calculation unit 110 performs a deformation process on each frame W to divide the known area into two areas.

In the transformation process, each frame W is transformed so that the total sum Q of the variance value σa of the pixel value in the frame and the variance value σb of the pixel value outside the frame shown in the following equation (2) becomes the smallest. Specifically, it is a process of repeating an inward concave process for indenting the frame W and an external convex process for projecting the frame W outward. In the indentation process, among the pixels at the inner edge of the frame W, pixels whose sum Q is reduced by taking the pixel out of the frame W are subjected to the indentation process, and the frame W is set at the position of the pixel. In the convex processing, out of the pixels at the outer edge of the frame W, a pixel whose total sum Q is reduced by placing the pixel in the frame W is set as the target of the convex processing. This is an outward convex process for projecting the frame W outward at the position. As shown in FIG. 9A, if the pixel A2 on the frame W is the pixel on the outer edge of the frame W, the pixel A1 in the frame W adjacent to the pixel A2 becomes the inner edge pixel. ), If the pixel A2 on the frame W is the inner edge pixel, the pixel A3 outside the frame W adjacent to the pixel A2 is the outer edge pixel. If there are no pixels on the frame, the two adjacent pixels across the frame become the inner edge pixel and the outer edge pixel, respectively. In the present embodiment, as shown in FIG. 9B, pixels on the frame W are pixels on the inner edge of the frame W, and pixels outside the frame W adjacent to the pixels are pixels on the outer edge. FIG. 9C shows an example in which the frame W is indented when the pixel A2 at the inner edge of the frame W is moved out of the frame W to reduce the above-described total sum Q of the dispersion values. 9 (d) shows an example in which the frame W is protruded outward when the pixel A3 at the outer edge of the frame W is placed in the frame W to reduce the above-described total sum Q of the dispersion values.

  The priority calculation means 110 repeats such inward and outward convex processing, and after the deformation of the frame W (and the combination of the frames resulting from the deformation), the known area is divided into 2 inside and outside of one frame. Divide into two areas. As a result of the division, a boundary line between these two areas is obtained.

  For the detected boundary line, the priority calculation unit 110 first extends from the contour line on the known area side of the priority calculation patch Fy to the edge of the restoration area in the priority calculation patch Fy. Check if it is a boundary line. Specifically, both ends of the boundary line are contact points of the boundary line, the contour line on the known area side of the priority calculation patch Fy, and the edge of the restoration area in the priority calculation patch. In other words, in other words, if the boundary line extends from the contour of the priority calculation patch Fy to the edge of the repair area, the boundary line mode and embedding direction (embedding direction) The priority higher than 0 is set based on the candidate), but if it is not a contact, the priority β of the pixel A is set to 0.

  FIG. 10 is a diagram for explaining setting of priority by the priority calculation means 110 when the boundary line is a boundary line extending to the edge of the restoration area in the priority calculation patch Fy. To the right, the order of priority is shown. As illustrated, the priority is set according to the following rules.

  Rule 1: The priority is higher as the position of the contact point between the boundary line and the edge of the repair area is closer to the center point of the priority calculation patch Fy.

  When there are two boundary lines as in the example (b) and example (c) shown in FIG. 10, the positions of the contact points are the contact points between the two boundary lines and the edge of the repair region. The average position of the positions.

  Rule 2: On the premise of Rule 1, the priority is higher when there is only one boundary line than when there are two boundary lines.

  Rule 3: When there are two boundary lines, the priority is higher as the area of the region surrounded by the two boundary lines is larger.

  Rule 4: On the premise of rules 1, 2, and 3, the priority is higher as the extending direction of the boundary line is closer to the embedding direction.

  The priority calculation means 110 sets the priority of the pixel at the position where the priority calculation patch Fy is arranged in this way. As shown in FIG. 6B, when there are two embedding candidate directions, the priority calculation unit 110 calculates the priorities for the two embedding candidates, and the larger priority is calculated. The embedding direction candidate is determined as the embedding direction, and the priority calculated for the direction is set as the priority of the pixel.

  The priority calculation means 110 performs the above-described processing on the pixel at the position where the priority calculation patch Fy is first arranged to calculate the priority β of the pixel. Then, the priority calculation patch Fy is arranged with its center point shifted by one pixel on the edge of the restoration area, and the priority β of the pixel at the newly arranged position is calculated.

  The priority calculation means 110 repeats the arrangement of the priority calculation patch Fy, the calculation of the priority, the movement of the priority calculation patch Fy, the calculation of the priority,... The pixel priority β is obtained.

  Here, based on the priority β calculated by the priority calculation unit 110, the embedding source image acquisition unit 115 selects the pixels in order from the pixel with the highest priority β among the pixels with the priority β higher than 0. The embedding source image at the position of is acquired. Here, a process of acquiring an embedded source image by the embedded source image acquiring unit 115 will be described.

<Process to acquire embedded image>
The embedding source image acquisition unit 115 checks the priority β obtained by using the image Di (i> 1) by the priority calculation unit 110, and if there is no pixel with the priority β greater than 0, the embedding source image If there is a pixel with a priority β greater than 0, the image used when the priority is calculated by the priority calculation unit 110 is first used. In the same image Di (i> 1), an embedding source image at the pixel position having the highest priority β is acquired. Specifically, the embedding source image acquisition unit 115 is a target pixel (a pixel having the highest priority β among the pixels having a priority β larger than 0 in the pixels on the edge of the restoration region in the image Di. ), An embedding patch Fu1 (a patch of 5 pixels × 5 pixels in this embodiment) that is smaller than the priority calculation patch Fy, and the center point thereof is the target pixel (pixel A shown in FIG. 11). Place it so that it is located in Then, in the arranged embedding patch Fu1, a known area portion is set as a matching area, and an embedding source image acquisition patch Fu2 for the pixel A shown in the lower part of FIG. 11 is obtained. For ease of understanding, the embedded source image acquisition patch Fu2 in the lower part of FIG. 11 is enlarged and displayed, but the size of the embedded source image acquisition patch Fu2 is 5 pixels × the size of the embedded patch Fu1. 5 pixels. Then, the embedding source image acquisition unit 115 performs matching in the embedding source image acquisition patch Fu2 at each moved position while moving the embedding source image acquisition patch Fu2 on the known area near the target pixel A. The similarity between the image of the area and the image of the known area in the embedding patch Fu1 is obtained. Then, an image in the embedding source image acquisition patch Fu2 arranged at a position having the highest similarity is acquired as an embedding source image.

  For the similarity, for example, the Euclidean distance of the luminance value between the image of the matching area in the embedding source image acquisition patch Fu2 and the image of the known area in the embedding patch Fu1 is obtained, and this distance is small. It can be calculated that the degree of similarity is higher.

  The embedding source image acquisition unit 115 acquires the embedding source image in this way.

On the other hand, although details will be described later, the embedding unit 120 of the restoration unit 100 embeds the restoration region in the order from the pixel position having the highest priority β in the image D ₁ having the highest resolution. On the other hand, the priority β calculated by the priority calculation unit 110 is a pixel on the edge of the restoration area in the image Di (i> 1) used for the calculation, and the embedding source The embedding source image obtained by the image acquisition unit 115 is an image in the image Di (i> 1) used when calculating the priority. Therefore, in order to provide the embedding means 120, determine the position of the thus calculated priority β position of pixels in the image D ₁ of the pixel corresponding with, and embedded original image on the image D ₁ corresponding image There is a need to. Hereinafter, the process for determining the position is referred to as a regression process. Here, the regression process will be described.

<Regression processing>
As described above, the regression process is performed on the pixel having the priority β calculated using the image Di (i> 1) greater than 0 and the embedding source image acquired for the pixel.

First, the lowest resolution image, the priority β calculated and embedded original image acquired for regression process as an example the case of using the image D ₄ will be described.

In performing the regression process, first, the priority calculation means 110 uses an image having a resolution one higher than that of the image (here, the image D ₄ ) used when calculating the priority, in this case, the image D ₃ . calculated priority β is to determine the position of greater than 0 pixels with D _4. Since the image D ₄ is obtained by reducing the image D ₃ by 1/2, the position of one pixel in the image D ₄ corresponds to the position of two pixels on the image D ₃ . The priority calculation unit 110 arranges and calculates the priority calculation patch Fy at the pixel position on the image D ₃ corresponding to the pixel position where the priority β calculated in the image D ₄ is greater than 0. Calculate the degree. Here, the priority calculation patch Fy and the priority are calculated by calculating the pixel at the position on the image D ₃ corresponding to the position of the pixel having the priority β of 0 or more calculated in the image D ₄ ( Since the number is calculated only for twice the number of pixels having a priority β greater than 0 calculated in the image D ₄ , it can be done in a short time. The embedded original image obtaining unit 115, the obtained embedded original image in the image D _4, the position in the image D _3, adjusted in accordance with the position of the image D ₃ obtained by the priority calculation unit 110, obtaining a position of the embedded original image on the image D _3.

Then, the embedding priority calculating unit 110 and the embedding source image obtaining unit 115 perform processing for determining the position on the low-resolution image as the corresponding position on the image on the one higher resolution in this way from D ₃ to D _2. , D ₂ to D ₁ , and having determined the position of the pixel whose embedding priority β is greater than 0 in the image D ₁ having the highest resolution, and the position of the embedding source image at the position of each of these pixels, End the regression process.

Note that here, the regression processing when the calculation of the priority and the acquisition of the embedding source image are performed using the image D ₄ has been described, but the calculation of the priority and the acquisition of the embedding source image are performed using the image D _3. If it is performed, as a regression process, an image whose resolution is one higher than that of the image D ₃ , that is, in the image D ₂ , the position of the pixel whose priority β calculated in the image D ₃ is greater than 0 and the corresponding embedding source After the position of the image is determined, an image having a resolution one higher than that of the image D ₂ , that is, in the image D ₁ , the position of the pixel whose priority β calculated in the image D ₂ is greater than 0 and the corresponding embedded original image The position of is fixed. Also, when performing acquisition priority calculation and embedded original image by using the image D _2, as the regression process, high image has one resolution than image D _2, i.e. in the image D _1, image D ₂ The position of the pixel whose priority β calculated in step S is greater than 0 and the position of the corresponding embedding source image are determined.

Next, the priority calculation unit 110 is processing for calculating the priority β will be described with reference to the image D _1.

<Calculation of priority β using the image D _1>
In calculating the priority β using the image D ₁ , the priority calculating unit 110 _first calculates the priority β shown in FIG. 4 in the same manner as when calculating the priority β using another image. the patch Fy, the central point is arranged so as to be located to the pixels on edge of repair area in the image D _1, to confirm the number of pixels known region within arranged priority calculating patch Fy. When the number of pixels in the known area in the priority calculation patch Fy is equal to or less than the threshold value K1, the priority calculation unit 110 sets the priority of the pixel as 0.

  On the other hand, when the number of pixels in the known area in the priority calculation patch Fy is larger than the threshold value K1, the priority calculation unit 110 determines the embedding direction at the pixel position where the priority calculation patch Fy is arranged. . Since the process of determining the embedding direction is the same as the determination of the embedding direction when calculating the priority β using another image Di (i> 1), detailed description thereof is omitted here. Also in this case, as shown in FIG. 6A, when there are two possibilities of embedding directions, the two directions are determined as embedding direction candidates, the priority is obtained for each, and the embedding direction candidate having a higher priority is obtained. Is determined as the embedding direction, and the priority calculated for the direction is set as the priority of the pixel.

  After determining the embedding direction (including embedding direction candidates), the priority calculation unit 110 detects an edge at the center point of the priority calculation patch Fy, and when no edge is detected at the center point. The priority β of the pixel located at the center point is set to 1, and when the edge is detected, the priority of the priority calculation patch Fy is set so that the higher the edge strength, the higher the priority. The priority β of the pixel at the point is set to a value larger than 1.

In this case, since the priority β is calculated in the image D _1, it is not performed regression process.

  Next, the overall image processing of the present embodiment will be further described.

FIG. 12 is a flowchart showing processing of the entire image processing apparatus of the present embodiment. As shown in the figure, the image to be processed obtained by the image input means 1 is acquired by the repair area acquisition means 10 to obtain a repair area as an image D0 and input to the repair means 100 (S10). In the restoration means 100, the multi-resolution image creation means 105 first creates i (ie, 4) images D ₁ (D0), D ₂ , D ₃ , and D ₄ having different resolutions with i being 4 ( S15, S20). Then, the priority calculation unit 110 calculates the priority using the image Di having the lowest resolution (here, D ₄ ), the priority β of each pixel on the edge of the restoration region in the image D ₄ , and these To obtain the embedding direction in the pixel (S30).
The embedding source image acquisition unit 115 checks whether or not there is a pixel with a priority β greater than 0. If there is a pixel with a priority β greater than 0 (S110: Yes, S111: Yes), among these pixels, An embedding source image is acquired at the position of the pixel having the highest priority β (S112). Then, the priority calculation unit 110 and the embedding source image acquisition unit 115 determine the position of the pixel having the priority β greater than 0 calculated in the image D ₄ and the position of the corresponding embedding source image in the image D ₁ . In the image D1, the pixel having the priority β larger than 0 and the embedding direction of these pixels and the embedding source image are obtained in the image D1 (S113). Then, the embedding unit 120 obtains the embedding source image used when acquiring the embedding source image using the embedding source image obtained for the pixel in order from the pixel having the highest priority in the image D1. Embedding is executed by replacing the image in the embedding patch Fu1 corresponding to the patch Fu2. Here, the size of the embedded patch Fu1 and embedded original image acquisition patch Fu2 is because it is 5 × 5 pixels in the image D _4, the size corresponding to the image D ₁ is 40 pixels × 40 pixels.

First, the embedding unit 120 performs embedding in a pixel having the highest priority β, and when embedding in a pixel having the highest priority is completed, embedding is performed in a pixel having the next highest priority β. In this order, pixels with a priority β higher than 0 are embedded in descending order of the priority β (S1S125: Yes, S118, S120). If all the embedding of pixels with the priority β greater than 0 is completed, the embedding based on the priority β calculated using the image Di (D _{4 in} this case) is terminated, and a new image D obtained by this embedding is completed. ₁ (an image in which the original image D ₁ is embedded in a pixel having a priority β greater than 0) is output to the multi-resolution image creating unit 105. The multi-resolution image creating means 105 uses a new image D ₁ to create a plurality of new images (up to a resolution one higher than the resolution of the image already used when obtaining the priority β (here, D ₄ )). Here, new images D ₁ , D ₂ , D ₃ ) are created, and the priority calculation means 110 uses an image having the lowest resolution among the new images (here, new image D ₃ ). The priority β is calculated, and the processing from step S30 is repeated ((S125: No, S128, S20, S30-).

On the other hand, in step S111, if there is no pixel whose priority β is greater than 0 (S111: No), the embedding source image acquisition unit 115 does not perform the process of acquiring the embedding source image, and the priority calculation unit 110 The priority β is calculated using an image (D ₃ here) that has one higher resolution than the image (D ₄ here) that has already been used to determine the priority β, and the processing from step S30 is repeated ( S114, S30 ~).

Such processing is performed until embedding is completed based on the priority β calculated using an image whose resolution is only one lower than the original image (image D _{2 at} that time) (S20 to S128).

When resolution is based on the β priority calculated using only one lower image than the original image embedded is completed, image embedding has been performed is obtained as a new image D _1. In this case, since the processing from the priority calculation to the embedding based on the calculated priority is performed for all images having a resolution lower than that of the original image, the priority calculation means 110 is the same as the original image. calculating a priority β using the new image _{D 1} with a resolution (S30, S110: Yes). The embedding source image acquisition unit 115 and the embedding unit 120 perform embedding from the pixel having the highest priority β, and the process ends when the embedding at the pixel position where the priority β is the lowest is 0 (S130). ~ S136).

  With respect to the image that has been embedded (the same resolution as the original image), the edge correction means 130 corrects an unsmoothed edge by the embedding and ends the repair process.

  FIG. 13 is a flowchart showing a process for calculating the priority β by the priority calculating means 110 (that is, the process P in step S30 in FIG. 12). As shown in the figure, when calculating the priority β using the image Di, the priority calculation unit 110 first selects the priority calculation patch Fy in the image Di, and the center point thereof is on the edge of the restoration area in the image Di. It arrange | positions so that it may be located in a pixel (S32). If the number of pixels in the known area in the priority calculation patch Fy is equal to or less than the threshold value K1, the priority β of the pixel at the center point of the priority calculation patch Fy is set to 0 (S34: No, S55). On the other hand, if the number of pixels in the known area in the priority calculation patch Fy is larger than the threshold value K1, the embedding direction is determined as one of the directions shown in FIG. 5 (S36). Then, for the image Di (i> 1) whose resolution is lower than that of the original image, the priority calculation means 110 obtains the variance value σ of the pixel values of the known area in the priority calculation patch Fy (S40: Yes, S42) If the variance value σ is equal to or smaller than the threshold value K2, the priority β of the pixel at the center point of the priority calculation patch Fy is set to 0 (S44: No, S55), while the variance value σ is greater than the threshold value K2. If it is larger, the boundary line is detected by dividing the area in the known area in the priority calculation patch Fy (S44: Yes, S46). If the detected boundary line is not a boundary line extending from the contour line on the known area side of the priority calculation patch Fy to the edge of the restoration area in the priority calculation patch Fy, the priority calculation unit 110 calculates the priority. The priority β of the pixel at the center point of the calculation patch Fy is set to 0 (S48: No, S55). On the other hand, the detected boundary line is prioritized from the contour line on the known area side of the priority calculation patch Fy. If the boundary line extends to the edge of the repair area in the calculation patch Fy, the priority β of the pixel is set to 0 based on the mode of the boundary line and the embedding direction of the pixel at the center point of the priority calculation patch Fy. The above value is set (S48: Yes, S50). The priority calculation means 110 is configured to arrange the priority calculation patch Fy and the pixel at the position where the priority calculation patch Fy is arranged (center of the priority calculation patch Fy) in each pixel on the edge of the restoration area in the image Di. The priority β is obtained for the pixel of the point (S60: No, S82, S34˜), and the obtained priority β of each pixel and the embedding direction are output to the embedding source image acquisition unit 115 (S60: Yes). , S100).

On the other hand, when the resolution is calculated priorities β using the same image D ₁ and the original image (S40: No), the priority calculation unit 110 performs the detection of the edge at the center point of the priority calculation patch Fy When the edge is not detected at the center point, the priority β of the pixel located at the center point is set to 1, while when the edge is detected, the higher the edge strength, the higher the priority. The priority β of the pixel at the center point of the priority calculation patch Fy is set to a value larger than 1, and the priority β set for each pixel is embedded along with its embedding direction. It outputs to the acquisition means 115 (S72-S110).

  Although not shown in FIG. 13, when the priority calculation unit 110 determines the embedding direction of the pixel at the center point of the priority calculation patch Fy, the embedding direction is 2 as shown in FIG. In the case where there are two possibilities, the priority is obtained for each of these two directions as embedding direction candidates, the embedding direction candidate having a higher priority is determined as the embedding direction, and the priority calculated for the direction is determined as the embedding direction candidate. The pixel priority.

  As described above, according to the image processing apparatus of the present embodiment, attention is paid to the fact that most of the dust attached by aging storage on print media such as photographic prints and paper, photographic films, etc. is yarn waste or mold. In accordance with such dust characteristics, the first morphological filter for detecting a microscopic region having a filamentous shape and a strong contrast with the background from an image obtained by reading the medium has a strong contrast with the background, and Since the defect area is detected and repaired by using the second morphological filter for detecting the point-like area that exists in the vicinity, the image can be restored without imposing a burden on the operator. Yes.

  In addition, by repairing a region obtained by expanding the detected defect region as a region to be repaired, it is possible to prevent a repair leak at the edge of the defect region.

  The preferred embodiment of the present invention has been described above. However, the image processing method and apparatus of the present invention and the program therefor are not limited to the above-described embodiment, and an embedding method is used without departing from the gist of the present invention. Various changes, such as the size of each patch and the number of multi-resolution pixels, can be added.

  For example, the image processing apparatus according to the present embodiment repairs the repair area automatically detected by the detection unit 10a in the repair area acquisition unit 10, but the repair area is acquired by the operator in the repair area acquisition unit 10. A defect area designating unit for designating may be provided so that a defect area (for example, large mold or dust) that could not be detected by the detection unit 10a can be acquired. In this way, the defect area can be acquired more reliably, and most of the defect area has already been detected by the detection means 10a before the designation by the operator, so the burden on the operator is small.

  For example, the image processing apparatus according to the present embodiment arranges a priority calculation patch having a size larger than that of the embedding patch so that the center point thereof is located at a pixel on the edge of the repair area in the image. In the known area in the calculation patch, a boundary line that extends from the contour line of the priority calculation patch to the edge of the restoration area in the priority calculation patch is detected, and the priority calculation for detecting the boundary line is performed. The embedding priority of the pixel at the center point of the patch is set higher than the embedding priority of the pixel at the center point of the priority calculation patch where the boundary line is detected. When embedding, the embedding patches are arranged on the pixels on the edge of the restoration area in order from the pixel with the highest embedding priority, and there is a boundary line dividing the area in the image. Although a good repair effect can be obtained, for example, as described in Patent Document 1, a patch of a predetermined size (for example, 3 pixels × 3 pixels) is arranged on the edge of the repair region, The average value of the pixel values of the pixels in the known area in the arranged patch may be obtained, and the repair may be performed by using the new pixel value of the pixel in the repair area in the same patch. Further, instead of the average value, a new pixel value of the pixel in the restoration area in the same patch may be obtained by performing interpolation processing on the pixel value of the pixel in the known area in the patch. In this method, when the repair area is large, the closer to the center of the repair area, the more the frequency component disappears. Therefore, the blur of the embedded repair area is noticeable, and the processed image has a sense of incongruity, but the repair area is small. When it can be done easily and quickly.

  The image processing apparatus according to the present embodiment performs processing such as calculation of embedding priority and acquisition of an embedding source image at a plurality of resolutions in order to achieve a good restoration effect and a high processing speed. Since the size of dust such as mold is small, it may be performed only on the original resolution image without dividing the image into a plurality of resolutions. Of course, when the size of the defect area designated by the operator is large, it is preferable to perform the processing by dividing the large defect area into a plurality of resolutions.

1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention. 1 is a block diagram showing a configuration of a restoration unit 100 in the image processing apparatus according to the embodiment shown in FIG. Diagram showing an example of a multi-resolution image The figure for demonstrating arrangement | positioning of the patch Fy for priority calculation Diagram showing embedding direction Diagram for explaining how to determine the embedding direction The figure which shows the example with few pixels of the known area | region in the patch Fy for priority calculation Diagram for explaining boundary detection The figure for demonstrating the deformation | transformation process of a frame The figure for explaining the setting of the priority when the boundary line is detected The figure for demonstrating the process of the embedding original image acquisition means 115 The flowchart which shows the process of the whole image processing apparatus shown in FIG. A flowchart showing processing for calculating priority by the priority calculation means 110

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image input means 10 Restoration area | region acquisition means 100 Restoration means 105 Multi-resolution image creation means 110 Priority calculation means 115 Embedding source image acquisition means 120 Embedding means 130 Edge modification means β Embedding priority

Claims (7)

In a digital image obtained by photoelectrically reading an image formed on a medium, in an image processing method for repairing a defective area caused by a minute substance that is not originally included in the image attached on the medium,
A first morphological filter for detecting a microscopic region that is filamentous and has a strong contrast with the background; and / or a second filter for detecting a point-like region that has a strong contrast with the background and that is close to the background. Detecting the defect area using a morphological filter,
An image processing method, wherein the detected defect area is set as a repair target area, and the defect area is repaired by replacing an image of the repair target area with an image other than the repair target area.   The image processing method according to claim 1, wherein an area obtained by expanding the detected defect area is set as the area to be repaired. In an image processing apparatus for repairing a defective area caused by a minute substance that is not originally included in the image attached to the medium with respect to a digital image obtained by photoelectrically converting an image formed on the medium,
A first morphological filter for detecting a microscopic region that is filamentous and has a strong contrast with the background; and / or a second filter for detecting a point-like region that has a strong contrast with the background and that is close to the background. A defect area detecting means for detecting the defect area using a morphological filter;
And a repair execution means for repairing the defective area by replacing the detected defective area as a repair target area and replacing an image of the repair target area with an image other than the repair target area. Image processing device. Having expansion means for expanding the defect area detected by the defect area detection means;
4. The image processing apparatus according to claim 3, wherein the repair execution unit sets the defect area expanded by the expansion unit as the repair target area. A defect area specifying means for allowing a user to specify a defect area in an area other than the defect area detected by the defect area detecting means;
5. The image processing apparatus according to claim 3, wherein the repair execution unit also sets the defect area designated by the defect area designation unit as the repair target area. For a digital image obtained by reading an image formed on a medium by photoelectric conversion, the computer performs image processing to repair a defective area caused by a minute substance that is not originally present on the medium. A program to
The image processing detects a first morphological filter for detecting a minute region having a string shape and a strong contrast with the background, and / or a dot-like region having a strong contrast with the background and existing in the vicinity. A defect area detection process for detecting the defect area using a second morphological filter for
A program comprising: a repair execution process for repairing the defective area by replacing the detected defective area with a repair target area and replacing an image of the repair target area with an image other than the repair target area. Causing the computer to further perform an expansion process for expanding the defective area detected by the defective area detection process;
The program according to claim 6, wherein the repair execution process sets the defective area expanded by the expansion process as the repair target area.

Images