JP2011035567A - Image processing apparatus and method thereof, and computer readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To recover damaged images at high speed for high quality. <P>SOLUTION: An image processing apparatus 1 performs image processing to recover damaged areas for input images including damaged areas, such as punch holes or stapler marks. The image processing apparatus 1 includes: a structuring means 21 for structuring pixel values of pixels in areas not including damaged areas; and a recovery means 22 for estimating pixel values of pixels included in damaged areas based on the structured pixel values and recovering the damaged areas. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, a method, and a computer-readable recording medium.

  When copying with a copying machine, there are many cases where an image is missing, such as a punch hole or a trace of a stapler. When trying to repair such a defective area, if the background color is known, it can be repaired neatly by simply painting with that color. In particular, since the background of the document image is likely to be overwhelmingly white, in most cases, it can be neatly restored by simply painting it with white.

  On the other hand, when the background color is unknown, if a process of painting with a specific color is performed, the defective area may be noticeable. In order to cope with such a scene, a method using a diffusion equation has been used in the past (for example, Non-Patent Document 1). This method is a method of smoothly interpolating a defective region by gradually propagating the luminance value of a pixel from the boundary of the defective region to the inside using a diffusion equation. Using this method, the continuity of brightness values is maintained, so that good results can be obtained for thin areas such as scratches on photographs, and the global structure and large edges can be obtained. There is a merit that it is strong against the gradient. On the other hand, there is a problem that when a large area is repaired, a fine texture cannot be expressed and an unclear image is generated.

  FIG. 11 is an image obtained by repairing a defective area by this method. The left is the original image, and the black circle at the center is the missing area. On the other hand, the image on the right is a restored image. As is apparent from the figure, there is a problem that when the image is repaired using this method, the images are smoothly connected but are blurred. This tendency is particularly strong when the luminance change of the image is steep. That is, it can be said that good defect repair cannot be performed by a technique such as that described in Non-Patent Document 1 in an area or an edge area where the influence of texture is large.

  Due to such problems, Non-Patent Document 2 proposes a method for estimating a texture of a defective region from a peripheral image. The outline of the algorithm is shown below. First, as shown in FIG. 12, a window having a certain size (rectangular region in FIG. 12) is set, and the moving range of the center of the window including the missing region is set to Ω ′. Subsequently, the pattern similarity with a window having a center in an area Φ (hereinafter referred to as a data area) other than Ω ′ in the image is calculated, and the energy function defined as a weighted sum of the pattern similarity is minimized. Search for the position of such a pattern.

  Here, SSD (Sum of Squared Differences) is used as the pattern similarity. Note that I (x) means a pixel value at x.

  The energy function can be described as a weighted sum of pattern similarity SSD as follows.

  Here, ωx is a weight for reflecting the reliability of the SSD, and it is possible to change the weight according to the distance from the defective pixel to be complemented. I will give it. In order to minimize the energy function defined above, a pattern similar to each pixel x in the defective area is searched for in the data area Φ, and the pixel value of the pixel x is determined by the pixel value of the center pixel of the pattern. Replace When replacement is completed for all the pixels in the defective region, search and replacement processing is performed for each pixel in the defective region from the beginning, and optimization is repeated until the energy becomes sufficiently small. By performing the above processing, a fine texture can be expressed, and a high-quality repair image can be obtained. For example, considering that repair processing is performed using a part of grass as a defective area, it is easily predicted that the pattern to fill the defective area exists in the peripheral image. For this reason, it is possible to express a fine texture of a defective area by this method.

  However, while a high-quality repaired image can be obtained, this method has a problem that it requires an extremely large amount of calculation time. For example, if the image is 200 × 200 pixels, the number of pixels in the defective area is 1000 pixels, and the window size is 3 × 3 = 9 pixels, the number of pixels in the data area is 39000 pixels. The SSD calculation requires 9 subtractions and multiplications, and the SSD calculation is performed 39000 times to finally fill one pixel. Further, the energy is calculated by performing the processing on 1000 pixels. However, in order for the energy to converge, the above calculation must be performed a plurality of times.

  As described above, in the conventional method, in order to complement one pixel, it is necessary to pattern-match all the data areas Φ, and there is a problem that the processing takes an extremely long time.

  In addition, as patent documents related to the present invention, there are Patent Documents 1 and 2, and Patent Document 1 describes switching between a plurality of complementing methods according to the content of an image (for example, paragraph 0037). . On the other hand, Patent Document 2 describes that when image data is restored, restoration is performed using a diffusion equation, but restoration processing is performed on a sufficiently small defect region by performing stepwise reduction processing. .

  The present invention has been made in view of the above, and provides an image processing apparatus, a method, and a computer-readable recording medium capable of repairing a defective image at high speed and high quality.

  To achieve the above object, according to a first aspect of the present invention, there is provided an image processing apparatus that performs image processing for repairing the defect area on an input image including the defect area, and includes the defect area. Structuring means for structuring the pixel values of the pixels in the non-existing region, and estimating the pixel values of the pixels included in the defective region based on the structured pixel values, and performing image processing for repairing the defective region An image processing apparatus comprising: a restoration unit.

  In order to achieve the above object, according to a second aspect of the present invention, there is provided an image processing method for performing image processing for repairing the defect area on an input image including the defect area, wherein the defect area A structuring step for structuring pixel values of pixels in a region not including the image, and image processing for estimating the pixel values of the pixels included in the defect region based on the structured pixel values and repairing the defect region And an image processing method.

  In order to achieve the above object, according to a third aspect of the present invention, there is provided a computer-readable recording medium in which a program for causing a computer to execute the image processing method is recorded.

  According to the present invention, it is possible to provide an image processing apparatus, a method, and a computer-readable recording medium that can repair a defective image at high speed and high quality.

It is a figure which shows an example of the hardware constitutions of the image processing apparatus 1 by this invention. It is a functional block diagram which shows the main functions of the image processing apparatus 1 by this invention. It is a figure which shows an example of the user interface displayed on the image output part 12 by the application which concerns on this embodiment. It is a flowchart of the use case of the application which concerns on this embodiment. It is a flowchart which shows operation | movement of the repair process which concerns on this embodiment. It is a figure for demonstrating the structuring process in the repair process which concerns on this embodiment. It is a figure for demonstrating the nearest neighbor search process in the repair process which concerns on this embodiment. It is a flowchart which shows the detailed flow of the repair process which concerns on this embodiment. It is a figure which illustrates the object structured by the structuring process in the repair process concerning this embodiment. It is a figure which shows the binary tree structure of the structuring process in the repair process which concerns on this embodiment. It is a figure which shows the example of the image restoration using the diffusion equation of a prior art. It is a figure which shows the outline | summary of the method of estimating the texture of a defect area | region from the peripheral image of a prior art.

  Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. Hereinafter, embodiments in which the present invention is applied to an image processing apparatus will be described.

<Configuration>
(hardware)
FIG. 1 is a diagram illustrating an example of a hardware configuration of an image processing apparatus 1 according to the present invention. The image processing apparatus 1 according to the present embodiment is an image processing apparatus that performs image processing for repairing a defect area on an input image including the defect area. Specifically, the image forming apparatus 1 includes an image input unit 11, an image output unit 12, a CPU 13, an I / O circuit 14, a ROM 15, a RAM 16, and an HDD 17.

  The image input unit 11 performs image input, and corresponds to, for example, an image reading unit in an image device such as a scanner, a copier, or a multifunction peripheral. Further, image data of a document read externally can be input via a medium (recording medium) or a network.

  The CPU 13, the I / O circuit 14, the ROM 15, the RAM 16, and the HDD 17 are configured to be communicable with each other via a bus line. The ROM 15 stores programs executed when the image processing apparatus 1 is started up and various data. The RAM 16 is a main storage device that temporarily stores various programs and data read from the ROM 15 and the HDD 17. Further, the CPU 13 controls the overall processing of the image processing apparatus including the arithmetic processing, and executes a program temporarily stored in the RAM 16. The I / O circuit 14 manages input / output with peripheral devices including the image input unit 11 and the image output unit 12. The HDD 17 stores image data processed under the control of the CPU 13 and image data captured from the outside.

  The image output unit 12 includes an output device such as a printer or a display and its control means. The image output unit 12 prints out image data having a defective area repaired by the processing by the image processing device 1 or displays it on a display screen. To do. It can also be output to the outside via a network.

(software)
The HDD 17 in FIG. 1 uses a hardware resource of the image processing apparatus 1 to perform predetermined information processing, and implements the following functions and operations (hereinafter, “application according to the present embodiment”). Is stored). The application according to the present embodiment may be provided to the image processing apparatus 1 by a computer-readable recording medium such as a magnetic or optical disk.

(function)
FIG. 2 is a functional block diagram showing main functions of the image processing apparatus 1. The image processing apparatus 1 is an image processing apparatus that performs image processing for repairing a defect area on an input image including the defect area, and has the following configuration. That is, the image processing apparatus 1 includes a structuring unit 21 and a restoration unit 22 as main functions.

  The structuring unit 21 structures the input image data including the missing area input by the image input unit 11 in advance before performing the repair process. The restoration means 22 performs image processing for restoring the structured input image data. Hereinafter, the process executed by the structuring unit 21 and the process executed by the repair unit 22 may be collectively referred to as “repair process”.

(Use Case)
FIG. 3 is a diagram illustrating an example of a user interface displayed on the image output unit 12 by the application according to the present embodiment. As shown in FIG. 3, the user interface of this application includes an “open button”, a “save button”, an “execute button”, and an “image display unit”.

  Next, the operation of the user and the operation of the application when using this application will be described with reference to the flowchart of FIG. In FIG. 4, the left shows the user's operation, and the right shows the application operation. The user starts the application (step 001). Subsequently, the user clicks the open button in FIG. 3 with the mouse. Then, a file selection dialog pops up, and the user selects a desired image file using the dialog (step 002). Then, the application displays the image designated by the user on the image display unit in FIG. 3 (step 003). In the image displayed on the image display unit, the user drags an area to be removed, that is, an area portion that will be a missing area in future processing (step 004). In the dragged area, a red line is drawn on the image to notify the user that the red line area is a missing area. Thereafter, when the user presses the start button in FIG. 3 (step 005), the application estimates and repairs the missing area from the other areas of the image (step 006). The restored image is replaced with the image displayed on the image display unit.

By performing the processing as described above, the user can delete unnecessary subjects in the image without a sense of incongruity. By doing this, there is an advantage that, for example, if a person who appears in an image to be uploaded to a web page is removed, the privacy of the person can be protected.
Hereinafter, the repair process indicated by step 006 will be described in detail.

<Operation>
(Outline of the repair process)
FIG. 5 is a flowchart showing the operation of the repair process indicated by step 006 in FIG. First, the color space of the image is converted from the RGB space to the L * a * b * space (step 201). Subsequently, the plane is decomposed into three planes of L * image, a * image, and b * image (step 202).

  The L * image is directly restored (step 203). On the other hand, the a * image and the b * image are reduced by half in the vertical and horizontal directions (step 204), and the restoration process is started (step 205). It is assumed that the same processing is performed for the repair processing described here. Although details will be described later, the restoration process is a process with a large amount of calculation. Subsequently, the a * image and the b * image are respectively enlarged, and only the region that is missing compared with the image before reduction is combined with the repair image and output. In each of the restored L * image a * image and b * image, the planes are integrated (step 207), converted into the original RGB color space, and output (step 208).

  The reason why the L * image and the a * image b * image are processed separately is to reduce the processing load. The L * a * b * color space includes an L * signal that is a luminance component and an a * b * component that is a chromaticity component. In general, the human eye is sensitive to changes in luminance components, but is insensitive to changes in chromaticity components. Therefore, even if the chromaticity component is not restored with such a high degree of accuracy, a high-quality image can be obtained by human eyes if the luminance component is restored accurately. Therefore, in the present embodiment, the chromaticity component is restored after being converted to a low resolution so that high-speed processing can be performed.

  In the present embodiment, the luminance component and the chromaticity component are the same processing, but the high speed is realized by the configuration of different resolutions, but various other methods are conceivable.

  For example, the luminance component may be restored by the method described in Non-Patent Document 2, and the chromaticity component may be restored by the method using the diffusion equation described in the related art. As described in the prior art, a technique using a diffusion equation is difficult to express a fine pattern or the like. However, a defective area can be repaired smoothly, and the processing is much faster than the technique of Non-Patent Document 2. Therefore, even if the luminance component and the chromaticity component are processed with the same resolution, a sufficiently high speed can be expected. Of course, the chromaticity component may be processed at a low resolution. Similarly, the same effect can be obtained even if the missing region is repaired by linear interpolation for the chromaticity component without using the diffusion equation.

  In addition, L * a * b * is used as the color space indicating the luminance and chromaticity components this time, but it may be YCbCr or the like.

(Details of the repair process)
The restoration process is performed by using a technique obtained by improving the technique of Non-Patent Document 2 described in the prior art. Three major improvements have been made. This improvement will be described.

  As described above, the method of Non-Patent Document 2 must perform a search by pattern matching for all the pixels included in the defect area. Furthermore, since the estimation of the defect region is not required once and is repeated until the energy becomes sufficiently small, a very large calculation amount is required. In contrast, in this method, the image data is structured in advance to increase the speed. This is the first improvement. The reason why the processing can be speeded up by structuring will be described below.

  For example, as shown in FIG. 6A, it is assumed that data indicated by □ is distributed in a certain space. Suppose that ○ that is input data is input here, I want to find the □ that is closest to the input data.

  For example, the distance may be calculated and compared for all rounds of ○ and all □, but it is very inefficient. So, for example, if you divide the space in advance as shown in (b) from the distribution of data and find out in which space the input data exists first, just calculate the distance between the two squares. Get better. As described above, since the distance calculation can be made efficient by structuring in advance, that is, by dividing the space from the data distribution, the speed can be increased.

  Although speeding up by such structuring is a general means, it is extremely compatible with the method of Non-Patent Document 2. For example, considering that structuring is performed by pattern matching processing used in normal image recognition or the like, the search processing itself can be speeded up by structuring. On the other hand, since a relatively large amount of calculation is required for the prior structuring, the speed is often not increased in a comprehensive sense including the structuring process and the search process. On the other hand, as described above, in the case of processing that repeats pattern matching for the same image many times as in the repair processing of Non-Patent Document 2, structured data is repeatedly used. Therefore, the processing speed can be increased.

Next, the second improvement point will be described. In the description of the example using FIG. 6, for simplicity of explanation, it is determined that the space in which the input data exists is determined, and the nearest neighbor only needs to be performed in the space. On the other hand, as shown in the following figure, the nearest neighbor data may exist in a different space from the determined space. Therefore, the closest data in the first determined space is searched, the distance l ₀ between the data and the input data is calculated, and included in the space (the upper left space in FIG. 7) that may be closer to the l ₀ . If the process of performing the nearest neighbor search is not performed on the data to be obtained, the exact nearest neighbor search cannot be performed.

  Therefore, the space division performed in advance is a very important process. That is, if the data in the space determined first is sufficiently close, the number of times of performing the nearest neighbor determination across the space is reduced. On the other hand, it is difficult to perform optimal space division in advance for all input data.

Therefore, Non-Patent Document 3 proposes a technique called Approximate Nearest Neighbor (ANN). ANN is characterized in that l smaller than l ₀ is used instead of the nearest neighbor search for a space that may be closer than l0. In order to obtain the exact nearest neighbor, the processing as described above is necessary. However, by allowing the nearest neighbor search to include an error, data close to the nearest neighbor can be obtained at high speed. The relationship between l and l ₀ can be expressed by the following mathematical formula. As ε increases, the error increases but the processing speed increases.

  The use of ANN is the second improvement, but here, it will be explained that the accuracy is low but the high speed is very suitable for this method. This is because the nearest neighbor search performed by this method is performed in order to determine a pixel for temporarily filling a missing area, and is not necessarily a strict nearest neighbor. In addition, it is often performed that ε is originally used in the range of about 0 to 1. However, in the case of an application that does not require a relatively high accuracy like the present method, even if ε = 10 or ε = 100, it is sufficiently practical. Can withstand

Further, it is effective to change ε every time estimation of the defect region is completed, that is, whenever energy is calculated. This is the third improvement. Although it depends on whether or not the missing area is initially filled, generally, in the initial stage of optimization, the energy, that is, the sum of the distance between the estimated pixel and the nearest pixel is large. In other words, even if the closest pattern is searched, it is not so close. As described above, if l ₀ calculated first is sufficiently small, the process for searching with high accuracy is small, but the fact that there is no very close pattern is that l ₀ is large. That is, the processing amount is increased with a considerable probability. On the other hand, after the end of the iteration, that is, after the energy has become sufficiently small, there is a high possibility that the processing will not increase even if the search is performed with high accuracy. Will not fall so much. Therefore, in this method, ε is set to 100 at the initial stage of optimization, and ε is changed according to the following formula when the defect region is estimated for the Nth time. However, the minimum value of ε was 5.

In the present embodiment, ε is changed with respect to N, but there is no problem even if it is changed according to energy. For example, the energy may be E, and ε may be changed by the following equation with respect to the initial energy E ₀ . However, the minimum value of ε is also set to 5. As described above, as the energy decreases, l ₀ is likely to be small. Therefore, if ε is changed according to the energy, the accuracy and speed can be controlled more finely.

  Based on the above description of the repair process, FIG. 8 shows a detailed flow of the repair process. In FIG. 8, first, image data included in an image is structured (step 301). The object to be structured here is a 9-dimensional vector created by arranging pixel values of 3 × 3 pixels around the pixel of interest (FIG. 9). In FIG. 9, a gray pixel (v5) is a target pixel, and its peripheral pixel values v1 to v9 are 9-dimensional vectors. Here, if the image is (n + 1) × (m + 1) pixels, n × m 9-dimensional vectors are to be structured. However, if even one of the nine pixels includes a defective region, it is not included in the structuring target, so the actual number of vectors is smaller than n × m.

  Further, although a structuring method will be described later, as shown in FIG. 10, a binary tree structure in which one vector is associated with one node and two nodes are hung from each node. To be structured. Specifically, a technique called kd-tree is used. kd-tree is a binary tree means for classifying a vector into two categories using the value of one element (one of nine elements in this example) at each node. It is possible to trace from the root node to one terminal leaf node with a very small amount of calculation.

Subsequently, ε is calculated (step 302). The calculation method of ε may be determined using Equation 3 (Equation 3) or Equation 4 (Equation 4). Thereafter, one pixel is selected from the missing area, and a vector is created by arranging elements including peripheral pixels in the same manner as described with reference to FIG. 9 (step 303). Thereafter, the nearest neighbor search is performed (step 304). Here, the ANN described above is used. That is, the terminal vector is traced from the root node to obtain a terminal leaf node. The distance l ₀ between the leaf node and the input vector is calculated. l is calculated from l ₀ . The leaf nodes are traced in reverse to enumerate leaf nodes that may be closer than l. Find the closest node to the input vector from the enumerated leaf nodes. The procedure will be followed.

  After performing the nearest neighbor search for all the pixels in the defective area in this way, energy is calculated (step 305). This energy is the sum of the SSDs described in the prior art, that is, the sum of the distance between the input vector obtained from the pixel in the defective region and its nearest node.

  The process is looped until the energy calculated in Step 305 is sufficiently small. Although there are various convergence determination methods, this time, the process is terminated when the difference between the energy calculated in the previous loop and the energy calculated this time is smaller than a predetermined magnitude.

  According to the configuration of the present embodiment described above, it is possible to perform high-speed and high-quality image restoration by providing a structuring unit that pre-structures pixel values of a region that does not include a defective region. It is. Further, by performing the structuring in advance, the speed can be extremely effectively increased.

  In addition, since the data is structured in a binary tree and each node is divided based on the value of one pixel value, a pattern close to the input pixel value pattern is searched from the structured pixel value pattern. It becomes possible.

  Further, by performing the nearest neighbor search approximately, it is possible to search for the nearest neighbor pattern at a higher speed.

In addition, by reducing the value of l relative to l ₀ for each optimization step, it is possible to search for the nearest neighbor at a high speed while suppressing deterioration in precision of the nearest neighbor search.

Further, by changing the value of l with respect to l ₀ according to the magnitude of energy, it becomes possible to search for the nearest neighbor at a high speed while suppressing deterioration in precision of the nearest neighbor search.

  In addition, by dividing the luminance component and the chromaticity component separately and performing different restoration processing, it is possible to finely control the processing speed and the quality of the image after restoration.

  Also, by restoring the chromaticity image after reducing it, it is possible to perform high-speed and high-quality restoration by performing high-quality restoration of heavy processing only for luminance images that are sensitive to human eyes. it can.

  In addition, by restoring the chromaticity component using the diffusion equation, only high-quality restoration is performed at high speed by performing high-quality restoration only for luminance images that are sensitive to the human eye, but at high speed. Can do.

  Also, the chromaticity component is restored using linear interpolation, and only a luminance image that is sensitive to the human eye is repaired at a high speed and with high quality by performing high-quality restoration with heavy processing. be able to.

  Although one mode for carrying out the present invention has been described above, the present invention can also be implemented with an image-dedicated IC or the like. For this reason, if it is installed in a Multi Function Printer (MFP) or the like, the scanned document is previewed on the operation liquid crystal of the MFP, and the user designates an area to be repaired such as a punch hole, and the area is repaired and output. Is also possible.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 11 Image input part 12 Image output part 13 CPU
14 I / O circuit 15 ROM
16 RAM
17 HDD
21 Structuring means 22 Repairing means

JP 2007-286734 A JP 2006-332785 A

M. Bertalmio, G. Sapiro, V. Caseelles and C. Ballester: Image Inpainting, In Proc, SIGGRAPH2000, pp.417-424, 2000. A. Criminisi, P. Perez and K. Toyama: Region Filling and Object Removal by Exemplar-Based Image Inpainting, IEEE Trance, on Image Processing, Vol. 13, No. 9, 2004. S. Arya, D. M. Mount, N. Netanyahu, R. Silverman, and A. Y. Wu.An optimal algorithm for approximate nearest neighbor searching in fixed dimensions.In 5th ACM-SIAM Symposium Discrete Algorithms, pp. 573582, 1994.

Claims (12)

An image processing apparatus that performs image processing for repairing the defect area with respect to an input image including the defect area,
Structuring means for structuring pixel values of pixels in a region not including the defective region;
A restoration unit that estimates a pixel value of a pixel included in the defect region based on the structured pixel value and performs image processing to repair the defect region;
An image processing apparatus comprising: The repair means includes
Nearest neighbor search means for inputting a peripheral pixel value pattern centered on a pixel in the defective area, and searching for and outputting a nearest pixel value pattern from an area not including the defective area of the input image;
Pixel value replacement means for replacing the pixel value of the defective region with the central pixel value of the pixel value pattern obtained by the search;
Performing the processing by the nearest neighbor searching means and the processing by the pixel value replacing means for all the missing regions as one step, and energy for each one step based on the difference between the input pattern and the output pattern in the nearest neighbor searching means. Energy calculating means for calculating
Optimization means for determining whether the energy has converged for each one step;
The image processing apparatus according to claim 1, further comprising: The nearest neighbor search means includes:
A temporary nearest neighbor search means for finding a temporary nearest neighbor by tracing a node of a tree structure;
Distance calculating means for calculating a distance l ₀ with the temporary nearest neighbor;
Search range determining means for determining the search range l based on the l ₀ ,
The image processing apparatus according to claim 2, wherein the nearest neighbor is selected from patterns that may be closer to the l. The image processing apparatus according to claim 3, wherein the search range determination unit decreases the value of l relative to the l ₀ for each one step. The image processing apparatus according to claim 3, wherein the search range determination unit changes the value of l with respect to the l ₀ according to the magnitude of the energy.   6. The structuring unit structuring a pixel value of a pixel in a region not including the missing region in a binary tree structure, and dividing each node based on one pixel value. The image processing apparatus according to claim 1. Comprising color conversion means for converting the input image into a luminance image and a chromaticity image;
The image according to any one of claims 1 to 6, wherein the repairing unit repairs the defective area separately at different resolutions for the converted luminance image and chromaticity image. Processing equipment.   The image according to claim 7, wherein the repairing unit performs a repairing process on the chromaticity image after performing a reduction process, and expands after the repairing process to repair the missing area. Processing equipment.   The image processing apparatus according to claim 7, wherein the repairing unit repairs the defective area using a diffusion equation for the chromaticity image.   10. The image processing apparatus according to claim 7, wherein the repair unit repairs the missing area using linear interpolation for the chromaticity image. 11. An image processing method for performing image processing for repairing the defect area on an input image including the defect area,
A structuring step for structuring pixel values of pixels in a region not including the defective region;
A repairing step of estimating a pixel value of a pixel included in the defective region based on the structured pixel value and performing image processing to repair the defective region;
An image processing method comprising:   A computer-readable recording medium having recorded thereon a program for causing a computer to execute the image processing method according to claim 11.

Images