JP2018060324A - Image processing system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent a certain pattern included in a background image having a brightness difference similar to that of a breakage part from being misdetected as the breakage part.SOLUTION: A background evaluation image generation section 110 generates a background evaluation image G110 representing an evaluation of a pixel to be a background of each pixel by conducting data smoothing to an original image G100. A flat evaluation image generation section 120 generates a flat evaluation image 120 representing an evaluation of flatness of the original image G100 by conducting edge detection processing and data smoothing to the original image G100. A breakage part detection image [1] generation section 200 generate a breakage part detection image [1] G200 by discriminating a breakage part on the basis of a differential value between the original image G100 and the background evaluation image G110 and a pixel value of the flat evaluation image G120.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing apparatus and an image processing method.

Paintings drawn with oil paints have been damaged by cracks, cracks, cracks, etc. on the surface due to changes over time, and the appearance of the paintings has been impaired. For this reason, there has been a demand for viewing an image before the damaged portion is generated.
Therefore, conventionally, an image processing technique is known in which a damaged portion is detected from an image of a picture in which the damaged portion is generated, and the detected damaged portion is repaired.
In Patent Document 1, for the purpose of removing abnormal parts such as dust and dirt from an image acquired via a scanner, an abnormal part is detected from a background image using an image processing technique, and the abnormal part is repaired. A method is disclosed.

However, in the conventional image processing technology, a pixel in which the change value between adjacent pixels is extremely larger than the average value of the surrounding pixel set is determined to be an abnormal portion.
Therefore, for example, in an image of a person painting in which a damaged part has occurred, there is a problem in that a specific pattern formed by the lace material is erroneously detected as a damaged part at a place where a lace material is applied against the background of the person's clothes. there were.
Patent Document 1 discloses that noise such as a damaged portion is removed using image processing. However, the problem of erroneously detecting a specific pattern included in the background image having the same brightness difference as the damaged portion as a damaged portion cannot be solved.
The present invention has been made in view of the above, and an object thereof is to prevent erroneous detection of a specific pattern included in a background image having a brightness difference similar to that of a damaged portion as a damaged portion. .

  In order to solve the above-mentioned problem, the invention according to claim 1 is an image processing device that detects a damaged portion existing in an original image, and performs a smoothing process on the original image to provide a background pixel for each pixel. A background evaluation image generating means for generating a background evaluation image representing an evaluation of the image, and a flat evaluation image representing an evaluation relating to flatness of the original image by performing an edge detection process and a smoothing process on the original image Damaged part detection image that identifies a damaged part and generates a damaged part detection image based on an evaluation image generation unit, a difference value between the original image and the background evaluation image, and a pixel value of the flat evaluation image Generating means.

  According to the present invention, it is possible to prevent a specific pattern included in the background image having the same brightness difference as the damaged portion from being erroneously detected as a damaged portion.

1 is a diagram illustrating an information processing apparatus and an image forming apparatus according to an embodiment of the present invention. It is a figure which shows the hardware constitutions of the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the function structure of the information processing apparatus which concerns on one Embodiment of this invention. 1 is a diagram illustrating a functional configuration of an image forming apparatus according to an embodiment of the present invention. It is a functional block diagram which shows the function structure of the image process part which concerns on one Embodiment of this invention. It is a flowchart for demonstrating the background evaluation image generation process by the background evaluation image generation part shown in FIG. (A) is a figure which shows the structure of a median filter, (b) is a figure which shows the structure of a Gaussian filter. It is a flowchart for demonstrating the flat evaluation image generation process by the flat evaluation image generation part shown in FIG. It is a flowchart for demonstrating the damage part detection image [1] production | generation process by the damage part detection image [1] production | generation part shown in FIG. 6 is a flowchart for explaining a subroutine of black broken portion detection processing by a broken portion detection image [1] generation unit shown in FIG. 5. 6 is a flowchart for explaining a subroutine of white broken part detection processing by a broken part detection image [1] generation unit shown in FIG. 5; It is a flowchart for demonstrating the damage part detection image [2] production | generation process by the damage part detection image [2] production | generation part shown in FIG. It is a graph for demonstrating the expansion process by the damaged part detection image [2] production | generation part shown in FIG. 6 is a flowchart for explaining a damaged portion repair image [1] generation process by a damaged portion repair image [1] generation portion shown in FIG. 5. (A) is a figure which shows the example of an image showing each pixel of damaged part detection image [2], (b) is a figure which shows a background evaluation pixel list. 6 is a flowchart for explaining a damaged portion repair image [2] generation process by a damaged portion repair image [2] generation portion shown in FIG. 5. It is a figure which shows an original image. It is a figure which shows the background evaluation image produced | generated from the original image. It is a figure which shows the flat evaluation image produced | generated from the original image. It is a figure which shows damaged part detection image [1] produced | generated from the original image and the flat evaluation image. It is a figure which shows damaged part detection image [2] produced | generated from damaged part detection image [1]. It is a figure which shows damaged part repair image [1] produced | generated from the original image and damaged part detection image [2]. It is a figure which shows damaged part repair image [2] produced | generated from damaged part repair image [1].

Hereinafter, the present invention will be described in detail with reference to embodiments shown in the drawings.
The present invention has the following configuration in order to prevent a specific pattern included in a background image having the same brightness difference as a damaged portion from being erroneously detected as a damaged portion.
In other words, the image processing apparatus of the present invention is an image processing apparatus that detects a damaged portion existing in an original image, and performs background processing that represents an evaluation of pixels that are the background of each pixel by performing a smoothing process on the original image. A background evaluation image generating means for generating an image, a flat evaluation image generating means for generating a flat evaluation image representing evaluation relating to flatness of the original image by performing edge detection processing and smoothing processing on the original image, and the original image And a damaged portion detection image generating means for identifying the damaged portion and generating a damaged portion detection image based on a difference value between the image and the background evaluation image and a pixel value of the flat evaluation image. .
With the above configuration, it is possible to prevent a specific pattern included in the background image having the same brightness difference as the damaged portion from being erroneously detected as a damaged portion.
Hereinafter, the features of the present invention will be described in detail with reference to the drawings.

Embodiments of the present invention will be described with reference to the drawings.
<Information processing apparatus and image forming apparatus>
FIG. 1 is a diagram showing an information processing apparatus 1 and an image forming apparatus 2 according to an embodiment of the present invention.
As shown in FIG. 1, the information processing apparatus 1 and the image forming apparatus 2 are connected via a network. As specific examples of the information processing apparatus 1 and the image forming apparatus 2, the information processing apparatus 1 is a general information processing terminal such as a PC (Personal Computer), and the image forming apparatus 2 is an ink jet used in homes. Besides the printer, the image forming apparatus 2 is a printing machine used in business printing, and the information processing apparatus 1 is a control information processing terminal for controlling such a business printing machine. Cases can be considered.

  The information processing apparatus 1 is a general-purpose information processing apparatus such as a PC. The information processing apparatus 1 is installed with a software program for configuring a printer driver that realizes a function for causing the image forming apparatus 2 to execute image forming output.

<Information processing device>
With reference to FIG. 2, the hardware configuration of the information processing apparatus 1 according to an embodiment of the present invention will be described. As illustrated in FIG. 2, the information processing apparatus 1 includes a configuration similar to that of a general server or PC. That is, in the information processing apparatus 1, a central processing unit (CPU) 11, a random access memory (RAM) 12, a read only memory (ROM) 13, a hard disk drive (HDD) 14, and an I / F 15 are connected via a bus 18. Has been. Further, an LCD (Liquid Crystal Display) 16 and an operation unit 17 are connected to the I / F 15.

The CPU 11 controls the overall operation of the information processing apparatus 1. The RAM 12 is a volatile storage medium capable of reading and writing information at high speed, and is used as a work area when the CPU 11 processes information.
The ROM 13 is a read-only nonvolatile storage medium, and stores programs such as firmware. The HDD 14 is a nonvolatile storage medium that can read and write information, and stores an OS (Operating System), various control programs, application programs, and the like. A software program for configuring the printer driver described above is also stored in the HDD 14.

  The I / F 15 connects and controls the bus 18 and various hardware and networks. The LCD 16 is a visual user interface for the user to check the state of the information processing apparatus 1. The operation unit 17 is a user interface for a user to input information to the information processing apparatus 1 such as a keyboard and a mouse.

  In such a hardware configuration, the software control unit reads out a program stored in a storage medium such as the ROM 13, HDD 14, or optical disk, writes the program in the RAM 12, and the CPU 11 performs calculations according to the programs read from the RAM 12. Composed. The function of the information processing apparatus 1 is realized by a combination of the software control unit configured as described above and hardware.

<Functional configuration of information processing apparatus>
Next, a functional configuration of the information processing apparatus 1 according to an embodiment of the present invention will be described.
As illustrated in FIG. 3, the information processing apparatus 1 includes a controller 20 and an external I / F 42. The external I / F 42 is an interface for the information processing apparatus 1 to exchange information with the image forming apparatus 2, and uses an Ethernet (registered trademark) or a USB (Universal Serial Bus) interface.

The controller 20 is a software control unit configured as described above, and includes an application 22, a communication control unit 40, and a printer driver 26.
The application 22 is software that implements a function for browsing and editing image data, document data, and the like.
The application 22 includes an image processing unit 24, and obtains an image in which the damaged portion is finally repaired from the original image by the image processing unit 24.

  The application 22 outputs a print instruction such as image data or document data being browsed or edited in accordance with a user operation. The communication control unit 40 performs processing for the controller 20 to exchange information with the image forming apparatus 2 via the external I / F 42.

  Upon receiving a print instruction from the application 22, the printer driver 26 generates drawing information that is information for the image forming apparatus 2 to execute image formation output. At this time, the printer driver 26 generates drawing information corresponding to the characteristics of the image forming mechanism included in the image forming apparatus 2 so that a highly accurate image as intended is formed.

<Printer driver>
As illustrated in FIG. 3, the printer driver 26 includes a print instruction acquisition unit 28, a drawing information generation unit 30, a drawing information transfer unit 32, a dot pattern generation unit 34, a read image acquisition unit 36, and a nozzle characteristic determination unit 38. The function of the printer driver 26 will be described in detail later.

<Image forming apparatus>
The image forming apparatus 2 includes an inkjet image forming mechanism, and executes image formation output according to the drawing information input from the information processing apparatus 1.
With reference to FIG. 4, a functional configuration of the image forming apparatus 2 according to the embodiment of the present disclosure will be described. As shown in FIG. 4, the image forming apparatus 2 includes a controller 70, an operation panel 30, a carriage 50, a main scanning motor 56, a sub scanning motor 60, a conveyance belt 62, and a charging roller 66.

  The operation panel 30 is a user interface that functions as an operation unit and a display unit for inputting and displaying information necessary for the image forming apparatus 1. The carriage 50 is equipped with a recording head 54 that ejects ink and a head driver 52 that drives the recording head 54. The carriage 50 is perpendicular to the sub-scanning direction, which is the conveyance direction of the sheet, with respect to the sheet conveyed by the conveyance belt 62. By moving in the main scanning direction, which is a normal direction, ink is ejected to the front surface of the paper to perform image formation output.

  The main scanning motor 56 is a motor that supplies power for moving the carriage 50 in the main scanning direction. The sub-scanning motor 60 is a motor that supplies power to a conveyance belt 62 that conveys a sheet that is an image output target. The rotations of the main scanning motor 56 and the sub scanning motor 60 are detected by the encoder sensor 58 and the encoder sensor 64, respectively, and the detection signals are input to the controller 70. The charging roller 66 charges the conveyance belt 62 to generate an electrostatic force for attracting the sheet, which is an image output target, to the conveyance belt 62.

  The controller 20 is a control unit that controls the operation of the image forming apparatus 1, and as shown in FIG. 2, a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an NVRAM. (Non Volatile RAM) 204, ASIC (Application Specific Integrated Circuit) 205, host I / F 82, print control unit 84, motor drive unit 86, AC bias supply unit 88, and I / O 90 are included.

  The CPU 72 controls the operation of each part of the controller 70. The ROM 74 is a read-only nonvolatile storage medium, and stores programs such as firmware. The RAM 76 is a volatile storage medium capable of reading and writing information at high speed, and is used as a work area when the CPU 72 processes information. The NVRAM 78 is a non-volatile storage medium capable of reading and writing information, and stores a control program and control parameters.

  The ASIC 80 is a hardware circuit that executes image processing necessary for image formation output. The host I / F 82 is an interface for receiving drawing data from the information processing apparatus 1, and uses an Ethernet (registered trademark) or a USB (Universal Serial Bus) interface. The I / O 90 is a port for inputting detection signals from various sensors such as the encoder sensors 58 and 28 to the controller 70.

  The print control unit 84 includes data transfer means for driving and controlling the recording head 54 included in the carriage 50, and drive waveform generation means for generating a drive waveform. The motor driving unit 86 drives the main scanning motor 56 and the sub scanning motor 60. The AC bias supply unit 88 supplies an AC bias to the charging roller 66.

  As described above, the drawing data input from the information processing apparatus 1 is input to the host I / F 82 in the controller 70 and stored in the reception buffer in the host I / F. The CPU 72 performs calculation according to the program loaded in the RAM 76 to read and analyze drawing data in the reception buffer included in the host I / F 82 and control the ASIC 80 to perform necessary image processing and data rearrangement processing. Etc. Thereafter, the CPU 72 controls the print control unit 84 to transfer the drawing data processed in the ASIC 80 to the head driver 52.

  The print control unit 84 transfers the drawing data described above to the head driver 52 as serial data, and transmits a transfer clock, a latch signal, a droplet control signal (mask signal), and the like necessary for transferring the drawing data and confirming the transfer. Output to the head driver 52. The print control unit 84 also includes a drive waveform generation unit and a head driver 52 including a D / A converter, a voltage amplifier, and a current amplifier that perform D / A conversion on the drive signal pattern data stored in the ROM 74. A drive waveform selection means including a drive pulse (drive signal) or a plurality of drive pulses (drive signal) is generated and output to the head driver 52.

  The head driver 52 generates energy for causing droplets to be ejected from the recording head 54 based on the drawing data for one line that is serially input, and a drive signal that constitutes a drive waveform supplied from the print control unit 84. The recording head 54 is driven by selectively applying the driving element. At this time, the head driver 52 selects a driving pulse that constitutes a driving waveform, so that, for example, large droplets (large dots), medium droplets (medium dots), small droplets (small dots), and the like having different sizes. Can be sorted out.

  The CPU 72 also detects a speed detection value and a position detection value obtained by sampling a detection pulse from the encoder sensor 58 constituting the linear encoder, and a speed target value and a position target value obtained from a previously stored speed / position profile. Based on the above, a drive output value (control value) for the main scanning motor 56 is calculated, and the main scanning motor 56 is driven via the motor driving unit 86. Similarly, the CPU 72 detects a speed detection value and a position detection value obtained by sampling a detection pulse from the encoder sensor 64 constituting the rotary encoder, and a speed target value and a position target value obtained from a previously stored speed / position profile. Based on the above, a drive output value (control value) for the sub-scanning motor 60 is calculated, and the sub-scanning motor 60 is driven via the motor driver 86 and the motor driver.

<Printer driver operation>
The information processing apparatus 1 and the image forming apparatus 2 as described above work together to execute image formation output. That is, a print instruction from the application 22 operating in the information processing apparatus 1 is processed by the printer driver 26 to generate a multi-valued recording dot pattern that can be output by the image forming apparatus 2, and is rasterized as drawing data. The image is transferred to the image forming apparatus 2, and image forming output is executed in the image forming apparatus 2.

  Specifically, in the information processing apparatus 1, an image drawing or character recording command from the application 22 or operating system (for example, a description of the position, thickness, and shape of a line to be recorded, or a typeface of a character to be recorded) Are described by the print instruction acquisition unit 28 and temporarily stored in the drawing data memory. Note that these instructions are written in a specific print language.

  The command stored in the drawing data memory is interpreted by the rasterizer included in the drawing information generation unit 30, and if it is a line recording command, it is converted into a recording dot pattern corresponding to the designated position and thickness. If the character recording command is used, the outline information of the corresponding character is called from the font outline data stored in the information processing apparatus 1 and converted into a recording dot pattern corresponding to the designated position and size. If it is data, it is converted into a recording dot pattern as it is.

  Thereafter, the drawing information generation unit 30 performs image processing on these recorded dot patterns and stores them in the raster data memory. At this time, the information processing apparatus 1 rasterizes the data of the recording dot pattern using the orthogonal lattice as the basic recording position. Examples of image processing include color management processing for adjusting colors, γ correction processing, halftone processing such as dithering and error diffusion, further background removal processing, and total ink amount regulation processing. Then, the recording dot pattern stored in the raster data memory is transferred to the inkjet printer 500 by the drawing information transfer unit 32 via the interface.

  Of the processes executed in the drawing information generation unit 30 as described above, the halftone process is also called a halftone process. This halftone process is a process of converting the color expression gradation in the image information to be subjected to image formation output according to the color expression gradation in the image formation output performance of the image forming apparatus 2.

  As described above, in the inkjet image forming apparatus 2, each pixel is formed by ejecting ink from the nozzles included in the recording head 54 and image forming output is executed. At this time, as described above, by switching the ink droplets to be ejected to small droplets, medium droplets, and large droplets, it is possible to express the density of about 4 gradations (2 bits) including one in a single pixel. However, on a PC such as the information processing apparatus 1, the image data is generally processed with 256 gradations (8 bits) or the like.

  Therefore, in the halftone processing, for example, image data expressed by 8 bits is converted into image data expressed by 2 bits without degrading the image quality recognized by human eyes. Comparing the image data before conversion and after conversion for each pixel, it is clearly different because the density gradation is reduced. Therefore, in the halftone process, when an image is compared for each predetermined area unit including a plurality of pixels, the image is converted by the area gradation method so that it is recognized as a similar image by human eyes. As a result, in the halftone process, a quantization error due to the area gradation method occurs.

  As a halftone process considering a quantization error, an error diffusion method, a DBS (Direct Binary Search) method, or the like is known. The error diffusion method is a method of weighting and accumulating the quantization error generated in each pixel and diffusing it to surrounding pixels. The error diffusion method is a method of performing quantization by comparison with a prescribed repetitive pattern such as dither method. In comparison, texture and moire are less likely to occur. However, there is also a feature that dot series on the worm due to the processing direction is likely to occur. DBS tentatively determines the dot arrangement, calculates the quantization error with the input by switching the dot arrangement for each pixel or section and switching the type, and updates the dot arrangement if the error is reduced By repeating this process, the error is reduced. The sharpness of the image is excellent, and regular patterns such as worms caused by error diffusion are less likely to occur.

  Since the above processing is related to data processing, it is theoretical. In the first place, each pixel is handled in a square shape, but each dot (hereinafter referred to as a landing image) formed by fixing the ink ejected from the nozzle on the paper is circular. It is. For this reason, how to fill the page differs depending on the presence or absence of dots at adjacent pixel positions and the type of dots.

  Ink is ejected from the nozzles by mechanical control. Therefore, the characteristics of the landing image for each nozzle differ depending on various factors such as the response characteristics of the piezoelectric element and the fine shape of the nozzle. Therefore, even when driven based on the same drive signal, the dot size of the landing image is different, the landing position is shifted, satellites are generated, and adjacent pixels are affected. There is. Here, the satellite is a minute droplet that is discharged along with the main droplet (main dot).

  As a result, even if you intend to print uniform data, depending on the characteristics of individual heads and nozzles, there will be differences in the amount of ink attached and the way the dots overlap, so this is an image defect such as color differences, streaks, and banding. , It may be recognized by human eyes.

  In order to cope with such a problem, the drawing information generation unit 30 calculates an error by forming a dot landing model instead of a digital value when performing dot replacement processing by error calculation by the DBS method. Create print data including engine characteristics.

  Therefore, when image formation output is performed using the image forming apparatus 2, as an advance process, an image patch generated by the dot pattern generation unit 34 is printed, and the patch is generated by reading it with a sensor, a scanner, or the like. The read image acquisition unit 36 acquires image data, and the nozzle characteristic determination unit 38 generates a landing model of dots formed by the nozzles of the image forming apparatus 2 based on the patch image data. Then, in the drawing data generation process in the subsequent image formation output, the drawing information generation unit 30 simulates the dots ejected by each nozzle using this landing model.

  First, an example of an image pattern generated by the dot pattern generation unit 34 will be described. The dot pattern generation unit 34 generates an image pattern and inputs it to the print instruction acquisition unit 28 to cause the image forming apparatus 2 to execute image formation output of the image pattern. The read image acquisition unit 36 acquires the image data generated by reading the paper output in this way by a scanner or the like, and the nozzle characteristic determination unit 38 acquires a model of dots formed by the nozzles.

  The nozzle characteristic determination unit 38 compares the dot pattern generated by the dot pattern generation unit 34 with image information obtained by reading the dot pattern acquired by the read image information acquisition unit 115, that is, dot pattern image information. As a result, the nozzle characteristic determination unit 38 attaches the ink ejected by each of the plurality of nozzles included in the image forming apparatus 2 on the sheet, which is the image formation output target medium, the size of the adhering ink, and the adhesion. Nozzle characteristic information including information on ink shape and satellite (distribution of attached ink) is generated.

  A case where a carriage 50 in which nozzles are arranged in 4 rows and 1 column is taken as an example and a pattern of 4 rows is formed is shown as an example. This is the most basic example in which one nozzle is provided for each color of CMYK, and the image pattern to be formed is set by the arrangement of nozzles such as a line type recording head.

  The layout of the image pattern generated by the dot pattern generation unit 34 is not particularly limited, but when the dot formation density of the pattern is high, it is combined with surrounding dots and which dot is ejected from which nozzle. Since it becomes difficult to distinguish, it is desirable that the pattern has a sufficient gap so that it can be distinguished which nozzle is ejected from which nozzle even if the size and landing position of the dot fluctuate.

  Also, in the case of a printer that can handle dots of a plurality of sizes, the size of the reference dot varies depending on the drop type. In addition, large droplets land as intended, but medium droplets tend to bend, and landing characteristics may vary depending on the type of droplet. For this reason, it is desirable to obtain an impact model by printing and reading an image pattern for each droplet type.

  Next, halftone processing by the drawing information generation unit 30 will be described. The drawing information generation unit 30 uses the DBS method. Specifically, the error of the input image is obtained for the dots in the attention area while changing the ON / OFF of the dots and the type, that is, the density within the gradation that the image forming apparatus 2 supports. If the error is smaller than before the change, the quantization error is reduced by updating the arrangement to the changed content.

  Here, the pixel of interest may be changed for each pixel and error determination processing may be performed, or a combination of ON / OFF of dots and replacement with other types of dots within a predetermined area may be performed, and errors within the section may be reduced. You may select arrangement | positioning on the combination conditions to become small. In addition, since actual data has a dot gain, the influence of adjacent pixels may not be reflected in the error in the context of processing. For this reason, after a round of data, a process of calculating an error or reexamining the arrangement may be entered again. At this time, the method of cutting the section for reviewing the arrangement may be changed.

  As the process termination condition, it is possible to adopt a configuration in which the process is terminated by the number of processes or when the error is not reduced with respect to the previous process. In the first time, halftone processing may be performed in advance by dithering, error diffusion processing, or the like, and processing may be started from that arrangement. In addition, it is also possible to use the case where the determination of the replacement process after the error comparison is not performed when the error is minimized, but is determined using a simulated annealing method or the like.

  When this error calculation is performed, the evaluation of the error proceeds by assuming that the acquired dot model is used instead of simple digital data. Therefore, the drawing information generation unit 30 converts the dot arrangement of the target pixel, that is, the image data to be output according to the gradation of the image forming apparatus 2 based on the nozzle characteristics acquired by the nozzle characteristic determination unit 38. The dot arrangement is replaced with a dot model of each nozzle to simulate the image after landing.

  For example, the drawing information generation unit 30 first quantizes each nozzle by performing halftone processing on input data, that is, original image data to be output. Furthermore, the drawing information generation unit 30 generates data on the basis of the nozzle characteristics acquired by the nozzle characteristic determination unit 38 and estimates the landing state when each nozzle ejects ink, that is, the ink distribution.

  Next, the drawing information generation unit 30 calculates an error between the image after landing and the input data. At this time, the drawing information generation unit 30 obtains the ink by comparing the ink adhesion within the area corresponding to the pixel of the input data with the level of the input image. For example, the multi-value level is made based on the ink coverage at a location corresponding to the pixel of the input data. For example, if the input data is a density gradation of 0 to 255 (here, 255 is black), and the coverage of the area of the corresponding pixel is 50%, the output at that location is half of 0 to 255. The difference from the input data is taken as an error.

  The drawing information generation unit 30 converts the input data into multi-value data so that the ink coverage 0% to 100% corresponds to the density gradations 0 to 255, and generates multi-value simulation data. By comparing the multi-value simulation data thus generated and the input data, the difference is made an error.

  The drawing information generation unit 30 may perform the calculation based on brightness and density instead of the covering amount (for example, conversion is performed so that the maximum density corresponds to 255 and the minimum density = paper density corresponds to 0). As described above, the drawing information generation unit 30 repeats error calculation / comparison while changing the quantization condition in consideration of the nozzle characteristics, so that the error with the input is reduced with the condition including the engine characteristics. Determine the quantization condition.

<Operation Example of Drawing Information Generation Unit 30>
A case will be described as an example where the number of pixels of the image to be output is “M” and the density gradation of the image forming apparatus 2 such as colorless, small, medium, and large droplets is “A”. First, the drawing information generation unit 30 provisionally determines the dot arrangement for all M pixels.

  By converting the gradation of the image to be image-formed output into the gradation of pixels that can be expressed by the recording head 54 of the image forming apparatus 2 from the multivalued digital data, like the digital data after halftone at the center, This is processing for generating an image after tone conversion. That is, the drawing information generation unit 30 functions as a gradation conversion unit.

  Next, the drawing information generation unit 30 applies the landing model to the tentatively determined dot arrangement to generate landing simulation data. In other words, this process is a process of generating simulation information of the image formation output result based on the information indicating the characteristics of the nozzles included in the recording head 54 of the image forming apparatus 2 and the image after gradation conversion. That is, the drawing information generation unit 30 functions as an output result simulation unit.

  Then, the drawing information generation unit 30 performs gradation value conversion on the result of the landing simulation and generates multi-value conversion data. This process is a process of converting the simulation information of the image formation output result into the gradation of the pixels constituting the image that is the original image formation output target.

  When the multi-value conversion data is generated, the drawing information generation unit 30 compares the pixels of the multi-value conversion data and the input data to calculate a quantization error. The drawing information generation unit 30 that has calculated the error compares the already calculated and recorded quantization error with the newly calculated error, and if the newly calculated error value decreases, The current dot arrangement and the newly calculated error value are recorded on the storage medium.

  On the other hand, if the error value has not decreased, the drawing information generation unit 30 discards the error value. The drawing information generation unit 30 repeats the above-described processing until all combinations of dot arrangements are completed, and ends the processing when all the combinations are completed.

  This is a process of determining, as drawing information, one of a plurality of gradation-converted images having different gradation conversion modes when generating a gradation-converted image based on a calculated error value. . That is, the drawing information generation unit 30 functions as a drawing information determination unit. By such processing, the dot arrangement with the smallest error value is recorded.

  As described above, according to the information processing apparatus 1 in which the printer driver 26 for causing the image forming apparatus 2 to execute image formation output is installed, the dot pattern quantized by the DBS method, the original image data, When an optimal dot pattern is selected by calculating the error value of the ink, the error value between the landing simulation based on the landing characteristics of ink ejection by the nozzles of the recording head 54 included in the image forming apparatus 2 and the original image data is calculated. Therefore, it is possible to enable image formation output in accordance with engine characteristics of image formation output.

  Note that the dot landing model may vary depending on the operating conditions of the apparatus. The operation conditions of the apparatus are, for example, the printing environment such as the paper type and mode used for printing, temperature and humidity. This is because, if the paper type is different, even when the same dot is ejected, the wet spread method on the paper surface is different. Furthermore, when the mode or printing environment changes, the ink viscosity determined according to the driving conditions of the head used for printing, the conveyance speed of the paper, the humidity, and the temperature are different, so the ejection characteristics of the head itself can also change.

  Therefore, it is preferable that acquisition and reflection of the landing model can be set / applied for each paper type, print mode, and printing environment. As a result, it is possible to perform image formation under conditions optimal for printing conditions.

  As for the timing to acquire the landing model, the operating environment of the device such as humidity and temperature fluctuates by more than a predetermined time at the start-up of the device, at the time of head replacement, every elapse of a predetermined time in the operation period of the device, every count of the number of printed sheets In this case, various factors can be used as a trigger, such as when the paper type or mode is changed or when the user designates it. This is because the landing characteristics such as the configuration of the apparatus, the usage environment, and changes with time are easily changed.

  In the case where the head is exchanged, it is the exchange head that can change the landing characteristics. Usually, since the head is exchanged for each color such as CMYK, the number of man-hours may be omitted by obtaining the landing model only for the exchanged head.

  These may be implemented through a series of processes from printing an image pattern to obtaining and reviewing a landing model, or may be configured to perform only a part. For example, patch image data is acquired by the image information acquisition unit 115 and a landing model is generated by the nozzle characteristic determination unit 38, and the difference between the newly generated landing model and the landing model that has been previously generated and applied is predetermined. If the threshold value is exceeded, the landing model may be updated to reduce the processing man-hours.

  Such processing can be realized by, for example, printing gradations and color patches when printing an image pattern, picking up characteristics, and revisiting the landing model when these characteristics change more than a predetermined value. Can do. The characteristics may be configured such that the landing model is updated when the characteristics change by a predetermined level or more using the brightness and density of information acquired by a sensor or scanner, RGB values, or the like.

  Further, as another method, a method of comparing fluctuations in the ink covering amount of the nozzle and revising when the amount deviates from a specified amount may be considered. The criteria for reviewing may be specified as absolute values, or by printing the pattern multiple times when printing an image pattern, picking up repeated variations during continuous printing, and if exceeding this, the characteristics change. For example, the landing model may be updated.

  In the above embodiment, as shown in FIG. 3, the case where the read image acquisition unit 36 and the nozzle characteristic determination unit 38 are included in the printer driver 26 of the information processing apparatus 1 has been described as an example. Such a configuration may be provided inside and the landing model may be downloaded from the image forming apparatus 2 to the information processing apparatus 1. In this way, if the image forming apparatus 2 is configured to be able to generate a landing model independently, it is possible to perform from creation to output of an image considering engine characteristics alone.

  As a specific aspect of the information processing apparatus 1 and the image forming apparatus 2, in the case of a business printing machine and a control apparatus, a dedicated apparatus or software called a RIP (raster image processor) is used in an apparatus used in the printing field. In many cases, a halftone image is created via an image forming apparatus, and the image is input to an image forming apparatus to print the image. In many cases, the image processing apparatus has advanced functions and a calculation mechanism for image processing.

Thus, any or all of chart output, landing model acquisition, and image data creation can be realized on the RIP side, that is, the information processing apparatus 1 side. In this case, since data processing including engine characteristics can be performed with RIP and flow to the apparatus side, even if the image forming apparatus does not have a function for correcting engine characteristics in particular, even RIP can be supported. This makes it possible to correct the engine characteristics.
Then, when generating the landing model based on the patch image data acquired by the read image acquisition unit 115, the nozzle characteristic determination unit 38 determines the landing model for each of the head A and the head B based on the above-described hitting information. Generate.

  Then, the drawing information generation unit 30 generates an impact simulation based on the hitting information when generating an impact simulation by applying the impact model. By such processing, even in an image forming apparatus using overlap processing, it is possible to obtain the same effect as described above by applying the landing model processing.

<Image processing unit>
FIG. 5 is a functional block diagram showing a functional configuration of the image processing unit 24 according to the embodiment of the present invention.
FIG. 5 shows a configuration in which the image processing unit 24 can obtain an image in which the damaged part is finally repaired from the original image, and further shows transition of the intermediate image.

<Image to be processed by image processing unit>
In the present embodiment, referring to FIGS. 17 to 23, the following units perform image processing.
FIG. 17 is a diagram showing an original image G100 acquired from a picture drawn with oil paint using, for example, a scanner or the HDD 14, and a damaged portion is generated in the original image G100.
FIG. 18 shows a background evaluation image G110 generated from the original image G100.
FIG. 19 shows a flatness evaluation image G120 generated from the original image G100.
FIG. 20 is a diagram illustrating a damaged portion detection image [1] G200 generated from the original image G100 and the flatness evaluation image G120.
FIG. 21 is a diagram showing a damaged portion detection image [2] G210 generated from the damaged portion detection image [1] G200.
FIG. 22 is a diagram illustrating a damaged portion repair image [1] G300 generated from the original image G100 and the damaged portion detection image [2] G210.
FIG. 23 is a diagram showing a damaged portion repair image [2] G310 generated from the damaged portion repair image [1] G300.

<Functional configuration of image processing unit>
Returning to FIG. 5, the image processing unit 24 generates the original image acquisition unit 100, the background evaluation image generation unit 110, the flat evaluation image generation unit 120, the damaged portion detection image [1] generation unit 200, and the damaged portion detection image [2]. Unit 210, damaged part repair image [1] generation unit 300, and damaged part repair image [2] generation unit 310.
The original image acquisition unit 100 acquires image data of a painting having a damaged portion as the original image G100 from the outside or the HDD 14 and stores the image data in the RAM 12.
The background evaluation image generation unit 110 generates a background evaluation image G110 from the original image G100. Specifically, the background evaluation image generation unit 110 generates a background evaluation image G110 that represents the evaluation of the pixel that is the background of each pixel by performing a smoothing process on the original image G100.
The flat evaluation image generation unit 120 generates a flat evaluation image G120 from the original image G100. Specifically, the flat evaluation image generation unit 120 generates a flat evaluation image G120 representing an evaluation related to flatness of the original image G100 by performing edge detection processing and smoothing processing on the original image G100.
The damaged portion detection image [1] generation unit 200 determines the presence or absence of a damaged portion in each pixel from the original image G100, the background evaluation image G110, and the flat evaluation image G120, and generates a damaged portion detection image [1] G200. Specifically, the damaged portion detection image [1] generation unit 200 identifies the damaged portion based on the difference value between the original image G100 and the background evaluation image G110 and the pixel value of the flat evaluation image G120, thereby detecting the damaged portion. Detected image [1] G200 is generated.
The damaged portion detection image [2] generation unit 210 performs pixel expansion processing of the damaged portion detection area from the damaged portion detection image [1] G200, and generates a damaged portion detection image [2] G210.

Next, the damaged part repaired image [1] generation unit 300 performs an interpolation process on the damaged part detected in the damaged part detection image [2] G210 using the pixel value of the original image G100, thereby repairing the damaged part. Image [1] G300 is generated.
Note that the damaged portion repaired image [1] generation unit 300 uses the pixel value of the background evaluation image G110 in order to cope with the case where interpolation at the boundary of the image or a pixel to be referred to at the time of interpolation is not found. Then, the damaged portion repair image [1] G300 may be generated by performing the complement processing.
Finally, the damaged portion repaired image [2] generation unit 310 applies a cross to the damaged portion repaired image [1] G300 in order to reduce a specific cross pattern that appears by performing interpolation on the damaged portion repaired image [1] G300. Pattern reduction sharpening filter processing is performed to generate a damaged portion repair image [2] G310.

<Background evaluation image generation processing>
FIG. 6 is a flowchart for explaining the background evaluation image generation processing by the background evaluation image generation unit 110 shown in FIG.
The background evaluation image generation unit 110 sets a processing area (RAM address space) for the original image G100 stored in the RAM 12 (S100a).
The background evaluation image generation unit 110 applies a smoothing process to the original image G100 by applying a smoothing filter to evaluate the pixel value that is the background of each pixel value of the original image G100. It is generated on the RAM 12 (S101).
The background evaluation image generation unit 110 stores the generated background evaluation image G110 in the HDD 14 (S110a).
As the smoothing filter, a median filter or a Gaussian filter that can minimize the influence of noise such as a damaged portion and can evaluate the background region is used.

FIG. 7A is a diagram illustrating the configuration of the median filter, and FIG. 7B is a diagram illustrating the configuration of the Gaussian filter.
Here, specific examples of the median filter and the Gaussian filter will be described with reference to FIGS.
The median filter shown in FIG. 7A has a median value when pixel values are arranged in order of size with respect to the pixel group in the filter centered on the pixel of interest (pixel value 45 shown in FIG. 7A). Is extracted and the pixel value of interest is replaced.
This makes it possible to efficiently replace spike noise using background pixels.
On the other hand, the Gaussian filter shown in FIG. 7B performs a process of replacing the target pixel by calculating an average value using a pixel group in the filter centered on the target pixel and a weighted coefficient called a Gaussian filter. . In processing using a Gaussian filter, for example, Equation (1),

・・・式（１）
により注目画素の値が求められる。

... Formula (1)
Thus, the value of the target pixel is obtained.

Here, characteristics of the median filter and the Gaussian filter will be described.
For example, if a noise image has a black line on the background image, applying a Gaussian filter to this image will reduce the noise, but the black line will still remain and the border of the background image will be Blur (blurring) occurs.
On the other hand, when a median filter is applied to the image, the black line is clearly removed from the background pixels, and the boundary portion is also retained.
It is suitable to use a median filter to evaluate background pixels for an image having a noise-like pattern.
The smoothing filter size is preferably about 5 to 7 times (-10 mm) the width of the damaged part (1-2 mm). This is because if it is too large, the background larger than the damaged part is not recognized as noise and cannot be evaluated correctly. Moreover, it is because a damaged part cannot be correctly evaluated as noise even if the filter size is too small.

<Flatness evaluation image generation processing>
FIG. 8 is a flowchart for explaining flatness evaluation image generation processing by the flatness evaluation image generation unit 120 shown in FIG.
The flat evaluation image generation unit 120 sets a processing area for the original image G100 stored in the RAM 12 (S100b).
First, a difference image between adjacent pixels of the target pixel is obtained.
The flat evaluation image generation unit 120 performs the Sobel filter processing in the X direction (S111) and the Sobel filter in the Y direction for each of the RGB images (k = r, g, b) of the original image G100 stored in the RAM 12. The process (S112) is applied.

Next, the flat evaluation image generation unit 120 calculates the square root of the square sum of the X-direction Sobel filter pixel (g _{k, x} ) and the Y-direction Sobel filter pixel (g _{k, y} ) (S113), and the RGB of the pixel A difference image (g _k ) is assumed.

・・・式（２）
また、平坦評価画像生成部１２０は、ＲＧＢを統一した差分画像（ｇ）を求めるのに、各画素のＲ、Ｇ、Ｂの最大値Ｍａｘを計算する（Ｓ１１４）。

... Formula (2)
Further, the flat evaluation image generation unit 120 calculates the maximum values Max of R, G, and B of each pixel in order to obtain a difference image (g) in which RGB are unified (S114).

・・・式（３）
次に、平坦評価画像生成部１２０は、周辺領域の平坦さを表す指標として、求めた差分画像を平滑化（Ｓ１１５）した画像の画素値で評価する。

... Formula (3)
Next, the flat evaluation image generation unit 120 evaluates the obtained difference image as an index representing the flatness of the peripheral region using the pixel value of the image obtained by smoothing (S115).

  Next, the flat evaluation image generation unit 120 generates a flat evaluation image G120 in order to evaluate how flat the change in pixel values around each pixel has (S120a). Here, the flat evaluation image generation unit 120 generates a flat evaluation image G120 by smoothing an image representing a difference value between adjacent pixels of the original image G100.

In step S115, the flatness evaluation image generation unit 120 may use a Gaussian filter as a smoothing filter if importance is attached to the flatness of the pixel of interest. In addition, an averaging filter, a median filter, or the like may be used. It may be used.
On the other hand, when it is desired to uniformly evaluate the flatness of surrounding pixels, an averaging filter is appropriate. Further, when it is desired to remove the influence of an abnormal value as a representative value of flatness, a median filter may be used similarly to the step (S120a) for generating the flatness evaluation image G120 described above.
As for the filter size, it is preferable to select about 5 to 7 times the filter size when the original image G100 is smoothed using a median filter.

<Damage detection image [1] generation process>
FIG. 9 is a flowchart for explaining the damaged portion detection image [1] generation process by the damaged portion detection image [1] generation unit 200 shown in FIG.
The damaged portion detection image [1] generation unit 200 sets a processing region for the original image G100 stored in the RAM 12 (S100c).
Next, the damaged part detection image [1] generation unit 200 sets a processing area for the background evaluation image G110 stored in the RAM 12 (S110b).
Next, the damaged portion detection image [1] generation unit 200 sets a processing region for the flat evaluation image G120 stored in the RAM 12 (S120b).
Next, the damaged part detection image [1] generation unit 200 designates the start address of the RAM 12 that stores the damaged part detection image [1] G200, and starts loop processing for each pixel (S125).
The damaged portion detection image [1] G200 [1] represents, as a binary value, whether or not a pixel is a damaged portion in each pixel of the original image G100, the background evaluation image G110, and the flat evaluation image G120. I remember the data.

Next, the damaged part detection image [1] generation unit 200 prepares for determining whether a certain pixel is a damaged part or not as an area for a background pattern in which it is difficult to determine whether a certain pixel is a damaged part. Sets a threshold value used for the damaged portion detection processing strictly, and sets a correction value to prevent the background pattern from being detected and removed by mistake (S130).
This correction value is determined by the pixel value in the flat evaluation image G120. The correction value is set by changing it in a binary manner with an arbitrary threshold as a boundary. However, in general, the correction value may be calculated as a function of the pixel value of the flat evaluation image G120.

Next, the damaged portion detection image [1] generation unit 200 enters the step of determining a damaged portion. As a clue to detect a damaged portion pixel from a picture image, the color difference value (ΔE in the L * a * b * space). *) To determine. Thereby, the accuracy of discrimination from the background pattern can be improved by using L * a * b * that is close to the color difference perceived by humans.
Next, the damaged part detection image [1] generation unit 200 compares the L * a * b * values of the pixels of the original image G100 and the background evaluation image G110 to determine whether the pixel is a part of the damaged part. Judge whether or not. At that time, the correction value based on the flatness set as the preparation in step S130 is applied to the damaged portion detection threshold. First, the damaged portion detection image [1] generation unit 200 converts the RGB values of each pixel of the original image G100 and the background evaluation image G110 into L * a * b * values (S140).
In step S140, the damaged portion detection image [1] generation unit 200 executes the RGB color space as sRGB, the light source as D65, and the conversion destination L * a * b * space as the light source D50.

Next, the process moves to a damaged portion detection process. Judgment is made by narrowing down the damaged part in the painting into two parts, “black damaged part” and “white damaged part” based on the visible features.
The damaged portion detection image [1] generation unit 200 initializes the flag as false (0 value) for the binary data value on the RAM 123 for storing the damaged portion detection image [1] G200 (S150). ).
Next, the damaged portion detection image [1] generation unit 200 calls a subroutine of black damaged portion detection processing (S160), and when returning from this subroutine, whether or not the pixel is a part of the damaged portion. Is determined (S168). If it is determined in step S168 that the pixel is a part of the damaged part (has a damaged part = 1), the process proceeds to step S180. If the pixel is not a part of the damaged part (no damaged part = 0), the process proceeds to step S170. move on.
In step S170, the damaged portion detection image [1] generation unit 200 calls a subroutine of white broken portion detection processing (S170), and when returning from this subroutine, whether or not the pixel is a part of the damaged portion. Is determined (S178). If it is determined in step S175 that the pixel is a part of the damaged part (there is a damaged part = 1), the process proceeds to step S180. If the pixel is not a part of the damaged part (there is no damaged part = 0), the process proceeds to step S185. move on.

In step S180, when the damaged part detection image [1] generation unit 200 determines that the pixel is a part of the damaged part, the damaged part binary value for the pixel is set as true in the RAM 12 (S180). The process proceeds to step S185.
In step S185, the damaged portion detection image [1] generation unit 200 determines whether the address of the target pixel represents the end point position and the loop processing has been completed. If it is determined that the loop process has been completed, the process proceeds to step S200. On the other hand, if it is determined that the loop process has not been completed, the process proceeds to step S190.

In step S190, the damaged portion detection image [1] generation unit 200 has not finished the loop process, so the address of the pixel of interest is incremented by one to set the next pixel as the pixel of interest, and the process returns to step S130.
In step S <b> 200, the damaged portion detection image [1] generation unit 200 stores the damaged portion detection image [1] G200 [1] generated on the RAM 12 in the HDD 14.
The details of the subroutine of the black broken part detection process (S160) and the white broken part detection process (S170) will be described later.

<Black breakage detection processing>
FIG. 10 is a flowchart for explaining a subroutine of black breakage detection processing by the breakage detection image [1] generation unit 200 shown in FIG.
Identification conditions B1 to B4 are set in a subroutine of black broken portion detection processing (S160).
B1: It is determined whether the pixel value (P) of the original image G100 has a lightness (L *) smaller than the pixel value (Q) of the background evaluation image G110 (S161).

L * (P) <L * (Q)
... Formula (4)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S162 when the equation (4) is satisfied, and proceeds to step S166 when the equation (4) is not satisfied.

B2: It is determined whether or not the color difference (ΔE *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T1) (S162).

ΔE * (P, Q)> T1
... Formula (5)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S163 when the equation (5) is satisfied, and proceeds to step S166 when the equation (5) is not satisfied.

B3: It is determined whether or not the saturation difference (ΔC *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold (T2) (S163).

ΔC * (P, Q) <T2
... Formula (6)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S164 when the equation (6) is satisfied, and proceeds to step S166 when the equation (6) is not satisfied.
B4: It is determined whether or not the saturation (C *) of the pixel value (P) of the original image is larger than the set threshold (T3) (S164).

C * (P)> T3
... Formula (7)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S165 when the equation (7) is satisfied, and proceeds to step S166 when the equation (7) is not satisfied.
In step S165, the damaged part detection image [1] generation unit 200 sets 1 in the register as a value indicating that there is a damaged part.
In step S166, the damaged portion detection image [1] generation unit 200 sets 0 in the register as a value indicating that there is no damaged portion.

The damaged part detection image [1] generation unit 200 detects a black broken part as a black broken part in the black broken part detection process.
The damaged portion detection image [1] generation unit 200 uses the most common CIE 1976 color difference calculated by the following equation for the color difference (ΔE *) of the identification condition B2.

・・・式（８）

別の色差表現として、より人間の目の色彩感覚に近いＣＩＥ２０００色差（ΔＥ^＊ _００）もあり、それを用いても良い。
彩度差ΔＣ＊は以下の式で計算される。

... Formula (8)

As another color difference expression, there is CIE2000 color difference (ΔE ^* ₀₀ ) closer to the color sense of human eyes, and this may be used.
The saturation difference ΔC * is calculated by the following formula.

・・・式（９）
黒破損部検知処理における各識別条件の可変閾値は、背景評価画像Ｇ１１０の画素値の明度、および平坦評価画像Ｇ１２０から決定した補正値の関数として設定している。
具体的には、背景評価画像Ｇ１１０の画素値の明度が低い領域、要するに背景が暗い領域に関しては、破損部検知が難しいので検知に用いる閾値を比較的ゆるく設定している。
破損部検知画像［１］生成部２００は、黒破損部検知処理（Ｓ１６０）において、破損部であると判断された場合は、破損部検知画像［１］Ｇ２００の当該画素に対するフラグをｔｒｕｅに設定（Ｓ１８０）し、次の画素に処理を移す（Ｓ１９０）。一方、検知の識別条件に一つでも当てはまらなかった場合は、白破損部検知処理（Ｓ１７０）に処理を移す。

... Formula (9)
The variable threshold value of each identification condition in the black broken part detection process is set as a function of the brightness of the pixel value of the background evaluation image G110 and the correction value determined from the flat evaluation image G120.
Specifically, in a region where the brightness of the pixel value of the background evaluation image G110 is low, in other words, in a region where the background is dark, it is difficult to detect a damaged portion, so the threshold used for detection is set relatively loose.
The damaged part detection image [1] generation unit 200 sets the flag for the pixel in the damaged part detection image [1] G200 to true when it is determined that the damaged part is detected in the black damaged part detection process (S160). (S180), and the process proceeds to the next pixel (S190). On the other hand, if even one of the detection identification conditions is not met, the process proceeds to the white broken part detection process (S170).

<White breakage detection processing>
FIG. 11 is a flowchart for explaining a subroutine of white broken part detection processing by the broken part detection image [1] generation unit 200 shown in FIG.
The damaged portion detection image [1] generation unit 200 sets identification conditions W1 to W5 in the subroutine of the white broken portion detection process (S170).
W1: It is determined whether the pixel value (P) of the original image has a lightness (L *) greater than the pixel value (Q) of the background evaluation image G110 (S171).

L * (P)> L * (Q)
... Formula (10)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S172 when the equation (10) is satisfied, and proceeds to step S176 when the equation (10) is not satisfied.

W2: It is determined whether or not the color difference (ΔE *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is greater than the set threshold (T4) (S172).

ΔE * (P, Q)> T4
... Formula (11)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S173 when the equation (11) is satisfied, and proceeds to step S176 when the equation (11) is not satisfied.

W3: It is determined whether or not the saturation difference (ΔC *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T5).

ΔC * (P, Q) <T5
... Formula (12)
And,
W4: It is determined whether the saturation (C *) of the pixel value (P) of the original image is smaller than the set threshold value (T6) (S173).

C * (P) <T6
... Formula (13)
Here, the damaged part detection image [1] generation unit 200 proceeds to step S175 when the expressions (12) and (13) are satisfied, and proceeds to step S175 when the expressions (12) and (13) are not satisfied. Proceed to step S174.

W5: It is determined whether or not the brightness difference (ΔL *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold (T7) (S174).

ΔL * (P, Q)> T7
... Formula (14)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S175 when the equation (14) is satisfied, and proceeds to step S176 when the equation (14) is not satisfied.
In step S175, the damaged part detection image [1] generation unit 200 sets 1 in the register as a value indicating that there is a damaged part.
In step S176, the damaged portion detection image [1] generation unit 200 sets 0 in the register as a value indicating that there is no damaged portion.

In order to avoid erroneous detection of the background pattern, the white broken portion detection processing detects W1 and W2 as essential identification conditions and detects those that fall under W3, W4, or W5 as broken portions.
Since the white damaged part is mainly caused by peeling of the paint, it does not depend on the brightness of the background evaluation image G110 and the correction value by the flat evaluation image G120, and the detection threshold values of each identification condition are all constant values. . When unevenness in detection occurs depending on the brightness of the background area, the threshold value for each identification condition may be set variably as in the case of black breakage detection.
If the damaged part detection image [1] generation unit 200 determines that the pixel is a damaged part pixel in the white broken part detection process (S170), the pixel of the damaged part detection image [1] G200 is set to true ( Then, the process proceeds to the next pixel (S190). On the other hand, if the damaged part detection image [1] generation unit 200 does not satisfy the detection identification condition, the process proceeds to the next pixel as it is (S190).
By applying the above-described damaged portion determination processing to all the pixels, the damaged portion detection image [1] G200 is generated.

<Damage detection image [2] generation process>
FIG. 12 is a flowchart for explaining a damaged portion detection image [2] generation process by the damaged portion detection image [2] generation unit 210 shown in FIG.
The damaged portion detection image [2] generation unit 210 sets a processing area for the damaged portion detection image [1] G210 stored in the RAM 12 (S200a).
The damaged part detection image [2] generation unit 210 performs an expansion process on the area detected as the damaged part in order to finally pass the damaged part detected image [1] G200 to the repair process, and generates the damaged part detection image [ 2] G210 is generated (S201).

Hereinafter, the expansion process performed by the damaged portion detection image [2] generation unit 210 illustrated in FIG. 5 will be described with reference to the graph illustrated in FIG. 13.
A broken line L202 indicates a change in the pixel value, and a dotted line L203 is a threshold value for the damaged portion detection process. Then, the region surrounded by the two vertical lines L204 is detected as a damaged portion, and is interpolated as a horizontal line L205 at the repair processing stage. The horizontal line 205 generally has a higher pixel value than the outer polygonal line, and the repaired portion becomes unnatural.
In order to prevent this, it is possible to interpolate with a pixel value closer to the background during the repair process by slightly expanding the damaged part.

The number of expansions can be adjusted depending on the resolution of the damaged part. If the number of expansions is too large, the non-damaged part to be referred to will be reduced during interpolation at the repair stage, so the image will be blurred (blurred) as a whole, so care must be taken. is there.
Specifically, the background evaluation image generation unit 110 may have a number of pixels that is about 1/10 of the size of the smoothing filter used when generating the background evaluation image G110.
Moreover, the detection with respect to the grain pattern of the surface which cut out the timber is mentioned as an application example of the damage part detection process using color difference (DELTA) E * in L * a * b * space.
It is also possible to generate an output image with a weakened wood pattern or an output image with a stronger wood pattern by detecting the background and color difference ΔE * based on the background and performing appropriate restoration processing. It is.

<Damage Repair Image [1] Generation Processing>
FIG. 14 is a flowchart for explaining the damaged portion repair image [1] generation processing by the damaged portion repair image [1] generation unit 300 shown in FIG.
The damaged portion repaired image [1] generation unit 300 sets a processing area for the damaged portion detection image [2] G210 stored in the RAM 12 (S210b).
Next, the damaged part repaired image [1] generation unit 300 designates the start address of the RAM 12 that stores the damaged part detection image [2] G210, and starts loop processing for each pixel (S211).
First, the damaged portion repaired image [1] generation unit 300 determines whether or not the target pixel is a damaged portion (S220).
Here, the damaged part repaired image [1] generation unit 300 proceeds to step S240 when the target pixel is a damaged part, and proceeds to step S275 when the target pixel is not a damaged part.

When the pixel of interest is a damaged portion, the damaged portion repaired image [1] generation unit 300 has a background evaluation pixel list that represents a pixel value, a distance between the pixels of interest, and the like regarding the pixel of the background evaluation image G110 at the position of the pixel of interest Is acquired (S240).
Next, the damaged portion repaired image [1] generation unit 300 interpolates the target pixel value from the background evaluation pixel list for the pixels of the acquired background evaluation image G110 (S250).
Next, the damaged portion repaired image [1] generation unit 300 performs brightness correction on the interpolated pixel value of interest (S260).

Next, the damaged portion repaired image [1] generation unit 300 performs noise addition correction on the target pixel value whose brightness has been corrected (S260).
Next, when the damaged portion repaired image [1] generation unit 300 determines that the target pixel is a damaged portion, the damaged portion repaired image [1] generation unit 300 determines whether or not the loop processing has ended (S275). If it is determined that the loop process has been completed, the process proceeds to step S300a. On the other hand, if it is determined that the loop process has not been completed, the process proceeds to step S230.
In step S230, since the damaged portion repaired image [1] generation unit 300 has not finished the loop process, the address of the target pixel is incremented by one to set the next pixel as the target pixel, and the process returns to step S220.
In step S300, the damaged portion repair image [1] generation unit 300 stores the damaged portion repair image [1] G300 generated on the RAM 12 in the HDD 14 (S200).

<Damage detection image [2]>
FIG. 15A is a diagram illustrating a 10 × 10 image example representing each pixel of the damaged portion detection image [2], and FIG. 15B is a diagram illustrating a background evaluation pixel list.
The background evaluation pixel list (FIG. 15 (b)) is obtained at the pixel (target pixel) of a different color in FIG. 15 (a).
The damaged portion repaired image [1] generation unit 300 scans pixels in the vertical and horizontal directions around the target pixel, and reaches a pixel that is not a damaged portion (a pixel whose flag is false in the damaged portion detection image). The pixel value (Value) of the original image G100 at that position and the distance (Distance) between the target pixels are stored in the background evaluation pixel list.

In the image example shown in FIG. 15A, as shown in FIG. 15B, information about the pixel A and the pixel C is stored in the background evaluation pixel list (L).
In addition, the following two situations are dealt with as exception processing by the damaged portion repaired image [1] generation unit 300.
[1] When a damaged part continues long in one direction and a pixel that is not a damaged part is not found [2] When the edge of the image is reached

The damaged portion repaired image [1] generation unit 300 sets a maximum scanning distance in [1], and if a pixel that is not a damaged portion is not found by performing an operation up to that distance, the background at that distance (maximum scanning distance) The pixel value (Value) and the maximum scanning distance (Distance) of the evaluation image G110 are stored in the background evaluation pixel list.
In the image example shown in FIG. 15A, information of the pixel B is stored in the background evaluation pixel list (L) as shown in FIG. 15B.
Similarly, in [2], the damaged portion repaired image [1] generating unit 300 scans the pixel value (Value) of the background evaluation image G110 at the position where the pixel reaches the edge pixel and the distance (Distance) between the target pixels. Are stored in the background evaluation pixel list.
In the image example shown in FIG. 15A, information of the pixel D is stored in the background evaluation pixel list (L) as shown in FIG. 15B.

Through the above processing, a background evaluation pixel list having four elements each of a value (Value) and a distance (Distance) is acquired.
As the maximum scanning distance, the size of the smoothing filter when generating the background evaluation image G110 is appropriate. The reason is that, if a value larger than that is set, background evaluation pixels of a portion having a different background are brought.
Next, in step S250 described above, the damaged portion repaired image [1] generation unit 300 interpolates the target pixel from the acquired background evaluation pixel list (S250). In the interpolation process, two-dimensional linear interpolation weighted by distance may be used.
The damaged portion repaired image [1] generation unit 300 calculates the interpolation value (V) according to the equations (15) and (16) based on the background evaluation pixel list (L) shown in FIG.

・・・式（１５）

... Formula (15)

・・・式（１６）

... Formula (16)

As described above, in order to make the repaired portion appear more natural for the interpolated pixel, two types of correction are performed for each pixel.
First, brightness correction was performed in step S260. This is because the closest pixel to be referred to when interpolating is close to the boundary of the damaged part, and therefore, by interpolating with a pixel whose brightness is slightly lower than that of the background pixel, the tendency of the brightness of the interpolated pixel of interest to be corrected is corrected. is there. The RGB value of the damaged portion repaired portion is increased by about 1 to 5%.
Second, noise addition correction was performed in step S270. Since the interpolated damaged portion is linear interpolation, the values of adjacent pixels change smoothly. On the other hand, for true background pixels that are not damaged, the values of adjacent pixels change somewhat randomly. In order to compensate for this difference and to express the reality to the pixels, the pixel values of all of the RGB in the region to be corrected are uniformly changed. The amount of noise is determined by calculating from the amount of difference between adjacent pixels of the target pixel. In addition, since there may be extremely large noise, the noise amount limit is set to about ± 5.

The reason why both the brightness correction (S260) and the noise addition correction (S270) are uniform in RGB is determined because the damaged portion has no difference in saturation from the background portion.
The repair process up to the correction process is applied to all pixels, and the damaged part repair image [1] G300 is generated. Further, a sufficient repaired image can be obtained without performing these two corrections.
In addition, when there is an impression that the repaired portion is darker than the background portion, lack of undulations, no inflection, or less unevenness, brightness correction (S260) and noise addition correction (S270) may be performed.
Further, as a variation of the restoration process, in the two-dimensional linear interpolation, scanning may be performed in an oblique direction in addition to the vertical and horizontal directions, and the interpolation process may be performed from a background evaluation pixel list having eight background evaluation pixel elements.
Alternatively, a damaged portion surrounding the target pixel may be defined, and the interpolation processing may be performed by adding all the pixel values to the background evaluation pixel list because the damaged portion is at the boundary.

As another example, when the expansion processing (S201) is performed on the damaged portion detection image [1] G200 [1], a pixel that has been damaged before the expansion processing and a pixel that has been newly damaged by the expansion processing On the other hand, the repair process can be freely changed.
For example, the restoration condition may be improved by restoring the pixel by normal two-dimensional linear interpolation for the former pixel and applying the pixel value of the background evaluation image G110 to the latter pixel.

<Damage Repair Image [2] Generation Unit>
FIG. 16 is a flowchart for explaining the damaged portion repair image [2] generation processing by the damaged portion repair image [2] generation unit 310 shown in FIG.
The damaged portion repaired image [2] generation unit 310 sets a processing area for the damaged portion repaired image [1] G300 stored in the RAM 12 (S300a).
Next, the damaged portion repaired image [2] generation unit 310 performs cross pattern reduction sharpening filter processing as post-processing in order to obtain the final damaged portion repaired image [2] G310 from the damaged portion repaired image [1] G300. I do. As a result, the repair process for obtaining the damaged part repair image [2] G310 is an improved version of the smoothing process in which the repair pixel value is calculated using the peripheral pixel value, so that the image is distorted. Giving to the viewer can be reduced by the sharpening process.

As described above, with the method of filling by linear interpolation with reference to the closest pixels in the upper, lower, left, and right directions (S240, S250), a cross-shaped pseudo pattern can be easily formed in the repaired image. When a simple sharpening process is performed, the cross pattern is conspicuous, resulting in an unnatural image.
In order to reduce such a cross pattern, in step S301, the damaged portion repaired image [2] generation unit 310 performs a cross pattern reduction sharpening filter process having the following anisotropy.

・・・（式１７）
上下左右方向を弱め、斜め方向を少し加える鮮鋭化処理によって、自然な修復画像が得られる。
フィルタの係数（鮮鋭化度、異方性の強さ）やフィルタサイズは変更しても良い。
また、修復画素の補間計算が、上下左右の最も近い破損部でない画素を参照して埋める方法でない場合は、単純な鮮鋭化フィルタでの処理でも良い。
以上により、破損部修復画像［２］生成部３１０は、破損部修復画像［２］Ｇ３１０が生成される。
ステップＳ３１０ｂでは、破損部修復画像［２］生成部３１０は、ＲＡＭ１２上に生成された破損部修復画像［２］をＨＤＤ１４に記憶する。

... (Formula 17)
A natural restoration image can be obtained by sharpening processing that weakens the vertical and horizontal directions and adds a little oblique direction.
The filter coefficient (sharpness, anisotropic strength) and filter size may be changed.
In addition, when the interpolation calculation of the repaired pixel is not a method of filling by referring to the pixel that is not the closest damaged part in the upper, lower, left, and right directions, a simple sharpening filter may be used.
As described above, the damaged portion repair image [2] generation unit 310 generates the damaged portion repair image [2] G310.
In step S <b> 310 b, the damaged part repair image [2] generation unit 310 stores the damaged part repair image [2] generated on the RAM 12 in the HDD 14.

In the present embodiment, the damaged portion repair image [2] G310 generated by the damaged portion repair image [2] generation unit 310 is output to the printer driver 26, and the printer driver 26 receives drawing information from the damaged portion repair image [2] G310. Generated and output to the image forming apparatus 2.
As a result, the damaged portion repair image [2] G310 in which the damaged portion is corrected is generated as the drawing information for the original image G100 having the damaged portion, and the drawing information is printed by the image forming apparatus 2. It is possible to meet the demand for viewing images before they occur.
In the above-described embodiment, the damaged portion repair image [2] G310 is printed using an ink jet printer as the image forming apparatus 2, but an electrophotography having a plurality of drums instead of the ink jet printer. The damaged part repair image [2] G310 may be printed using a printer.

<Configuration, operation and effect of exemplary embodiment of the present invention>
<First aspect>
The image processing apparatus according to this aspect is an image processing apparatus that detects a damaged portion that exists in the original image G100, and performs background processing that represents the evaluation of pixels that are the background of each pixel by performing smoothing processing on the original image G100. A background evaluation image generation unit 110 that generates an image G110, and a flat evaluation image generation that generates a flat evaluation image G120 that represents an evaluation related to flatness of the original image G100 by performing edge detection processing and smoothing processing on the original image G100. The damaged part that identifies the damaged part and generates the damaged part detection image [1] G200 based on the difference value between the part 120, the original image G100 and the background evaluation image G110, and the pixel value of the flat evaluation image G120 And a detected image [1] generation unit 200.
According to this aspect, the background evaluation image generation unit 110 generates a background evaluation image G110 that represents the evaluation of the pixel that is the background of each pixel by performing a smoothing process on the original image G100. The flat evaluation image generation unit 120 generates a flat evaluation image G120 representing the evaluation related to flatness of the original image G100 by performing edge detection processing and smoothing processing on the original image G100. The damaged part detection image [1] generation unit 200 identifies the damaged part based on the difference value between the original image G100 and the background evaluation image G110 and the pixel value of the flat evaluation image G120, and detects the damaged part detection image [ 1] G200 is generated.
As described above, by performing smoothing processing on the original image G100 and the original image G100, the difference value between the background evaluation image G110 representing the evaluation of the pixel serving as the background of each pixel, and the edge detection processing on the original image G100. By performing the smoothing process, the damaged portion is identified based on the pixel value of the flat evaluation image G120 representing the evaluation related to the flatness of the original image G100, and the damaged portion detection image [1] G200 is generated. It is possible to prevent a specific pattern included in the image having the same brightness difference as the damaged portion from being erroneously detected as a damaged portion.
For example, in order to reduce the erroneous detection of the background pattern by determining the damaged pixel based on the L * a * b * color difference between the original image G100 and the background evaluation image G110 generated based on the original image G100, the original image G100 Since the criterion for judging the damaged portion is adjusted from the flat evaluation image G120 generated based on the image, and the correction processing is applied to the original image G100 using the generated damaged portion detection image [1] G200, the damaged portion is generated. The damaged part can be removed from the existing painting, and the painting without the damaged part at that time can be reproduced. In addition, it is possible to make adjustments by parameters according to the size of the damaged portion, the degree of repair, and the like, and to try the repair condition. With respect to a region where there is a detection failure or erroneous detection of a damaged portion, a better effect can be obtained by performing filter processing using another parameter only in that region.

<Second aspect>
The background evaluation image generation unit 110 of this aspect generates a background evaluation image G110 by performing a smoothing process on the original image G100 using a median filter.
According to this aspect, the background evaluation image generation unit 110 generates the background evaluation image G110 by performing a smoothing process on the original image G100 using the median filter.
Thereby, the background evaluation image G110 can be generated by performing a smoothing process on the original image G100 using a median filter.

<Third aspect>
The flat evaluation image generation unit 120 of this aspect generates an adjacent difference image by performing edge detection processing on the original image G100 using a Sobel filter, and performs a Gaussian filter, an averaging filter, a median on the adjacent difference image. A flat evaluation image G120 is generated by performing a smoothing process using any one of the filters.
According to this aspect, the flat evaluation image generation unit 120 generates an adjacent difference image by performing edge detection processing on the original image G100 using a Sobel filter, and performs Gaussian filtering and averaging on the adjacent difference image. A flatness evaluation image G120 is generated by performing a smoothing process using any one of a filter and a median filter.
Thereby, an adjacent difference image is generated by performing edge detection processing on the original image G100 using a Sobel filter, and any one of a Gaussian filter, an averaging filter, and a median filter is used for the adjacent difference image. By performing the smoothing process, the flat evaluation image G120 can be generated.

<4th aspect>
The damaged portion detection image [1] generation unit 200 of this aspect distinguishes the background pixel from the damaged portion pixel based on the original image G100, the background evaluation image G110, and the flat evaluation image G120, and has at least binary information. Damaged part detection image [1] G200 is generated.
According to this aspect, the damaged part detection image [1] generation unit 200 distinguishes the background pixel from the damaged part pixel based on the original image G100, the background evaluation image G110, and the flat evaluation image G120, and at least binary. A damaged portion detection image [1] G200 having information is generated.
Thereby, based on the original image G100, the background evaluation image G110, and the flat evaluation image G120, the background pixel and the damaged portion pixel are distinguished, and the damaged portion detection image [1] G200 having at least binary information is generated. Can do.

<5th aspect>
The damaged portion detection image [1] generation unit 200 according to the present aspect performs an expansion process on a region that is a damaged portion pixel on the damaged portion detection image [1] G200 that stores at least binary information for each pixel. It is characterized by.
According to this aspect, the damaged portion detection image [1] generation unit 200 performs an expansion process on a region that is a damaged portion pixel with respect to the damaged portion detection image [1] G200 that stores at least binary information for each pixel. I do.
Thereby, the expansion process can be performed on the area that is the damaged part pixel with respect to the damaged part detection image [1] G200 in which at least binary information is stored for each pixel.

<Sixth aspect>
The damaged portion detection image [1] generation unit 200 according to this aspect is characterized in that each pixel of the damaged portion detection image [1] G200 is identified as two types of black damaged portion pixels or white damaged portion pixels.
According to this aspect, the damaged portion detection image [1] generation unit 200 identifies each pixel of the damaged portion detection image [1] G200 as two types of black damaged portion pixels or white damaged portion pixels.
Thereby, each pixel of damaged part detection image [1] G200 can be identified as two types, a black damaged part pixel or a white damaged part pixel.

<Seventh aspect>
The damaged portion detection image [1] generation unit 200 of this aspect is
As an identification condition used to identify a pixel having a damaged portion detection image [1] G200 as a black damaged portion pixel,
B1: The pixel value (P) of the original image G100 has a lightness (L *) smaller than the pixel value (Q) of the background evaluation image G110 L * (P) <L * (Q)
B2: The color difference (ΔE *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T1). ΔE * (P, Q)> T1
B3: The saturation difference (ΔC *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T2). ΔC * (P, Q) <T2
B4: The saturation (C *) of the pixel value (P) of the original image G100 is larger than the set threshold value (T3). C * (P)> T3
Have
When all of B1 to B4 are satisfied, a certain pixel is identified as a black damaged pixel.
According to this aspect, the damaged portion detection image [1] generation unit 200
As an identification condition used to identify a pixel having a damaged portion detection image [1] G200 as a black damaged portion pixel,
B1: The pixel value (P) of the original image G100 has a lightness (L *) smaller than the pixel value (Q) of the background evaluation image G110 L * (P) <L * (Q)
B2: The color difference (ΔE *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T1). ΔE * (P, Q)> T1
B3: The saturation difference (ΔC *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T2). ΔC * (P, Q) <T2
B4: The saturation (C *) of the pixel value (P) of the original image G100 is larger than the set threshold value (T3). C * (P)> T3
Have
When all of B1 to B4 are satisfied, a certain pixel is identified as a black damaged pixel.
As a result, when an identification condition used to identify a pixel in the damaged portion detection image [1] G200 as a black damaged portion pixel is satisfied, all pixels B1 to B4 are satisfied. Can be identified.

<Eighth aspect>
The damaged portion detection image [1] generation unit 200 of this aspect is
As an identification condition used to identify a pixel having a damaged portion detection image [1] G200 as a white damaged portion pixel,
W1: The pixel value (P) of the original image G100 is lighter (L *) than the pixel value (Q) of the background evaluation image G110. L * (P)> L * (Q)
W2: The color difference (ΔE *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T4). ΔE * (P, Q)> T4
W3: The saturation difference (ΔC *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T5). ΔC * (P, Q) <T5
W4: The saturation (C *) of the pixel value (P) of the original image G100 is smaller than the set threshold value (T6). C * (P) <T6
W5: The brightness difference (ΔL *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T7). ΔL * (P, Q)> T7
Have
If W1 and W2 are satisfied and W3 and W4 are satisfied, or W1 and W2 are satisfied and W. 5 is satisfied, a certain pixel is identified as a white damaged pixel.
According to this aspect, the damaged portion detection image [1] generation unit 200
As an identification condition used to identify a pixel having a damaged portion detection image [1] G200 as a white damaged portion pixel,
W1: The pixel value (P) of the original image G100 is lighter (L *) than the pixel value (Q) of the background evaluation image G110. L * (P)> L * (Q)
W2: The color difference (ΔE *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T4). ΔE * (P, Q)> T4
W3: The saturation difference (ΔC *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T5). ΔC * (P, Q) <T5
W4: The saturation (C *) of the pixel value (P) of the original image G100 is smaller than the set threshold value (T6). C * (P) <T6
W5: The brightness difference (ΔL *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T7). ΔL * (P, Q)> T7
Have
If W1 and W2 are satisfied and W3 and W4 are satisfied, or W1 and W2 are satisfied and W. 5 is satisfied, a certain pixel is identified as a white damaged portion pixel. Thus, the identification condition used to identify a certain pixel in the damaged portion detection image [1] G200 as a white damaged portion pixel is described above. Satisfy W1 and W2 and satisfy W3 and W4, or satisfy W1 and W2 and When 5 is satisfied, a certain pixel can be identified as a white damaged pixel.

<Ninth aspect>
The damaged part detection image [1] generation unit 200 according to the present aspect corrects each setting threshold related to each identification condition based on the brightness of the background evaluation image G110 and the pixel value of the flat evaluation image G120. .
According to this aspect, the damaged portion detection image [1] generation unit 200 corrects each setting threshold value related to each identification condition based on the brightness of the background evaluation image G110 and the pixel value of the flat evaluation image G120.
Thereby, each setting threshold value related to each identification condition can be corrected based on the brightness of the background evaluation image G110 and the pixel value of the flat evaluation image G120.

<10th aspect>
The damaged portion detection image [1] generation unit 200 of this aspect does not correspond to the identification condition related to the black damaged portion pixel or the identification condition related to the white damaged portion pixel for a pixel in the damaged portion detected image [1] G200. In some cases, a certain pixel is used as a background pixel.
According to this aspect, the damaged part detection image [1] generation unit 200 corresponds to the identification condition related to the black damaged part pixel or the identification condition related to the white damaged part pixel in which the damaged part detected image [1] G200 is located. If not, a certain pixel is set as a background pixel.
As a result, when a pixel in the damaged portion detection image [1] G200 does not meet the identification condition related to the black damaged portion pixel or the identification condition related to the white damaged portion pixel, the certain pixel is set as the background pixel. it can.

<Eleventh aspect>
The image processing method according to this aspect is an image processing method by an image processing apparatus that detects a damaged portion in the original image G100, and performs smoothing processing on the original image to evaluate the pixel that is the background of each pixel. A background evaluation image generation step (S100a to S110a) for generating a background evaluation image G110 to be expressed, and a flat evaluation image G120 that represents an evaluation related to flatness of the original image G100 by performing edge detection processing and smoothing processing on the original image G100. The damaged portion region is identified and damaged based on the flat evaluation image generation step (S100b to S120a) for generating the image, the difference value between the original image G100 and the background evaluation image G110, and the pixel value of the flat evaluation image G120. A damaged portion detection image generation step (S100c to S200) for generating a portion detection image [1] G200; That.
Since the operation and effect of the eleventh aspect are the same as those of the first aspect, description thereof will be omitted.

24 ... Image processing unit, 100 ... Original image acquisition unit, 110 ... Background evaluation image generation unit, 120 ... Flat evaluation image generation unit, 200 ... Damaged part detection image [1] generation part, 210 ... Damaged part detection image [2] Generation unit 300 ... Damaged part repair image [1] generation unit 310 ... Damaged part repair image [2] generation unit

Japanese Patent No. 3877916

Claims (11)

An image processing apparatus for detecting a damaged portion existing in an original image,
Background evaluation image generation means for generating a background evaluation image representing an evaluation of a pixel as a background of each pixel by performing a smoothing process on the original image;
Flat evaluation image generation means for generating a flat evaluation image representing evaluation relating to flatness of the original image by performing edge detection processing and smoothing processing on the original image;
Damaged part detection image generating means for identifying a damaged part and generating a damaged part detection image based on a difference value between the original image and the background evaluation image and a pixel value of the flat evaluation image;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the background evaluation image generation unit generates the background evaluation image by performing a smoothing process on the original image using a median filter.   The flat evaluation image generation means generates an adjacent difference image by performing edge detection processing on the original image using a Sobel filter, and applies a Gaussian filter, an averaging filter, and a median filter to the adjacent difference image. The image processing apparatus according to claim 1, wherein the flatness evaluation image is generated by performing a smoothing process using any one of them.   The damaged portion detection image generating means distinguishes a background pixel and a damaged portion pixel based on the original image, the background evaluation image, and the flatness evaluation image, and generates a damaged portion detection image having at least binary information. The image processing apparatus according to claim 1, wherein:   2. The damaged part detection image generation unit performs expansion processing on an area that is a damaged part pixel on the damaged part detection image that stores at least binary information for each pixel. Image processing apparatus.   The image processing apparatus according to claim 5, wherein the damaged portion detection image generation unit identifies each pixel of the damaged portion detection image into two types of black damaged portion pixels and white damaged portion pixels. The damaged portion detection image generating means is
As an identification condition used to identify a pixel having the damaged portion detection image as a black damaged portion pixel,
B1: The pixel value (P) of the original image has a lightness (L *) smaller than the pixel value (Q) of the background evaluation image L * (P) <L * (Q)
B2: The color difference (ΔE *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is greater than the set threshold value (T1) ΔE * (P, Q)> T1
B3: The saturation difference (ΔC *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is smaller than the set threshold value (T2). ΔC * (P, Q) <T2
B4: The saturation (C *) of the pixel value (P) of the original image is larger than the set threshold value (T3). C * (P)> T3
Have
The image processing apparatus according to claim 6, wherein the pixel is identified as a black damaged pixel when all of the B1 to B4 are satisfied. The damaged portion detection image generating means is
As an identification condition used to identify a pixel having the damaged portion detection image as a white damaged portion pixel,
W1: The pixel value (P) of the original image is lighter (L *) than the pixel value (Q) of the background evaluation image L * (P)> L * (Q)
W2: The color difference (ΔE *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is larger than the set threshold value (T4) ΔE * (P, Q)> T4
W3: The saturation difference (ΔC *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is smaller than the set threshold value (T5). ΔC * (P, Q) <T5
W4: The saturation (C *) of the pixel value (P) of the original image is smaller than the set threshold value (T6). C * (P) <T6
W5: The lightness difference (ΔL *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is larger than the set threshold value (T7). ΔL * (P, Q)> T7
Have
When W1 and W2 are satisfied and W3 and W4 are satisfied, or when W1 and W2 are satisfied and W. 7. The image processing apparatus according to claim 6, wherein when the condition 5 is satisfied, the certain pixel is identified as a white damaged pixel.   The said damaged part detection image generation means correct | amends each setting threshold value which concerns on each said identification condition based on the brightness of the said background evaluation image, and the pixel value of the said flat evaluation image, The said threshold value is characterized by the above-mentioned. An image processing apparatus according to 1.   The damaged portion detection image generation means is configured to detect the pixel when the pixel of the damaged portion detection image does not satisfy the identification condition related to the black damaged portion pixel or the identification condition related to the white damaged portion pixel. The image processing apparatus according to claim 7, wherein the image processing apparatus is a background pixel. An image processing method by an image processing apparatus for detecting a damaged portion existing in an original image,
A background evaluation image generating step for generating a background evaluation image representing an evaluation of a pixel as a background of each pixel by performing a smoothing process on the original image;
A flat evaluation image generating step for generating a flat evaluation image representing evaluation relating to flatness of the original image by performing edge detection processing and smoothing processing on the original image;
A damaged part detection image generation step for identifying a damaged part region and generating a damaged part detection image based on a difference value between the original image and the background evaluation image and a pixel value of the flat evaluation image;
The image processing method characterized by performing.

Images