JP6798229B2 - Image processing device and image processing method - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method.

Paintings drawn with oil paints had cracks, cracks, cracks, and other damaged parts on the surface due to aging, which spoiled the appearance of the paintings. For this reason, there has been a request to appreciate the image before the damaged portion is generated.
Therefore, conventionally, there is known an image processing technique in which a damaged portion is detected from an image of a painting in which a damaged portion is generated and the detected damaged portion is repaired.
In Patent Document 1, for the purpose of removing abnormal parts such as dust and dirt from an image acquired via a scanner, an abnormal part is detected from a background image by using an image processing technique, and the abnormal part is repaired. The method is disclosed.

However, in the conventional image processing technique, it is determined that a pixel in which the change value between adjacent pixels is extremely large compared to the average value of the peripheral pixel set is an abnormal portion.
Therefore, for example, in an image of a person's painting in which a damaged part is generated, there is a problem that a specific pattern formed by the lace material is erroneously detected as a damaged part in a place where a lace material is attached against the background of the person's clothes. there were.
Patent Document 1 discloses that noise such as a damaged portion is removed by using image processing. However, the problem of erroneously detecting a specific pattern included in the background image having a difference in brightness similar to that of the damaged portion as the damaged portion has not been solved.
The present invention has been made in view of the above, and an object of the present invention is to prevent erroneous detection of a specific pattern included in a background image having a difference in brightness similar to that of a damaged portion as a damaged portion. ..

The invention according to claim 1 is an image processing device that detects a damaged portion existing in the original image in order to solve the above problem, and is a pixel that becomes a background of each pixel by performing smoothing processing on the original image. A background evaluation image generating means for generating a background evaluation image representing the evaluation of the above, and a flatness that generates a flatness evaluation image representing the evaluation related to the flatness of the original image by performing edge detection processing and smoothing processing on the original image. A damaged portion detection image that identifies a damaged portion and generates a damaged portion detection image based on the difference value between the evaluation image generating means, the original image and the background evaluation image, and the pixel value of the flat evaluation image. It is characterized by comprising a generation means.

According to the present invention, it is possible to prevent erroneous detection of a specific pattern included in a background image having a difference in brightness similar to that of a damaged portion as a damaged portion.

It is a figure which shows the information processing apparatus which concerns on one Embodiment of this invention, and the image forming apparatus. It is a figure which shows the hardware configuration of the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the functional structure of the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the functional structure of the image forming apparatus which concerns on one Embodiment of this invention. It is a functional block diagram which shows the functional structure of the image processing part which concerns on one Embodiment of this invention. It is a flowchart for demonstrating the background evaluation image generation processing by the background evaluation image generation part shown in FIG. (A) is a diagram showing the configuration of a median filter, and (b) is a diagram showing the configuration of a Gaussian filter. It is a flowchart for demonstrating the flat evaluation image generation processing by the flat evaluation image generation part shown in FIG. It is a flowchart for demonstrating the damage part detection image [1] generation process by the damage part detection image [1] generation part shown in FIG. It is a flowchart for demonstrating the subroutine of the black break part detection processing by the damage part detection image [1] generation part shown in FIG. It is a flowchart for demonstrating the subroutine of the white damage part detection processing by the damage part detection image [1] generation part shown in FIG. It is a flowchart for demonstrating the damage part detection image [2] generation process by the damage part detection image [2] generation part shown in FIG. FIG. 5 is a graph for explaining the expansion process by the damaged portion detection image [2] generation portion shown in FIG. It is a flowchart for demonstrating the damage part repair image [1] generation process by the damage part repair image [1] generation part shown in FIG. (A) is a diagram showing an image example showing each pixel of the damaged portion detection image [2], and (b) is a diagram showing a background evaluation pixel list. It is a flowchart for demonstrating the damage part repair image [2] generation process by the damage part repair image [2] generation part shown in FIG. It is a figure which shows the original image. It is a figure which shows the background evaluation image generated from the original image. It is a figure which shows the flat evaluation image generated from the original image. It is a figure which shows the damaged part detection image [1] generated from the original image and the flatness evaluation image. It is a figure which shows the broken part detection image [2] generated from the broken part detection image [1]. It is a figure which shows the damaged part repair image [1] generated from the original image and the damaged part detection image [2]. It is a figure which shows the broken part repair image [2] generated from the broken part repair image [1].

Hereinafter, the present invention will be described in detail with reference to the embodiments shown in the drawings.
The present invention has the following configuration in order to prevent erroneous detection of a specific pattern included in the background image having a difference in brightness similar to that of the damaged portion as the damaged portion.
That is, the image processing device of the present invention is an image processing device that detects a damaged portion existing in the original image, and is a background evaluation that represents the evaluation of the pixels that are the background of each pixel by performing smoothing processing on the original image. A background evaluation image generation means for generating an image, a flat evaluation image generation means for generating a flat evaluation image representing an evaluation related to the flatness of the original image by performing edge detection processing and smoothing processing on the original image, and an original image. It is characterized by comprising a damaged portion detection image generation means for identifying a damaged portion and generating a damaged portion detection image based on a difference value between the image and the background evaluation image and a pixel value of the flat evaluation image. ..
By providing the above configuration, it is possible to prevent erroneous detection of a specific pattern included in the background image having a difference in brightness similar to that of the damaged portion as the damaged portion.
The above-mentioned features of the present invention will be described in detail below with reference to the drawings.

An embodiment of the present invention will be described with reference to the drawings.
<Information processing device and image forming device>
FIG. 1 is a diagram showing an information processing device 1 and an image forming device 2 according to an embodiment of the present invention.
As shown in FIG. 1, the information processing device 1 and the image forming device 2 are connected via a network. As a specific example of the information processing device 1 and the image forming device 2, the information processing device 1 is a general information processing terminal such as a PC (Personal Computer), and the image forming device 2 is an inkjet used at home or the like. In addition to the case of a printer, the image forming apparatus 2 is a printing machine used in commercial printing, and the information processing apparatus 1 is a control information processing terminal for controlling such a commercial printing machine. There are cases.

The information processing device 1 is a general-purpose information processing device such as a PC. A software program for configuring a printer driver that realizes a function for causing the image forming apparatus 2 to execute an image forming output is installed in the information processing apparatus 1.

<Information processing device>
The hardware configuration of the information processing apparatus 1 according to the embodiment of the present invention will be described with reference to FIG. As shown in FIG. 2, the information processing device 1 includes a configuration similar to that of a general server, PC, or the like. That is, in the information processing device 1, the CPU (Central Processing Unit) 11, the RAM (Random Access Memory) 12, the ROM (Read Only Memory) 13, the HDD (Hard Disk Drive) 14, and the I / F 15 are connected via the bus 18. Has been done. Further, an LCD (Liquid Crystal Display) 16 and an operation unit 17 are connected to the I / F 15.

The CPU 11 controls the operation of the entire information processing device 1. The RAM 12 is a volatile storage medium capable of reading and writing information at high speed, and is used as a work area when the CPU 11 processes the information.
The ROM 13 is a read-only non-volatile storage medium, and stores programs such as firmware. The HDD 14 is a non-volatile storage medium capable of reading and writing information, and stores an OS (Operating System), various control programs, application programs, and the like. The software program for configuring the printer driver described above is also stored in the HDD 14.

The I / F 15 connects and controls the bus 18 with various hardware, networks, and the like. The LCD 16 is a visual user interface for the user to confirm the state of the information processing device 1. The operation unit 17 is a user interface such as a keyboard and a mouse for the user to input information to the information processing device 1.

In such a hardware configuration, the software control unit reads a program stored in a storage medium such as a ROM 13, an HDD 14, or an optical disk, writes it to the RAM 12, and performs an operation according to the program read from the RAM 12 by the CPU 11. It is composed. The function of the information processing apparatus 1 is realized by the combination of the software control unit configured in this way and the hardware.

<Functional configuration of information processing device>
Next, the functional configuration of the information processing apparatus 1 according to the embodiment of the present invention will be described.
As shown in FIG. 3, the information processing device 1 includes a controller 20 and an external I / F 42. The external I / F 42 is an interface for the information processing device 1 to exchange information with the image forming device 2, and an Ethernet (registered trademark) or USB (Universal Serial Bus) interface is used.

The controller 20 is a software control unit configured as described above, and includes an application 22, a communication control unit 40, and a printer driver 26.
The application 22 is software that realizes a function for viewing and editing image data, document data, and the like.
The application 22 has an image processing unit 24, and the image processing unit 24 obtains an image in which the damaged portion is finally repaired from the original image.

The application 22 outputs a print instruction such as image data or document data being viewed or edited according to the user's operation. The communication control unit 40 performs a process for the controller 20 to exchange information with the image forming apparatus 2 via the external I / F 42.

The printer driver 26 receives a print instruction from the application 22 and generates drawing information which is information for the image forming apparatus 2 to execute the image forming output. At this time, the printer driver 26 generates drawing information according to the characteristics of the image forming mechanism included in the image forming apparatus 2 so that a highly accurate image as intended is formed.

<Printer driver>
As shown in FIG. 3, the printer driver 26 includes a print instruction acquisition unit 28, a drawing information generation unit 30, a drawing information transfer unit 32, a dot pattern generation unit 34, a scanned image acquisition unit 36, and a nozzle characteristic determination unit 38. The function of the printer driver 26 will be described in detail later.

<Image forming device>
The image forming apparatus 2 includes an inkjet type image forming mechanism, and executes an image forming output according to drawing information input from the information processing apparatus 1.
The functional configuration of the image forming apparatus 2 according to the embodiment of the present invention will be described with reference to FIG. As shown in FIG. 4, the image forming apparatus 2 includes a controller 70, an operation panel 30, a carriage 50, a main scanning motor 56, a sub scanning motor 60, a conveyor belt 62, and a charging roller 66.

The operation panel 30 is a user interface that functions as an operation unit and a display unit for inputting and displaying information necessary for the image forming apparatus 1. The carriage 50 is equipped with a recording head 54 for ejecting ink and a head driver 52 for driving the recording head 54, and is perpendicular to the sub-scanning direction, which is the paper conveying direction, with respect to the paper conveyed by the conveying belt 62. Ink is ejected to the front surface of the paper to output an image by being moved in the main scanning direction.

The main scanning motor 56 is a motor that supplies power for moving the carriage 50 in the main scanning direction. The sub-scanning motor 60 is a motor that supplies power to a transport belt 62 that conveys paper that is an image output target. The rotations of the main scanning motor 56 and the sub scanning motor 60 are detected by the encoder sensor 58 and the encoder sensor 64, respectively, and the detection signals are input to the controller 70. The charging roller 66 charges the transport belt 62 to generate an electrostatic force for attracting the paper to be output of the image to the transport belt 62.

The controller 20 is a control unit that controls the operation of the image forming apparatus 1, and as shown in FIG. 2, the CPU (Central Processing Unit) 201, the ROM (Read Only Memory) 202, the RAM (Random Access Memory) 203, and the NVRAM. (Non Volatile RAM) 204, ASIC (Application Specific Integrated Circuit) 205, host I / F 82, print control unit 84, motor drive unit 86, AC bias supply unit 88 and I / O 90 are included.

The CPU 72 controls the operation of each part of the controller 70. The ROM 74 is a read-only non-volatile storage medium, and stores programs such as firmware. The RAM 76 is a volatile storage medium capable of reading and writing information at high speed, and is used as a work area when the CPU 72 processes the information. The NVRAM 78 is a non-volatile storage medium capable of reading and writing information, and stores a control program and control parameters.

The ASIC 80 is a hardware circuit that executes image processing necessary for image formation output. The host I / F 82 is an interface for receiving drawing data from the information processing device 1, and an Ethernet (registered trademark) or USB (Universal Serial Bus) interface is used. The I / O 90 is a port for inputting detection signals from various sensors such as encoder sensors 58 and 28 to the controller 70.

The print control unit 84 includes a data transfer means for driving and controlling the recording head 54 included in the carriage 50, and a driving waveform generating means for generating a driving waveform. The motor drive unit 86 drives the main scanning motor 56 and the sub-scanning motor 60. The AC bias supply unit 88 supplies the AC bias to the charging roller 66.

As described above, the drawing data input from the information processing device 1 is input to the host I / F 82 in the controller 70 and stored in the reception buffer in the host I / F. The CPU 72 reads and analyzes the drawing data in the receive buffer included in the host I / F 82 by performing an operation according to the program loaded in the RAM 76, and controls the ASIC 80 to perform necessary image processing and data sorting processing. And so on. After that, the CPU 72 controls the print control unit 84 to transfer the drawing data processed by the ASIC 80 to the head driver 52.

The print control unit 84 transfers the above-mentioned drawing data as serial data to the head driver 52, and also transfers a transfer clock, a latch signal, a drop control signal (mask signal), and the like necessary for transferring the drawing data and confirming the transfer. Output to the head driver 52. Further, the print control unit 84 is used as a drive waveform generator and a head driver 52 composed of a D / A converter that D / A-converts the pattern data of the drive signal stored in the ROM 74, a voltage amplifier, a current amplifier, and the like. The drive waveform selection means to be given is included, and a drive waveform composed of one drive pulse (drive signal) or a plurality of drive pulses (drive signals) is generated and output to the head driver 52.

The head driver 52 generates energy for ejecting droplets from the recording head 54 as a drive signal constituting a drive waveform given from the print control unit 84 based on the drawing data for one line continuously input. The recording head 54 is driven by selectively applying it to the driving element. At this time, the head driver 52 selects dots that form a drive waveform, so that dots having different sizes, such as large droplets (large dots), medium droplets (medium dots), and small droplets (small dots), are used. Can be separated.

Further, the CPU 72 has a speed detection value and a position detection value obtained by sampling detection pulses from an encoder sensor 58 constituting a linear encoder, and a speed target value and a position target value obtained from a speed / position profile stored in advance. The drive output value (control value) for the main scanning motor 56 is calculated based on the above, and the main scanning motor 56 is driven via the motor drive unit 86. Similarly, the CPU 72 has a speed detection value and a position detection value obtained by sampling a detection pulse from an encoder sensor 64 constituting a rotary encoder, and a speed target value and a position target value obtained from a speed / position profile stored in advance. The drive output value (control value) for the sub-scanning motor 60 is calculated based on the above, and the sub-scanning motor 60 is driven via the motor drive unit 86 and the motor driver.

<Printer driver operation>
The image formation output is executed by interlocking the information processing device 1 and the image forming device 2 as described above. That is, the print instruction from the application 22 operating in the information processing apparatus 1 is processed by the printer driver 26 to generate a multi-valued recording dot pattern that can be output by the image forming apparatus 2, which is rasterized and used as drawing data. It is transferred to the image forming apparatus 2, and the image forming output is executed in the image forming apparatus 2.

Specifically, in the information processing device 1, an image drawing or character recording command from the application 22 or the operating system (for example, a description of the position, thickness, and shape of a line to be recorded, or a typeface of characters to be recorded) The size and position are described) is acquired by the print instruction acquisition unit 28, and is temporarily stored in the drawing data memory. It should be noted that these instructions are written in a specific print language.

Then, the instruction stored in the drawing data memory is interpreted by the rasterizer included in the drawing information generation unit 30, and if it is a line recording instruction, it is converted into a recording dot pattern according to a designated position, thickness, and the like. If it is a character recording command, the outline information of the corresponding character is called from the font outline data stored in the information processing device 1 and converted into a recording dot pattern according to the specified position and size, and the image is displayed. If it is data, it is directly converted into a recording dot pattern.

After that, the drawing information generation unit 30 performs image processing on these recorded dot patterns and stores them in the raster data memory. At this time, the information processing apparatus 1 rasterizes the data of the recording dot pattern with the orthogonal grid as the basic recording position. Image processing includes, for example, color management processing and γ correction processing for adjusting colors, halftone processing such as dither method and error diffusion method, background removal processing, and total ink amount regulation processing. Then, the recording dot pattern stored in the raster data memory is transferred to the inkjet printer 500 by the drawing information transfer unit 32 via the interface.

Among the processes executed by the drawing information generation unit 30 as described above, the halftone process is also called a halftone process. This halftone processing is a process of converting the gradation of color expression in the image information of the target to be output for image formation according to the gradation of color expression in the image formation output performance by the image forming apparatus 2.

As described above, in the inkjet type image forming apparatus 2, each pixel is formed by ejecting ink from a nozzle included in the recording head 54, and image forming output is executed. At this time, as described above, by switching the ink droplets to be ejected to small droplets, medium droplets, and large droplets, it is possible to express the density of about 4 gradations (2 bits) including colorless in one pixel. However, on a PC such as the information processing device 1, the image data is generally processed with 256 gradations (8 bits) or the like.

Therefore, in the halftone processing, for example, the image data represented by 8 bits is converted into the image data represented by 2 bits without deteriorating the image quality perceived by human eyes. Comparing the image data before and after conversion for each pixel, it is clearly different because the density gradation is small. Therefore, in the halftone processing, the image is converted by the area gradation method so that when the images are compared for each predetermined area unit including a plurality of pixels, the images are recognized as similar images by human eyes. As a result, in the halftone processing, a quantization error due to the area gradation method occurs.

The error diffusion method and the DBS (Direct Binary Search) method are known as halftone processing in consideration of the quantization error. The error diffusion method is a method in which the quantization error generated in each pixel is weighted and accumulated, and then diffused to surrounding pixels.It is a method of performing quantization by comparing with a specified repeating pattern such as the dither method. In comparison, texture and moire are less likely to occur. However, there is also a feature that a series of dots on the worm is likely to occur due to the direction of processing. DBS tentatively determines the dot arrangement, calculates the quantization error with the input by turning the dot ON / OFF and switching the type for each pixel or section, and updates the dot arrangement when the error is reduced. The error is reduced by repeating the process, and the sharpness of the image is excellent, and regular patterns such as worms caused by error diffusion are unlikely to occur.

Since the above processing is related to data processing, it is only theoretical. In the first place, each pixel of digital data is treated as a square shape, but each dot (hereinafter referred to as a landing image) formed by fixing the ink ejected from the nozzle on the paper has a circular shape. Is. Therefore, how to fill the paper surface differs depending on the presence or absence of dots at adjacent pixel positions and the type of dots.

Ink is ejected from the nozzle by mechanical control. Therefore, the characteristics of the landing image for each nozzle differ depending on various factors such as the response characteristics of the piezoelectric element and the fine shape of the nozzle. Therefore, even if they are driven based on the same drive signal, the dot size of the landing image may be different, the landing position may shift, satellites may occur, and adjacent pixels may be affected. There is. Here, the satellite is a minute drop that is ejected in association with the main drop (main dot).

As a result, even if you intend to print uniform data, the amount of ink adhered and the way dots overlap will differ depending on the characteristics of each head and nozzle, and this is an image defect such as a difference in color and streaks and banding. , It may be recognized visually by humans.

In order to deal with such a problem, the drawing information generation unit 30 forms a dot landing model instead of a digital value and calculates an error when performing dot replacement processing by error calculation by the DBS method. Create print data including engine characteristics.

Therefore, when performing image formation output using the image forming apparatus 2, as a preliminary process, a patch generated by printing an image patch generated by the dot pattern generation unit 34 and reading it with a sensor, a scanner, or the like. The image data is acquired by the scanned image acquisition unit 36, and the nozzle characteristic determination unit 38 generates a landing model of dots formed by the nozzle of the image forming apparatus 2 based on the patch image data. Then, in the subsequent drawing data generation processing in the image formation output, the drawing information generation unit 30 simulates the dots ejected by each nozzle using this landing model.

First, an example of an image pattern generated by the dot pattern generation unit 34 will be described. The dot pattern generation unit 34 generates an image pattern and inputs it to the print instruction acquisition unit 28 to cause the image forming apparatus 2 to execute an image forming output of the image pattern. The scanned image acquisition unit 36 acquires the image data generated by reading the paper output in this manner with a scanner or the like, and the nozzle characteristic determination unit 38 acquires a dot model formed by the nozzle.

The nozzle characteristic determination unit 38 compares the dot pattern generated by the dot pattern generation unit 34 with the image information obtained by reading the dot pattern acquired by the read image information acquisition unit 115, that is, the dot pattern image information. As a result, the nozzle characteristic determination unit 38 adheres to the position where the ink ejected by each of the plurality of nozzles included in the image forming apparatus 2 adheres to the paper which is the medium for image formation output, the size of the adhered ink, and the adhesion. Generates nozzle characteristic information including information on ink shape and satellite (distribution of adhered ink).

Taking the carriage 50 in which the nozzles are arranged in 4 rows and 1 column as an example, a case where a pattern of 4 rows is formed is shown as an example. This is the most basic example in which one nozzle is provided for each CMYK color, and the image pattern to be formed is set by the arrangement of nozzles such as a line type recording head.

The layout of the image pattern generated by the dot pattern generation unit 34 is not particularly limited, but if the dot formation density of the pattern is high, it is combined with the surrounding dots and which dot is ejected from which nozzle. Since it is difficult to distinguish them, it is desirable that the pattern has a sufficient gap so that it can be distinguished which dot is ejected from which nozzle even if the dot size and the landing position fluctuate.

Further, in the case of a printer capable of handling dots of a plurality of sizes, the standard dot size differs depending on the drop type. In addition, large drops land as intended, but medium drops tend to bend, and the characteristics of landing may differ depending on the type of drop. Therefore, it is desirable to print and read the image pattern for each drop type to obtain the landing model.

Next, the halftone processing by the drawing information generation unit 30 will be described. The drawing information generation unit 30 uses the DBS method. Specifically, the error between the dots in the region of interest and the input image is obtained while changing the dot ON / OFF and the type, that is, changing the density within the gradation supported by the image forming apparatus 2. If the error is smaller than before the change, the quantization error is reduced by updating the arrangement according to the changed content.

Here, the pixel of interest may be changed or the error may be determined for each pixel, or the dot may be turned ON / OFF or replaced with another type of dot within a predetermined area, and the error in the section may be reduced. The arrangement may be selected according to the combination condition that becomes smaller. Further, since the actual data has a dot gain, the influence of adjacent pixels may not be reflected in the error due to the context of the processing. For this reason, it is possible to add a process of calculating the error and reviewing the arrangement again after going through the data, and at this time, the method of cutting the section for reviewing the arrangement may be changed.

As the end condition of the process, it is possible to adopt a configuration in which the process ends with the number of processes or when the error is no longer reduced with respect to the previous process. Further, at the first time, halftone processing may be performed in advance by dither or error diffusion processing, and the processing may be performed starting from the arrangement. In addition, it is also possible to use a simulated annealing method or the like to determine the replacement process after error comparison, instead of determining the case where the error is minimized each time.

When performing this error calculation, the error is evaluated assuming that the acquired dot model is hit instead of simple digital data. Therefore, the drawing information generation unit 30 is based on the nozzle characteristics acquired by the nozzle characteristic determination unit 38, and the dot arrangement of the pixel of interest, that is, the image data to be output is converted according to the gradation of the image forming apparatus 2. The dot arrangement is replaced with the dot model of each nozzle to simulate the image after landing.

For example, the drawing information generation unit 30 first quantizes the input data, that is, the original image data to be output, by performing halftone processing on each nozzle. Further, the drawing information generation unit 30 generates data in which the landing state when each nozzle ejects ink, that is, the ink distribution is estimated, based on the nozzle characteristics acquired by the nozzle characteristic determination unit 38.

Next, the drawing information generation unit 30 calculates an error between the image after landing and the input data. At this time, the drawing information generation unit 30 obtains the ink adhesion in the area corresponding to the pixels of the input data by comparing the level of the input image. For example, the multi-value level is increased based on the ink coating amount at the location corresponding to the pixel of the input data. For example, if the input data has a density gradation of 0 to 255 (here, 255 is black) and the area coverage of the corresponding pixel is 50%, the output at that location is half of 0 to 255. 127, which corresponds to, and the difference from the input data is used as an error.

The drawing information generation unit 30 converts the input data into multi-value data so that the ink coating amount of 0% to 100% corresponds to the density gradations 0 to 255, respectively, and generates multi-value simulation data. By comparing the multi-valued simulated data generated in this way with the input data, the difference is used as an error.

The drawing information generation unit 30 may perform the operation based on the brightness and the density instead of the covering amount (for example, the maximum density is converted to 255, and the minimum density = the paper surface density is converted to 0). In this way, the drawing information generation unit 30 repeats the calculation / comparison of the error while changing the quantization condition after adding the nozzle characteristics, so that the error with the input is small due to the conditions including the engine characteristics. The quantization condition is determined.

<Operation example of drawing information generation unit 30>
The case where the total number of pixels of the image to be output is "M", the density gradation of the image forming apparatus 2 such as colorless, small droplets, medium droplets, and large droplets is "A" will be described as an example. First, the drawing information generation unit 30 tentatively determines the dot arrangement for all the M pixels.

By converting the gradation of the image to be output for image formation from the multi-valued digital data to the gradation of the pixels that can be expressed by the recording head 54 of the image forming apparatus 2, such as the digital data after the halftone in the center, the floor This is a process for generating an image after key conversion. That is, the drawing information generation unit 30 functions as a gradation conversion unit.

Next, the drawing information generation unit 30 applies the landing model to the tentatively determined dot arrangement to generate landing simulation data. In other words, this process is a process of generating simulation information of the image formation output result based on the information indicating the characteristics of the nozzle included in the recording head 54 of the image forming apparatus 2 and the image after gradation conversion. That is, the drawing information generation unit 30 functions as an output result simulation unit.

Then, the drawing information generation unit 30 converts the result of the landing simulation into a gradation value to generate multi-value conversion data. This process is a process of converting the simulation information of the image formation output result into the gradation of the pixels constituting the image of the image formation output target which is the original image.

When the multi-value conversion data is generated, the drawing information generation unit 30 compares the pixels of the multi-value conversion data with the input data and calculates the quantization error. The drawing information generation unit 30 that has calculated the error compares the quantization error that has already been calculated and recorded with the newly calculated error, and if the error value is reduced by the newly calculated error, The current dot arrangement and the newly calculated error value are recorded in the storage medium.

On the other hand, if the error value is not reduced, the drawing information generation unit 30 discards the error value. The drawing information generation unit 30 repeats the above-described processing until all the combinations of dot arrangements are completed, and ends the processing when all the combinations are completed.

This is a process of determining one of a plurality of images after gradation conversion having different gradation conversion modes when generating an image after gradation conversion as drawing information based on the calculated error value. .. That is, the drawing information generation unit 30 functions as a drawing information determination unit. By such processing, the dot arrangement with the smallest error value is recorded.

As described above, according to the information processing apparatus 1 in which the printer driver 26 for causing the image forming apparatus 2 to execute the image forming output is installed, the dot pattern quantized by the DBS method and the original image data When calculating the error value of and selecting the optimum dot pattern, the error value between the landing simulation based on the landing characteristics of ink ejection by the nozzle of the recording head 54 included in the image forming apparatus 2 and the original image data is calculated. Therefore, it is possible to enable image forming output according to the characteristics of the image forming output engine.

The dot landing model may change depending on the operating conditions of the device. The operating conditions of the device are, for example, a printing environment such as a paper type and mode used for printing, temperature and humidity. This is because different paper types have different ways of getting wet and spreading on the paper surface even if the same dots are ejected. Furthermore, when the mode or printing environment changes, the ink viscosity determined according to the driving conditions of the head used for printing, the paper transport speed, the humidity and the temperature differs, so that the ejection characteristics of the head itself may change.

Therefore, it is preferable that the acquisition and reflection of the landing model can be set / applied for each of these paper types, printing modes, and printing environments. As a result, the image can be formed under the optimum conditions for the printing conditions.

The timing for acquiring the landing model is when the device is started up, when the head is replaced, every predetermined time elapses during the operation period of the device, every predetermined number of prints is counted, and the operating environment of the device such as humidity and temperature fluctuates by a predetermined value or more. If this is the case, various factors can be used as triggers, such as when the paper type or mode is changed, or when the user is specified. This is because the landing characteristics are likely to change, such as the configuration of the device, the usage environment, and changes over time.

When the head is replaced, it is the replacement head that can change the impact characteristics. Since the heads are usually replaced for each color such as CMYK, the man-hours may be omitted by acquiring the landing model only for the replaced heads.

These may be carried out through a series of processes from printing the image pattern to acquiring the landing model and reviewing, or may be performed only partially. For example, the image information acquisition unit 115 acquires the patch image data and the nozzle characteristic determination unit 38 generates the impact model, and the difference between the newly generated impact model and the previously generated and applied impact model is determined. The processing man-hours may be reduced by updating the landing model when the threshold value of is exceeded.

Such processing can be realized, for example, by printing gradations and color patches when printing an image pattern, picking up characteristics, and reviewing the landing model when these characteristics change more than a predetermined value. Can be done. As the characteristic, the brightness and density of the information acquired by the sensor or the scanner, the RGB value, and the like may be used to update the landing model when the characteristic changes by a predetermined value or more.

Further, as another method, it is conceivable to compare the fluctuation of the ink coating amount of the nozzle and review it when it deviates from the specified amount. The criteria for review may be specified by an absolute value, or by printing the pattern multiple times when printing the image pattern, repeated variations during continuous printing are picked up, and if this is exceeded, the characteristics change. As a matter of fact, the landing model may be updated.

In the above embodiment, as shown in FIG. 3, the case where the printer driver 26 of the information processing apparatus 1 includes the read image acquisition unit 36 and the nozzle characteristic determination unit 38 has been described as an example, but the image forming apparatus 2 Such a configuration may be provided internally, and the landing model may be downloaded from the image forming apparatus 2 to the information processing apparatus 1. In this way, if the image forming apparatus 2 is configured to be able to generate the landing model independently, it is possible to independently create and output an image in consideration of the engine characteristics.

As a specific embodiment of the information processing device 1 and the image forming device 2, in the case of a commercial printing machine and a control device, in the device used in the printing field, a dedicated device or software called RIP (raster image processor) is used. In many cases, a halftone image is created through the image, and the image is input to an image forming apparatus to print the image, and many of them have advanced functions and calculation mechanisms for image processing.

Therefore, any or all of the output of the chart, the acquisition of the landing model, and the creation of the image data can be realized on the RIP side, that is, on the information processing device 1 side. In this case, data processing including engine characteristics can be performed by RIP and sent to the device side, so even if the image forming device does not have a function to correct engine characteristics, even RIP should be supported. If so, it becomes possible to correct the engine characteristics.
Then, when the nozzle characteristic determination unit 38 generates the landing model based on the patch image data acquired by the read image acquisition unit 115, the nozzle characteristic determination unit 38 creates a landing model for each of the head A and the head B based on the above-mentioned sorting information. Generate.

Then, when the drawing information generation unit 30 applies the landing model to generate the landing simulation, the drawing information generation unit 30 generates the landing simulation based on the above-mentioned hitting information. By such a process, it is possible to apply the process of the landing model to the image forming apparatus using the overlap process and obtain the same effect as described above.

<Image processing unit>
FIG. 5 is a functional block diagram showing a functional configuration of the image processing unit 24 according to the embodiment of the present invention.
FIG. 5 shows a configuration in which the image processing unit 24 can finally obtain an image in which the damaged portion is repaired from the original image, and further shows the transition of the intermediate image.

<Image to be processed by the image processing unit>
In the present embodiment, referring to FIGS. 17 to 23, the following parts perform image processing.
FIG. 17 is a diagram showing an original image G100 acquired from a painting drawn with oil paint using, for example, a scanner or HDD 14, and the original image G100 has a damaged portion.
FIG. 18 is a diagram showing a background evaluation image G110 generated from the original image G100.
FIG. 19 is a diagram showing a flat evaluation image G120 generated from the original image G100.
FIG. 20 is a diagram showing a damaged portion detection image [1] G200 generated from the original image G100 and the flat evaluation image G120.
FIG. 21 is a diagram showing a damaged portion detection image [2] G210 generated from the damaged portion detection image [1] G200.
FIG. 22 is a diagram showing a damaged portion repair image [1] G300 generated from the original image G100 and the damaged portion detection image [2] G210.
FIG. 23 is a diagram showing a damaged portion repair image [2] G310 generated from the damaged portion repair image [1] G300.

<Functional configuration of image processing unit>
Returning to FIG. 5, the image processing unit 24 generates the original image acquisition unit 100, the background evaluation image generation unit 110, the flat evaluation image generation unit 120, the damaged part detection image [1] generation unit 200, and the damaged part detection image [2]. A unit 210, a damaged portion repair image [1] generation unit 300, and a damaged portion repair image [2] generation unit 310 are provided.
The original image acquisition unit 100 acquires the image data of the painting in which the damaged portion is generated as the original image G100 from the outside or the HDD 14, and stores it in the RAM 12.
The background evaluation image generation unit 110 generates a background evaluation image G110 from the original image G100. Specifically, the background evaluation image generation unit 110 generates a background evaluation image G110 representing the evaluation of pixels that are the background of each pixel by performing smoothing processing on the original image G100.
The flat evaluation image generation unit 120 generates a flat evaluation image G120 from the original image G100. Specifically, the flat evaluation image generation unit 120 generates a flat evaluation image G120 representing an evaluation related to the flatness of the original image G100 by performing edge detection processing and smoothing processing on the original image G100.
The damaged portion detection image [1] generation unit 200 determines the presence or absence of a damaged portion in each pixel from the original image G100, the background evaluation image G110, and the flat evaluation image G120, and generates the damaged portion detection image [1] G200. Specifically, the damaged portion detection image [1] generation unit 200 identifies the damaged portion based on the difference value between the original image G100 and the background evaluation image G110 and the pixel value of the flat evaluation image G120, and identifies the damaged portion. Detection image [1] G200 is generated.
The damaged portion detection image [2] generation unit 210 performs pixel expansion processing of the damaged portion detection region from the damaged portion detection image [1] G200 to generate the damaged portion detection image [2] G210.

Next, the damaged portion repair image [1] generation unit 300 performs interpolation processing on the damaged portion detected in the damaged portion detection image [2] G210 using the pixel values of the original image G100 to repair the damaged portion. Image [1] G300 is generated.
The damaged portion repair image [1] generation unit 300 uses the pixel value of the background evaluation image G110 in order to deal with the case where interpolation is performed at the boundary of the image or a pixel to be referred to is not found when interpolating. The completion process may be performed to generate the damaged portion repair image [1] G300.
Finally, the damaged portion repair image [2] generation unit 310 crosses the damaged portion repair image [1] G300 in order to reduce the peculiar cross pattern that appears by performing the interpolation processing on the damaged portion repair image [1] G300. The pattern reduction sharpening filter processing is performed to generate a damaged portion repair image [2] G310.

<Background evaluation image generation process>
FIG. 6 is a flowchart for explaining the background evaluation image generation process by the background evaluation image generation unit 110 shown in FIG.
The background evaluation image generation unit 110 sets a processing area (RAM address space) for the original image G100 stored in the RAM 12 (S100a).
The background evaluation image generation unit 110 applies a smoothing filter to the original image G100 to perform smoothing processing in order to evaluate the pixel values that are the background of each pixel value of the original image G100, and the background evaluation image G110 is generated. It is generated on the RAM 12 (S101).
The background evaluation image generation unit 110 stores the generated background evaluation image G110 in the HDD 14 (S110a).
As the smoothing filter, a median filter or a Gaussian filter that can evaluate the background area while minimizing the influence of noise such as a damaged part is used.

FIG. 7A is a diagram showing a configuration of a median filter, and FIG. 7B is a diagram showing a configuration of a Gaussian filter.
Here, specific examples of the median filter and the Gaussian filter will be described with reference to FIGS. 7A and 7B.
The median filter shown in FIG. 7A is the median value when the pixel values are arranged in order of size with respect to the pixel group in the filter centered on the pixel of interest (pixel value 45 shown in FIG. 7A). Is extracted and a process of replacing the pixel value of interest is performed.
This makes it possible to efficiently replace spike-shaped noise with background pixels.
On the other hand, the Gaussian filter shown in FIG. 7B calculates an average value using a group of pixels in the filter centered on the pixel of interest and a weighted coefficient called a Gaussian filter, and performs a process of replacing the pixel of interest. .. In the processing using the Gaussian filter, for example, Eq. (1),

・・・式（１）
により注目画素の値が求められる。

... Equation (1)
The value of the pixel of interest is obtained.

Here, the characteristics of the median filter and the Gaussian filter will be described.
For example, as a noise image, if the background image has noise with black lines, applying a Gaussian filter to this image will reduce the noise, but the black lines will still remain, and the boundary part of the background image will also be contoured. Bleeding (blurring) occurs.
On the other hand, when a median filter is applied to the above image, black lines are neatly removed from the background pixels, and the boundary portion is also retained.
It is suitable to use a median filter to evaluate the background pixels for an image having a pattern such as noise.
Regarding the smoothing filter size, it is preferable that the width of the damaged portion (1 to 2 mm) is 5 to 7 times (10 mm). The reason for this is that if it is too large, the background larger than the damaged part will be recognized as noise and cannot be evaluated correctly. Further, even if the filter size is too small, the damaged portion cannot be correctly evaluated as noise.

<Flat evaluation image generation processing>
FIG. 8 is a flowchart for explaining the flat evaluation image generation process by the flat evaluation image generation unit 120 shown in FIG.
The flat evaluation image generation unit 120 sets a processing area for the original image G100 stored in the RAM 12 (S100b).
First, the difference image of the target pixel from the adjacent pixel is obtained.
The flat evaluation image generation unit 120 performs a Sobel filter process (S111) in the X direction and a Sobel filter in the Y direction for each of the RGB images (k = r, g, b) of the original image G100 stored in the RAM 12. The process (S112) is applied.

Next, the flat evaluation image generation unit 120 calculates the square root of the sum of squares of the X-direction Sobel filter pixels (g _{k, x} ) and the Y-direction Sobel filter pixels (g _{k, y} ) (S113), and RGB of the pixels. Let it be a difference image (g _k ).

・・・式（２）
また、平坦評価画像生成部１２０は、ＲＧＢを統一した差分画像（ｇ）を求めるのに、各画素のＲ、Ｇ、Ｂの最大値Ｍａｘを計算する（Ｓ１１４）。

... Equation (2)
Further, the flat evaluation image generation unit 120 calculates the maximum value Max of R, G, and B of each pixel in order to obtain the difference image (g) in which RGB is unified (S114).

・・・式（３）
次に、平坦評価画像生成部１２０は、周辺領域の平坦さを表す指標として、求めた差分画像を平滑化（Ｓ１１５）した画像の画素値で評価する。

... Equation (3)
Next, the flatness evaluation image generation unit 120 evaluates the obtained difference image with the pixel value of the smoothed (S115) image as an index showing the flatness of the peripheral region.

Next, the flat evaluation image generation unit 120 generates a flat evaluation image G120 in order to evaluate how flat the change in the pixel value around each pixel has. (S120a). Here, the flat evaluation image generation unit 120 generates a flat evaluation image G120 by smoothing an image representing a difference value between adjacent pixels of the original image G100.

In step S115, the flatness evaluation image generation unit 120 may use a Gaussian filter as the smoothing filter if the flatness of the pixel of interest is important, and may also use an averaging filter, a median filter, or the like. You may use it.
On the other hand, when it is desired to uniformly evaluate the flatness of peripheral pixels, an averaging filter is appropriate. Further, when it is desired to remove the influence of an abnormal value as a representative value of flatness, a median filter may be used in the same manner as in the step (S120a) for generating the flatness evaluation image G120 described above.
As for the filter size, it is preferable to select about 5 times or 7 times the filter size when the original image G100 is smoothed by using a median filter.

<Damage detection image [1] generation process>
FIG. 9 is a flowchart for explaining the damaged portion detection image [1] generation process by the damaged portion detection image [1] generation unit 200 shown in FIG.
The damaged portion detection image [1] generation unit 200 sets a processing area for the original image G100 stored in the RAM 12 (S100c).
Next, the damaged portion detection image [1] generation unit 200 sets a processing area for the background evaluation image G110 stored in the RAM 12 (S110b).
Next, the damaged portion detection image [1] generation unit 200 sets a processing area for the flat evaluation image G120 stored in the RAM 12 (S120b).
Next, the damaged portion detection image [1] generation unit 200 specifies the start address of the RAM 12 that stores the damaged portion detection image [1] G200, and starts the loop process for each pixel (S125).
The damaged portion detection image [1] G200 [1] represents whether or not a certain pixel is a damaged portion in each pixel of the original image G100, the background evaluation image G110, and the flat evaluation image G120 as a binary value. I remember the data.

Next, the damaged portion detection image [1] generation unit 200 is an area for a background pattern in which it is difficult to determine whether or not a pixel is a damaged portion in preparation for determining whether or not a pixel is a damaged portion. Strictly sets the threshold value used for the damaged portion detection process, and sets a correction value in order to prevent the background pattern from being mistakenly detected and removed (S130).
The correction value is determined by the pixel value in the flat evaluation image G120. This correction value is set by changing it binaryally with an arbitrary threshold value as a boundary, but in general, the correction value may be calculated as a function of the pixel value of the flat evaluation image G120.

Next, the damaged portion detection image [1] generation unit 200 enters the step of determining the damaged portion, but as a clue for detecting the damaged portion pixel from the painting image, the color difference value (ΔE) in the L * a * b * space. *) Is used for discrimination. As a result, the accuracy of discrimination from the background pattern can be improved by using L * a * b *, which is close to the color difference felt by humans.
Next, the damaged portion detection image [1] generation unit 200 compares the L * a * b * values of each pixel of the original image G100 and the background evaluation image G110 to see if the pixel is a part of the damaged portion. Judge whether or not. At that time, the correction value based on the flatness set as the preparation in step S130 is applied to the damage portion detection threshold value. First, the damaged portion detection image [1] generation unit 200 converts the RGB values of the pixels of the original image G100 and the background evaluation image G110 into L * a * b * values (S140).
In step S140, the damaged portion detection image [1] generation unit 200 implements the RGB color space as sRGB, the light source as D65, and the light source in the conversion destination L * a * b * space as D50.

Next, the process of detecting the damaged portion is started. Based on the visible characteristics of the damaged part in the painting, the judgment is made by narrowing down to two, "black damaged part" and "white damaged part".
The damaged portion detection image [1] generation unit 200 initializes the flag as false (0 value) with respect to the binary data value on the RAM 123 for storing the damaged portion detection image [1] G200 (S150). ).
Next, the damaged portion detection image [1] generation unit 200 calls the subroutine of the black damaged portion detection process (S160), and when returning from this subroutine, whether or not the pixel is a part of the damaged portion. Is determined (S168). In step S168, if the pixel is a part of the damaged portion (damaged portion = 1), the process proceeds to step S180, and if the pixel is not a part of the damaged portion (no damaged portion = 0), the process proceeds to step S170. move on.
In step S170, the damaged portion detection image [1] generation unit 200 calls the subroutine of the white damaged portion detection process (S170), and when returning from this subroutine, whether or not the pixel is a part of the damaged portion. Is determined (S178). In step S175, if the pixel is a part of the damaged portion (damaged portion = 1), the process proceeds to step S180, and if the pixel is not a part of the damaged portion (no damaged portion = 0), the process proceeds to step S185. move on.

In step S180, when the damaged portion detection image [1] generation unit 200 determines that the pixel is a part of the damaged portion, the damaged portion binary value for the pixel is set in the RAM 12 as a true (S180). , Step S185.
In step S185, the damaged portion detection image [1] generation unit 200 determines whether or not the loop processing is completed because the address of the pixel of interest represents the end point position. If it is determined that the loop processing is completed, the process proceeds to step S200, while if it is determined that the loop processing is not completed, the process proceeds to step S190.

In step S190, since the loop processing is not completed in the damaged portion detection image [1] generation unit 200, the address of the pixel of interest is incremented by one to make the next pixel the pixel of interest, and the process returns to step S130.
In step S200, the damaged portion detection image [1] generation unit 200 stores the damaged portion detection image [1] G200 [1] generated on the RAM 12 in the HDD 14.
The details of the subroutines of the black damaged portion detection process (S160) and the white damaged portion detection process (S170) will be described later.

<Black damaged part detection processing>
FIG. 10 is a flowchart for explaining a subroutine of black damaged portion detection processing by the damaged portion detection image [1] generation unit 200 shown in FIG.
The identification conditions B1 to B4 are set in the subroutine of the black damaged portion detection process (S160).
B1: The pixel value (P) of the original image G100 determines whether or not the brightness (L *) is smaller than the pixel value (Q) of the background evaluation image G110 (S161).

L * (P) <L * (Q)
... Equation (4)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S162 if the equation (4) is satisfied, and proceeds to step S166 if the equation (4) is not satisfied.

B2: It is determined whether or not the color difference (ΔE *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T1) (S162).

ΔE * (P, Q)> T1
... Equation (5)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S163 if the equation (5) is satisfied, and proceeds to step S166 if the equation (5) is not satisfied.

B3: It is determined whether or not the saturation difference (ΔC *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T2) (S163).

ΔC * (P, Q) <T2
... Equation (6)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S164 when the equation (6) is satisfied, and proceeds to step S166 when the equation (6) is not satisfied.
B4: It is determined whether or not the saturation (C *) of the pixel value (P) of the original image is larger than the set threshold value (T3) (S164).

C * (P)> T3
... Equation (7)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S165 when the equation (7) is satisfied, and proceeds to step S166 when the equation (7) is not satisfied.
In step S165, the damaged portion detection image [1] generation unit 200 sets 1 in the register as a value indicating the presence of the damaged portion.
In step S166, the damaged portion detection image [1] generation unit 200 sets 0 in the register as a value indicating no damaged portion.

Damaged portion detection image [1] In the black damaged portion detection process, the generation unit 200 detects a black damaged portion that applies to all of B1 to B4 as a black damaged portion.
The damaged portion detection image [1] generation unit 200 uses the most common CIE1976 color difference calculated by the following formula with respect to the color difference (ΔE *) of the identification condition B2.

・・・式（８）

別の色差表現として、より人間の目の色彩感覚に近いＣＩＥ２０００色差（ΔＥ^＊ _００）もあり、それを用いても良い。
彩度差ΔＣ＊は以下の式で計算される。

... Equation (8)

As another color difference expression, there is also CIE2000 color difference (ΔE ^* ₀₀ ), which is closer to the color sensation of the human eye, and it may be used.
The saturation difference ΔC * is calculated by the following formula.

・・・式（９）
黒破損部検知処理における各識別条件の可変閾値は、背景評価画像Ｇ１１０の画素値の明度、および平坦評価画像Ｇ１２０から決定した補正値の関数として設定している。
具体的には、背景評価画像Ｇ１１０の画素値の明度が低い領域、要するに背景が暗い領域に関しては、破損部検知が難しいので検知に用いる閾値を比較的ゆるく設定している。
破損部検知画像［１］生成部２００は、黒破損部検知処理（Ｓ１６０）において、破損部であると判断された場合は、破損部検知画像［１］Ｇ２００の当該画素に対するフラグをｔｒｕｅに設定（Ｓ１８０）し、次の画素に処理を移す（Ｓ１９０）。一方、検知の識別条件に一つでも当てはまらなかった場合は、白破損部検知処理（Ｓ１７０）に処理を移す。

... Equation (9)
The variable threshold value of each identification condition in the black breakage detection process is set as a function of the brightness of the pixel value of the background evaluation image G110 and the correction value determined from the flat evaluation image G120.
Specifically, in the region where the brightness of the pixel value of the background evaluation image G110 is low, that is, the region where the background is dark, it is difficult to detect the damaged portion, so the threshold value used for detection is set relatively loosely.
When the damaged portion detection image [1] generation unit 200 is determined to be a damaged portion in the black damaged portion detection process (S160), the flag for the pixel of the damaged portion detection image [1] G200 is set to true. (S180), and the process is transferred to the next pixel (S190). On the other hand, if even one of the detection identification conditions is not met, the process is transferred to the white damaged portion detection process (S170).

<White damaged part detection processing>
FIG. 11 is a flowchart for explaining a subroutine of the white damaged portion detection process by the damaged portion detection image [1] generation unit 200 shown in FIG.
The damaged portion detection image [1] generation unit 200 sets the identification conditions W1 to W5 in the subroutine of the white damaged portion detection process (S170).
W1: The pixel value (P) of the original image determines whether or not the brightness (L *) is larger than the pixel value (Q) of the background evaluation image G110 (S171).

L * (P)> L * (Q)
... Equation (10)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S172 when the equation (10) is satisfied, and proceeds to step S176 when the equation (10) is not satisfied.

W2: It is determined whether or not the color difference (ΔE *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T4) (S172).

ΔE * (P, Q)> T4
... Equation (11)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S173 if the equation (11) is satisfied, and proceeds to step S176 if the equation (11) is not satisfied.

W3: It is determined whether or not the saturation difference (ΔC *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T5).

ΔC * (P, Q) <T5
... Equation (12)
And,
W4: It is determined whether or not the saturation (C *) of the pixel value (P) of the original image is smaller than the set threshold value (T6) (S173).

C * (P) <T6
... Equation (13)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S175 when the equation (12) and the equation (13) are satisfied, and when the equation (12) and the equation (13) are not satisfied, The process proceeds to step S174.

W5: It is determined whether or not the brightness difference (ΔL *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T7) (S174).

ΔL * (P, Q)> T7
... Equation (14)
Here, the damaged portion detection image [1] generation unit 200 proceeds to step S175 when the equation (14) is satisfied, and proceeds to step S176 when the equation (14) is not satisfied.
In step S175, the damaged portion detection image [1] generation unit 200 sets 1 in the register as a value indicating the presence of the damaged portion.
In step S176, the damaged portion detection image [1] generation unit 200 sets 0 in the register as a value indicating no damaged portion.

In the white damaged portion detection process, in order to avoid erroneous detection of the background pattern, W1 and W2 are set as essential identification conditions, and those corresponding to W3, W4, or W5 are detected as damaged portions.
Since the white damaged portion is mainly caused by the peeling of the paint, the detection threshold value of each identification condition is set to a constant value without depending on the brightness of the background evaluation image G110 and the correction value by the flat evaluation image G120. .. If the detection is uneven due to the brightness of the background area, the threshold value for each identification condition may be set variably as in the case of detecting the black damaged portion.
When the damaged portion detection image [1] generation unit 200 is determined to be a damaged portion pixel by the white damaged portion detection process (S170), the damaged portion detection image [1] G200 is set to true (the pixel is set to true). S180), and the process is transferred to the next pixel (S190). On the other hand, if the damaged portion detection image [1] generation unit 200 does not meet the detection identification conditions, the processing is directly transferred to the next pixel (S190).
By applying the above-mentioned damaged portion determination process to all pixels, the damaged portion detection image [1] G200 is generated.

<Damage detection image [2] generation process>
FIG. 12 is a flowchart for explaining the damage portion detection image [2] generation process by the damage portion detection image [2] generation unit 210 shown in FIG.
The damaged portion detection image [2] generation unit 210 sets a processing area for the damaged portion detection image [1] G210 stored in the RAM 12 (S200a).
The damaged part detection image [2] generation unit 210 performs expansion processing on the area detected as the damaged part in order to finally pass the damaged part detection image [1] G200 to the repair process, and the damaged part detection image [1] 2] Generate G210 (S201).

Hereinafter, the expansion process by the damaged portion detection image [2] generation unit 210 shown in FIG. 5 will be described with reference to the graph shown in FIG.
The polygonal line L202 indicates the change in the pixel value, and the dotted line L203 is the threshold value for the damaged portion detection process. Then, the area surrounded by the two vertical lines L204 is detected as a damaged portion, and is interpolated like the horizontal line L205 at the repair processing stage. The pixel value of the horizontal line 205 is generally higher than that of the outer polygonal line, and the repaired portion becomes unnatural.
In order to prevent this, by slightly widening the damaged portion, it becomes possible to interpolate with pixel values closer to the background during the repair process.

The number of expansions can be adjusted depending on the resolution of the damaged portion. If the number of expansions is set too large, the part that is not the damaged part to be referred to will decrease during interpolation at the repair stage, so the image will have bleeding (blurring) of the outline as a whole, so be careful. is there.
Specifically, the background evaluation image generation unit 110 preferably has a pixel count of about 1/10 of the size of the smoothing filter used when generating the background evaluation image G110.
Further, as an application example of the damaged portion detection process using the color difference ΔE * in the L * a * b * space, the detection of the grain pattern on the surface of the cut out wood can be mentioned.
It is also possible to generate an output image with a weakened wood grain pattern or an output image with a stronger wood grain pattern by detecting the wood grain pattern based on the background and the color difference ΔE * and performing appropriate repair processing. Is.

<Damage repair image [1] generation process>
FIG. 14 is a flowchart for explaining the damaged portion repair image [1] generation process by the damaged portion repair image [1] generation unit 300 shown in FIG.
The damaged portion repair image [1] generation unit 300 sets a processing area for the damaged portion detection image [2] G210 stored in the RAM 12 (S210b).
Next, the damaged portion repair image [1] generation unit 300 specifies the start address of the RAM 12 that stores the damaged portion detection image [2] G210, and starts the loop process for each pixel (S211).
First, the damaged portion repair image [1] generation unit 300 determines whether or not the pixel of interest is the damaged portion (S220).
Here, the damaged portion repair image [1] generation unit 300 proceeds to step S240 when the pixel of interest is a damaged portion, while proceeding to step S275 when the pixel of interest is not a damaged portion.

When the pixel of interest is a damaged portion, the damaged portion repair image [1] generation unit 300 represents a pixel value for the pixel of the background evaluation image G110 at the position of the pixel of interest, a distance between the pixels of interest, and the like. (S240).
Next, the damaged portion repair image [1] generation unit 300 interpolates the attention pixel value from the background evaluation pixel list for the acquired background evaluation image G110 pixels (S250).
Next, the damaged portion repair image [1] generation unit 300 corrects the brightness of the interpolated pixel value of interest (S260).

Next, the damaged portion repair image [1] generation unit 300 performs additional noise correction on the brightness-corrected pixel value of interest (S260).
Next, the damaged portion repair image [1] generation unit 300 determines whether or not the loop processing is completed when it is determined that the pixel of interest is the damaged portion (S275). If it is determined that the loop processing is completed, the process proceeds to step S300a, while if it is determined that the loop processing is not completed, the process proceeds to step S230.
In step S230, since the loop processing is not completed in the damaged portion repair image [1] generation unit 300, the address of the pixel of interest is incremented by one to make the next pixel the pixel of interest, and the process returns to step S220.
In step S300, the damaged portion repair image [1] generation unit 300 stores the damaged portion repair image [1] G300 generated on the RAM 12 in the HDD 14 (S200).

<Damage detection image [2]>
FIG. 15A is a diagram showing an example of a 10 × 10 image representing each pixel of the damaged portion detection image [2], and FIG. 15B is a diagram showing a background evaluation pixel list.
The background evaluation pixel list (FIG. 15 (b)) of pixels having different colors (pixels of interest) in FIG. 15 (a) is acquired.
Damaged portion repair image [1] When the generation unit 300 scans pixels in the vertical and horizontal directions centering on the pixel of interest and reaches a pixel that is not a damaged portion (a pixel whose flag is false in the damaged portion detection image), The pixel value (Value) of the original image G100 at that position and the distance (Distance) between the pixels of interest are stored in the background evaluation pixel list.

In the image example shown in FIG. 15 (a), as shown in FIG. 15 (b), information about the pixels A and C is stored in the background evaluation pixel list (L).
In addition, as exception handling by the damaged portion repair image [1] generation unit 300, the following two situations are dealt with.
[1] When the damaged part continues for a long time in one direction and no pixel that is not the damaged part is found [2] When the edge of the image is reached

The damaged part repair image [1] generation unit 300 sets the maximum scanning distance in [1], operates up to that distance, and if a pixel that is not the damaged part is not found, the background at that distance (maximum scanning distance). The pixel value (Value) and the maximum scanning distance (Distance) of the evaluation image G110 are stored in the background evaluation pixel list.
In the image example shown in FIG. 15A, the information of the pixel B is stored in the background evaluation pixel list (L) as shown in FIG. 15B.
Similarly in [2], the damaged portion repair image [1] generation unit 300 scans and reaches the edge pixel, and the pixel value (Value) of the background evaluation image G110 and the distance between the pixels of interest (Distance). Is stored in the background evaluation pixel list.
In the image example shown in FIG. 15A, the information of the pixel D is stored in the background evaluation pixel list (L) as shown in FIG. 15B.

By the above processing, a background evaluation pixel list having four elements each of a value (Value) and a distance (Distance) is acquired.
For the maximum scanning distance, the size of the smoothing filter when generating the background evaluation image G110 is appropriate. The reason is that if the value is set larger than that, the background evaluation pixels of the parts having different backgrounds will be brought.
Next, in step S250 described above, the damaged portion repair image [1] generation unit 300 interpolates the pixel of interest from the acquired background evaluation pixel list (S250). In the interpolation process, two-dimensional linear interpolation weighted by distance may be used.
The damaged portion repair image [1] generation unit 300 calculates the interpolation value (V) according to the equations (15) and (16) based on the background evaluation pixel list (L) shown in FIG. 15 (b).

・・・式（１５）

... Equation (15)

・・・式（１６）

... Equation (16)

As described above, in order to make the restored portion look more natural for the interpolated pixels, two types of corrections were performed for each pixel.
First, the brightness was corrected in step S260. This is because the closest pixel to be referred to when interpolating is close to the boundary of the damaged part, so by interpolating with a pixel whose brightness is slightly lower than the background pixel, the tendency of the interpolated attention pixel to become low is corrected. is there. The RGB value of the damaged part repaired part is increased by about 1 to 5%.
Second, noise additional correction was performed in step S270. Since the interpolated damaged part is linearly interpolated, the values between adjacent pixels change smoothly. On the other hand, in the true background pixel that is not the damaged portion, the values of the adjacent pixels change somewhat randomly. In order to make up for this difference and let the pixels express reality, the pixel values are uniformly changed in all of RGB in the correction area. The amount of noise is determined by calculating from the amount of difference between adjacent pixels of the pixel of interest. Further, since the noise may be extremely large, the limit of the amount of noise is set to about ± 5.

The reason why both the brightness correction (S260) and the noise addition correction (S270) are uniform in RGB is determined by the fact that the damaged portion has no difference in saturation from the background portion.
The repair process up to the correction process is applied to all the pixels, and the damaged portion repair image [1] G300 is generated. Further, a sufficient restored image can be obtained without performing these two corrections.
When the repaired portion has an impression that it is darker, has less undulations, has no intonation, and has less unevenness than the background portion, brightness correction (S260) and noise addition correction (S270) may be performed.
Further, as a variation of the restoration processing, in the two-dimensional linear interpolation, scanning may be performed in the diagonal direction in addition to the vertical / horizontal direction, and the interpolation processing may be performed from the background evaluation pixel list having eight elements of the background evaluation pixels.
Further, a damaged portion surrounding the pixel of interest may be defined, and all the pixel values may be added to the background evaluation pixel list because the damaged portion is at the boundary, and interpolation processing may be performed.

As another example, when the damaged part detection image [1] G200 [1] is subjected to the expansion process (S201), the pixels that were damaged before the expansion process and the pixels that were newly damaged by the expansion process. On the other hand, the repair process can be changed freely.
For example, the repair condition may be improved by repairing the former pixel by ordinary two-dimensional linear interpolation and applying the pixel value of the background evaluation image G110 to the latter pixel.

<Damage repair image [2] Generation>
FIG. 16 is a flowchart for explaining the damaged portion repair image [2] generation process by the damaged portion repair image [2] generation unit 310 shown in FIG.
The damaged portion repair image [2] generation unit 310 sets a processing area for the damaged portion repair image [1] G300 stored in the RAM 12 (S300a).
Next, in order to obtain the final damaged portion repair image [2] G310 from the damaged portion repair image [1] G300, the damaged portion repair image [2] generation unit 310 undergoes a cross pattern reduction sharpening filter process as post-processing. I do. As a result, the repair process for obtaining the damaged portion repair image [2] G310 is an improved version of the smoothing process such as calculating the repair pixel value using the peripheral pixel value, so that the image looks dull. The amount given to the viewer can be reduced by the sharpening process.

As described above, in the method (S240, S250) of linearly interpolating and filling the restored image with reference to the closest pixels in the vertical and horizontal directions, a cross-shaped pseudo pattern is likely to be formed in the restored image. When a simple sharpening process is performed, the cross pattern is conspicuous, resulting in an unnatural image.
In order to reduce such a cross pattern, in step S301, the damaged portion repair image [2] generation unit 310 performs a cross pattern reduction sharpening filter process having the following anisotropy.

・・・（式１７）
上下左右方向を弱め、斜め方向を少し加える鮮鋭化処理によって、自然な修復画像が得られる。
フィルタの係数（鮮鋭化度、異方性の強さ）やフィルタサイズは変更しても良い。
また、修復画素の補間計算が、上下左右の最も近い破損部でない画素を参照して埋める方法でない場合は、単純な鮮鋭化フィルタでの処理でも良い。
以上により、破損部修復画像［２］生成部３１０は、破損部修復画像［２］Ｇ３１０が生成される。
ステップＳ３１０ｂでは、破損部修復画像［２］生成部３１０は、ＲＡＭ１２上に生成された破損部修復画像［２］をＨＤＤ１４に記憶する。

... (Equation 17)
A natural restoration image can be obtained by the sharpening process that weakens the vertical and horizontal directions and adds a little diagonal direction.
The coefficient of the filter (sharpness, strength of anisotropy) and the filter size may be changed.
Further, if the interpolation calculation of the repaired pixels is not a method of filling by referring to the pixels that are not the closest damaged portions in the vertical and horizontal directions, a simple sharpening filter may be used.
As described above, the damaged portion repair image [2] generation unit 310 generates the damaged portion repair image [2] G310.
In step S310b, the damaged portion repair image [2] generation unit 310 stores the damaged portion repair image [2] generated on the RAM 12 in the HDD 14.

In the present embodiment, the damaged part repair image [2] generated by the generation unit 310 outputs the damaged part repair image [2] G310 to the printer driver 26, and the printer driver 26 outputs drawing information from the damaged part repair image [2] G310. It is generated and output to the image forming apparatus 2.
As a result, with respect to the original image G100 having the damaged portion, the damaged portion repair image [2] G310 in which the damaged portion is corrected is generated as drawing information, and the drawing information is printed by the image forming apparatus 2, so that the damaged portion is formed. It is possible to meet the desire to appreciate the image before it occurs.
In the above-described embodiment, the image forming apparatus 2 is configured to print the damaged portion repair image [2] G310 by using an inkjet printer. However, instead of the inkjet printer, an electrophotographic having a plurality of drums. A damaged portion repair image [2] G310 may be printed using a formula printer.

<Structure, Action, Effect of Examples of Embodiments of the Present Invention>
<First aspect>
The image processing device of this embodiment is an image processing device that detects a damaged portion existing in the original image G100, and is a background evaluation representing an evaluation of pixels that are the background of each pixel by performing smoothing processing on the original image G100. The background evaluation image generation unit 110 that generates the image G110 and the flatness evaluation image generation that generates the flatness evaluation image G120 that represents the evaluation related to the flatness of the original image G100 by performing edge detection processing and smoothing processing on the original image G100. Based on the difference value between the unit 120, the original image G100 and the background evaluation image G110, and the pixel value of the flat evaluation image G120, the damaged portion is identified and the damaged portion detection image [1] G200 is generated. It is characterized by including a detection image [1] generation unit 200.
According to this aspect, the background evaluation image generation unit 110 generates the background evaluation image G110 representing the evaluation of the pixels that are the background of each pixel by performing the smoothing process on the original image G100. The flat evaluation image generation unit 120 generates a flat evaluation image G120 that represents an evaluation related to the flatness of the original image G100 by performing edge detection processing and smoothing processing on the original image G100. Damaged portion detection image [1] The generation unit 200 identifies the damaged portion based on the difference value between the original image G100 and the background evaluation image G110 and the pixel value of the flat evaluation image G120, and the damaged portion detection image [1] 1] Generate G200.
In this way, the difference value between the original image G100 and the background evaluation image G110 representing the evaluation of the background pixel of each pixel by performing the smoothing process on the original image G100, and the edge detection process on the original image G100. By performing the smoothing process, the damaged portion is identified based on the pixel value of the flat evaluation image G120 representing the evaluation related to the flatness of the original image G100, and the damaged portion detection image [1] G200 is generated to generate the background. It is possible to prevent erroneous detection of a specific pattern included in an image having a difference in brightness similar to that of a damaged portion as a damaged portion.
For example, in order to determine the damaged pixel based on the L * a * b * color difference between the original image G100 and the background evaluation image G110 generated based on the original image G100 and reduce the false detection of the background pattern, the original image G100 The criteria for determining the damaged portion is adjusted from the flat evaluation image G120 generated based on the above, and the generated damaged portion detection image [1] G200 is used to perform correction processing on the original image G100, so that the damaged portion is generated. It is possible to remove the damaged part from the existing painting and reproduce the painting without the damaged part at that time. In addition, it is possible to adjust by parameters according to the size of the damaged part and the degree of repair, and to test the repair condition. A better effect can be obtained by performing a filter process using another parameter only in the area where there is an omission of detection or a false detection of the damaged part.

<Second aspect>
The background evaluation image generation unit 110 of this aspect is characterized in that the background evaluation image G110 is generated by performing a smoothing process on the original image G100 using a median filter.
According to this aspect, the background evaluation image generation unit 110 generates the background evaluation image G110 by performing a smoothing process on the original image G100 using a median filter.
As a result, the background evaluation image G110 can be generated by performing a smoothing process on the original image G100 using a median filter.

<Third aspect>
The flat evaluation image generation unit 120 of this embodiment generates an adjacent difference image by performing edge detection processing on the original image G100 using a sobel filter, and the Gaussian filter, the averaging filter, and the median for the adjacent difference image. It is characterized in that a flat evaluation image G120 is generated by performing a smoothing process using any one of the filters.
According to this aspect, the flat evaluation image generation unit 120 generates an adjacent difference image by performing edge detection processing on the original image G100 using a sobel filter, and a Gaussian filter and averaging the adjacent difference image. A flat evaluation image G120 is generated by performing a smoothing process using any one of a filter and a median filter.
As a result, an adjacent difference image is generated by performing edge detection processing on the original image G100 using a Sobel filter, and any one of a Gaussian filter, an averaging filter, and a median filter is used for the adjacent difference image. The flat evaluation image G120 can be generated by performing the smoothing process.

<Fourth aspect>
The damaged portion detection image [1] generation unit 200 of this embodiment has at least binary information by distinguishing the background pixel and the damaged portion pixel based on the original image G100, the background evaluation image G110, and the flat evaluation image G120. It is characterized in that a damaged portion detection image [1] G200 is generated.
According to this aspect, the damaged portion detection image [1] generation unit 200 distinguishes between the background pixel and the damaged portion pixel based on the original image G100, the background evaluation image G110, and the flat evaluation image G120, and has at least two values. Damaged portion detection image [1] G200 having information is generated.
As a result, based on the original image G100, the background evaluation image G110, and the flat evaluation image G120, the background pixel and the damaged portion pixel are distinguished, and the damaged portion detection image [1] G200 having at least binary information is generated. Can be done.

<Fifth aspect>
The damaged portion detection image [1] generation unit 200 of the present embodiment performs expansion processing on the damaged portion detection image [1] G200 in which at least binary information is stored for each pixel in a region that is a damaged portion pixel. It is characterized by.
According to this aspect, the damaged portion detection image [1] generation unit 200 expands the damaged portion detection image [1] G200, which stores at least binary information for each pixel, into a region that is a damaged portion pixel. I do.
As a result, the damaged portion detection image [1] G200, which stores at least binary information for each pixel, can be expanded in the region of the damaged portion pixel.

<Sixth aspect>
The damaged portion detection image [1] generation unit 200 of the present embodiment is characterized in that each pixel of the damaged portion detection image [1] G200 is identified into two types, a black damaged portion pixel and a white damaged portion pixel.
According to this aspect, the damaged portion detection image [1] generation unit 200 identifies each pixel of the damaged portion detection image [1] G200 into two types, a black damaged portion pixel and a white damaged portion pixel.
As a result, each pixel of the damaged portion detection image [1] G200 can be identified into two types, a black damaged portion pixel and a white damaged portion pixel.

<7th aspect>
The damaged part detection image [1] generation part 200 of this aspect is
Damaged portion detection image [1] As an identification condition used to identify a pixel of G200 as a black damaged portion pixel.
B1: The pixel value (P) of the original image G100 has a brightness (L *) smaller than the pixel value (Q) of the background evaluation image G110 L * (P) <L * (Q)
B2: The color difference (ΔE *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T1) ΔE * (P, Q)> T1
B3: The saturation difference (ΔC *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T2) ΔC * (P, Q) <T2
B4: The saturation (C *) of the pixel value (P) of the original image G100 is larger than the set threshold value (T3) C * (P)> T3
Have,
When all of B1 to B4 are satisfied, it is characterized in that a certain pixel is identified as a black damaged portion pixel.
According to this aspect, the damaged portion detection image [1] generation unit 200
Damaged portion detection image [1] As an identification condition used to identify a pixel of G200 as a black damaged portion pixel.
B1: The pixel value (P) of the original image G100 has a brightness (L *) smaller than the pixel value (Q) of the background evaluation image G110 L * (P) <L * (Q)
B2: The color difference (ΔE *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T1) ΔE * (P, Q)> T1
B3: The saturation difference (ΔC *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T2) ΔC * (P, Q) <T2
B4: The saturation (C *) of the pixel value (P) of the original image G100 is larger than the set threshold value (T3) C * (P)> T3
Have,
When all of B1 to B4 are satisfied, a certain pixel is identified as a black damaged portion pixel.
As a result, when all of the above-mentioned B1 to B4 are satisfied as the identification condition used to identify a pixel of the damaged portion detection image [1] G200 as a black damaged portion pixel, a certain pixel is a black damaged portion pixel. Can be identified as.

<8th aspect>
The damaged part detection image [1] generation part 200 of this aspect is
Damaged portion detection image [1] As an identification condition used to identify a pixel of G200 as a white damaged portion pixel.
W1: The pixel value (P) of the original image G100 has a higher brightness (L *) than the pixel value (Q) of the background evaluation image G110 L * (P)> L * (Q)
W2: The color difference (ΔE *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T4) ΔE * (P, Q)> T4
W3: The saturation difference (ΔC *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T5) ΔC * (P, Q) <T5.
W4: The saturation (C *) of the pixel value (P) of the original image G100 is smaller than the set threshold value (T6) C * (P) <T6
W5: The brightness difference (ΔL *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T7) ΔL * (P, Q)> T7
Have,
When W1 and W2 are satisfied and W3 and W4 are satisfied, or W1 and W2 are satisfied and W. When the condition 5 is satisfied, it is characterized in that a certain pixel is identified as a white-damaged portion pixel.
According to this aspect, the damaged portion detection image [1] generation unit 200
Damaged portion detection image [1] As an identification condition used to identify a pixel of G200 as a white damaged portion pixel.
W1: The pixel value (P) of the original image G100 has a higher brightness (L *) than the pixel value (Q) of the background evaluation image G110 L * (P)> L * (Q)
W2: The color difference (ΔE *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T4) ΔE * (P, Q)> T4
W3: The saturation difference (ΔC *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is smaller than the set threshold value (T5) ΔC * (P, Q) <T5.
W4: The saturation (C *) of the pixel value (P) of the original image G100 is smaller than the set threshold value (T6) C * (P) <T6
W5: The brightness difference (ΔL *) between the pixel value (P) of the original image G100 and the pixel value (Q) of the background evaluation image G110 is larger than the set threshold value (T7) ΔL * (P, Q)> T7
Have,
When W1 and W2 are satisfied and W3 and W4 are satisfied, or W1 and W2 are satisfied and W. When the condition 5 is satisfied, a certain pixel is identified as a white damaged portion pixel. As a result, the above-mentioned identification condition used for identifying a certain pixel of the damaged portion detection image [1] G200 as a white damaged portion pixel is described above. When W1 and W2 are satisfied and W3 and W4 are satisfied, or W1 and W2 are satisfied and W. When 5 is satisfied, it can be identified that a certain pixel is a white-damaged portion pixel.

<9th aspect>
The damaged portion detection image [1] generation unit 200 of the present embodiment is characterized in that each setting threshold value related to each identification condition is corrected based on the brightness of the background evaluation image G110 and the pixel value of the flat evaluation image G120. ..
According to this aspect, the damaged portion detection image [1] generation unit 200 corrects each setting threshold value related to each identification condition based on the brightness of the background evaluation image G110 and the pixel value of the flat evaluation image G120.
Thereby, each setting threshold value related to each identification condition can be corrected based on the brightness of the background evaluation image G110 and the pixel value of the flat evaluation image G120.

<10th aspect>
In the damaged portion detection image [1] generation unit 200 of this aspect, the pixel of the damaged portion detection image [1] G200 does not correspond to the identification condition related to the black damaged portion pixel or the identification condition related to the white damaged portion pixel. In some cases, it is characterized in that a certain pixel is used as a background pixel.
According to this aspect, the damaged portion detection image [1] generation unit 200 corresponds to the identification condition in which a pixel of the damaged portion detection image [1] G200 corresponds to the black damaged portion pixel or the white damaged portion pixel. If not, a certain pixel is used as a background pixel.
As a result, when a certain pixel of the damaged portion detection image [1] G200 does not meet the identification condition related to the black damaged portion pixel or the identification condition related to the white damaged portion pixel, the certain pixel can be used as the background pixel. it can.

<11th aspect>
The image processing method of this embodiment is an image processing method using an image processing device that detects a damaged portion existing in the original image G100, and evaluates the background pixel of each pixel by performing smoothing processing on the original image. The flat evaluation image G120 representing the evaluation related to the flatness of the original image G100 by performing the background evaluation image generation steps (S100a to S110a) for generating the background evaluation image G110 to be represented and the edge detection process and the smoothing process on the original image G100. Based on the difference value between the flat evaluation image generation steps (S100b to S120a), the original image G100 and the background evaluation image G110, and the pixel value of the flat evaluation image G120, the damaged portion region is identified and damaged. Part detection image [1] It is characterized in that the damaged part detection image generation step (S100c to S200) for generating G200 is executed.
Since the operation and effect of the eleventh aspect are the same as those of the first aspect, the description thereof will be omitted.

24 ... Image processing unit, 100 ... Original image acquisition unit, 110 ... Background evaluation image generation unit, 120 ... Flat evaluation image generation unit, 200 ... Damaged part detection image [1] Generation unit, 210 ... Damaged part detection image [2] Generation part, 300 ... Damaged part repair image [1] Generation part, 310 ... Damaged part repair image [2] Generation part

Japanese Patent No. 3877916

Claims (11)

An image processing device that detects damaged parts existing in the original image.
A background evaluation image generation means for generating a background evaluation image representing the evaluation of pixels that are the background of each pixel by performing smoothing processing on the original image.
A flat evaluation image generation means for generating a flat evaluation image representing an evaluation related to the flatness of the original image by performing edge detection processing and smoothing processing on the original image.
A damaged portion detection image generating means that identifies a damaged portion and generates a damaged portion detection image based on a difference value between the original image and the background evaluation image and a pixel value of the flat evaluation image.
An image processing device characterized by comprising. The image processing apparatus according to claim 1, wherein the background evaluation image generation means generates the background evaluation image by performing a smoothing process on the original image using a median filter. The flat evaluation image generation means generates an adjacent difference image by performing edge detection processing on the original image using a sobel filter, and the Gaussian filter, the averaging filter, and the median filter on the adjacent difference image. The image processing apparatus according to claim 1, wherein the flat evaluation image is generated by performing a smoothing process using any one of them. The damaged portion detection image generation means distinguishes between background pixels and damaged portion pixels based on the original image, the background evaluation image, and the flat evaluation image, and generates a damaged portion detection image having at least binary information. The image processing apparatus according to claim 1, wherein the image processing apparatus is used. The damaged portion detection image generation means according to claim 1, wherein the damaged portion detection image in which at least binary information is stored for each pixel is expanded in a region that is a damaged portion pixel. Image processing equipment. The image processing apparatus according to claim 5, wherein the damaged portion detection image generation means identifies each pixel of the damaged portion detection image into two types, a black damaged portion pixel and a white damaged portion pixel. The damaged part detection image generation means
As an identification condition used to identify a pixel having the damaged portion detection image as a black damaged portion pixel,
B1: The pixel value (P) of the original image is smaller in brightness (L *) than the pixel value (Q) of the background evaluation image L * (P) <L * (Q)
B2: The color difference (ΔE *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is larger than the set threshold value (T1) ΔE * (P, Q)> T1
B3: The saturation difference (ΔC *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is smaller than the set threshold value (T2) ΔC * (P, Q) <T2
B4: The saturation (C *) of the pixel value (P) of the original image is larger than the set threshold value (T3) C * (P)> T3
Have,
The image processing apparatus according to claim 6, wherein when all of B1 to B4 are satisfied, the certain pixel is identified as a black damaged portion pixel. The damaged part detection image generation means
As an identification condition used to identify a pixel having the damaged portion detection image as a white damaged portion pixel,
W1: The pixel value (P) of the original image has a higher brightness (L *) than the pixel value (Q) of the background evaluation image L * (P)> L * (Q)
W2: The color difference (ΔE *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is larger than the set threshold value (T4) ΔE * (P, Q)> T4
W3: The saturation difference (ΔC *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is smaller than the set threshold value (T5) ΔC * (P, Q) <T5
W4: The saturation (C *) of the pixel value (P) of the original image is smaller than the set threshold value (T6) C * (P) <T6
W5: The brightness difference (ΔL *) between the pixel value (P) of the original image and the pixel value (Q) of the background evaluation image is larger than the set threshold value (T7) ΔL * (P, Q)> T7
Have,
When the W1 and the W2 are satisfied and the W3 and the W4 are satisfied, or when the W1 and W2 are satisfied and the W. The image processing apparatus according to claim 6, wherein when the condition 5 is satisfied, the certain pixel is identified as a white-damaged portion pixel. Claim 7 or 8 is characterized in that the damaged portion detection image generation means corrects each setting threshold value related to each of the identification conditions based on the brightness of the background evaluation image and the pixel value of the flat evaluation image. The image processing apparatus according to. When the pixel with the damaged portion detection image does not meet the identification condition related to the black damaged portion pixel or the identification condition related to the white damaged portion pixel, the damaged portion detection image generation means obtains the certain pixel. The image processing apparatus according to claim 7 or 8, wherein the background pixel is used. This is an image processing method using an image processing device that detects damaged parts existing in the original image.
A background evaluation image generation step of generating a background evaluation image representing the evaluation of pixels that are the background of each pixel by performing smoothing processing on the original image, and
A flat evaluation image generation step of generating a flat evaluation image representing an evaluation related to the flatness of the original image by performing edge detection processing and smoothing processing on the original image, and
A damaged portion detection image generation step of identifying a damaged portion region and generating a damaged portion detection image based on a difference value between the original image and the background evaluation image and a pixel value of the flat evaluation image.
An image processing method characterized by executing.

Images