JP2006031440A - Image processing method, image processing apparatus, image processing program and image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize image processing adaptable to various preferences about photographic printing. <P>SOLUTION: An image processing apparatus 200 calculates predetermined characteristic values from image information acquired by an image information acquisition part 4, and calculates tone correction parameters by using a first neural network whose input signals are the calculated characteristic values and whose output signals are the tone correction parameters of tone conversion of the image information. According to the calculated tone correction parameters, the image information is tone-converted to create a corrected image. If an operator operates a command input part 9 to command a correction to the created corrected image, a neuro-learning part 7 executes learning of the first neural network, using levels of correction to the created corrected image as teacher data, and recalculates the tone correction parameters by using the learned first neural network. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing method, an image processing apparatus, an image processing program, and an image processing system that perform image processing on image information.

  In recent years, a technique for obtaining a photographic print using a photographic printing machine or a digital printer from image information (photographed image) obtained by photographing means such as a silver salt film or a digital still camera (hereinafter referred to as DSC) has been widely spread. ing. In general, a captured image does not have a preferable image quality as a photographic print as it is, and therefore it is necessary to perform various image quality adjustments on the captured image to generate a preferable image.

  For example, in the case of a photographic printer, the actual image quality adjustment is performed by adjusting the exposure amount (exposure time, exposure intensity, color tone of the exposure light source) when printing the developed film image on photographic paper, and adjusting the exposure amount. This is done by printing on photographic paper. Further, image quality adjustment is performed by applying predetermined image processing to a photographed image obtained by photographing with DSC or a photographed image obtained by reading a developed silver salt film with a film scanner. The captured image that has undergone image quality adjustment is printed out by a digital printer or the like, or stored in a recording medium such as a memory card.

  In the image quality adjustment, it is necessary to acquire information on how to adjust to obtain a preferable image, that is, an image adjustment condition. Various methods have been proposed for automatically acquiring the image adjustment conditions. For example, Patent Document 1 acquires information on a skin color area from a photographed image, detects an area corresponding to a human face from the acquired skin color area information, and corresponds to the detected face. An image processing technique for performing gradation conversion for finishing an image of a region to a predetermined color tone is disclosed.

Hereinafter, conventional image processing will be described with reference to the flowchart of FIG.
First, image information input to the image processing apparatus is acquired (step T1). Next, a skin color area is extracted from the image information acquired in step T1, and a human face is extracted from the extracted skin color area (step T2).

  Next, a characteristic value of the person's face area extracted in step T2 (for example, a statistical value such as an average or variance of pixel signal values) is calculated (step T3) and input based on the calculated characteristic value. Gradation correction is performed on the obtained image information (step T4). Next, an additional correction value for the image information is set (step T5). The initial value of this additional correction value is 0, and the value input by the operator is set as the additional correction value.

  Next, a corrected image is created according to the additional correction value set in step T5 (step T6), and the created corrected image is displayed on the display unit (monitor) (step T7). The operator looks at the correction image displayed on the display unit, determines whether or not the correction result is good, and inputs the determination result.

When a determination result indicating that the correction result of the correction image is good is input (step T8; YES), the image processing ends. When the correction result of the corrected image is not good (step T8; NO), an additional correction value is input by the operator (step T9), the input additional correction value is set (step T5), and steps T6 to T8 are performed. The process is repeated.
JP-A-6-67320

  However, with the conventional image processing technology, the face image can be finished to a uniform gradation by automatic image processing, but a uniform finished image ignoring the subject's environment and the individual characteristics of the subject There was a problem that it could only be obtained. In particular, when the technique described in Patent Document 1 is applied to a photo print service, it is difficult to perform image processing applied to each store because the preference for preferred printing differs depending on the store providing the photo print service. Met. On the other hand, if the image adjustment result changes dramatically with the improvement of image processing, there is a risk of confusion for stores and customers that provide photo print services.

  An object of the present invention is to realize image processing that can be adapted to various preferences relating to photographic prints.

  In order to solve the above problem, the invention according to claim 1 is a characteristic value calculating step for calculating a predetermined characteristic value from the input image information, and the calculated characteristic value is used as an input signal, and a step for the image information is calculated. A correction parameter calculating step of calculating a gradation correction parameter using a first neural network having a gradation correction parameter indicating a degree of tone conversion as an output signal; and the image information based on the calculated gradation correction parameter. A correction image generation step of generating a correction image by performing tone conversion, and when the correction for the generated correction image is instructed, the correction amount for the generated correction image is used as teacher data for the first neural network. And a first learning step for performing network learning.

  The invention according to claim 2 includes a face area detection step of detecting a face area of a person from the input image information in the image processing method according to claim 1, wherein the characteristic value calculation step includes: A characteristic value relating to the detected face area is calculated.

  The invention according to claim 3 includes a face candidate area extracting step of extracting a face candidate area having a possibility of a human face from the input image information in the image processing method according to claim 2, In the face area detection step, a face area is detected from the extracted face candidate areas using a second neural network that has learned human face detection processing, and in the characteristic value calculation step, the extracted face The response intensity of the output signal of the second neural network and / or the characteristic value of the detected face area, which indicates the possibility that the candidate area is a face area, is calculated.

  According to a fourth aspect of the present invention, in the image processing method according to any one of the first to third aspects, an acquisition step of acquiring a third neural network having a configuration different from the first neural network; The second learning step of learning the third neural network using the output signal of the first neural network as teacher data, and the learning result of the learned third neural network satisfy a predetermined condition A replacement step of replacing the first neural network with the third neural network.

  According to a fifth aspect of the present invention, in the image processing method according to any one of the first to third aspects, an acquisition step of acquiring a third neural network having a configuration different from the first neural network; An input step of inputting a first correction value for further correcting the corrected image created based on the gradation correction parameter calculated by the first neural network, and based on the first correction value A correction value calculating step of calculating a second correction value for correcting the corrected image created based on the gradation correction parameter calculated by the third neural network, and the calculated second correction value , Learning data of the third neural network as teacher data, the first correction value, the learned third neural network If the relationship of the second correction value calculated by using a predetermined condition is satisfied, and characterized in that it comprises a replacement step of replacing the first neural network to the third neural network.

  A sixth aspect of the present invention is the image processing method according to any one of the first to fifth aspects, wherein the characteristic value calculating step indicates a light distribution condition when the input image is photographed. It is characterized in that a value is calculated.

  The invention according to claim 7 is the image processing method according to claim 6, wherein the light distribution condition includes a backlight state and / or a flash light utilization state.

  The invention according to claim 8 is the image processing method according to any one of claims 1 to 7, wherein the input image information is a color image, and the gradation calculated in the correction parameter calculation step. A division parameter calculation step of dividing the correction parameter at a predetermined ratio and calculating the first correction parameter and the second correction parameter; and in the correction image creation step, the input image information is converted into luminance-color difference information. By applying the first correction parameter to the luminance information when expressed, the luminance information is subjected to gradation conversion, and each input color information is expressed as RGB information. On the other hand, the color information is subjected to gradation conversion by applying the second correction parameter.

  According to a ninth aspect of the present invention, in the image processing method according to any one of the first to eighth aspects, the correction image creating step uses a correction coefficient set in advance based on a shading characteristic in a negative positive method. The image information is converted.

The invention according to claim 10 is the image processing method according to any one of claims 1 to 8, based on an attribute information acquisition step of acquiring attribute information of the image information, and the acquired attribute information. Determining whether the image information is a natural image obtained by photographing, and
If it is determined in the determination step that the image information is a natural image, the correction information creation step converts the image information using a correction coefficient set in advance based on a shading characteristic in a negative-positive method. It is characterized by that.

  The invention according to claim 11 is a characteristic value calculating means for calculating a predetermined characteristic value from the input image information, and a level indicating the degree of gradation conversion with respect to the image information using the calculated characteristic value as an input signal. A correction parameter calculation means for calculating a gradation correction parameter using a first neural network having a tone correction parameter as an output signal, and gradation conversion of the image information based on the calculated gradation correction parameter. When a correction image generation unit for generating a correction image and correction for the generated correction image are instructed, the first neural network is learned using the correction amount for the generated correction image as teacher data. 1 learning means.

  A twelfth aspect of the present invention is the image processing apparatus according to the eleventh aspect, further comprising face area detection means for detecting a face area of a person from the input image information, wherein the characteristic value calculation means A characteristic value related to the detected face area is calculated.

  The invention according to claim 13 is the image processing apparatus according to claim 12, further comprising face candidate region extraction means for extracting a face candidate region that may be a human face from the input image information. The face area detecting means detects a face area from the extracted face candidate areas using a second neural network that has learned human face detection processing, and the characteristic value calculating means is configured to detect the extracted face. It is characterized in that the response intensity of the output signal of the second neural network and / or the characteristic value of the detected face area, which indicates the possibility that the candidate area is a face area, is calculated.

  According to a fourteenth aspect of the present invention, in the image processing device according to any one of the eleventh to thirteenth aspects, an acquisition unit that acquires a third neural network having a configuration different from that of the first neural network; , Second learning means for performing learning of the third neural network using the output signal of the first neural network as teacher data, and a learning result of the learned third neural network satisfying a predetermined condition In this case, a replacement means for replacing the first neural network with the third neural network is provided.

  According to a fifteenth aspect of the present invention, in the image processing apparatus according to any one of the eleventh to thirteenth aspects, the acquisition means acquires a third neural network having a configuration different from the first neural network. , Input means for inputting a first correction value for further correcting a correction image created based on the gradation correction parameter calculated by the first neural network, and based on the first correction value Correction value calculating means for calculating a second correction value for correcting the corrected image created based on the gradation correction parameter calculated by the third neural network, and the calculated second correction value Using the second learning means for learning the third neural network as teacher data, the first correction value, and the learned third neural network. If the relationship of the second correction value calculated by using the work satisfies a predetermined condition, is characterized by and a replacement means for replacing the first neural network to the third neural network.

  According to a sixteenth aspect of the present invention, in the image processing apparatus according to any one of the eleventh to fifteenth aspects, the characteristic value calculating means indicates a light distribution condition when the input image is taken. It is characterized by calculating a value.

  According to a seventeenth aspect of the present invention, in the image processing apparatus according to the sixteenth aspect, the light distribution condition includes a backlight state and / or a flash light utilization state.

  According to an eighteenth aspect of the present invention, in the image processing device according to any one of the eleventh to eleventh aspects, the input image information is a color image, and the gradation calculated by the correction parameter calculating unit. Division parameter calculation means is provided for dividing the correction parameter at a predetermined ratio and calculating the first correction parameter and the second correction parameter, and the correction image creation means converts the input image information into luminance-color difference information. By applying the first correction parameter to the luminance information when expressed, the luminance information is subjected to gradation conversion, and for each color information when the input image information is expressed by RGB information. In this case, the color information is subjected to gradation conversion by applying the second correction parameter.

  According to a nineteenth aspect of the present invention, in the image processing device according to any one of the eleventh to eleventh aspects, the correction image creating means uses a correction coefficient that is set in advance based on a shading characteristic in a negative positive system. The image information is converted.

  According to a twentieth aspect of the present invention, in the image processing device according to any one of the eleventh to eleventh aspects, an attribute information acquisition unit that acquires attribute information of the image information and the acquired attribute information. Determining means for determining whether or not the image information is a natural image obtained by photographing, and if the determination means determines that the image information is a natural image, the correction The image creating means converts the image information using a correction coefficient set in advance based on the shading characteristics in the negative-positive method.

  According to a twenty-first aspect of the present invention, a computer that executes image processing has a characteristic value calculation function for calculating a predetermined characteristic value from input image information, and the calculated characteristic value is used as an input signal. A correction parameter calculation function for calculating a gradation correction parameter using a first neural network that uses a gradation correction parameter indicating the degree of gradation conversion as an output signal, and the image based on the calculated gradation correction parameter When a correction image generation function for generating a corrected image by gradation conversion of information and correction for the generated correction image are instructed, the correction amount for the generated correction image is used as teacher data as the first data. And a first learning function for performing learning of a neural network.

  According to a twenty-second aspect of the present invention, in the image processing program according to the twenty-first aspect, a face area detecting function for detecting a human face area from the input image information is realized, and the characteristic value calculating function is realized. When performing, a characteristic value related to the detected face area is calculated.

  According to a twenty-third aspect of the present invention, in the image processing program according to the twenty-second aspect, a face candidate area extracting function for extracting a face candidate area having a possibility of a human face from the input image information is realized. When the face area detection function is realized, the characteristic value calculation function is realized by detecting a face area from the extracted face candidate areas using a second neural network that has learned human face detection processing. Calculating the reaction intensity of the output signal of the second neural network and / or the characteristic value of the detected face area, which indicates the possibility that the extracted face candidate area is a face area. It is a feature.

  According to a twenty-fourth aspect of the present invention, in the image processing program according to any one of the twenty-first to twenty-third aspects, an acquisition function for acquiring a third neural network having a configuration different from the first neural network; The second learning function for performing learning of the third neural network using the output signal of the first neural network as teacher data, and the learning result of the learned third neural network satisfy a predetermined condition In this case, a replacement function for replacing the first neural network with the third neural network is realized.

  According to a twenty-fifth aspect of the present invention, in the image processing program according to any one of the twenty-first to twenty-third aspects, an acquisition function for acquiring a third neural network having a configuration different from the first neural network; An input function for inputting a first correction value for further correcting a correction image created based on the gradation correction parameter calculated by the first neural network, and based on the first correction value A correction value calculation function for calculating a second correction value for correcting the corrected image created based on the gradation correction parameter calculated by the third neural network, and the calculated second correction value , The second learning function for performing learning of the third neural network using the teacher data as the training data, the first correction value, and the learned third neural network. If the relationship of the second correction value calculated by using the network satisfies a predetermined condition, to realize a replacement function to replace the first neural network to the third neural network.

  According to a twenty-sixth aspect of the present invention, in the image processing program according to any one of the twenty-first to twenty-fifth aspects, when the characteristic value calculation function is realized, an arrangement when the input image is taken is provided. It is characterized by calculating a value indicating the light condition.

  According to a twenty-seventh aspect of the present invention, in the image processing program according to the twenty-sixth aspect, the light distribution condition includes a backlight state and / or a flash light utilization state.

  The invention according to claim 28 is the image processing program according to any one of claims 21 to 27, wherein the input image information is a color image, and the gradation calculated by the correction parameter calculation function. When the correction parameter is divided at a predetermined ratio, the division parameter calculation function for calculating the first correction parameter and the second correction parameter is realized, and the correction image creation function is realized, the input image information is Each of the cases where the first correction parameter is applied to the luminance information when expressed by luminance-chrominance information to perform gradation conversion of the luminance information, and the input image information is expressed by RGB information. The color information is subjected to gradation conversion by applying the second correction parameter to the color information.

  According to a twenty-ninth aspect of the present invention, in the image processing program according to any one of the twenty-first to twenty-eighth aspects, when the correction image creation function is realized, the image processing program is preset based on a shading characteristic in a negative positive method. The image information is converted using a correction coefficient.

  A thirty-third aspect of the invention is the image processing program according to any one of the twenty-first to twenty-eighth aspects, based on an attribute information acquisition function for acquiring attribute information of the image information and the acquired attribute information. A determination function for determining whether or not the image information is a natural image obtained by photographing, and when the correction image creation function is realized, the determination function allows the image information to be When it is determined that the image is a natural image, the image information is converted using a correction coefficient set in advance based on a shading characteristic in the negative-positive method.

  The invention described in claim 31 is an image input device for inputting image information to be processed, an image processing device for performing various image processing on the input image information, and the image processing. An image processing system including an image output device that outputs image information, wherein the image processing device includes a characteristic value calculation unit that calculates a predetermined characteristic value from image information input from the image input device; Correction parameter calculation means for calculating a gradation correction parameter using a first neural network having the calculated characteristic value as an input signal and a gradation correction parameter indicating the degree of gradation conversion for image information as an output signal; A corrected image creating means for creating a corrected image by gradation-converting the image information based on the calculated gradation correction parameter; and the created correction If correction for the image is instructed, it is characterized by comprising a first learning means for performing learning of the first neural network a correction amount for the created corrected image as teacher data.

  According to a thirty-second aspect of the present invention, in the image processing system according to the thirty-first aspect, the image processing device includes a face area detecting unit that detects a face area of a person from the input image information, and the characteristic The value calculating means calculates a characteristic value related to the detected face area.

  According to a thirty-third aspect of the present invention, in the image processing system according to the thirty-second aspect, the image processing device extracts a face candidate region that may be a human face from the input image information. The face area detecting means detects a face area from the extracted face candidate areas using a second neural network that has learned human face detection processing, and the characteristic value calculating means includes: , Calculating the reaction intensity of the output signal of the second neural network and / or the characteristic value of the detected face area, which indicates the possibility that the extracted face candidate area is a face area. .

  The invention according to claim 34 is the image processing system according to any one of claims 31 to 33, wherein the image processing device has a configuration different from that of the first neural network. , Second learning means for learning the third neural network using the output signal of the first neural network as teacher data, and a learning result of the learned third neural network And a replacement means for replacing the first neural network with the third neural network when a predetermined condition is satisfied.

  The invention according to claim 35 is the image processing system according to any one of claims 31 to 33, wherein the image processing apparatus has a configuration different from that of the first neural network. Acquisition means for acquiring the first correction value, input means for inputting a first correction value for further correcting the correction image created based on the gradation correction parameter calculated by the first neural network, and the first Correction value calculation means for calculating a second correction value for correcting a correction image created based on the tone correction parameter calculated by the third neural network based on the correction value of Second learning means for performing learning of the third neural network using the second correction value as teacher data, the first correction value, and the learning value. Replacement means for replacing the first neural network with the third neural network when the relationship between the second correction values calculated using the third neural network satisfies a predetermined condition. It is characterized by.

  A thirty-sixth aspect of the present invention is the image processing system according to any one of the thirty-first to thirty-fifth aspects, wherein the characteristic value calculating means indicates a light distribution condition when the input image is taken. It is characterized by calculating a value.

  A thirty-seventh aspect of the present invention is the image processing system according to the thirty-sixth aspect, wherein the light distribution condition includes a backlight state and / or a flash light utilization state.

  According to a thirty-eighth aspect of the present invention, in the image processing system according to any one of the thirty-first to thirty-seventh aspects, the input image information is a color image, and the image processing apparatus includes the correction parameter calculating unit. And a division parameter calculation unit that divides the gradation correction parameter calculated in step 1 at a predetermined ratio and calculates a first correction parameter and a second correction parameter, and the correction image creation unit includes the input image information When the first correction parameter is applied to the luminance information when the image is expressed by luminance-color difference information, the luminance information is subjected to gradation conversion, and the input image information is expressed by RGB information. The color information is subjected to gradation conversion by applying the second correction parameter to each color information.

  According to a thirty-ninth aspect of the present invention, in the image processing system according to any one of the thirty-first to thirty-eighth aspects, the correction image creating means uses a correction coefficient that is set in advance based on a shading characteristic in a negative positive system. The image information is converted.

  The invention according to claim 40 is the image processing system according to any one of claims 31 to 38, wherein the image processing apparatus acquires attribute information acquisition means for acquiring attribute information of the image information, and the acquisition Determination means for determining whether the image information is a natural image obtained by photographing based on the attribute information thus obtained, and the determination means determines that the image information is a natural image. In this case, the corrected image creating means converts the image information using a correction coefficient set in advance based on the shading characteristic in the negative-positive method.

  According to the present invention, when the correction for the created corrected image is instructed, the correction amount for the created corrected image is learned as the teacher data of the first neural network. By correcting the image information using a neural network, an image that suits the operator's preference can be obtained.

  Further, the characteristic value of the person's face area and the reaction intensity of the second neural network are calculated using the second neural network that has learned the process of detecting the face of the person included in the input image information, and the calculated face area By using the characteristic value and the reaction intensity as input signals of the first neural network, it is possible to obtain an image with higher quality and suitable for the operator's preference.

  Further, before using the third neural network having the new configuration, the third neural network is learned using the output signal of the first neural network learned according to the preference of the operator. Therefore, it is possible to smoothly shift to a new neural network that matches the operator's preference.

  Further, by determining the light distribution condition at the time of image capturing from the input image information and making the determination result one of the input signals to the first neural network, it is possible to obtain gradation correction parameters suitable for the light distribution condition. And a higher quality image can be obtained.

  Further, the gradation correction parameter calculated by the first neural network is divided into two correction parameters, the first correction parameter is applied to the luminance information of the correction target image, and gradation conversion is performed. The second correction parameter Is applied to the RGB color information of the image to be corrected to convert the gradation, thereby reducing the change in saturation (image vividness) that accompanies the change in gradation and facilitating additional correction according to operator instructions. become.

  In addition, in the correction image creation process, it is possible to obtain a correction image that approximates negative-positive printing by performing processing that reproduces the shading characteristics at the time of printing in the negative-positive method, so that additional correction by the operator is easy. Become.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
First, the configuration in the present embodiment will be described.

  FIG. 1 shows a configuration of an image processing apparatus 100 according to an embodiment of the present invention. The image processing apparatus 100 is a photographic print system that optically prints a developed image photographed on a film on photographic paper. As shown in FIG. 1, a light source unit 1, an imaging unit 2, an exposure control unit 3, and image information. The acquisition unit 4, the image determination unit 5, the additional correction data storage unit 6, the neuro learning unit 7, the image processing unit 8, the instruction input unit 9, and the display unit 10 are configured.

  The light emitted from the light source unit 1 is transmitted through the color adjustment filters (Y, M, C), and the color tone is adjusted. The light is diffusely reflected in the light diffusing member, homogenized, and irradiated onto the film. The light transmitted through the film passes through the lens, forms an image on the film on the photographic paper, and is exposed under the control of the exposure control unit 3. The exposed photographic paper is developed by a development processing device (not shown) to produce a photograph. The light transmitted through the film is appropriately guided to the imaging unit 2 by a movable mirror. The imaging unit 2 collects the optical image input from the movable mirror and outputs it to the image information acquisition unit 4 as film image information.

  The exposure control unit 3 receives the gradation correction parameter (parameter indicating the degree of gradation conversion) calculated by the image determination unit 5 and the additional correction value input by the instruction input unit 9 from the image processing unit 8, The exposure time of the film and the color balance of the light source are determined based on the tone correction parameter and the additional correction value, and the color adjustment filter and the shutter are controlled according to the determined exposure time and color balance.

  The image information acquisition unit 4 acquires the image information input from the imaging unit 2 and outputs it to the image determination unit 5.

  The image determination unit 5 determines various characteristic values from the image information acquired from the image information acquisition unit 4 (additional information such as average pixel signal values, statistical values such as dispersion, lens focal length, focus position, etc., light distribution conditions) Etc.), a first hierarchical neural network having the calculated characteristic value as an input signal and a tone correction parameter for tone conversion of image information as an output signal is acquired from the neuro-learning unit 7, A gradation correction parameter is calculated using the first hierarchical neural network. The processing executed in the image determination unit 5 will be described in detail later with reference to FIG.

  The additional correction data storage unit 6 stores teacher data (described later) necessary for learning of the neural network, including the additional correction value (correction amount) input by the instruction input unit 9.

  The neuro learning unit 7 learns the first hierarchical neural network using the additional correction value (correction amount) stored in the additional correction data storage unit 6 as teacher data. In addition, the neuro-learning unit 7 performs learning of a third hierarchical neural network (described later) having a configuration different from that of the first hierarchical neural network. Further, the neuro-learning unit 7 executes a neuro replacement process for replacing the first hierarchical neural network with the newly configured third hierarchical neural network (see FIGS. 18 and 19). The hierarchical neural network applied in this embodiment will be described later with reference to FIGS.

  The image processing unit 8 performs gradation conversion on the image information to be corrected based on the gradation correction parameter calculated by the image determination unit 5 and / or the additional correction value input by the instruction input unit 9, thereby correcting the corrected image. And the generated corrected image is output to the display unit 10. When there is an instruction to output a corrected image, the image processing unit 8 outputs corrected image information (tone correction parameter, additional correction value information, etc.) to the exposure control unit 3. The correction image creation process will be described in detail later with reference to FIGS.

  The instruction input unit 9 includes an input device such as a keyboard and a mouse, and outputs an operation signal generated by operating the input device to the image processing unit 8. The instruction input unit 9 may include a touch panel that is provided so as to cover the display screen of the display unit 10. The touch panel detects coordinates instructed by a touch based on a reading principle such as an electromagnetic induction type, a magnetostriction type, and a pressure sensitive type, and outputs the detected coordinates to the image processing unit 8 as a position signal.

  The display unit 10 has a display screen such as an LCD (Liquid Crystal Display), and performs a required display according to a display control signal input from the image processing unit 8.

  The image processing apparatus to which the present invention is applied is not limited to the image processing apparatus 100 that is a photo print system for optical printing as shown in FIG. FIG. 2 shows a configuration of an image processing apparatus 200 as another example of the image processing apparatus to which the present invention is applied. Hereinafter, in the image processing apparatus 200, the same components as those of the image processing apparatus 100 of FIG. 1 are denoted by the same reference numerals, and the description of the functions of the components having the same functions is omitted.

  As shown in FIG. 2, the image processing apparatus 200 includes an image information acquisition unit 4, an image determination unit 5, an additional correction data storage unit 6, a neuro learning unit 7, an image processing unit 8, an instruction input unit 9, a display unit 10, A reduced image creation unit 11, a DSC (digital still camera) 12, a document scanner 13, a recording media driver 14, a communication control unit 15, an image information output unit 16, a printer unit 17, and a recording media writing unit 18 are configured.

  The image information acquisition unit 4 acquires digital image information input via the DSC 12, the document scanner 13, the recording media driver 14, and the communication control unit 15, and outputs the digital image information to the reduced image creation unit 11 or the image processing unit 8.

  The image processing unit 8 performs gradation conversion on the image information to be corrected based on the gradation correction parameter calculated by the image determination unit 5 and / or the additional correction value input by the instruction input unit 9, thereby correcting the corrected image. And the generated corrected image is output to the display unit 10. If there is an instruction to output a corrected image, the image processing unit 8 creates a corrected image corresponding to the designated output destination and outputs the corrected image to the image information output unit 16.

  The reduced image creating unit 11 creates a reduced image (thumbnail image) from the image information acquired from the image information acquiring unit 4 as necessary (for example, when an instruction is given from the operator), and sends it to the image determining unit 5. Output.

  The DSC 12 includes an optical lens unit for photographing including a focusing lens, a zoom lens, a shutter, a diaphragm, a photoelectric conversion unit including an imaging element such as a CCD (Charge Coupled Device), and the like, and is obtained by photographing a subject. The captured image is photoelectrically converted to acquire a digital image signal, and the acquired digital image signal is output to the image information acquisition unit 4.

  The document scanner 13 includes a light source, a CCD (Charge-Coupled Device), an A / D converter, and the like. The document scanner 13 irradiates a document such as a photographic print placed on a document table with light from the light source, and reflects the reflected light by the CCD. The digital image signal is converted into an electrical signal (analog signal), and the analog signal is converted into a digital signal by an A / D converter to acquire a digital image signal, and the acquired digital image signal is output to the image information acquisition unit 4.

  The recording media driver 14 is a CD-R, a memory stick (registered trademark), a smart media (registered trademark), a compact flash (registered trademark), a multimedia card (registered trademark), an SD memory card (registered trademark), and the like. It is configured to be able to mount media, reads image signals recorded on these recording media, and outputs the read results to the image information acquisition unit 4.

  The communication control unit 15 controls communication between an image processing apparatus 200 and an external device connected to a communication network N such as a LAN (Local Area Network) or the Internet. Specifically, the communication control unit 15 performs control for outputting the image information received from the external device to the image information acquisition unit 4, or the image information acquired from the image information output unit 16 is designated. Control to send to address.

  The image information output unit 16 outputs the image information processed by the image processing unit 8 to a designated output destination (any one of the communication control unit 15, the printer unit 17, and the storage medium writing unit 18).

The printer unit 17 exposes the photosensitive material based on the image information input from the image information output unit 16, develops the exposed photosensitive material, and dries it to create a photographic print.
The storage medium writing unit 18 is configured to be able to mount various recording media, and records the image information processed by the image processing unit 8 on the mounted recording medium.

<neural network>
Next, the neural network used in the image processing of this embodiment will be briefly described. As shown in FIG. 3, this neural network is a hierarchical neural network (hereinafter simply referred to as a neural network) having an input layer, an intermediate layer, and an output layer. The neural network preferably uses a digital neural network chip, but can also be realized by using a general-purpose DSP (Digital Signal Processor) and a dedicated emulation program, or using a normal CPU and an emulation program. It doesn't matter.

ここで、ｆ（ｘ）は一般に式（２）に示すような非線形のシグモイド関数が用いられるが、必ずしもこれに限定されるわけではない。

Each layer of the neural network is composed of structural units called neurons. Except for the input layer, an arbitrary neuron receives a large number of inputs called synapses as shown in FIG. 4 and multiplies each input value x _i by a predetermined weight w _i called bond strength to obtain the sum, It functions as a functional element that gives the output y as a result of evaluating it with a predetermined output function f. The output function f is expressed as Equation (1).

Here, f (x) is generally a non-linear sigmoid function as shown in Equation (2), but is not necessarily limited thereto.

Here, the input weight w _i of each neuron is important information for determining the relationship between the input and output of the neural network, in other words, the operation of the neural network, and is determined by performing a learning operation on the neural network. In FIG. 3, only one intermediate layer is shown, but this can be configured in one or more arbitrary layers. Hierarchical networks are known to be able to achieve any function that can be synthesized if there is only one intermediate layer, but increasing the number of intermediate layers is advantageous in terms of learning efficiency and the number of neurons. It has been known.

The neural network has the following characteristics.
(1) A multi-input / multi-output nonlinear system can be realized with a relatively simple configuration. (2) Each neuron in each layer can be operated independently, and high-speed operation can be expected by parallel processing.
(3) Arbitrary input / output relations can be realized by providing appropriate teacher data for learning.
(4) The system has generalization ability. That is, there is an ability to give a generally correct output even to an unlearned input pattern that is not necessarily given as teacher data.

  Here, the teacher data is a pair of an input pattern and a desired output pattern for the input pattern, and a plurality of teacher data are usually prepared. The operation of determining a desired output pattern for a specific input pattern is generally determined by relying on the subjective judgment of an expert. Major applications of neural networks include (a) nonlinear function approximation and (b) clustering (identification, recognition, classification).

ここで、式（３）の関数ｆ（ｘ）は、式（２）に示した非線形のシグモイド関数、Ｙ_iLはニューロンの出力、Ｕ_iLはニューロンの内部状態、Ｙ_i＜Ｌ−１＞はL−１層のニューロンの出力＝Ｌ層のニューロンの入力、ｗ_ij＜Ｌ＞は結合強度である。 Next, a learning method using teacher data for the neural network will be described. The state of the i-th neuron in the L-th layer with respect to the p-th input pattern is expressed as Equation (3) and Equation (4).

Here, the function f (x) in Expression (3) is the nonlinear sigmoid function shown in Expression (2), Y _iL is the output of the neuron, U _iL is the internal state of the neuron, and Y _i <L−1> is L-1 layer neuron output = L layer neuron input, w _ij <L> is the connection strength.

ただしｋは出力層の番号であり、Ｔ_i（ｐ）はｐ番目の入力パターンに対する望ましい出力パターンである。 In such a definition, the widely used error back propagation learning method (back propagation learning method, hereinafter referred to as BP method) uses the mean square error as an error evaluation function, and the error Define E.

Where k is the output layer number and T _i (p) is the desired output pattern for the pth input pattern.

ここで、∂は偏微分記号（ラウンド）、ηは学習率を表す係数である。Δｗ_ij＜Ｌ＞を各々のｗ_ij＜Ｌ＞に加算することによって学習が進む。なお、この形式のＢＰ法では学習が進んだ結果、教師データとの誤差の絶対値が小さくなってくると、Δｗの絶対値も小さくなり、学習が進まなくなるという現象が指摘されており、その問題点を回避するために、種々の改良が提案されている。 In this case, the correction amount Δw of each bond strength based on the BP method is determined by the following equation.

Here, ∂ is a partial differential symbol (round), and η is a coefficient representing a learning rate. Learning proceeds by adding Δw _ij <L> to each w _ij <L>. In addition, as a result of learning progressed in the BP method of this form, if the absolute value of the error from the teacher data becomes small, the absolute value of Δw also becomes small, and the phenomenon that learning does not progress has been pointed out. Various improvements have been proposed in order to avoid problems.

式（７）の第二項は慣性項（モーメンタム項）と呼ばれ、現在の修正量だけでなく、過去の修正量も考慮することで収束時の振動を抑え、学習を高速化する効果がある。安定化定数αは1．0以下の正の実数で、過去の修正量の考慮の度合いを規定する係数である。 For example, there is a proposal to change the definition of Δw as follows.

The second term in equation (7) is called the inertial term (momentum term), and it has the effect of suppressing the vibration at the time of convergence by considering not only the current correction amount but also the past correction amount, and speeding up learning. is there. The stabilization constant α is a positive real number of 1.0 or less, and is a coefficient that defines the degree of consideration of past correction amounts.

Alternatively, there is also a proposal of dynamically changing the learning rate η so as to satisfy the following expression according to the number of learnings n.

  Furthermore, there is also a proposal that the error evaluation is in a form that cancels out the derivative of the sigmoid function instead of the mean square error. In any case, an output approximate to the teacher data can be obtained by performing the learning operation a sufficient number of times using the BP method.

Next, the operation in this embodiment will be described.
First, the overall flow of image processing executed in the image processing apparatus 100 or 200 will be described with reference to the flowchart of FIG.

  First, image information input to the image information acquisition unit 4 (hereinafter referred to as input image information) is acquired (step S1). Next, the image determination unit 5 calculates various characteristic values from the image information acquired in step S1 (step S2), and extracts a human face candidate region (skin color hue region) (step S4). . In step S2, for example, statistical values such as average and variance of pixel signal values are calculated, and additional information such as lens focal length, in-focus position, and light source information at the time of photographing is acquired.

  When the characteristic value of the input image information is calculated, a light distribution condition determination process for determining a light distribution condition (light source state) at the time of image capturing is performed from the input image information (step S3). The light distribution conditions at the time of photographing include forward light, backlight, and strobe (flash light use state). The light distribution condition determination process in step S3 will be described in detail later with reference to FIG.

  When a human face candidate area is extracted, a signal indicating the face candidate area is used as an input signal, and a second hierarchical neural network that has learned human face detection processing (hereinafter referred to as “neural”). By using (abbreviated), a process for detecting a face area from the extracted face candidate area is performed (step S5). For the face detection, for example, various characteristic values such as [area area / peripheral length] representing the form of the skin color hue area, [shortest diameter / longest diameter], and dispersion of pixel signal values are extracted in step S4. It is calculated for each face candidate region. In step S5, the reaction intensity of the second neuron is also calculated. If the reaction intensity is greater than a predetermined value, it is determined that the face is a “face”.

  Next, the characteristic value of the face area detected in step S5 is acquired (step S6), and the second neuronal reaction intensity with respect to the detected face area is acquired (step S7).

  Next, as shown in FIG. 6, the characteristic value calculated in step S2, the determination result in step S3, the characteristic value acquired in step S6, and the reaction intensity acquired in step S7 are input signals (input parameters). The gradation correction parameter is calculated using the first neuron having the gradation correction parameter as an output signal (step S8).

  FIG. 6 shows a case where the gradation correction parameters that are the output signals of the first neuron are the brightness correction parameter and the contrast correction parameter. In the case of the image processing apparatus 100 shown in FIG. 1, the opening / closing time of the shutter is controlled by the brightness correction parameter calculated by the first neuro. In the case of the image processing apparatus 200 shown in FIG. 2, control of signal value conversion characteristics as will be described later with reference to FIG. 20 based on the brightness correction parameter and / or contrast correction parameter calculated by the first neuron. And gradation correction of the image information is performed.

  Note that the gradation correction parameter calculated in the first neuro may be either the brightness correction parameter, the contrast correction parameter, or another gradation correction parameter, and the number of gradation correction parameters to be output. Accordingly, the number of neurons in the output layer is adjusted.

  Next, an additional correction value for the input image information is set based on the tone correction parameter calculated in step S8 (step S9), and tone conversion processing is performed on the input image information based on the set correction value. As a result, a corrected image is created (step S10). The initial setting of the additional correction value is 0. The correction image creation process (gradation conversion process) in step S10 will be described in detail later with reference to FIGS.

  Next, the corrected image created in step S10 is displayed on the display unit 10 (step S11). The operator observes the correction image displayed on the display unit 10, determines whether or not the correction result is good, and inputs the determination result through the instruction input unit 9.

  When the correction result of the corrected image is not good (step S12; NO), an additional correction value is input by the operation of the instruction input unit 9 by the operator (step S13), and the process returns to step S9 and the input additional correction value is input. Is set (step S9), and the processes of steps S10 to S12 are repeated.

  When the determination result that the correction result of the correction image is good is input (step S12; YES), it is determined whether or not the additional correction amount is 0 (step S14). If it is determined in step S14 that the additional correction amount is not 0 (step S14; NO), the additional correction data is added with the gradation correction parameter and the additional correction amount (referred to as “correction amount”) as teacher data. Accumulated in the storage unit 6 (step S15). As described above, the teacher data includes the desired value of the neuro output (that is, the correction amount in step S15) and the corresponding input signal group (that is, various signal values calculated in steps S2, S3, S6, and S7). In step S15, these input signal groups are also stored in the additional correction data storage unit 6.

  When the number of teacher data accumulated in the additional correction data storage unit 6 exceeds a predetermined number, learning of the first neuro is performed based on these teacher data (step S16), and the learned first neuro The process of step S8 is performed using. The learning in step S16 may be performed periodically.

  In step S14, when it is determined that the additional correction amount is 0 (step S14; YES), the main image processing ends.

  Next, the light distribution condition determination process shown in step S3 of FIG. 5 will be described with reference to the flowchart of FIG.

  First, the captured image data is divided into predetermined image areas, and an occupancy ratio calculation process is performed to calculate an occupancy ratio indicating the ratio of each divided area to the entire captured image data (step S20). The occupation rate calculation process will be described in detail later with reference to FIGS.

  Next, based on the occupation ratio calculated in step S20 and a coefficient set in advance according to the imaging condition, an index (indicator 1 to 5) that specifies the light distribution condition (quantitatively represents the light source state) is calculated. (Step S21). The index calculation process in step S21 will be described in detail later.

  Next, the light distribution condition of the captured image data is determined based on the index calculated in step S21 (step S22), and the main light distribution condition determination process ends. A method for determining the light distribution condition will be described in detail later.

  Next, the occupation rate calculation process shown in step S20 of FIG. 7 will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step S30). FIG. 9 shows an example of a conversion program (HSV conversion program) that obtains a hue value, a saturation value, and a lightness value by converting from RGB to the HSV color system using program code (c language). In the HSV conversion program shown in FIG. 9, the values of the digital image data as the input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, and the saturation The value is OutS, the brightness value is OutV, and the unit is defined as 0 to 255.

  Next, the photographed image data is divided into regions composed of combinations of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S31). Hereinafter, the area division of the captured image data will be described in detail.

The lightness value (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 (v6) , 225 to 255 (v7). Hue (H) is a skin hue hue area (H1 and H2) with a hue value of 0 to 39, 330 to 359, a green hue area (H3) with a hue value of 40 to 160, and a blue hue area with a hue value of 161 to 250 It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the determination of the light distribution condition is small. The flesh-color hue area is further divided into a flesh-color area (H1) and other areas (H2). Hereinafter, among the flesh color hue regions (H = 0 to 39, 330 to 359), a hue color '(H) that satisfies the following equation (9) is defined as a flesh color region (H1), and a region that does not satisfy the equation (9) is ( H2).
10 <Saturation (S) <175,
Hue '(H) = Hue (H) + 60 (when 0 ≤ Hue (H) <300),
Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),
Luminance Y = InR × 0.30 + InG × 0.59 + InB × 0.11
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (9)
Therefore, the number of divided areas of the captured image data is 4 × 7 = 28. Note that brightness (V) can also be used in equation (9).

When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S32), and the main occupancy ratio calculation is performed. The process ends. Assuming that the first occupancy ratio calculated in the divided area composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy ratio in each divided area is expressed as shown in Table 1.

Next, a method for calculating the index 1 and the index 2 will be described.
Table 2 shows the first coefficient necessary for calculating the index 1 that quantitatively indicates the accuracy of strobe shooting obtained by discriminant analysis, that is, the lightness state of the face area at the time of strobe shooting. Show. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupation ratio Rij of each divided area shown in Table 1 is multiplied.

  FIG. 10 shows the lightness (v) -hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (r1) distributed in the skin color hue region of high brightness in FIG. 10, and blue other than that is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region (r2). FIG. 12 shows a first coefficient in the skin color area (H1) and a first coefficient in the other area (green hue area (H3)) as a curve (coefficient curve) that continuously changes over the entire brightness. It is shown. According to Table 2 and FIG. 12, in the region of high brightness (V = 170 to 224), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of the two are different.

従って、H1〜H4領域の和は、下記の式(10-1)〜式(10-4)のように表される。
H1領域の和＝R11×(-44.0)＋R21×(-16.0)＋（中略）...＋R71×(-11.3) (10-1)
H2領域の和＝R12×0.0＋R22×8.6＋（中略）... ＋R72×(-11.1) (10-2)
H3領域の和＝R13×0.0＋R23×(-6.3)＋（中略）...＋R73×(-10.0) (10-3)
H4領域の和＝R14×0.0＋R24×(-1.8)＋（中略）...＋R74×(-14.6) (10-4) When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (10).

Therefore, the sum of the H1 to H4 regions is expressed by the following formulas (10-1) to (10-4).
Sum of H1 area = R11 x (-44.0) + R21 x (-16.0) + (omitted) ... + R71 x (-11.3) (10-1)
Sum of H2 area = R12 x 0.0 + R22 x 8.6 + (omitted) ... + R72 x (-11.1) (10-2)
Sum of H3 area = R13 x 0.0 + R23 x (-6.3) + (omitted) ... + R73 x (-10.0) (10-3)
Sum of H4 area = R14 x 0.0 + R24 x (-1.8) + (omitted) ... + R74 x (-14.6) (10-4)

The index 1 is defined as in Expression (11) using the sum of the H1 to H4 regions shown in Expressions (10-1) to (10-4).
Index 1 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 4.424 (11)

Table 3 shows, for each divided region, the second coefficient necessary for calculating the index 2 that quantitatively indicates the accuracy of backlight photographing obtained by discriminant analysis, that is, the brightness state of the face region at the time of backlight photographing. Show. The coefficient of each divided area shown in Table 3 is a weighting coefficient that is multiplied by the first occupation ratio Rij of each divided area shown in Table 1.

  FIG. 11 shows a lightness (v) -hue (H) plane. According to Table 3, a negative (-) coefficient is used for the occupation ratio calculated from the region (r4) distributed in the intermediate lightness of the flesh color hue region in FIG. 11, and the low lightness (shadow) region ( A positive (+) coefficient is used for the occupation ratio calculated from r3). FIG. 13 shows the second coefficient in the skin color region (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and FIG. 13, the sign of the second coefficient of the intermediate lightness region of the flesh color hue region having the lightness value of 85 to 169 (v4) is negative (−), and the lightness value is 26 to 84 (v2, It can be seen that the sign of the second coefficient in the low brightness (shadow) region of v3) is positive (+), and the sign of the coefficient in both regions is different.

従って、H1〜H4領域の和は、下記の式(12-1)〜式(12-4)のように表される。
H1領域の和＝R11×(-27.0)＋R21×4.5＋（中略）...＋R71×(-24.0) (12-1)
H2領域の和＝R12×0.0＋R22×4.7＋（中略）... ＋R72×(-8.5) (12-2)
H3領域の和＝R13×0.0＋R23×0.0＋（中略）...＋R73×0.0 (12-3)
H4領域の和＝R14×0.0＋R24×(-5.1)＋（中略）...＋R74×7.2 (12-4) When the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 2 is defined as in Expression (12).

Therefore, the sum of the H1 to H4 regions is expressed by the following formulas (12-1) to (12-4).
H1 area sum = R11 x (-27.0) + R21 x 4.5 + (omitted) ... + R71 x (-24.0) (12-1)
Sum of H2 area = R12 x 0.0 + R22 x 4.7 + (omitted) ... + R72 x (-8.5) (12-2)
Sum of H3 area = R13 x 0.0 + R23 x 0.0 + (omitted) ... + R73 x 0.0 (12-3)
H4 area sum = R14 x 0.0 + R24 x (-5.1) + (omitted) ... + R74 x 7.2 (12-4)

The index 2 is defined as in Expression (13) using the sum of the H1 to H4 regions shown in Expressions (12-1) to (12-4).
Index 2 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 1.554 (13)
Since the index 1 and the index 2 are calculated based on the brightness of the captured image data and the distribution amount of the hue, it is effective for determining the light distribution condition when the captured image data is a color image.

  Next, the occupation rate calculation process (step S20 in FIG. 7) for calculating the index 3 will be described in detail with reference to the flowchart in FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step S40). Next, the photographed image data is divided into regions each composed of a combination of the distance from the outer edge of the photographed image screen and brightness, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S41). Hereinafter, the area division of the captured image data will be described in detail.

  FIGS. 15A to 15D show four regions n1 to n4 divided according to the distance from the outer edge of the screen of the captured image data. A region n1 shown in FIG. 15A is an outer frame, a region n2 shown in FIG. 15B is a region inside the outer frame, and a region n3 shown in FIG. A region n4 shown in FIG. 15D is an inner region and is a central region of the captured image screen. Further, the lightness is divided into seven regions v1 to v7 as described above. Therefore, when the captured image data is divided into regions composed of combinations of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S42), and this occupancy ratio calculation is performed. The process ends. If the second occupancy calculated in the divided area composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each divided area is expressed as shown in Table 4.

図１６は、画面領域ｎ１〜ｎ４における第３の係数を、明度全体に渡って連続的に変化する曲線（係数曲線）として示したものである。 Next, a method for calculating the index 3 will be described.
Table 5 shows the third coefficient necessary for calculating the index 3 for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy Qij of each divided area shown in Table 4 is multiplied, and is obtained by discriminant analysis.

FIG. 16 shows the third coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

従って、n1〜n4領域の和は、下記の式(14-1)〜式(14-4)のように表される。
n1領域の和＝Q11×40.1＋Q21×37.0＋（中略）...＋Q71×22.0 (14-1)
n2領域の和＝Q12×(-14.8)＋Q22×(-10.5)＋（中略）...＋Q72×0.0 (14-2)
n3領域の和＝Q13×24.6＋Q23×12.1＋（中略）...＋Q73×10.1 (14-3)
n4領域の和＝Q14×1.5＋Q24×(-32.9)＋（中略）...＋Q74×(-52.2) (14-4) If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (14).

Therefore, the sum of the n1 to n4 regions is expressed by the following formulas (14-1) to (14-4).
n1 area sum = Q11 x 40.1 + Q21 x 37.0 + (omitted) ... + Q71 x 22.0 (14-1)
Sum of n2 areas = Q12 x (-14.8) + Q22 x (-10.5) + (omitted) ... + Q72 x 0.0 (14-2)
n3 area sum = Q13 x 24.6 + Q23 x 12.1 + (omitted) ... + Q73 x 10.1 (14-3)
n4 area sum = Q14 x 1.5 + Q24 x (-32.9) + (omitted) ... + Q74 x (-52.2) (14-4)

The index 3 is defined as in Expression (15) using the sum of the N1 to H4 regions shown in Expressions (14-1) to (14-4).
Index 3 = sum of n1 regions + sum of n2 regions + sum of n3 regions + sum of n4 regions−12.6201 (15)
The index 3 is calculated on the basis of compositional characteristics (distance from the outer edge of the screen of the photographed image data) based on the lightness distribution position of the photographed image data. It is also effective to do.

The index 4 is defined as Expression (16) using the indices 1 and 3, and the index 5 is defined as Expression (17) using the indices 1 to 3.
Index 4 = 0.565 x Index 1 + 0.565 x Index 3 + 0.457 (16)
Indicator 5 = (-0.121) x Indicator 1 + 0.91 x Indicator 2 + 0.113 x Indicator 3-0.072 (17)
Here, the weighting coefficient by which each index is multiplied in Expression (16) and Expression (17) is set in advance according to the shooting conditions.

このように指標４、５の値により配光条件を定量的に判別することができる。 <Determination method of light distribution conditions>
Next, a method for determining light distribution conditions will be described.
FIG. 17 shows 60 images taken under each light distribution condition of forward light, backlight, and strobe, and indexes 4 and 5 are calculated for a total of 180 digital image data, and the values of indexes 4 and 5 under each light source condition are calculated. Are plotted. According to FIG. 17, when the value of the index 4 is larger than 0.5, there are many strobe scenes, and when the value of the index 4 is 0.5 or less and the value of the index 5 is larger than −0.5, the backlight scene is I understand that there are many. Table 6 shows the determination contents of the light distribution conditions based on the values of the indexes 4 and 5.

Thus, the light distribution condition can be determined quantitatively based on the values of the indices 4 and 5.

  Next, referring to the flowcharts of FIGS. 18 and 19, the first neuron (hereinafter referred to as the old neuron) is changed to a third neuron having a new configuration different from the first neuron (hereinafter referred to as the new neuron). The neuro replacement process to be replaced with.) Will be described. In this embodiment, two types of neuro replacement processing will be described.

First, the neuro replacement process 1 executed in the neuro learning unit 7 will be described with reference to the flowchart of FIG.
First, a new neuro is acquired (step S50), and an old neuro is acquired (step S51). Next, determination image information used to determine the output results of the new neuron and the old neuron is acquired (step S52).

  Next, for the determination image information acquired in step S52, the tone correction parameter is acquired using the new neuron, the tone correction parameter is acquired using the old neuron, and is output from the new neuron and the old neuron. The difference information of the tone correction parameters obtained is acquired (step S53).

  Next, the average value of the difference information of the predetermined number of past scenes is calculated (step S54), the average value calculated last time is compared with the amount of change of the average value calculated this time (step S55), and the average value of the difference information It is determined whether or not the amount of change has converged (step S56).

  In step S56, when it is determined that the change amount of the average value of the difference information has not converged (step S56; NO), the teacher data using the old neuron gradation correction parameter as a desirable value of the output signal of the new neuron Is used to learn new neuro (step S58). Then, a new neuro that is finely corrected by the execution of learning in step S58 is set (step S59), and the process returns to step S52, and the processes in steps S52 to S56 are repeated for the set new neuro.

  If it is determined in step S56 that the change amount of the average value of the difference information has converged (step S56; YES), the old neuron is replaced with the new neuron (step S57), and the present neuroreplacement process 1 ends.

Next, the neuro replacement process 2 executed in the neuro learning unit 7 will be described with reference to the flowchart of FIG.
First, a new neuro is acquired (step S60), and an old neuro is acquired (step S61). Next, determination image information used to determine the output results of the new neuron and the old neuron is acquired (step S62).

  Next, for the determination image information acquired in step S52, the tone correction parameter is acquired using the new neuron, and the tone correction parameter is acquired using the old neuron (step S63). Next, the image information for determination is corrected based on the old neuron gradation correction parameters (step S64), and the corrected image is displayed on the display unit 10 (step S65).

When the operator observes the corrected image displayed on the display unit 10 and determines that correction is necessary, the operator inputs the first correction value by operating the instruction input unit 9 (step S66). Next, as shown in Expression (18), the correct value for correction is calculated by adding the old neurotone correction parameter and the first correction value (step S67).
(Correct correction value) = (old neurotone gradation correction parameter) + (first correction value) (18)

Next, a predicted value of the second correction value input by the operator is calculated based on the correct correction value and the new neurotone correction parameter (step S68). A formula for calculating the second correction value is shown in Formula (19).
(Second correction value) = (correction correct value) − (new neurotone correction parameter) (19)

  Next, an average value of the first correction values and an average value of the second correction values for the past predetermined number of scenes are calculated (step S69), and the average value of the first correction values is calculated based on the average value of the second correction values. It is determined whether it is larger (step S70).

  In step S70, when it is determined that the average value of the first correction value is equal to or less than the average value of the second correction value (step S70; NO), the correct value of the correction is set as a desirable value of the output signal of the new neuron. New neuro learning is performed using the teacher data used (step S72). Then, a new neuro that is finely corrected by the execution of learning in step S72 is set (step S73), and the process returns to step S62, and the processes in steps S62 to S70 are repeated for the set new neuro.

  In Step S70, when it is determined that the average value of the first correction value is larger than the average value of the second correction value (Step S70; YES), the old neuron is replaced with the new neuron (Step S71), and the current neuron. The replacement process 2 ends.

  Note that a process obtained by connecting the neuro replacement process 1 in FIG. 18 and the neuro replacement process 2 in FIG. 19 may be executed. In this case, when the average value of the difference information is converged in step S56 in FIG. 18 (step S56; YES), the process proceeds to the neuro replacement process 2 in FIG. 19, and in the neuro replacement process 2, the neuro replacement process 1 is performed as a new neuro. A new neuron that has already been learned is acquired. As described above, when the neuro replacement process 1 and the neuro replacement process 2 are combined, the neuro replacement process 1 completes a large amount of learning at a high speed. In the neuro replacement process 2, a small amount of more advanced learning is performed. Therefore, advanced learning at the time of neuro replacement can be performed in a short time.

<Correction image creation process>
Next, with reference to FIGS. 20 to 25, the corrected image creation process (gradation conversion process) shown in step S10 of FIG. 5 will be described.

  In the digital image, for example, brightness is corrected by a tone curve (tone conversion curve) shown in FIG. 20A, and contrast is corrected by a tone curve shown in FIG. For an operator who observes an image on the display unit 10 and instructs additional correction, it is preferable to simply perform additional correction.

  For example, it is preferable that the contrast correction result reliably adjusts the contrast and that there is no subjective change in other elements such as image vividness (saturation). However, in actual correction, for example, when correction for increasing the contrast is performed, applying the tone curve of “high contrast” in FIG. 20B to each of the RGB elements of the RGB color image simultaneously increases the saturation. It is necessary to adjust the saturation further. Similarly, for example, when the tone curve of “high contrast” in FIG. 20B is applied to the L * information of the L * a * b * color image, the saturation is observed to be low, and the saturation is further increased. Need to be adjusted.

  Due to these problems, if the accuracy of additional correction by the operator decreases, the learning efficiency of the neuron decreases, and it takes longer to reach performance that matches the operator's preference. It is necessary to mitigate changes. An example of such gradation conversion processing is shown in FIG.

First, the tone correction parameter (for example, contrast correction parameter) calculated in step S8 of FIG. 5 is acquired (step S80). Next, the acquired gradation correction parameter is divided into two correction parameters (first correction parameter and second correction parameter) at a predetermined ratio (step S81). The following relationships (20-1) and (20-2) are established between the gradation correction parameter acquired in step S80 and the two correction parameters divided in step S81.
(First correction parameter) = P1 × (gradation correction parameter) (20-1)
(Second correction parameter) = P2 × (gradation correction parameter) (20-1)
Here, P1 + P2 = 1 (for example, P1 = P2 = 0.5)

  Further, image information to be corrected is acquired (step S82), and the image information is coordinate-converted into the L * a * b * color space (step S83). Next, gradation conversion of the L * information is performed by applying the first correction parameter to the L * information (step S84).

  Next, the image information to be corrected is coordinate-converted into the RGB color space (step S85), and the second correction parameter is applied to each of the RGB color information, whereby gradation conversion of each color information is performed ( Step S86), the gradation conversion process ends. In FIG. 21, the gradation conversion of RGB information is performed after the gradation conversion of L * information. However, the order of these two gradation conversion conversions is not limited to this, and the gradation of RGB information is not limited to this. You may make it perform the gradation conversion of L * information after conversion.

  As described above, when the tone conversion process shown in FIG. 21 is performed for each tone correction parameter, the change in saturation due to the tone change is reduced, so that additional correction by the operator is facilitated, and the first neuron is corrected. The learning efficiency of.

  Next, shading correction will be described as another example of the corrected image creation processing with reference to FIGS.

  In many cases, digital image information is printed out using dedicated digital printers according to the purpose. In order to print out digital signals such as computer graphics accurately, these printers generally perform high-precision shading correction. Has been done. In the image processing apparatus 100 (optical printing apparatus) shown in FIG. 1, for example, generally from the center of the image due to the light source irradiation intensity of the film surface and the shading characteristics of the lens (Cos 4 power law or change in opening efficiency due to vignetting). There is a gradual shading error toward the periphery, and the amount of light applied to the photographic paper gradually decreases around the periphery.

  With a negative-positive method, the surroundings become brighter, producing a brighter and lighter screen. With the positive-positive method, the surroundings become darker, the screen is darker, producing a profound feeling and effective drawing. Has been used. Reproducing such characteristics is expected to improve the appearance of the image and facilitate additional correction by the operator. In the following, two methods (FIGS. 22 and 24) for reproducing the optical printing shading characteristics will be described.

  First, an example of the optical printing shading characteristic reproduction process will be described with reference to the flowchart of FIG.

  First, image information to be corrected is acquired (step S90). Next, the pixel signal value x for one pixel to be corrected is acquired from the image information acquired in step S90 (step S91). Next, the distance d from the center of the pixel signal value x acquired in step S91 is calculated (step S92).

Next, using the first correction coefficient f (d) and the second correction coefficient g (x) set in advance based on the shading characteristics in the negative-positive method, the pixel correction value is expressed as shown in Expression (21). Calculated (step S93).
(Pixel correction value) = 1 / {g (x) × f (d) + (1−g (x))} (21)
Here, the first correction coefficient f (d) is a function of the distance d from the center, and has characteristics as shown in FIG. The second correction coefficient g (x) is a function of the pixel signal value x and has characteristics as shown in FIG.

  Next, a corrected correction value (a value obtained by correcting the pixel signal value x) is calculated by taking the product of the pixel correction value calculated in step S93 and the pixel signal value x (step S94). Next, it is determined whether or not the processing in steps S91 to S94 has been performed for all the pixels to be corrected (step S95).

  If it is determined in step S95 that the processing has not been completed for all the pixels (step S95; NO), the process proceeds to calculation processing for the next pixel to be corrected (step S96), and steps S91 to S94 are performed for the corresponding pixel. The process is repeated. If it is determined in step S95 that the processing for all the pixels has been completed (step S95; YES), the reproduction process for the present optical printing shading characteristic is completed.

  Next, another example of the optical printing shading characteristic reproduction process will be described with reference to the flowchart of FIG.

  First, image attribute information is acquired from the image information to be corrected (step S100). Here, the image attribute information is information relating to image generation means, information on the purpose of use, and finish preference. Next, it is determined whether or not the image information to be corrected is a natural image obtained by photographing (step S101). Computer graphics images, design images, text images, and the like do not require the addition of shading characteristics and are therefore considered not natural images.

  In step S101, when it is determined that the image information to be corrected is not a natural image (step S101; NO), the reproduction process of the optical printing shading characteristics is completed. If it is determined in step S101 that the image information to be corrected is a natural image (step S101; YES), it is determined whether or not a negative print tone is selected (designated) (step S102).

  When the negative print tone is selected in step S102 (step S102; YES), the first correction coefficient f (d) and the second correction coefficient g (x) shown in FIG. 23 are used as the negative print tone correction coefficients. ) Is selected (step S103), and the reproduction process of the optical printing shading characteristics shown in FIG. 22 is executed using these correction coefficients (step S105).

  When the positive print tone is selected (designated) in step S102 (step S102; NO), the first correction coefficient f ′ (d) and the second correction coefficient g ′ (positive print tone correction coefficient) are used. x) is selected (step S104). The first correction coefficient f ′ (d) and the second correction coefficient g ′ (x) are set in advance based on the shading characteristics in the positive / positive system, and are respectively shown in FIGS. 25 (a) and 25 (b). It has the following characteristics. Unlike the first correction coefficient f (d) applied in the negative print tone, the first correction coefficient f ′ (d) is a monotonically increasing function that increases from 1.0 as the distance from the center increases. It has become.

  When the positive print tone correction coefficient is selected, the reproduction process of the optical printing shading characteristic shown in FIG. 22 is executed using the correction coefficient (step S105). In this case, in the calculation formula of the pixel correction value of Expression (21), f (d) → f ′ (d) and g (x) → g ′ (x) are replaced.

  As described above, according to the image processing apparatus of this embodiment, the additional correction value instructed by the operator is learned as the first neuro teacher data, and the image information is corrected using the learned first neuro. As a result, an image suitable for the operator's preference can be obtained.

  The characteristic value of the person's face area and the response intensity of the second neuron are calculated using the second neuron that has learned the human face detection process included in the input image information, and the calculated characteristic of the face area is calculated. By using the value and the reaction intensity as the input signal of the first neuron, it is possible to obtain an image that matches the preference of the operator with higher quality.

  Further, since the third neuro learning is performed using the output signal of the first neuro learned according to the operator's preference before using the third neuro having the new configuration, the operator It is possible to make a smooth transition to a new neuro that suits the taste of people.

  Also, by determining the light distribution condition at the time of image capturing from the input image information and making the determination result one of the input signals to the first neuron, it is possible to obtain a gradation correction parameter suitable for the light distribution condition. A higher quality image can be obtained.

  Further, the image information to be corrected is coordinate-converted to the L * a * b * color space and coordinate-converted to the RGB color space, and the gradation correction parameter calculated by the first neuro is divided into two correction parameters. The gradation change is performed by applying the first correction parameter to the luminance information (L *) to perform gradation conversion, and applying the second correction parameter to each of the RGB unit colors to perform gradation conversion. The change in the saturation (the vividness of the image) that accompanies is reduced, and additional correction according to an instruction from the operator becomes easy.

  In addition, in the correction image creation process, it is possible to obtain a correction image that approximates negative-positive printing by performing processing that reproduces the shading characteristics at the time of printing in the negative-positive method, so that additional correction by the operator is easy. Become.

  Note that the description in the present embodiment can be changed as appropriate without departing from the spirit of the present invention.

  For example, in the image processing apparatus 100 or 200 of the present embodiment, each component is housed in a single housing. For example, the image processing apparatus 200 is divided into a plurality of blocks, and each block is physically independent. The present embodiment can also be applied to a system that is an apparatus (device).

  FIG. 26 shows a configuration of an image processing system 300 including a plurality of physically independent devices. As shown in FIG. 26, the image processing system 300 includes an image input device 20, an image processing device 21, an image output device 22, a system control device 23, and a communication control unit 15. In FIG. 26, the same components as those in the image processing apparatus 200 in FIG. 2 are denoted by the same reference numerals, and the description of the functions of the components having the same functions is omitted.

  The image input device 20 includes an image information acquisition unit 4, a DSC 12, a document scanner 13, and a recording media driver 14.

  The image processing device 21 includes an image determination unit 5, an additional correction data storage unit 6, a neuro learning unit 7, an instruction input unit 9, a display unit 10, a reduced image creation unit 11, and an image processing unit 24. The image processing unit 24 performs gradation conversion for display on the display unit 10 based on the gradation correction parameter calculated by the image determination unit 5 and / or the additional correction value input from the instruction input unit 9. A corrected image is created, and the created corrected image is output to the display unit 10.

  The image output device 21 includes an image information output unit 16, a printer unit 17, a recording medium writing unit 18, and an image processing unit 25. The image processing unit 25 is designated based on the gradation correction parameter calculated by the image determination unit 5 and / or the additional correction value input from the instruction input unit 9 according to the control signal input from the system control device 23. The tone conversion for output to the output destination is performed to create a corrected image, and the generated corrected image is output to the image information output unit 16.

  The system control device 23 comprehensively controls various processes in the communication control unit 15, the image input device 20, and the image processing device 21.

  Note that a plurality of each of the image input device 20, the image processing device 21, and the image output device 2 may be arranged as necessary. For example, when the processing capability of the image input device 20 is inferior to the image processing capability of the entire system, a plurality of image input devices 20 can be arranged and operated in parallel under the control of the system control device 23.

1 is a block diagram illustrating an example of a configuration of an image processing apparatus according to an embodiment of the present invention. The block diagram which shows the other structural example of the image processing apparatus in embodiment of this invention. The figure which shows the structure of a hierarchical neural network typically. The figure which shows the structure of the neuron which comprises a neural network. 6 is a flowchart showing image processing executed in the image processing apparatus according to the present embodiment. The figure which shows the 1st hierarchical neural network and 3rd hierarchical neural network applied by this embodiment. The flowchart which shows a light distribution condition discrimination | determination process. The flowchart which shows the occupation rate calculation process which calculates the 1st occupation rate for every area | region of brightness and hue. The figure which shows an example of the program which converts from RGB to HSV color system. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r1 and the area | region r2 on a VH plane. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r3 and the area | region r4 on a VH plane. The figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. The figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. The flowchart which shows the occupation rate calculation process which calculates a 2nd occupation rate based on the composition of picked-up image data. The figure which shows the area | regions n1-n4 determined according to the distance from the outer edge of the screen of picked-up image data. The figure which shows the curve showing the 3rd coefficient for multiplying the 2nd occupation rate for calculating the parameter | index 3 according to area | region (n1-n4). The plot figure of the parameter | index 4 and the parameter | index 5 calculated according to light distribution conditions (forward light, strobe, backlight). The flowchart which shows the neuro replacement | exchange process 1 which replaces a 1st hierarchical neural network with a 3rd hierarchical neural network. The flowchart which shows the neuro replacement | exchange process 2 which replaces a 1st hierarchical neural network with a 3rd hierarchical neural network. The figure which shows the gradation conversion curve (a) which shows brightness correction, and the gradation conversion curve (b) which shows contrast correction. The flowchart which shows a gradation conversion process. The flowchart which shows the reproduction process of an optical printing shading characteristic. The figure which shows the characteristic of 1st correction coefficient f (d) and 2nd correction coefficient g (x) used by the reproduction process of the optical printing shading characteristic of FIG. The figure which shows the reproduction process of the optical printing shading characteristic according to an image attribute. The figure which shows the characteristic of 1st correction coefficient f '(d) and 2nd correction coefficient g' (x) used when picked-up image data is a positive print tone. The figure which shows the modification of the image processing apparatus of FIG. The flowchart which shows the conventional image processing.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Light source part 2 Imaging part 3 Exposure control part 4 Image information acquisition part 5 Image determination part 6 Additional correction data storage part 7 Neuro learning part 8 Image processing part 9 Instruction input part 10 Display part 11 Reduced image creation part 12 DSC
13 Document Scanner 14 Recording Media Driver 15 Communication Control Unit 16 Image Information Output Unit 17 Printer Unit 18 Recording Media Writing Unit 20 Image Input Device 21 Image Processing Device 22 Image Output Device 23 System Control Devices 24 and 25 Image Processing Units 100 and 200 Images Processing device 300 Image processing system N Communication network

Claims (40)

A characteristic value calculating step for calculating a predetermined characteristic value from the input image information;
A correction parameter calculating step of calculating a gradation correction parameter using a first neural network having the calculated characteristic value as an input signal and a gradation correction parameter indicating a degree of gradation conversion for image information as an output signal; ,
A corrected image creating step of creating a corrected image by gradation-converting the image information based on the calculated gradation correction parameter;
A first learning step of performing learning of the first neural network using, as teacher data, a correction amount for the created corrected image when an instruction to correct the created corrected image is given;
An image processing method comprising: A face area detection step of detecting a face area of a person from the input image information,
The image processing method according to claim 1, wherein in the characteristic value calculation step, a characteristic value related to the detected face area is calculated. A face candidate area extracting step of extracting a face candidate area that may be a human face from the input image information,
In the face area detection step, a face area is detected from the extracted face candidate areas using a second neural network that has learned human face detection processing;
In the characteristic value calculating step, a reaction intensity of the output signal of the second neural network and / or a characteristic value of the detected face area, which indicates the possibility that the extracted face candidate area is a face area, is calculated. The image processing method according to claim 2, wherein: An acquisition step of acquiring a third neural network having a configuration different from that of the first neural network;
A second learning step of performing learning of the third neural network using the output signal of the first neural network as teacher data;
A replacement step of replacing the first neural network with the third neural network when a learning result of the learned third neural network satisfies a predetermined condition;
The image processing method according to claim 1, further comprising: An acquisition step of acquiring a third neural network having a configuration different from that of the first neural network;
An input step of inputting a first correction value for further correcting the corrected image created based on the gradation correction parameter calculated by the first neural network;
A correction value calculation step of calculating a second correction value for correcting a correction image created based on the gradation correction parameter calculated by the third neural network based on the first correction value;
A second learning step of performing learning of the third neural network using the calculated second correction value as teacher data;
When the relationship between the first correction value and the second correction value calculated using the learned third neural network satisfies a predetermined condition, the first neural network is changed to the third neural network. A replacement step to replace with a neural network;
The image processing method according to claim 1, further comprising:   The image processing method according to claim 1, wherein in the characteristic value calculation step, a value indicating a light distribution condition when the input image is taken is calculated.   The image processing method according to claim 6, wherein the light distribution condition includes a backlight state and / or a flash light utilization state. The input image information is a color image,
A division parameter calculation step of dividing the gradation correction parameter calculated in the correction parameter calculation step by a predetermined ratio and calculating a first correction parameter and a second correction parameter;
In the corrected image creating step, the luminance information is gradation-converted by applying the first correction parameter to the luminance information when the input image information is expressed by luminance-color difference information. The color information is subjected to gradation conversion by applying the second correction parameter to each color information when the inputted image information is expressed by RGB information. 8. The image processing method according to any one of 7.   The said correction | amendment image creation process converts the said image information using the correction coefficient preset based on the shading characteristic in a negative positive system, The Claim 1 characterized by the above-mentioned. Image processing method. An attribute information acquisition step of acquiring attribute information of the image information;
A determination step of determining whether the image information is a natural image obtained by photographing based on the acquired attribute information,
If it is determined in the determination step that the image information is a natural image, the correction information creation step converts the image information using a correction coefficient set in advance based on a shading characteristic in a negative-positive method. The image processing method according to claim 1, wherein the image processing method is an image processing method. Characteristic value calculation means for calculating a predetermined characteristic value from the input image information;
Correction parameter calculation means for calculating a gradation correction parameter using a first neural network having the calculated characteristic value as an input signal and a gradation correction parameter indicating the degree of gradation conversion for image information as an output signal; ,
A corrected image creating means for creating a corrected image by gradation-converting the image information based on the calculated gradation correction parameter;
A first learning means for performing learning of the first neural network using a correction amount for the generated corrected image as teacher data when an instruction to correct the generated corrected image is given;
An image processing apparatus comprising: A face area detecting means for detecting a face area of a person from the input image information;
The image processing apparatus according to claim 11, wherein the characteristic value calculation unit calculates a characteristic value related to the detected face area. Face candidate area extraction means for extracting a face candidate area that may be a human face from the input image information,
The face area detection means detects a face area from the extracted face candidate areas using a second neural network that has learned human face detection processing;
The characteristic value calculation means calculates a reaction intensity of the output signal of the second neural network and / or a characteristic value of the detected face area, which indicates the possibility that the extracted face candidate area is a face area. The image processing apparatus according to claim 12, wherein: Obtaining means for obtaining a third neural network having a configuration different from that of the first neural network;
Second learning means for performing learning of the third neural network using the output signal of the first neural network as teacher data;
Replacement means for replacing the first neural network with the third neural network when a learning result of the learned third neural network satisfies a predetermined condition;
The image processing apparatus according to claim 11, further comprising: Obtaining means for obtaining a third neural network having a configuration different from that of the first neural network;
Input means for inputting a first correction value for further correcting a correction image created based on the gradation correction parameter calculated by the first neural network;
Correction value calculation means for calculating a second correction value for correcting a correction image created based on the gradation correction parameter calculated by the third neural network based on the first correction value;
Second learning means for performing learning of the third neural network using the calculated second correction value as teacher data;
When the relationship between the first correction value and the second correction value calculated using the learned third neural network satisfies a predetermined condition, the first neural network is changed to the third neural network. Replacement means to replace with a neural network;
The image processing apparatus according to claim 11, further comprising:   The image processing apparatus according to claim 11, wherein the characteristic value calculation unit calculates a value indicating a light distribution condition when the input image is taken.   The image processing apparatus according to claim 16, wherein the light distribution condition includes a backlight state and / or a flash light use state. The input image information is a color image,
A division parameter calculation unit that divides the gradation correction parameter calculated by the correction parameter calculation unit at a predetermined ratio, and calculates a first correction parameter and a second correction parameter;
The corrected image creating means performs gradation conversion of the luminance information by applying the first correction parameter to luminance information when the input image information is expressed by luminance-color difference information, and 18. The color information is subjected to gradation conversion by applying the second correction parameter to each color information when the input image information is expressed by RGB information. The image processing apparatus according to any one of the above.   The image according to any one of claims 11 to 18, wherein the corrected image creating means converts the image information using a correction coefficient set in advance based on a shading characteristic in a negative-positive method. Processing equipment. Attribute information acquisition means for acquiring attribute information of the image information;
Determination means for determining whether the image information is a natural image obtained by photographing based on the acquired attribute information,
When the determination unit determines that the image information is a natural image, the correction image creation unit converts the image information using a correction coefficient set in advance based on shading characteristics in a negative-positive method. The image processing apparatus according to claim 11, wherein the image processing apparatus is an image processing apparatus. On the computer that performs image processing,
A characteristic value calculation function for calculating a predetermined characteristic value from the input image information;
A correction parameter calculation function for calculating a gradation correction parameter using a first neural network having the calculated characteristic value as an input signal and a gradation correction parameter indicating the degree of gradation conversion for image information as an output signal; ,
A corrected image creation function for creating a corrected image by gradation-converting the image information based on the calculated gradation correction parameter;
A first learning function for performing learning of the first neural network using, as teacher data, a correction amount for the created corrected image when an instruction to correct the created corrected image is given;
An image processing program for realizing Realizing a face area detection function for detecting a human face area from the input image information,
The image processing program according to claim 21, wherein a characteristic value related to the detected face area is calculated when the characteristic value calculation function is realized. Realizing a face candidate area extraction function that extracts a face candidate area that may be a human face from the input image information;
When realizing the face area detection function, a face area is detected from the extracted face candidate areas using a second neural network that has learned human face detection processing;
When realizing the characteristic value calculation function, the response intensity of the output signal of the second neural network and / or the detected face area indicating the possibility that the extracted face candidate area is a face area. The image processing program according to claim 22, wherein a characteristic value is calculated. An acquisition function for acquiring a third neural network having a configuration different from that of the first neural network;
A second learning function for performing learning of the third neural network using the output signal of the first neural network as teacher data;
A replacement function for replacing the first neural network with the third neural network when a learning result of the learned third neural network satisfies a predetermined condition;
24. The image processing program according to any one of claims 21 to 23, wherein: An acquisition function for acquiring a third neural network having a configuration different from that of the first neural network;
An input function for inputting a first correction value for further correcting the corrected image created based on the gradation correction parameter calculated by the first neural network;
A correction value calculation function for calculating a second correction value for correcting a correction image created based on the gradation correction parameter calculated by the third neural network based on the first correction value;
A second learning function for performing learning of the third neural network using the calculated second correction value as teacher data;
When the relationship between the first correction value and the second correction value calculated using the learned third neural network satisfies a predetermined condition, the first neural network is changed to the third neural network. Replacement function to replace with neural network,
24. The image processing program according to any one of claims 21 to 23, wherein:   The image according to any one of claims 21 to 25, wherein when the characteristic value calculation function is realized, a value indicating a light distribution condition when the input image is photographed is calculated. Processing program.   27. The image processing program according to claim 26, wherein the light distribution condition includes a backlight state and / or a flash light utilization state. The input image information is a color image,
Dividing the gradation correction parameter calculated in the correction parameter calculation function at a predetermined ratio to realize a division parameter calculation function for calculating the first correction parameter and the second correction parameter;
When the corrected image creation function is realized, the luminance information is converted into gradation by applying the first correction parameter to the luminance information when the input image information is expressed by luminance-color difference information. In addition, the color information is subjected to gradation conversion by applying the second correction parameter to each color information when the input image information is expressed by RGB information. The image processing program according to any one of 21 to 27.   29. The image information is converted by using a correction coefficient set in advance based on a shading characteristic in a negative-positive method when the correction image creation function is realized. The image processing program described in 1. An attribute information acquisition function for acquiring attribute information of the image information;
A determination function for determining whether the image information is a natural image obtained by photographing based on the acquired attribute information; and
When realizing the corrected image creation function, if the image information is determined to be a natural image by the determination function, the image is calculated using a correction coefficient set in advance based on a shading characteristic in a negative-positive method. The image processing program according to any one of claims 21 to 28, wherein information is converted. An image input device that inputs image information to be processed, an image processing device that performs various types of image processing on the input image information, and an image output device that outputs the image information subjected to the image processing An image processing system comprising:
The image processing apparatus includes:
A characteristic value calculating means for calculating a predetermined characteristic value from the image information input from the image input device;
Correction parameter calculation means for calculating a gradation correction parameter using a first neural network having the calculated characteristic value as an input signal and a gradation correction parameter indicating the degree of gradation conversion for image information as an output signal; ,
A corrected image creating means for creating a corrected image by gradation-converting the image information based on the calculated gradation correction parameter;
A first learning means for performing learning of the first neural network using a correction amount for the generated corrected image as teacher data when an instruction to correct the generated corrected image is given;
An image processing system comprising: The image processing apparatus includes:
A face area detecting means for detecting a face area of a person from the input image information;
32. The image processing system according to claim 31, wherein the characteristic value calculation unit calculates a characteristic value related to the detected face area. The image processing apparatus includes:
Face candidate area extraction means for extracting a face candidate area that may be a human face from the input image information,
The face area detection means detects a face area from the extracted face candidate areas using a second neural network that has learned human face detection processing;
The characteristic value calculation means calculates a reaction intensity of the output signal of the second neural network and / or a characteristic value of the detected face area, which indicates the possibility that the extracted face candidate area is a face area. The image processing system according to claim 32, wherein: The image processing apparatus includes:
Obtaining means for obtaining a third neural network having a configuration different from that of the first neural network;
Second learning means for performing learning of the third neural network using the output signal of the first neural network as teacher data;
Replacement means for replacing the first neural network with the third neural network when a learning result of the learned third neural network satisfies a predetermined condition;
The image processing system according to any one of claims 31 to 33, comprising: The image processing apparatus includes:
Obtaining means for obtaining a third neural network having a configuration different from that of the first neural network;
Input means for inputting a first correction value for further correcting a correction image created based on the gradation correction parameter calculated by the first neural network;
Correction value calculation means for calculating a second correction value for correcting a correction image created based on the gradation correction parameter calculated by the third neural network based on the first correction value;
Second learning means for performing learning of the third neural network using the calculated second correction value as teacher data;
When the relationship between the first correction value and the second correction value calculated using the learned third neural network satisfies a predetermined condition, the first neural network is changed to the third neural network. Replacement means to replace with a neural network;
The image processing system according to any one of claims 31 to 33, comprising:   36. The image processing system according to claim 31, wherein the characteristic value calculation unit calculates a value indicating a light distribution condition when the input image is taken.   The image processing system according to claim 36, wherein the light distribution condition includes a backlight state and / or a flash light utilization state. The input image information is a color image,
The image processing apparatus includes:
A division parameter calculation unit that divides the gradation correction parameter calculated by the correction parameter calculation unit at a predetermined ratio, and calculates a first correction parameter and a second correction parameter;
The corrected image creating means performs gradation conversion of the luminance information by applying the first correction parameter to luminance information when the input image information is expressed by luminance-color difference information, and 38. The tone conversion of the color information is performed by applying the second correction parameter to each color information when the input image information is expressed by RGB information. The image processing system according to any one of claims.   The image according to any one of claims 31 to 38, wherein the corrected image creating means converts the image information using a correction coefficient set in advance based on a shading characteristic in a negative-positive method. Processing system. The image processing apparatus includes:
Attribute information acquisition means for acquiring attribute information of the image information;
Determination means for determining whether the image information is a natural image obtained by photographing based on the acquired attribute information,
When the determination unit determines that the image information is a natural image, the correction image creation unit converts the image information using a correction coefficient set in advance based on shading characteristics in a negative-positive method. The image processing system according to any one of claims 31 to 38.

Images