JP4449619B2 - Image processing method, image processing apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing method, an image processing apparatus, and an image processing program for performing image processing on captured image data.

  In recent years, digital still cameras (including toys, cameras incorporated in devices such as mobile phones and laptop computers, general-purpose applications for general users, and high-functional professional cameras. Hereinafter, abbreviated as DSC). Widely used and images taken by DSC are viewed as being output as hard copy images or displayed on output media such as CRT (Cathode Ray Tube), as with color photographic film.

  In addition, a technique is widely used in which an image formed on a color photographic film is photoelectrically read by a CCD (Charge Coupled Device) sensor or the like and converted into an image signal. Such an image signal is subjected to various image processing represented by negative / positive reversal, luminance adjustment, color balance adjustment, grain removal, and sharpness enhancement, and then CD-R (CD-Recordable), CD-RW ( CD-ReWritable), floppy (registered trademark) disk, memory card, or other recording media, distributed via the Internet, and output as a hard copy on silver halide photographic paper, inkjet printers, thermal printers, etc. It can be viewed on an output medium such as a liquid crystal display or plasma display.

  By the way, in any output form, the viewer's attention is high and the evaluation standard is strict in a scene in which a person is photographed as a subject, and in particular, the face is focused. However, in actual image shooting, it is common that a person's face is not in an environment where it is shot as an image having an appropriate color, brightness, sharpness, noise, and stereoscopic effect. In order to obtain a high-quality image for a scene including a person in response to such a background, a technique for extracting a face area from captured image data and performing a correction process is required. Techniques and techniques for improving the determination accuracy of face determination processing are disclosed.

  For example, in Patent Document 1, a face candidate area corresponding to the shape of a person's face is determined from photographed image data, and a face area is determined using a predetermined threshold value or the like based on a feature amount in the face candidate area. Techniques to do this are disclosed. In Patent Document 1, attention is paid to “eyes” as one of the feature amounts, and the brightness histogram of the face candidate region is composed of a high brightness skin portion and two peaks corresponding to low brightness eyes. It is assumed that the low lightness peak corresponds to “eyes”. Then, the number of pixels constituting a low brightness peak with respect to the number of pixels in the entire face candidate region is used as the appearance rate of the eye pixel, and the appearance rate of the eye pixel is used as the feature amount. Further, a template matching method is also exemplified for eye extraction.

  Patent Document 2 describes a method for determining a light source state (light distribution condition) at the time of photographing in order to compensate for the extraction accuracy of a face region. In the method described in Patent Document 2, first, a human face candidate area is extracted from captured image data, a bias of the average luminance of the extracted face candidate area with respect to the entire image is calculated, and depending on the magnitude of the bias amount, The light source state (backlight shooting or flash close-up shooting) is determined, and the allowable range of the determination criterion for the face area is adjusted. Japanese Patent Laid-Open No. 6-67320 discloses a method of using a two-dimensional histogram of hue and saturation, Japanese Patent Laid-Open No. 8-122944, and Japanese Patent Laid-Open No. 8-184925. The pattern matching described in Japanese Patent Laid-Open No. 9-138471 and the pattern search method described in Japanese Patent Laid-Open No. 9-138471 are cited.

Further, in Patent Document 2, as a method for removing a background area other than the face, the ratio of the straight line portion, the line object property, and the outer edge of the screen described in JP-A-8-122944 and JP-A-8-184925 are described. A method of determining using a contact rate, density contrast, density change pattern and periodicity is cited. In addition, a method using a one-dimensional histogram of brightness is described for determining the light source state (light distribution condition). This method is based on an empirical rule that the face area is dark and the background area is bright in the case of backlight, and the face area is bright and the background area is dark in the case of close-up flash photography.
JP-A-8-63597 JP 2000-148980 A

  A natural image taken by a general user has various shooting environments such as a light source type and an exposure state. Therefore, even if the subject is the same, the luminance distribution, that is, the characteristic of the brightness histogram changes. However, the technique described in Patent Document 1 does not take into account the fact that the luminance distribution changes depending on the shooting environment, and as a result, there is a problem that the face discrimination accuracy is not sufficient. In addition, the technique described in Patent Document 2 can achieve the effect of compensating for the identification of the face area in the case of typical backlighting or close-up flash photography, but if it does not apply to the typical composition, There was a problem that the effect could not be obtained.

  An object of the present invention is to improve face discrimination accuracy in photographed image data regardless of the photographing environment.

  In order to solve the above-described problem, the invention according to claim 1 is an acquisition step of acquiring an image signal from captured image data, and the captured image data is converted to a predetermined brightness and hue based on the acquired image signal. An occupancy ratio calculating step of dividing the area into a combination area and calculating an occupancy ratio indicating a ratio of the entire captured image data for each of the divided areas, and the occupancy ratio of each of the calculated areas. A light distribution condition determining step for determining a light distribution condition at the time of shooting by performing an operation of multiplying a coefficient set in advance according to the above, and a person from the captured image data according to the determined light distribution condition A determination step of determining a processing condition for determining the face of the person, and a face determination step of performing a process of determining a person's face from the captured image data according to the determined processing condition. There.

  The invention according to claim 2 is an acquisition step of acquiring an image signal from the captured image data, and based on the acquired image signal, the captured image data is converted to a distance from the outer edge of the screen of the captured image data. An occupancy ratio calculating step for dividing an area into light intensity combinations and calculating an occupancy ratio indicating a ratio of the entire captured image data for each of the divided areas, and the calculated occupancy ratio of each area A light distribution condition determining step for determining a light distribution condition at the time of photographing by performing an operation of multiplying a coefficient set in advance according to the photographing condition, and the photographing according to the determined light distribution condition. A determination step for determining a processing condition for determining a person's face from image data; and a face determination step for performing a process for determining a person's face from the captured image data according to the determined processing condition. It is characterized by a door.

  According to a third aspect of the present invention, an acquisition step of acquiring an image signal from the captured image data, and the captured image data is divided into regions composed of combinations of predetermined brightness and hue based on the acquired image signal. For each of the divided areas, a first occupancy ratio indicating the ratio of the entire captured image data is calculated, and the captured image data is calculated from a combination of the distance from the outer edge of the captured image data and the brightness. An occupancy ratio calculating step of calculating a second occupancy ratio indicating the ratio of the entire captured image data for each of the divided areas, and the calculated first occupancy ratio and A light distribution condition determining step for determining a light distribution condition at the time of photographing by performing an operation of multiplying the second occupation ratio by a coefficient set in advance according to the photographing conditions, and the determined light distribution condition. Depending on A determination step of determining a processing condition for determining a person's face from the captured image data; and a face determination step of performing a process of determining a person's face from the captured image data according to the determined processing condition. It is characterized by including.

  According to a fourth aspect of the present invention, in the image processing method according to any one of the first to third aspects, in the light distribution condition determining step, the light distribution condition is specified by performing the calculation. An index is calculated, and a light distribution condition is determined based on the calculated index. In the determination step, processing conditions for face determination processing are switched and / or adjusted according to the calculated index value. It is characterized by being.

  According to a fifth aspect of the present invention, in the image processing method according to any one of the first to fourth aspects, the captured image data is predetermined according to a determination result in the face determination step and / or a light distribution condition. And an image processing step for performing the image processing.

  According to a sixth aspect of the present invention, in the image processing method according to any one of the first to fifth aspects, in the determining step, a person's image is determined from the captured image data according to the determined light distribution condition. An extraction condition for extracting a face candidate area is determined, and in the face determination step, a human face candidate area is extracted from the photographed image data in accordance with the determined face candidate area extraction condition, and the extracted face candidate area is extracted. It is characterized in that face discrimination processing is performed on the face candidate area.

  According to a seventh aspect of the present invention, in the image processing method according to the sixth aspect, in the determining step, an extraction condition for the feature amount extracted from the face candidate region is determined according to the determined light distribution condition. In the face discrimination step, a feature quantity is extracted from the face candidate area according to the determined feature quantity extraction condition, and a face discrimination process is performed based on the extracted feature quantity. .

  According to an eighth aspect of the present invention, in the image processing method according to the seventh aspect, in the determining step, a feature amount to be extracted from the face candidate region is determined according to the determined light distribution condition. It is characterized by.

  According to a ninth aspect of the present invention, in the image processing method according to the seventh or eighth aspect, in the determination step, the face candidate region is a person based on the feature amount according to the determined light distribution condition. A face determination process condition for determining whether or not the face is determined is determined, and in the face determination step, face determination processing is performed according to the determined face determination process condition.

  According to a tenth aspect of the present invention, in the image processing method according to any one of the first to ninth aspects, the face discrimination process is performed using a neural network in the face discrimination step.

  According to an eleventh aspect of the present invention, an acquisition unit that acquires an image signal from captured image data, and the captured image data are divided into regions including combinations of predetermined brightness and hue based on the acquired image signal. An occupancy ratio calculating unit that calculates an occupancy ratio indicating a ratio of the entire captured image data for each of the divided areas, and the calculated occupancy ratio of each area are set in advance according to the imaging conditions. A light distribution condition determining means for determining a light distribution condition at the time of shooting by performing an operation of multiplying by a coefficient, and for determining a person's face from the captured image data according to the determined light distribution condition; The image processing apparatus includes: a determination unit that determines a processing condition; and a face determination unit that performs a process of determining a person's face from the captured image data according to the determined processing condition.

  According to a twelfth aspect of the present invention, an acquisition unit that acquires an image signal from captured image data, and based on the acquired image signal, the captured image data is converted into a distance from an outer edge of a screen of the captured image data. Occupancy rate calculating means that divides into predetermined regions each consisting of a combination of brightness and calculates an occupancy rate indicating the proportion of the entire captured image data for each of the divided regions, and the calculated occupancy rate of each region And a light distribution condition determining means for determining a light distribution condition at the time of shooting by performing an operation of multiplying a preset coefficient according to the shooting condition, and the shooting according to the determined light distribution condition. Determining means for determining processing conditions for determining a person's face from image data; and face determining means for performing processing for determining a person's face from the captured image data according to the determined processing conditions. It is characterized in Rukoto.

  According to the thirteenth aspect of the present invention, the acquisition means for acquiring the image signal from the captured image data, and the captured image data are divided into regions composed of combinations of predetermined brightness and hue based on the acquired image signal. For each of the divided areas, a first occupancy ratio indicating the ratio of the entire captured image data is calculated, and the captured image data is calculated from a combination of the distance from the outer edge of the captured image data and the brightness. Occupancy ratio calculating means for calculating a second occupancy ratio indicating the ratio of the entire captured image data for each of the divided areas, and the calculated first occupancy ratio and A light distribution condition determining unit that determines a light distribution condition at the time of photographing by performing an operation of multiplying the second occupation ratio by a coefficient set in advance according to the photographing condition, and the determined light distribution condition. According Determining means for determining processing conditions for determining a person's face from the captured image data; and face determining means for performing processing for determining a person's face from the captured image data according to the determined processing conditions; It is characterized by having.

  The invention according to claim 14 is the image processing apparatus according to any one of claims 11 to 13, wherein the light distribution condition determining means performs the calculation to identify the light distribution condition. An index is calculated, the light distribution condition is determined based on the calculated index, and the determination unit switches and / or adjusts the processing condition of the face determination process according to the calculated index value. It is characterized by that.

  According to a fifteenth aspect of the present invention, in the image processing device according to any one of the eleventh to fourteenth aspects, the captured image data is predetermined according to a determination result by the face determination unit and / or a light distribution condition. The image processing means for performing the image processing is provided.

  According to a sixteenth aspect of the present invention, in the image processing device according to any one of the eleventh to fifteenth aspects, the determining unit is configured to detect a person from the captured image data according to the determined light distribution condition. An extraction condition for extracting a face candidate area is determined, and the face discriminating means extracts a human face candidate area from the photographed image data in accordance with the determined face candidate area extraction condition, and the extracted It is characterized in that face discrimination processing is performed on the face candidate area.

  According to a seventeenth aspect of the present invention, in the image processing apparatus according to the sixteenth aspect, the determination unit determines a feature amount extraction condition to be extracted from the face candidate region in accordance with the determined light distribution condition. The face discriminating means extracts a feature amount from the face candidate region in accordance with the determined feature amount extraction condition, and performs face discrimination processing based on the extracted feature amount.

  According to an eighteenth aspect of the present invention, in the image processing apparatus according to the seventeenth aspect, the determining unit determines a feature amount to be extracted from the face candidate region according to the determined light distribution condition. It is a feature.

  According to a nineteenth aspect of the present invention, in the image processing apparatus according to the seventeenth or eighteenth aspect, the determining unit determines that the face candidate region is a person based on the feature amount according to the determined light distribution condition. A face determination process condition for determining whether or not the face is determined is determined, and the face determination unit performs a face determination process according to the determined face determination process condition.

  According to a twentieth aspect of the present invention, in the image processing device according to any one of the eleventh to nineteenth aspects, the face discrimination means performs a face discrimination process using a neural network.

  According to a twenty-first aspect of the present invention, an acquisition function for acquiring an image signal from captured image data is transmitted to a computer that executes image processing, and the captured image data is converted into predetermined brightness and hue based on the acquired image signal. An occupancy ratio calculation function for calculating an occupancy ratio indicating a ratio of the entire captured image data for each of the divided areas, and the calculated occupancy ratio of each area. A light distribution condition determining function for determining a light distribution condition at the time of photographing by performing an operation of multiplying a coefficient set in advance according to the condition, and from the photographed image data according to the determined light distribution condition A determination function for determining a processing condition for determining a person's face and a face determination function for performing a process of determining a person's face from the captured image data according to the determined processing condition are realized. Is because image processing program.

  According to a twenty-second aspect of the present invention, in a computer that executes image processing, an acquisition function for acquiring an image signal from captured image data, and the captured image data is converted into the captured image data based on the acquired image signal. An occupancy ratio calculating function that divides the predetermined area composed of a combination of the distance from the outer edge of the screen and brightness and calculates an occupancy ratio indicating the ratio of the entire captured image data for each of the divided areas; and A light distribution condition determining function for determining a light distribution condition at the time of photographing by performing an operation of multiplying the calculated occupancy of each area by a coefficient set in advance according to the photographing condition, and the determined distribution A determination function for determining a processing condition for determining a person's face from the photographed image data according to light conditions, and a person's face from the photographed image data according to the determined processing condition. A face discrimination function for performing a process of an image processing program for realizing.

  According to a twenty-third aspect of the present invention, an acquisition function for acquiring an image signal from captured image data is transmitted to a computer that executes image processing, and the captured image data is converted into predetermined brightness and hue based on the acquired image signal. And a first occupancy ratio indicating the ratio of the entire captured image data for each of the divided regions is calculated, and the captured image data is converted into an outer edge of the screen of the captured image data. An occupancy ratio calculation function that divides a predetermined area composed of a combination of a distance from the image and a brightness and calculates a second occupancy ratio indicating the ratio of the entire captured image data for each of the divided areas; A light distribution condition discriminator for determining a light distribution condition at the time of photographing by performing an operation of multiplying the first occupancy ratio and the second occupancy ratio by a coefficient set in advance according to the photographing condition. And a determination function for determining a processing condition for determining a person's face from the photographed image data according to the determined light distribution condition, and a person's character from the photographed image data according to the determined processing condition. An image processing program for realizing a face discrimination function for performing face discrimination processing.

  According to a twenty-fourth aspect of the present invention, in the image processing program according to any one of the twenty-first to twenty-third aspects, the light distribution condition is determined by performing the calculation when the light distribution condition determining function is realized. An index for identification is calculated, a light distribution condition is determined based on the calculated index, and when realizing the determination function, the processing condition of the face determination process according to the calculated index value Switching and / or adjustment.

  According to a twenty-fifth aspect of the present invention, in the image processing program according to any one of the twenty-first to twenty-fourth aspects, the captured image data is predetermined according to a determination result by the face determination function and / or a light distribution condition. It is characterized by realizing an image processing function for performing the image processing.

  According to a twenty-sixth aspect of the present invention, in the image processing program according to any one of the twenty-first to twenty-fifth aspects, when the determination function is realized, the captured image is selected according to the determined light distribution condition. When an extraction condition for extracting a human face candidate area from data is determined and the face discrimination function is realized, the human face candidate area is extracted from the photographed image data according to the determined face candidate area extraction condition. Is extracted, and face discrimination processing is performed on the extracted face candidate region.

  According to a twenty-seventh aspect of the present invention, in the image processing program according to the twenty-sixth aspect, the feature amount extracted from the face candidate region according to the determined light distribution condition when the determination function is realized. When an extraction condition is determined and the face discrimination function is realized, a feature quantity is extracted from the face candidate region according to the determined feature quantity extraction condition, and a face discrimination process is performed based on the extracted feature quantity. It is characterized by performing.

  According to a twenty-eighth aspect of the present invention, in the image processing program according to the twenty-seventh aspect, when the determination function is realized, a feature amount to be extracted from the face candidate region according to the determined light distribution condition is calculated. It is characterized by deciding.

  According to a twenty-ninth aspect of the present invention, in the image processing program according to the twenty-seventh or twenty-eighth aspect, when the determination function is realized, the face is based on the feature amount according to the determined light distribution condition. Determining a face determination processing condition for determining whether or not the candidate region is a person's face, and performing the face determination processing according to the determined face determination processing condition when realizing the face determination function It is a feature.

  The invention according to claim 30 is the image processing program according to any one of claims 21 to 29, wherein the face discrimination function is performed using a neural network when the face discrimination function is realized. It is said.

  According to the first, eleventh, and twenty-first aspects of the present invention, the light distribution condition at the time of shooting is determined from the captured image data, and the person's face is determined from the captured image data according to the determined light distribution condition. By determining the processing conditions, it is possible to improve the face discrimination accuracy. In particular, by determining the light distribution condition from the brightness and hue distribution state of the captured image data, it is possible to improve the determination accuracy of the light distribution condition in the color image.

  According to the second, twelfth, and twenty-second aspects of the present invention, the light distribution condition at the time of shooting is determined from the captured image data, and the person's face is determined from the captured image data according to the determined light distribution condition. By determining the processing conditions, it is possible to improve the face discrimination accuracy. In particular, by determining the light distribution condition from the compositional feature of the brightness distribution position of the photographed image data, it is possible to improve the determination accuracy of the light distribution condition even if the photographed image data is a monochrome image.

  According to the third, thirteenth and twenty-third aspects of the invention, the light distribution condition at the time of photographing is determined from the photographed image data, and the person's face is determined from the photographed image data according to the determined light distribution condition. By determining the processing conditions, it is possible to improve the face discrimination accuracy. In particular, the photographic image data is either a color image or a monochrome image by discriminating the light distribution conditions from the compositional characteristics of the brightness and hue distribution state of the photographic image data and the lightness distribution position of the photographic image data. In addition, it is possible to improve the discrimination accuracy of the light distribution condition.

  According to the invention described in claims 4, 14, and 24, an index for specifying the light distribution condition is calculated, and the processing condition of the face discrimination process is stepwise or continuous according to the calculated index value. By making the adjustment possible, it is possible to perform a finer face discrimination process, thereby improving the face discrimination accuracy.

  According to the fifth, fifteenth and twenty-fifth aspects of the present invention, predetermined image processing is performed on the photographed image data in accordance with the face discrimination result and / or the light distribution condition discriminated with high accuracy. High-quality images can be obtained.

  According to the invention described in claims 6, 16, and 26, a face candidate area is extracted from photographed image data according to a light distribution condition, and it is determined whether or not the extracted face candidate area is a human face. Thus, since the face can be determined from the face candidate area suitable for the light distribution condition, the face candidate area extraction accuracy can be improved and the face discrimination accuracy can be improved.

  According to the invention described in claims 7, 17 and 27, the extraction condition suitable for the light distribution condition is determined by determining the extraction condition of the feature amount extracted from the face candidate region according to the light distribution condition at the time of photographing. Accordingly, it is possible to extract the feature amount from the face candidate region with high accuracy. Thereby, the discrimination accuracy of a face can be improved.

  According to the invention described in claims 8, 18, and 28, the feature amount extracted from the face candidate region is determined according to the light distribution condition at the time of photographing, so that the feature amount suitable for the light distribution condition can be obtained with high accuracy. Can be extracted. Thereby, the discrimination accuracy of a face can be improved.

  According to the invention described in claims 9, 19 and 29, by determining the face determination processing condition for determining whether or not the face candidate region is a face according to the light distribution condition at the time of shooting, The face discrimination accuracy can be further improved.

  According to the tenth, twentieth and thirty aspects of the present invention, by performing the face discrimination process using a neural network, the face discrimination process can be realized with simple means with a low processing load.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
First, the configuration in the present embodiment will be described.

<Appearance Configuration of Image Processing Apparatus 1>
FIG. 1 is a perspective view showing an external configuration of an image processing apparatus 1 according to an embodiment of the present invention. As shown in FIG. 1, the image processing apparatus 1 is provided with a magazine loading unit 3 for loading a photosensitive material on one side of a housing 2. An exposure processing unit 4 that exposes the photosensitive material and a print creation unit 5 that develops the exposed photosensitive material, dries it, and creates a print are provided inside the housing 2. The print created by the print creation unit 5 is discharged to a tray 6 provided on the other side of the housing 2.

  In addition, a CRT (Cathode Ray Tube) 8 serving as a display device, a film scanner unit 9 serving as a device for reading a transparent document, a reflective document input device 10, and an operation unit 11 are provided on the upper portion of the housing 2. The housing 2 further includes an image reading unit 14 that can read image data recorded on various recording media, and an image writing unit 15 that can write (record) image data on various recording media. In addition, a control unit 7 that centrally controls these units is provided inside the housing 2.

  The image reading unit 14 is provided with a PC card adapter 14a and a floppy (registered trademark) disk adapter 14b, and a PC card 13a and a floppy (registered trademark) disk 13b can be respectively inserted therein. For example, image data captured by a digital camera is recorded on the PC card 13a. For example, image data captured by a digital camera is recorded on the floppy (registered trademark) disk 13b. In addition, examples of the recording medium capable of recording image data include a multimedia card (registered trademark), a memory stick (registered trademark), an MD (Mini Disc), and a CD-ROM (Compact Disc Read Only Memory).

  The image writing unit 15 includes a floppy (registered trademark) disk adapter 15a, an MO adapter 15b, and an optical disk adapter 15c. A floppy (registered trademark) disk 16a is used as a recording medium capable of recording image data. , MO 16b and optical disc 16c can be inserted. Examples of the optical disk 16c include a CD-R (Compact Disc-Recordable), a DVD-R (Digital Versatile Disk-Recordable), and the like.

  In FIG. 1, the operation unit 11, the CRT 8, the film scanner unit 9, the reflection original input device 10, and the image reading unit 14 are integrally provided in the housing 2. One or more may be provided separately.

  Further, in the image processing apparatus 1 shown in FIG. 1, there is exemplified an apparatus that creates a print by exposing to a photosensitive material and developing it, but the print creation system is not limited to this, for example, an inkjet system, an electronic system, etc. A method such as a photographic method, a thermal method, or a sublimation method may be used.

<Internal Configuration of Image Processing Apparatus 1>
FIG. 2 shows a main part configuration inside the image processing apparatus 1. As shown in FIG. 2, the image processing apparatus 1 includes a control unit 7, an exposure processing unit 4, a print creation unit 5, a CRT 8, a film scanner unit 9, a reflective original input device 10, an operation unit 11, an image reading unit 14, and an image. The writing unit 15 includes a communication unit (input) 32, a communication unit (output) 33, and a data storage unit 71.

  The control unit 7 includes a microcomputer, and various control programs such as an image processing program stored in a storage unit (not shown) such as a ROM (Read Only Memory), and a CPU (Central Processing Unit) (not shown). The operation of each part constituting the image processing apparatus 1 is comprehensively controlled in cooperation with the above.

  The control unit 7 includes an image processing unit 70, and reads image data acquired by the film scanner unit 9 or the reflection original input device 10 from the image reading unit 14 based on an input signal (command information) from the operation unit 11. The image data and the image data input from the external device via the communication means (input) 32 are subjected to predetermined image processing to generate output image data, which are output to the exposure processing unit 4. Further, the image processing unit 70 performs a conversion process corresponding to the output form on the image processed image data, and outputs the converted image data. Output destinations of the image processing unit 70 include the CRT 8, the image writing unit 15, the communication means (output) 33, and the like.

  The exposure processing unit 4 exposes an image on the photosensitive material, and outputs the exposed photosensitive material to the print creating unit 5. The print creating unit 5 develops and exposes the exposed photosensitive material to create prints P1, P2, and P3. The print P1 is a service size, high-definition size, panoramic size print, the print P2 is an A4 size print, and the print P3 is a business card size print.

  The film scanner unit 9 reads an image recorded on a transparent original such as a developed negative film N or a reversal film taken by an analog camera.

  The reflection original input device 10 reads an image formed on a print P (photo print, document, various printed materials) by a flat bed scanner (not shown).

  The operation unit 11 includes information input means 12. The information input unit 12 is configured by a touch panel, for example, and outputs a pressing signal of the information input unit 12 to the control unit 7. Note that the operation unit 11 may be configured to include a keyboard, a mouse, and the like. The CRT 8 performs a required display process according to the display control signal input from the control unit 7.

  The image reading unit 14 includes, as the image transfer means 30, a PC card adapter 14a, a floppy (registered trademark) disk adapter 14b, and the like, and the PC card 13a inserted into the PC card adapter 14a or a floppy (registered trademark). ) Image data recorded on the floppy (registered trademark) disk 13b inserted into the disk adapter 14b is read out and transferred to the control unit 7. For example, a PC card reader or a PC card slot is used as the PC card adapter 14a.

  The image writing unit 15 includes, as the image conveying unit 31, a floppy (registered trademark) disk adapter 15a, an MO adapter 15b, and an optical disk adapter 15c. In accordance with a write signal input from the control unit 7, the image writing unit 15 includes a floppy (registered trademark) disk 16a inserted into the floppy (registered trademark) disk adapter 15a, an MO 16b inserted into the MO adapter 15b, The generated image data is written to the optical disk 16c inserted into the optical disk adapter 15c.

  The communication means (input) 32 receives image data representing a captured image and a print command signal from another computer in the facility where the image processing apparatus 1 is installed, or a distant computer connected via the Internet or the like. .

  The communication means (output) 33 is connected to the image data representing the photographed image after the image processing and the order information via another computer in the facility where the image processing apparatus 1 is installed, the Internet, or the like. Send to a remote computer.

  The data storage means 71 stores and sequentially stores image data and corresponding order information (information about how many prints are to be created from images of which frames, information on print sizes, etc.).

<Configuration of Image Processing Unit 70>
FIG. 3 shows a main part configuration inside the image processing unit 70. As shown in FIG. 3, the image processing unit 70 includes a film scan data processing unit 701, a reflection original scan data processing unit 702, an image data format decoding processing unit 703, an image adjustment processing unit 704, a CRT specific processing unit 705, a printer specific A processing unit (1) 706, a printer specific processing unit (2) 707, and an image data format creation processing unit 708 are configured.

  A film scan data processing unit 701 performs a calibration operation specific to the film scanner unit 9, negative / positive reversal in the case of a negative document, dust flaw removal, gray balance adjustment, contrast adjustment, and granular noise for the image data input from the film scanner unit 9. Removal, sharpening, and the like are performed, and the result is output to the image adjustment processing unit 704. The film scan data processing unit 701 also includes information on film size, negative / positive type, ISO (International Organization for Standardization) sensitivity recorded on the film optically or magnetically, manufacturer name, information on main subject, and shooting conditions ( For example, the description information content of APS (Advanced Photo System) is also output to the image adjustment processing unit 704.

  The reflection document scan data processing unit 702 performs a calibration operation unique to the reflection document input device 10, negative / positive reversal in the case of a negative document, dust flaw removal, gray balance adjustment, and contrast adjustment for the image data input from the reflection document input device 10. , Noise removal, sharpening enhancement, and the like, and output to the image adjustment processing unit 704.

  An image data format decoding processing unit 703 performs compression code restoration, color data expression method conversion, and the like according to the data format of the image data input from the image transfer unit 30 and the communication unit (input) 32, and the image processing unit The data is converted into a data format suitable for the calculation in 70 and output to the image adjustment processing unit 704.

  The image adjustment processing unit 704 receives the image received from the film scanner unit 9, the reflective original input device 10, the image transfer unit 30, and the communication unit (input) 32 based on the command information input from the operation unit 11 or the control unit 7. Various image processing is performed on the data, and the CRT specific processing unit 705, the printer specific processing unit (1) 706, the printer specific processing unit (2) 707, the image data format creation processing unit 708, and the data storage unit 71 have been processed. Output image data. Image processing executed by the image adjustment processing unit 704 of the present embodiment will be described in detail later with reference to FIGS.

  The CRT specific processing unit 705 performs processing such as changing the number of pixels and color matching on the image data input from the image adjustment processing unit 704, and information that needs to be displayed, such as control information, and a combined display image. Data is output to CRT8.

  A printer-specific processing unit (1) 706 performs printer-specific calibration processing, color matching, pixel number change, and the like on the image data input from the image adjustment processing unit 704, and outputs the result to the exposure processing unit 4.

  When an external printer 34 such as a large-format ink jet printer is connected to the image processing apparatus 1, a printer specific processing unit (2) 707 is provided for each connected printer. The printer-specific processing unit (2) 707 performs appropriate printer-specific calibration processing, color matching, pixel number change, and the like on the image data input from the image adjustment processing unit 704, and outputs the result to the external printer 34. .

  The image data format creation processing unit 708 is represented by JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), Exif (Exchangeable Image File Format) and the like for the image data input from the image adjustment processing unit 704. Are converted into various general-purpose image formats, and output to the image transport unit 31 and the communication means (output) 33.

  A film scan data processing unit 701, a reflection original scan data processing unit 702, an image data format decoding processing unit 703, an image adjustment processing unit 704, a CRT specific processing unit 705, a printer specific processing unit (1) 706, a printer specific processing unit. (2) The category 707 and the image data format creation processing unit 708 are provided to help understanding of the function of the image processing unit 70, and need not be realized as a physically independent device. Alternatively, it may be realized as a type of software processing in a single CPU. Further, the image processing apparatus 1 in the present embodiment is not limited to the above-described configuration, and can be applied to various forms such as a digital photo printer, a printer driver, and various image processing software plug-ins.

Next, the operation in this embodiment will be described.
First, the overall flow of image processing executed by the image adjustment processing unit 704 will be described with reference to the flowchart of FIG.

  First, an image signal of photographic image data to be image processed is acquired via the film scanner unit 9, the reflective original input device 10, the image transfer means 30, the communication means (input) 32, and the like (step S1). Next, based on the image signal acquired in step S1, the captured image data is divided into predetermined image areas, and an occupation ratio calculation process is performed to calculate an occupation ratio indicating the ratio of each divided area to the entire captured image data. (Step S2). The occupation rate calculation process in step S2 will be described in detail later with reference to FIGS.

  Next, based on the occupation ratio calculated in step S2 and a coefficient set in advance according to the shooting conditions, an index (index 1 to 5) for specifying the light distribution condition (light source state) is calculated. Based on the calculated index, a light distribution condition (any one of forward light, backlight, strobe) at the time of photographing is determined (step S3). Here, the index for specifying the light distribution condition is a numerical value that quantitatively represents the light distribution condition at the time of photographing. The index calculation process and the light distribution condition determination process in step S3 will be described in detail later.

  Next, in accordance with the light distribution conditions determined in step S3, processing conditions for face determination processing for determining a person's face from the captured image data are determined (step S4), and face determination processing is performed in accordance with the determined processing conditions. Performed (step S5). The determination of the processing conditions in step S4 will be described in detail later. In addition, the face discrimination process in step S5 will be described in detail later with reference to FIGS.

<Occupancy rate calculation process>
Next, the occupation rate calculation process shown in step S2 of FIG. 4 will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step S10). FIG. 6 shows an example of a conversion program (HSV conversion program) that obtains a hue value, a saturation value, and a lightness value by converting from RGB to the HSV color system using program code (c language). In the HSV conversion program shown in FIG. 6, the values of the digital image data as the input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, and the saturation The value is OutS, the brightness value is OutV, and the unit is defined as 0 to 255.

  Next, the photographed image data is divided into regions composed of combinations of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S11). Hereinafter, the area division of the captured image data will be described in detail.

The lightness value (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 (v6) , 225 to 255 (v7). Hue (H) is a skin hue hue area (H1 and H2) with a hue value of 0 to 39, 330 to 359, a green hue area (H3) with a hue value of 40 to 160, and a blue hue area with a hue value of 161 to 250 It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the determination of the light distribution condition is small. The flesh-color hue area is further divided into a flesh-color area (H1) and other areas (H2). Hereinafter, among the flesh-colored hue regions (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following equation (1) is defined as the flesh-colored region (H1), and the region that does not satisfy the equation (1) is ( H2).
10 <Saturation (S) <175,
Hue '(H) = Hue (H) + 60 (when 0 ≤ Hue (H) <300),
Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),
Luminance Y = InR × 0.30 + InG × 0.59 + InB × 0.11
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (1)
Therefore, the number of divided areas of the captured image data is 4 × 7 = 28. In addition, it is also possible to use the brightness (V) in the formula (1).

When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S12), and the main occupancy ratio calculation is performed. The process ends. Assuming that the first occupancy ratio calculated in the divided area composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy ratio in each divided area is expressed as shown in Table 1.

Next, a method for calculating the index 1 and the index 2 will be described.
Table 2 shows the first coefficient necessary for calculating the index 1 that quantitatively indicates the accuracy of strobe shooting obtained by discriminant analysis, that is, the lightness state of the face area at the time of strobe shooting. Show. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupation ratio Rij of each divided area shown in Table 1 is multiplied.

  FIG. 7 shows a lightness (v) -hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (r1) distributed in the skin color hue region of high brightness in FIG. A negative (−) coefficient is used for the first occupancy calculated from the hue region (r2). FIG. 9 shows the first coefficient in the skin color area (H1) and the first coefficient in the other area (green hue area (H3)) as a curve (coefficient curve) that continuously changes over the entire brightness. It is shown. According to Table 2 and FIG. 9, in the area of high brightness (V = 170 to 224), the sign of the first coefficient in the skin color area (H1) is positive (+), and other areas (for example, the green hue area) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of the two are different.

従って、H1〜H4領域の和は、下記の式(2-1)〜式(2-4)のように表される。
H1領域の和＝R11×(-44.0)＋R21×(-16.0)＋（中略）...＋R71×(-11.3) (2-1)
H2領域の和＝R12×0.0＋R22×8.6＋（中略）... ＋R72×(-11.1) (2-2)
H3領域の和＝R13×0.0＋R23×(-6.3)＋（中略）...＋R73×(-10.0) (2-3)
H4領域の和＝R14×0.0＋R24×(-1.8)＋（中略）...＋R74×(-14.6) (2-4) When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (2).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (2-1) to (2-4).
H1 area sum = R11 x (-44.0) + R21 x (-16.0) + (omitted) ... + R71 x (-11.3) (2-1)
Sum of H2 area = R12 x 0.0 + R22 x 8.6 + (omitted) ... + R72 x (-11.1) (2-2)
Sum of H3 area = R13 x 0.0 + R23 x (-6.3) + (omitted) ... + R73 x (-10.0) (2-3)
Sum of H4 area = R14 x 0.0 + R24 x (-1.8) + (omitted) ... + R74 x (-14.6) (2-4)

The index 1 is defined as Expression (3) using the sum of the H1 to H4 regions shown in Expressions (2-1) to (2-4).
Index 1 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 4.424 (3)

Table 3 shows, for each divided region, the second coefficient necessary for calculating the index 2 that quantitatively indicates the accuracy of backlight photographing obtained by discriminant analysis, that is, the brightness state of the face region at the time of backlight photographing. Show. The coefficient of each divided area shown in Table 3 is a weighting coefficient that is multiplied by the first occupation ratio Rij of each divided area shown in Table 1.

  FIG. 8 shows a lightness (v) -hue (H) plane. According to Table 3, a negative (-) coefficient is used for the occupancy calculated from the region (r4) distributed in the intermediate lightness of the flesh color hue region in FIG. 8, and the low lightness (shadow) region ( A positive (+) coefficient is used for the occupation ratio calculated from r3). FIG. 10 shows the second coefficient in the skin color area (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and FIG. 10, the sign of the second coefficient of the intermediate lightness area of the flesh color hue area having the lightness value of 85 to 169 (v4) is negative (−), and the lightness value is 26 to 84 (v2, It can be seen that the sign of the second coefficient in the low brightness (shadow) region of v3) is positive (+), and the sign of the coefficient in both regions is different.

従って、H1〜H4領域の和は、下記の式(4-1)〜式(4-4)のように表される。
H1領域の和＝R11×(-27.0)＋R21×4.5＋（中略）...＋R71×(-24.0) (4-1)
H2領域の和＝R12×0.0＋R22×4.7＋（中略）... ＋R72×(-8.5) (4-2)
H3領域の和＝R13×0.0＋R23×0.0＋（中略）...＋R73×0.0 (4-3)
H4領域の和＝R14×0.0＋R24×(-5.1)＋（中略）...＋R74×7.2 (4-4) When the second coefficient in the lightness area vi and the hue area Hj is Dij, the sum of the Hk areas for calculating the index 2 is defined as in Expression (4).

Therefore, the sum of the H1 to H4 regions is expressed by the following formulas (4-1) to (4-4).
H1 area sum = R11 x (-27.0) + R21 x 4.5 + (omitted) ... + R71 x (-24.0) (4-1)
Sum of H2 area = R12 x 0.0 + R22 x 4.7 + (omitted) ... + R72 x (-8.5) (4-2)
Sum of H3 area = R13 x 0.0 + R23 x 0.0 + (omitted) ... + R73 x 0.0 (4-3)
H4 area sum = R14 x 0.0 + R24 x (-5.1) + (omitted) ... + R74 x 7.2 (4-4)

The index 2 is defined as in Expression (5) using the sum of the H1 to H4 regions shown in Expressions (4-1) to (4-4).
Index 2 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 1.554 (5)
Since the index 1 and the index 2 are calculated based on the brightness of the captured image data and the distribution amount of the hue, it is effective for determining the light distribution condition when the captured image data is a color image.

  Next, the occupation rate calculation process (step S2 in FIG. 4) for calculating the index 3 will be described in detail with reference to the flowchart in FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step S20). Next, the photographed image data is divided into regions each composed of a combination of the distance from the outer edge of the photographed image screen and the brightness, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S21). Hereinafter, the area division of the captured image data will be described in detail.

  FIGS. 12A to 12D show four regions n1 to n4 divided according to the distance from the outer edge of the screen of the captured image data. A region n1 shown in FIG. 12A is an outer frame, a region n2 shown in FIG. 12B is a region inside the outer frame, and a region n3 shown in FIG. A region n4 shown in FIG. 12D is an inner region and is a central region of the captured image screen. Further, the lightness is divided into seven regions v1 to v7 as described above. Therefore, when the captured image data is divided into regions composed of combinations of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S22), and the main occupancy ratio calculation is performed. The process ends. If the second occupancy calculated in the divided area composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each divided area is expressed as shown in Table 4.

図１３は、画面領域ｎ１〜ｎ４における第３の係数を、明度全体に渡って連続的に変化する曲線（係数曲線）として示したものである。 Next, a method for calculating the index 3 will be described.
Table 5 shows the third coefficient necessary for calculating the index 3 for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy Qij of each divided area shown in Table 4 is multiplied, and is obtained by discriminant analysis.

FIG. 13 shows the third coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

従って、n1〜n4領域の和は、下記の式(6-1)〜式(6-4)のように表される。
n1領域の和＝Q11×40.1＋Q21×37.0＋（中略）...＋Q71×22.0 (6-1)
n2領域の和＝Q12×(-14.8)＋Q22×(-10.5)＋（中略）...＋Q72×0.0 (6-2)
n3領域の和＝Q13×24.6＋Q23×12.1＋（中略）...＋Q73×10.1 (6-3)
n4領域の和＝Q14×1.5＋Q24×(-32.9)＋（中略）...＋Q74×(-52.2) (6-4) If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (6).

Accordingly, the sum of the n1 to n4 regions is expressed as the following formulas (6-1) to (6-4).
n1 area sum = Q11 x 40.1 + Q21 x 37.0 + (omitted) ... + Q71 x 22.0 (6-1)
Sum of n2 area = Q12 x (-14.8) + Q22 x (-10.5) + (omitted) ... + Q72 x 0.0 (6-2)
n3 area sum = Q13 x 24.6 + Q23 x 12.1 + (omitted) ... + Q73 x 10.1 (6-3)
n4 area sum = Q14 x 1.5 + Q24 x (-32.9) + (omitted) ... + Q74 x (-52.2) (6-4)

The index 3 is defined as in Expression (7) using the sum of the N1 to H4 regions shown in Expressions (6-1) to (6-4).
Index 3 = sum of n1 regions + sum of n2 regions + sum of n3 regions + sum of n4 regions−12.6201 (7)
The index 3 is calculated on the basis of compositional characteristics (distance from the outer edge of the screen of the photographed image data) based on the lightness distribution position of the photographed image data, so that the light distribution condition of the monochrome image as well as the color image is discriminated. It is also effective to do.

The index 4 is defined as Expression (8) using the indices 1 and 3, and the index 5 is defined as Expression (9) using the indices 1 to 3.
Index 4 = 0.565 x Index 1 + 0.565 x Index 3 + 0.457 (8)
Indicator 5 = (-0.121) x Indicator 1 + 0.91 x Indicator 2 + 0.113 x Indicator 3-0.072 (9)
Here, the weighting coefficient by which each index is multiplied in Expression (8) and Expression (9) is set in advance according to the shooting conditions.

このように指標４、５の値により配光条件を定量的に判別することができる。 <Determination method of light distribution conditions>
Next, a method for determining light distribution conditions (step S3 in FIG. 4) will be described.
In FIG. 14, 60 images are taken under each light distribution condition of forward light, backlight, and strobe, and indexes 4 and 5 are calculated for a total of 180 digital image data, and the indexes 4 and 5 under each light distribution condition are calculated. The values are plotted. According to FIG. 14, when the value of index 4 is greater than 0.5, there are many strobe scenes, and when the value of index 4 is 0.5 or less and the value of index 5 is greater than −0.5, the backlight scene is I understand that there are many. Table 6 shows the determination contents of the light distribution conditions based on the values of the indexes 4 and 5.

Thus, the light distribution condition can be determined quantitatively based on the values of the indexes 4 and 5.

<Determination of processing conditions for face discrimination processing>
Next, the method for determining the processing conditions for the face discrimination process shown in step S4 in FIG. 4 will be described in detail.

  The determination of the processing condition of the face discrimination process executed according to the light distribution condition at the time of shooting is performed in order to improve the discrimination accuracy of the face discrimination process. As a processing method of face discrimination processing, a face candidate area is extracted from captured image data, a method for determining whether or not the extracted face candidate area is a human face, and a human face is included in the face candidate area For example, there is a method for determining whether or not it is. When such a method is used, the discrimination accuracy of the face discrimination process includes the extraction accuracy of the face candidate region and whether or not the face candidate region is a human face (or whether a face is included in the face candidate region). Both of the determination accuracy contribute to determining whether or not the processing conditions of the face discrimination process are determined, the extraction conditions of the face candidate area, and the processing conditions of the process of determining a person's face from the face candidate area It is preferable to determine either one or both in accordance with the determination result of the light distribution condition.

  Hereinafter, the process of extracting a face candidate area from captured image data and the process of determining whether or not the extracted face candidate area is a person's face will be referred to as a “face discrimination process”. Of these, the part of the process for determining whether or not the face candidate area is a person's face is referred to as “face determination process”.

First, a method for determining the extraction condition of the face candidate area will be described.
As a method for extracting a face candidate area, a method can be used in which an initial point corresponding to a skin color is searched from captured image data, and a skin color area is extracted as a face candidate area using a simple area expansion method. The simple area expansion method is an image processing method for extracting a specific image area by expanding adjacent areas with pixels adjacent to each other whose data difference between pixels is equal to or less than a threshold value belonging to the same image area. . In other words, the simple area expansion method starts from an initial pixel that matches a specified condition, and a data difference between adjacent pixels (either 4-connected or 8-connected) is equal to or less than a threshold value. In this case, it is assumed that the adjacent pixel and the initial pixel belong to the same image area, and the same determination is performed for a pixel adjacent to the pixel that belongs to the same pixel. In this manner, starting from the initial pixel, a specific image region is extracted by gradually expanding the same image region. In the present embodiment, it is preferable to further use a process for extracting a face component region by extracting an edge component in an image and ending simple region expansion when the image edge is reached.

  When a skin color area is extracted as the face candidate area, even if the subject is the same person, the color of the skin color area is significantly different depending on the light distribution conditions at the time of shooting. For example, based on the skin color of an image taken in front light, the skin tone becomes whiter (especially the closer it is) in strobe shooting, and if it is shooting in backlight, Skin tone becomes darker. In consideration of such a situation, it is preferable to switch and / or adjust (stepwise or continuously) the search condition for the initial pixel according to the determination result of the light distribution condition.

  For example, for a predetermined basic search condition for searching for an initial pixel, if the light distribution condition is “forward light”, the basic search condition is used. If the light distribution condition is “strobe”, the basic search condition is used. If the search condition with high (brighter) correction is used and the light distribution condition is “backlight”, the search condition with lower (darker) correction from the basic search condition may be used. As described above, when extracting a face candidate area using the simple area expansion method, the search condition for the initial pixel may be switched according to the light distribution condition at the time of photographing. Moreover, since the edge intensity | strength which complete | finishes simple area expansion changes with light distribution conditions, it is preferable to adjust the edge extraction conditions for ending simple area expansion according to light distribution conditions.

  When the determination result of the light distribution condition is obtained as an index (index 4, 5) indicating the light distribution condition stepwise or continuously, such as strobe degree, backlight intensity, etc., depending on the value of the obtained index The search condition for the initial pixel may be determined by adjusting the degree of increasing the brightness (in the case of a strobe) and the degree of decreasing in the brightness (in the case of backlight). Therefore, the simple region expansion method using the initial pixel search condition can be determined as the extraction condition for the face candidate region.

  In the above description, the example in which the search condition for the initial pixel is corrected using the fact that the brightness difference is generated in the skin color area according to the light distribution condition is shown. However, not only the brightness but also saturation is used. Thus, the search condition correction requirement may be increased. For example, with flash and flash photography, the saturation is lower than with direct light. Further, not only the initial pixel search condition but also the area expansion condition may be corrected. Furthermore, it is preferable to correct the initial pixel search condition and the area expansion condition for various processing conditions for extracting the face candidate area, reflecting the difference in the characteristics of the skin color area obtained for each light distribution condition.

  Next, a method for determining a face determination processing condition for determining a face from a face candidate area will be described. In the following, a method for determining a face determination processing condition when a feature amount is extracted from a face candidate region and face determination processing is performed using a neural network based on the extracted feature amount will be described.

  First, prior to a specific description of a method for determining face determination processing conditions, a neural network used in face determination processing will be briefly described. As shown in FIG. 15, this neural network is a hierarchical neural network having an input layer, an intermediate layer, and an output layer (hereinafter simply referred to as a neural network). The neural network preferably uses a digital neural network chip, but can also be realized by using a general-purpose DSP (Digital Signal Processor) and a dedicated emulation program, or using a normal CPU and an emulation program. It doesn't matter.

ここで、ｆ（ｘ）は一般に式（１１）に示すような非線形のシグモイド関数が用いられるが、必ずしもこれに限定されるわけではない。

Each layer of the neural network is composed of structural units called neurons. Except for the input layer, as shown in FIG. 16, an arbitrary neuron receives a large number of inputs called synapses, and multiplies each input value x _i by a predetermined weight w _i called connection strength to sum up the sum. It functions as a functional element that obtains and evaluates a result obtained by evaluating it with a predetermined output function f as output y. The output function f is expressed as in Expression (10).

Here, f (x) is generally a non-linear sigmoid function as shown in Equation (11), but is not necessarily limited thereto.

Here, the input weight w _i of each neuron is important information for determining the relationship between the input and output of the neural network, in other words, the operation of the neural network, and is determined by performing a learning operation on the neural network. In FIG. 15, only one intermediate layer is shown, but this may have a configuration of one or more arbitrary hierarchies. Hierarchical networks are known to be able to achieve any function that can be synthesized if there is only one intermediate layer, but increasing the number of intermediate layers is advantageous in terms of learning efficiency and the number of neurons. It has been known.

The neural network has the following characteristics.
(1) A multi-input / multi-output nonlinear system can be realized with a relatively simple configuration. (2) Each neuron in each layer can be operated independently, and high-speed operation can be expected by parallel processing.
(3) Arbitrary input / output relations can be realized by providing appropriate teacher data for learning.
(4) The system has generalization ability. That is, there is an ability to give a generally correct output even to an unlearned input pattern that is not necessarily given as teacher data.

  Here, the teacher data is a pair of an input pattern and a desired output pattern for the input pattern, and a plurality of teacher data are usually prepared. The operation of determining a desired output pattern for a specific input pattern is generally determined by relying on the subjective judgment of an expert. Major applications of neural networks include (a) nonlinear function approximation and (b) clustering (identification, recognition, classification).

  In general, nonlinear function approximation is easy to implement and it is easy to obtain a system with high generalization ability. On the other hand, application to clustering is highly dependent on teacher data, and there is a high possibility that there is a contradiction between a plurality of teacher data for complex input. For this reason, when applied to clustering, unless the input is very simple, the result is not only a low generalization ability, but also an unsatisfactory system in terms of suitability for specific teacher data. There are many things that are not.

ここで、式（１２）の関数ｆ（ｘ）は、式（１１）に示した非線形のシグモイド関数、Ｙ_iLはニューロンの出力、Ｕ_iLはニューロンの内部状態、Ｙ_i＜Ｌ−１＞はL−１層のニューロンの出力＝Ｌ層のニューロンの入力、ｗ_ij＜Ｌ＞は結合強度である。 Next, a learning method using teacher data for the neural network will be described. The state of the i-th neuron in the L-th layer with respect to the p-th input pattern is expressed as Equation (12) and Equation (13).

Here, the function f (x) in Expression (12) is the nonlinear sigmoid function shown in Expression (11), Y _iL is the output of the neuron, U _iL is the internal state of the neuron, and Y _i <L−1> is L-1 layer neuron output = L layer neuron input, w _ij <L> is the connection strength.

ただしｋは出力層の番号であり、Ｔ_i（ｐ）はｐ番目の入力パターンに対する望ましい出力パターンである。 In such a definition, the widely used error back propagation learning method (back propagation learning method, hereinafter referred to as BP method) uses the mean square error as an error evaluation function, and the error Define E.

Where k is the output layer number and T _i (p) is the desired output pattern for the pth input pattern.

ここで、∂は偏微分記号（ラウンド）、ηは学習率を表す係数である。Δｗ_ij＜Ｌ＞を各々のｗ_ij＜Ｌ＞に加算することによって学習が進む。なお、この形式のＢＰ法では学習が進んだ結果、教師データとの誤差の絶対値が小さくなってくると、Δｗの絶対値も小さくなり、学習が進まなくなるという現象が指摘されており、その問題点を回避するために、種々の改良が提案されている。 In this case, the correction amount Δw of each bond strength based on the BP method is determined by the following equation.

Here, ∂ is a partial differential symbol (round), and η is a coefficient representing a learning rate. Learning proceeds by adding Δw _ij <L> to each w _ij <L>. In addition, as a result of learning progressed in the BP method of this form, if the absolute value of the error from the teacher data becomes small, the absolute value of Δw also becomes small, and the phenomenon that learning does not progress has been pointed out. Various improvements have been proposed in order to avoid problems.

式（１６）の第二項は慣性項（モーメンタム項）と呼ばれ、現在の修正量だけでなく、過去の修正量も考慮することで収束時の振動を抑え、学習を高速化する効果がある。安定化定数αは1．0以下の正の実数で、過去の修正量の考慮の度合いを規定する係数である。 For example, there is a proposal to change the definition of Δw as follows.

The second term in equation (16) is called an inertial term (momentum term), and has the effect of suppressing the vibration at the time of convergence by considering not only the current correction amount but also the past correction amount, and speeding up learning. is there. The stabilization constant α is a positive real number of 1.0 or less, and is a coefficient that defines the degree of consideration of past correction amounts.

Alternatively, there is also a proposal of dynamically changing the learning rate η so as to satisfy the following expression according to the number of learnings n.

  Furthermore, there is also a proposal that the error evaluation is in a form that cancels out the derivative of the sigmoid function instead of the mean square error. In any case, an output approximate to the teacher data can be obtained by performing the learning operation a sufficient number of times using the BP method.

  Next, a method for determining the face determination processing condition will be specifically described. If the same face determination processing conditions are used even though the light distribution conditions are different, a difference occurs in the face determination accuracy. This is because the feature amount contributing to the face determination process is different for each light distribution condition, and the extraction condition for optimally extracting the feature amount is different. Therefore, it is preferable to determine the face determination processing condition according to the light distribution condition.

  Specifically, it is preferable to determine the learning result of the neural network used for the design of the neural network and the face determination process according to the light distribution condition. Here, the design of the neural network indicates, for example, designing the type and number of input parameters of the neural network, the number of intermediate layers, and the like. The input parameter includes a feature amount extracted from the face candidate region. The learning result of the neural network represents a weight (Weight value) assigned to each neuron constituting the neural network, a slope value of the sigmoid function, and the like.

  Regardless of the nature of the feature quantity used as the input parameter of the neural network and the light distribution condition, if the feature quantity is extracted from the face candidate region under the same extraction condition, a difference occurs in the extraction accuracy. In addition, depending on the light distribution conditions, there is a difference in the distinguishability between a person's face and a portion other than the face depending on the nature of the feature amount. For this reason, it is preferable to change the feature amount extracted from the face candidate region and the feature amount extraction condition in accordance with the light distribution condition.

  For example, using features that make up the face such as eyes and mouth as feature values, and extraction conditions for feature values, differences in skin and eyes or mouth color (skin color and almost black, skin color and redness) and contours of parts In the case of using the extraction condition using the edge of the backlight, there is a tendency for the backlight to be backlit and the backlight to be less accurate in extracting feature quantities. That is, in the backlight, the entire face of the person is photographed darker than the normal light, so that the difference between the color values of the skin and the part is small, and the edge of the contour is difficult to extract compared to the normal light.

  In this way, the feature quantity extraction conditions may be switched or the type and number of input parameters of the neural network may be changed according to the light distribution conditions. Further, the feature quantity extraction condition may be adjusted stepwise or continuously in accordance with the calculated index value (strobe degree, backlight intensity, etc.). It is also preferable to change the number of intermediate layers and the number of neurons in the neural network according to the number of input parameters.

  Further, when determining the face determination processing conditions, it is preferable to switch the neural network learned in advance for each of the forward light, strobe, and backlight images according to the light distribution conditions. In addition, if the result of determining the light distribution condition is a system in which the light distribution condition is obtained as an index indicating stepwise or continuously, such as strobe degree, backlight intensity, etc., the neural network is used for the intensity strobe, weak If it is prepared step by step according to the degree of the light distribution condition, such as for the strobe for light, for the strong backlight, for the weak backlight, etc., the determination accuracy of the face determination process can be improved. In addition, it is preferable to operate the system by appropriately updating it using a version upgrade method of the self-learning neural network.

<Face detection process>
Next, the face discrimination process shown in step S5 of FIG. 4 will be described in detail with reference to the flowchart of FIG.

  First, a face candidate area is extracted from the photographed image data according to the face candidate area extraction condition determined in step S4 of FIG. 4 (step S30), and it is determined whether or not there is a face candidate area in the photographed image data. (Step S31).

  If it is determined in step S31 that there is no face candidate area in the captured image data (step S31; NO), the face discrimination process ends. If it is determined in step S31 that there is a face candidate area in the captured image data (step S31; YES), face determination is performed to determine whether the face candidate area extracted in step S30 is a human face. Processing is performed (step S32), and the face discrimination processing ends. The face determination process in step S32 will be described in detail later with reference to FIG.

  In the above description, the simple area expansion method is used as an example of the face candidate area extraction process. However, the face candidate area extraction method is not limited to this. For example, it is possible to use a method of extracting an edge from photographed image data and extracting a specific shape pattern such as an ellipse using a technique such as Hough transform or template matching.

  Furthermore, as disclosed in JP-A-4-346332, an original image (photographed image data) is divided into a large number of pixels, and a histogram of hue values and saturation values is created from the RGB values of each pixel, A method is used in which a histogram is divided from the shape, an image is divided into areas composed of pixels corresponding to the divided parts, and an area corresponding to a human face is estimated from the divided areas. be able to.

  Further, as disclosed in JP-A-6-309433, JP-A-6-67320, JP-A-5-158164, and JP-A-5-165120, a hue value, a saturation value, and a luminance value are disclosed. A method of determining a skin color area based on a histogram such as the above and using the skin color area as a face candidate area can be used. Further, a line process as disclosed in JP-A-8-221567, JP-A-2000-20694, JP-A-2000-32272, JP-A-2000-201158, and JP-A-2000-20769. The face candidate region can be extracted using the method.

  In addition, an edge and a low frequency image are generated from a color digital image, and predetermined parameters (for example, saturation value, hue value, and lightness value) are initially matched with predetermined initial conditions (values in each predetermined range). Extract a pixel, perform simple area expansion from the extracted initial pixel for the low-frequency image, and forcibly end simple area expansion when the image edge is reached to extract a face candidate area May be.

  Next, the face determination process shown in step S32 of FIG. 17 will be described with reference to the flowchart of FIG.

  First, a feature value is extracted from the face candidate region according to the feature value determined in step S4 of FIG. 4 and its extraction condition (step S40). Next, a neural network determined according to the light distribution condition (for example, when the light distribution condition is backlight), using the feature amount extracted in step S40 as an input signal (input parameter) and a parameter representing facialness as an output signal. The neural network learned for backlighting is used to determine whether the face candidate area is a human face (step S41), and the face determination process ends.

  When the face determination process ends, predetermined image processing for improving the image quality of the image is performed on the captured image data in accordance with the determination result of the face determination and / or the light distribution condition. For example, when it is determined that the face candidate area is a person's face, density correction and color correction suitable for the person's face are performed. When the light distribution condition is a strobe, the face area of the photographed image data is white, so that a process for adding color is performed. When the light distribution condition is backlight, correction is performed to brighten the face area and darken the background area.

  As described above, according to the image processing apparatus 1 of the present embodiment, the light distribution condition at the time of shooting is determined from the captured image data, and the person's face is determined from the captured image data according to the determined light distribution condition. By determining the processing conditions for this, the face discrimination accuracy can be improved.

  In particular, by calculating the index for specifying the light distribution condition from the brightness and hue distribution state of the captured image data, and determining the light distribution condition based on the index, the accuracy of determining the light distribution condition in the color image Can be improved. Further, an index for specifying the light distribution condition is calculated from the distribution position (compositional feature) of the brightness of the photographed image data, and the photographed image data is obtained by determining the light distribution condition based on the index. Even in the case of an image, it is possible to improve the light distribution condition discrimination accuracy.

  In addition, according to the value of the index for specifying the light distribution condition, by making it possible to adjust the processing conditions of the face discrimination process stepwise or continuously, it is possible to perform a finer face discrimination process, The face discrimination accuracy can be further improved.

  Furthermore, a face candidate area is extracted from the captured image data according to the light distribution condition, and it is determined whether or not the extracted face candidate area is a human face. Since the face can be determined from the candidate area, the accuracy of extracting the face candidate area can be improved and the face discrimination accuracy (determination accuracy) can be further improved.

  In addition, by extracting feature values from the face candidate region according to the light distribution conditions and performing face determination based on the extracted feature values, the feature amount extraction accuracy is improved and face discrimination is performed. The accuracy (judgment accuracy) can be further improved.

  In particular, the face discrimination process is performed using a neural network using the extracted feature quantity as an input signal (input parameter), so that the face discrimination process can be realized with a simple means with a low processing load. . Further, since the neural network to be used is determined according to the light distribution condition, the face discrimination accuracy (determination accuracy) can be further improved.

  Further, since predetermined image processing is performed on the captured image data in accordance with the face discrimination result and / or the light distribution condition determined with high accuracy, a high-quality image can be obtained.

  Note that the description in the present embodiment can be changed as appropriate without departing from the spirit of the present invention.

  For example, in this embodiment, an example in which a neural network is used for the face determination process has been described, but the method of the face determination process is not limited to this. For example, a method of performing face determination using template matching for an image obtained by binarizing a face candidate area, or extracting a feature part characterizing a face such as an eye or a mouth from the face candidate area, and the position of the face candidate area and feature part Any method such as a method of performing face determination based on the relationship can be employed.

  In this embodiment, in the face discrimination process, the face candidate area is extracted from the captured image data, and it is determined whether or not the extracted face candidate area is a face. It is possible to discriminate the face without the need. For example, it is possible to apply a process of scanning the image area of the photographed image data with a predetermined face template and determining that there is a person's face at that location if there is a matching location.

1 is a perspective view showing an external configuration of an image processing apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram showing the main configuration of the image processing apparatus according to the present embodiment. The block diagram which shows the principal part structure of the image processing part of FIG. 6 is a flowchart showing the overall flow of image processing executed in the image adjustment processing unit of the present embodiment. The flowchart which shows the occupation rate calculation process which calculates the 1st occupation rate for every area | region of brightness and hue. The figure which shows an example of the program which converts from RGB to HSV color system. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r1 and the area | region r2 on a VH plane. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r3 and the area | region r4 on a VH plane. The figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. The figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. The flowchart which shows the occupation rate calculation process which calculates a 2nd occupation rate based on the composition of picked-up image data. The figure which shows the area | regions n1-n4 determined according to the distance from the outer edge of the screen of picked-up image data. The figure which shows the curve showing the 3rd coefficient for multiplying the 2nd occupation rate for calculating the parameter | index 3 according to area | region (n1-n4). The plot figure of the parameter | index 4 and the parameter | index 5 calculated according to light distribution conditions (forward light, strobe, backlight). The figure which shows the structure of a hierarchical neural network typically. The figure which shows the structure of the neuron which comprises a neural network. The flowchart which shows a face discrimination | determination process. The flowchart which shows a face determination process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 4 Exposure processing part 5 Print preparation part 7 Control part 8 CRT
DESCRIPTION OF SYMBOLS 9 Film scanner part 10 Reflective original input device 11 Operation part 14 Image reading part 15 Image writing part 30 Image transfer means 31 Image conveyance part 32 Communication means (input)
33 Communication means (output)
34 External Printer 70 Image Processing Unit 701 Film Scan Data Processing Unit 702 Reflected Original Scan Data Processing Unit 703 Image Data Format Decoding Processing Unit 704 Image Adjustment Processing Unit 705 CRT Specific Processing Unit 706 Printer Specific Processing Unit (1)
707 Printer-specific processing unit (2)
708 Image data format creation processing unit 71 Data storage means

Claims (30)

An acquisition step of acquiring an image signal from the captured image data;
Based on the acquired image signal, the photographed image data is divided into areas composed of a combination of predetermined brightness and hue, and an occupation ratio indicating a ratio of the whole photographed image data is obtained for each of the divided areas. An occupancy calculation step to calculate,
A light distribution condition determining step of determining a light distribution condition at the time of photographing by performing an operation of multiplying the calculated occupation ratio of each region by a coefficient set in advance according to the photographing condition;
A determination step for determining processing conditions for determining a person's face from the captured image data in accordance with the determined light distribution conditions;
A face discrimination step for performing a process of discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing method comprising: An acquisition step of acquiring an image signal from the captured image data;
Based on the acquired image signal, the photographed image data is divided into predetermined regions composed of a combination of the distance from the outer edge of the screen of the photographed image data and brightness, and the photographing is performed for each of the divided regions. An occupancy ratio calculating step for calculating an occupancy ratio indicating the ratio of the entire image data;
A light distribution condition determining step of determining a light distribution condition at the time of photographing by performing an operation of multiplying the calculated occupation ratio of each region by a coefficient set in advance according to the photographing condition;
A determination step for determining processing conditions for determining a person's face from the captured image data in accordance with the determined light distribution conditions;
A face discrimination step for performing a process of discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing method comprising: An acquisition step of acquiring an image signal from the captured image data;
Based on the acquired image signal, the photographed image data is divided into regions composed of a combination of a predetermined brightness and hue, and a first ratio indicating a ratio of the entire photographed image data for each divided region. The occupancy ratio is calculated, and the captured image data is divided into a predetermined area composed of a combination of the distance from the outer edge of the screen of the captured image data and the brightness, and for each of the divided areas, the entire captured image data is divided. An occupancy ratio calculating step of calculating a second occupancy ratio indicating the occupying ratio;
A light distribution condition determining step of determining a light distribution condition at the time of photographing by performing an operation of multiplying the calculated first occupancy and second occupancy by a coefficient set in advance according to the photographing condition. When,
A determination step for determining processing conditions for determining a person's face from the captured image data in accordance with the determined light distribution conditions;
A face discrimination step for performing a process of discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing method comprising: In the light distribution condition determining step, an index for specifying the light distribution condition is calculated by performing the calculation, and the light distribution condition is determined based on the calculated index.
The image according to any one of claims 1 to 3, wherein in the determination step, switching and / or adjustment of processing conditions for face discrimination processing is performed according to the calculated index value. Processing method.   5. The image processing step according to claim 1, further comprising an image processing step of performing predetermined image processing on the captured image data in accordance with a determination result and / or a light distribution condition in the face determination step. Image processing method. In the determining step, an extraction condition for extracting a human face candidate region from the captured image data is determined according to the determined light distribution condition,
In the face discrimination step, a human face candidate area is extracted from the captured image data in accordance with the determined face candidate area extraction condition, and a face discrimination process is performed on the extracted face candidate area. The image processing method according to claim 1, wherein the image processing method is characterized. In the determination step, an extraction condition for a feature amount to be extracted from the face candidate area is determined according to the determined light distribution condition,
The feature determination is characterized in that, in the face discrimination step, a feature quantity is extracted from the face candidate region according to the determined feature quantity extraction condition, and a face discrimination process is performed based on the extracted feature quantity. 6. The image processing method according to 6.   The image processing method according to claim 7, wherein in the determination step, a feature amount extracted from the face candidate region is determined according to the determined light distribution condition. In the determining step, a face determination processing condition for determining whether or not the face candidate region is a person's face based on the feature amount is determined according to the determined light distribution condition,
The image processing method according to claim 7 or 8, wherein in the face discrimination step, face discrimination processing is performed according to the determined face determination processing conditions.   The image processing method according to claim 1, wherein in the face discrimination step, face discrimination processing is performed using a neural network. Acquisition means for acquiring an image signal from captured image data;
Based on the acquired image signal, the photographed image data is divided into areas composed of a combination of predetermined brightness and hue, and an occupation ratio indicating a ratio of the whole photographed image data is obtained for each of the divided areas. Occupancy ratio calculating means for calculating;
A light distribution condition determining means for determining a light distribution condition at the time of photographing by performing an operation of multiplying the calculated occupancy of each region by a coefficient set in advance according to the photographing condition;
Determining means for determining a processing condition for discriminating a person's face from the photographed image data in accordance with the discriminated light distribution condition;
Face discriminating means for performing processing for discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing apparatus comprising: Acquisition means for acquiring an image signal from captured image data;
Based on the acquired image signal, the photographed image data is divided into predetermined regions composed of a combination of the distance from the outer edge of the screen of the photographed image data and brightness, and the photographing is performed for each of the divided regions. An occupancy ratio calculating means for calculating an occupancy ratio indicating the ratio of the entire image data;
A light distribution condition determining means for determining a light distribution condition at the time of photographing by performing an operation of multiplying the calculated occupancy of each region by a coefficient set in advance according to the photographing condition;
Determining means for determining a processing condition for discriminating a person's face from the photographed image data in accordance with the discriminated light distribution condition;
Face discriminating means for performing processing for discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing apparatus comprising: Acquisition means for acquiring an image signal from captured image data;
Based on the acquired image signal, the photographed image data is divided into regions composed of a combination of a predetermined brightness and hue, and a first ratio indicating a ratio of the entire photographed image data for each divided region. The occupancy ratio is calculated, and the captured image data is divided into a predetermined area composed of a combination of the distance from the outer edge of the screen of the captured image data and the brightness, and for each of the divided areas, the entire captured image data is divided. An occupancy ratio calculating means for calculating a second occupancy ratio indicating the occupying ratio;
Light distribution condition determining means for determining the light distribution condition at the time of photographing by performing an operation of multiplying the calculated first occupancy and second occupancy by a coefficient set in advance according to the photographing condition. When,
Determining means for determining a processing condition for discriminating a person's face from the photographed image data in accordance with the discriminated light distribution condition;
Face discriminating means for performing processing for discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing apparatus comprising: The light distribution condition determining means calculates the index for specifying the light distribution condition by performing the calculation, determines the light distribution condition based on the calculated index,
The image processing according to any one of claims 11 to 13, wherein the determination unit performs switching and / or adjustment of processing conditions for face discrimination processing according to the calculated index value. apparatus.   15. The image processing device according to claim 11, further comprising an image processing unit configured to perform predetermined image processing on the captured image data in accordance with a determination result by the face determination unit and / or a light distribution condition. Image processing apparatus. The determining means determines an extraction condition for extracting a human face candidate region from the captured image data according to the determined light distribution condition,
The face discriminating means extracts a human face candidate area from the captured image data in accordance with the determined face candidate area extraction condition, and performs face discrimination processing on the extracted face candidate area. The image processing apparatus according to any one of claims 11 to 15. The determining unit determines an extraction condition for a feature amount to be extracted from the face candidate region according to the determined light distribution condition,
The face discrimination means extracts a feature quantity from the face candidate region according to the determined feature quantity extraction condition, and performs face discrimination processing based on the extracted feature quantity. An image processing apparatus according to 1.   The image processing apparatus according to claim 17, wherein the determining unit determines a feature amount to be extracted from the face candidate region according to the determined light distribution condition. The determining means determines a face determination processing condition for determining whether the face candidate region is a human face based on the feature amount according to the determined light distribution condition,
The image processing apparatus according to claim 17, wherein the face determination unit performs a face determination process according to the determined face determination processing condition.   The image processing apparatus according to claim 11, wherein the face determination unit performs a face determination process using a neural network. On the computer that performs image processing,
An acquisition function for acquiring image signals from captured image data;
Based on the acquired image signal, the photographed image data is divided into areas composed of a combination of predetermined brightness and hue, and an occupation ratio indicating a ratio of the whole photographed image data is obtained for each of the divided areas. Occupancy rate calculation function to calculate,
A light distribution condition determining function for determining a light distribution condition at the time of photographing by performing an operation of multiplying the calculated occupation ratio of each region by a coefficient set in advance according to the photographing condition;
A determination function for determining a processing condition for determining a person's face from the photographed image data in accordance with the determined light distribution condition;
A face discrimination function for performing a process of discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing program for realizing On the computer that performs image processing,
An acquisition function for acquiring image signals from captured image data;
Based on the acquired image signal, the photographed image data is divided into predetermined regions composed of a combination of the distance from the outer edge of the screen of the photographed image data and brightness, and the photographing is performed for each of the divided regions. An occupancy ratio calculation function for calculating an occupancy ratio indicating the ratio of the entire image data;
A light distribution condition determining function for determining a light distribution condition at the time of photographing by performing an operation of multiplying the calculated occupation ratio of each region by a coefficient set in advance according to the photographing condition;
A determination function for determining a processing condition for determining a person's face from the photographed image data in accordance with the determined light distribution condition;
A face discrimination function for performing a process of discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing program for realizing On the computer that performs image processing,
An acquisition function for acquiring image signals from captured image data;
Based on the acquired image signal, the photographed image data is divided into regions composed of a combination of a predetermined brightness and hue, and a first ratio indicating a ratio of the entire photographed image data for each divided region. The occupancy ratio is calculated, and the captured image data is divided into a predetermined area composed of a combination of the distance from the outer edge of the screen of the captured image data and the brightness, and for each of the divided areas, the entire captured image data is divided. An occupancy ratio calculating function for calculating a second occupancy ratio indicating the occupying ratio;
A light distribution condition determining function for determining the light distribution condition at the time of photographing by performing an operation of multiplying the calculated first occupancy and second occupancy by a coefficient set in advance according to the photographing condition. When,
A determination function for determining a processing condition for determining a person's face from the photographed image data in accordance with the determined light distribution condition;
A face discrimination function for performing a process of discriminating a person's face from the captured image data in accordance with the determined processing conditions;
An image processing program for realizing When realizing the light distribution condition determination function, by performing the calculation, an index for specifying the light distribution condition is calculated, the light distribution condition is determined based on the calculated index,
24. When realizing the determination function, switching and / or adjustment of processing conditions of face discrimination processing is performed according to the calculated index value. The image processing program described.   The image processing function for performing predetermined image processing on the captured image data according to a determination result by the face determination function and / or a light distribution condition is realized. The image processing program described. When realizing the determination function, in accordance with the determined light distribution condition, determine an extraction condition for extracting a human face candidate region from the captured image data,
When realizing the face discrimination function, a human face candidate region is extracted from the captured image data in accordance with the determined face candidate region extraction condition, and face discrimination processing is performed on the extracted face candidate region. The image processing program according to any one of claims 21 to 25, wherein the image processing program is executed. When realizing the determination function, in accordance with the determined light distribution condition, to determine a feature amount extraction condition to be extracted from the face candidate region,
When realizing the face discrimination function, a feature amount is extracted from the face candidate region according to the determined feature amount extraction condition, and face discrimination processing is performed based on the extracted feature amount. The image processing program according to claim 26.   28. The image processing program according to claim 27, wherein when the determination function is realized, a feature amount extracted from the face candidate region is determined according to the determined light distribution condition. When realizing the determination function, a face determination processing condition for determining whether or not the face candidate region is a person's face is determined based on the feature amount according to the determined light distribution condition. And
The image processing program according to claim 27 or 28, wherein when the face discrimination function is realized, face discrimination processing is performed according to the determined face determination processing conditions.   The image processing program according to any one of claims 21 to 29, wherein a face discrimination process is performed using a neural network when the face discrimination function is realized.

Images