JP2008040578A - Image processor, printer, image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor for quickly and accurately executing processing relating to a red eye phenomenon. <P>SOLUTION: This image processor is provided with: an acquisition means (image acquisition part 70c) for acquiring image data as the object of processing; a red eye processing means (read eye detection processing part 70g, red eye reduction processing part 70h) for executing processing for reducing a red eye to image data acquired by the acquisition means; a discrimination means (discrimination part 70e) for discriminating the type of the primary object of the image data acquired by the acquisition means; and a stop means (control part 70b) for stopping the processing of the red eye processing means when it is discriminated that the primary object of the image data is any thing other than a person by the discrimination means. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, a printing apparatus, an image processing method, and an image processing program.

  Conventionally, a so-called red-eye phenomenon is known in which when a person is photographed with a strobe light in a dark environment, the photographed person's eyes turn red. In the red-eye phenomenon, when the strobe emits light in a dark environment with the pupil open, the light reflected by the retina is incident on the imaging device through the open pupil, and the capillaries existing in the retina are imaged. This is a phenomenon that occurs because the image is captured.

  As a method for correcting image data in which such a red-eye phenomenon has occurred, for example, as shown in Patent Document 1, an operator designates an area including red-eye and an inside of red-eye, and in an area including a designated red-eye, A technique has been proposed in which a boundary is obtained by comparing information on red color of matching pixels, and a correction process is performed by obtaining a red-eye region depending on whether the inside of the red-eye exists inside or outside the boundary.

  In recent years, there are many color printers on the market that can insert a recording medium such as a memory card into the main body and select and print recorded image data. Some of such color printers automatically detect a red-eye area from image data and perform a correction process when a red-eye is detected.

JP 2003-304555 A (abstract, claim)

  By the way, in the above-described color printer, processing for detecting whether or not the red-eye phenomenon has occurred is uniformly performed on all image data. In the red-eye detection process, for example, an eye contour is detected, and when the hue of a pixel included in the eye contour is within a predetermined range and the saturation is equal to or higher than a predetermined value, it is determined as red-eye.

  However, since such red-eye detection processing takes time, there is a problem that a waiting time from when an instruction to print predetermined image data is actually executed until printing is performed becomes long. In addition, even in the case of image data that does not include a person such as a landscape image, there is a problem that an area having a hue similar to that of red eyes is erroneously detected as red eyes and unintended correction is performed.

  The present invention has been made based on the above circumstances, and an object of the present invention is to provide an image processing device, a printing device, an image processing method, and an image processing device capable of quickly and accurately executing processing relating to the red-eye phenomenon. It is intended to provide a processing program.

  In order to achieve the above object, an image processing apparatus according to the present invention executes an acquisition unit that acquires image data to be processed, and a process for reducing red eyes on the image data acquired by the acquisition unit. A red-eye processing means, a discrimination means for discriminating the main subject type of the image data acquired by the acquisition means, and a red-eye processing means when the discrimination means determines that the main subject of the image data is other than a person. And stopping means for stopping the operation. For this reason, it is possible to provide an image processing apparatus capable of executing a process relating to the red-eye phenomenon quickly and accurately.

  In addition to the above-described invention, the image processing apparatus of another invention further includes face recognition means for determining whether or not a human face is included in the image data by face recognition processing. The type of main subject of the image data is determined based on the recognition result of the face recognition unit, and the stop unit determines the operation of the red-eye processing unit when the determination unit determines that the main subject of the image data is other than a person. To stop. Therefore, when the human face is not included in the image data, the processing speed can be improved by omitting the red-eye processing (red-eye detection processing and red-eye reduction processing).

  In addition to the above-described invention, in the image processing apparatus according to another aspect of the invention, the red-eye processing unit performs a process for reducing red eyes by referring to the face area of the person detected by the face recognition processing unit. I am doing so. For this reason, it is possible to increase the processing speed and prevent the occurrence of erroneous detection as compared with the case where the red-eye region is searched from the entire image data.

  In addition to the above-described invention, in the image processing apparatus of another invention, the determining unit determines the type of main subject of the image data based on information input from the user, and the stopping unit is determined by the determining unit from the user. When it is determined that the main subject is other than a person based on the input information, the operation of the red-eye processing unit is stopped. For this reason, when the user designates that the main subject of the image data is other than a person, the processing speed can be improved and the occurrence of erroneous detection can be prevented by omitting the red-eye process.

  The printing apparatus of the present invention includes the above-described image processing apparatus. For this reason, it is possible to provide a printing apparatus capable of executing processing relating to the red-eye phenomenon quickly and accurately.

  The image processing method of the present invention acquires image data to be processed, determines the type of main subject of the acquired image data, and determines that the main subject of the image data is other than a person The red-eye reduction process is executed, and if it is determined that the person is not a person, the red-eye reduction process is not executed on the image data. For this reason, the image processing method which can perform the process regarding a red eye phenomenon rapidly and correctly can be provided.

  The image processing program according to the present invention also includes an acquisition unit that acquires image data to be processed, a red-eye reduction processing unit that executes red-eye reduction processing on the image data acquired by the acquisition unit, and an acquisition unit. Determining means for determining the type of the main subject of the image data obtained by the step, and when the determining means determines that the main subject of the image data is other than a person, a stopping means for stopping the operation of the red-eye reduction processing means, To function as. For this reason, it is possible to provide an image processing program capable of executing a process relating to the red-eye phenomenon quickly and accurately.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration example of a printing apparatus 11 including an image processing apparatus according to an embodiment of the present invention. The image processing apparatus of the present invention will be described by taking the printing apparatus 11 as an example. The image processing method of the present invention will be described by taking the operation of the printing apparatus 11 as an example. Furthermore, the image processing program of the present invention will be described using a program executed by the printing apparatus 11 as an example.

  A printing apparatus 11 shown in FIG. 1 is a so-called composite printing apparatus in which a scanner device, a printing device, and a copying device are integrated. The printing apparatus 11 includes a CPU (Central Processing Unit) 50, a ROM (Read Only Memory) 51, a RAM (Random Access Memory) 52, an EEPROM (Electrically Erasable and Programmable ROM) 53, a GP (Graphic Processor) 54, an I / F (I / F). Interface) 55, bus 56, LCD 15, operation button 16, card slot 18 into which the memory card M is inserted, card I / F circuit 60, printer engine controller 62, paper feed motor 63, roller 64, carriage motor 65, drive belt 66, a carriage 67, and a recording head 68.

  Here, the CPU 50 controls each unit. The ROM 51 stores various programs executed by the CPU 50. The RAM 52 temporarily stores programs and data. The EEPROM 53 is a ROM that can rewrite data. The GP 54 performs a drawing process and supplies the obtained image to the LCD 15 to display it.

  The I / F 55 appropriately converts the data expression format. The bus 56 is a signal line group for enabling exchange of information. The operation button 16 generates information corresponding to a user operation. The memory card M is a recording medium that stores image data as print data. The card slot 18 is provided in a part of a case (not shown) of the printing apparatus 11, and the memory card M is inserted into this part. The card I / F circuit 60 is an interface for reading / writing information from / to the memory card M.

  The printer engine controller 62 is a control device for controlling the paper feed motor 63, the carriage motor 65, and the recording head 68. The paper feed motor 63 moves the printing paper in the sub-scanning direction by rotating the roller 64. The roller 64 is configured by a cylindrical member, and moves the printing paper in the sub-scanning direction. The carriage motor 65 reciprocates the carriage 67 in the main scanning direction by applying a driving force to a driving belt 66 whose one end is fixed to the carriage 67. The recording head 68 has a plurality of nozzles formed on the surface facing the printing paper, and records information on the printing paper by discharging color ink from the plurality of nozzles.

  FIG. 2 shows functional blocks for performing correction processing including red-eye reduction processing when printing predetermined image data stored in the memory card M by the printing apparatus 11 shown in FIG. The functional block 70 shown in this figure is realized by the cooperation of software as a program stored in the ROM 51 or the EEPROM 53 shown in FIG. 1 and hardware such as the CPU 50.

  The input unit 70a illustrated in FIG. 2 inputs information output from the operation button 16. The control unit 70b serving as a stopping unit controls each unit based on information input from the input unit 70a, and based on the recognition result of the face recognition processing unit 70d, the red-eye detection processing unit 70g and the red-eye reduction processing unit 70h. Stop operation. The image acquisition unit 70c as an acquisition unit acquires a predetermined image from the memory card M. The face recognition processing unit 70d as a face recognition processing unit performs face recognition processing on the acquired image data, and specifies a person's face. The determination unit 70e serving as a determination unit specifies the main subject type of the image data based on the result of the face recognition process. The correction amount calculation processing unit 70f calculates a correction amount that makes the color of each subject appropriate according to the type of the main subject specified by the determination unit 70e. A red-eye detection processing unit 70g as a part of the red-eye processing unit detects a red-eye region of a person in the image data. The red-eye reduction processing unit 70h as a part of the red-eye processing means performs processing for reducing red-eye in the image data. The correction processing unit 70i performs correction processing based on the correction amount calculated by the correction amount calculation processing unit 70f.

  Next, the operation of the embodiment of the present invention will be described. FIG. 3 is a flowchart showing an example of processing executed in the embodiment shown in FIG. When the flowchart of this figure is started, the following steps are executed.

  Step S10: When the user operates the operation button 16 to select predetermined image data stored in the memory card M, the control unit 70b detects this and detects it in the image acquisition unit 70c. Notice.

  Step S11: The image acquisition unit 70c acquires the image data corresponding to the notification from the control unit 70b from the memory card M.

  Step S12: The face recognition processing unit 70d performs face recognition processing on the acquired image data. Here, as the face recognition processing, for example, a plurality of face templates with different human orientations (for example, front, left, right, top, and bottom directions) and sizes are prepared, and each template and image data are prepared. A matching process with a part of the area is performed, and it is determined that a face exists in a part with a high matching rate. Then, information indicating the position and range in the detected face image data is acquired and supplied to the control unit 70b. Instead of using a template, for example, feature parts such as eyes, nose and mouth may be extracted and the face may be recognized from these positional relationships.

  Step S13: The determination unit 70e determines the scene (the main subject type of the image data) based on the processing result of the face recognition processing unit 70d. That is, the determination unit 70e determines whether the main subject of the image data is a person, a landscape, or a night view. Whether or not the person is a person is determined from the processing result of the face recognition processing unit 70d, and the scenery and the night scene are determined based on the distribution of hues of data generated by sampling image data at a certain rate.

  Step S14: The correction amount calculation processing unit 70f calculates the correction amount of the image data based on the determination result by the determination unit 70e. Specifically, when the scene is a person, the correction amount is calculated so that the skin color of the person falls within the optimum range. When the scene is a landscape, the correction amount is calculated so that the green color of the plant and the blue color of the sky or the sea fall within the optimum range. Further, when the scene is a night view, the amount of correction for color cast and contrast is calculated so that the night view becomes vivid.

  Step S15: Based on the determination result of the determination unit 70e, the control unit 70b proceeds to step S16 if the main subject of the image data is a person, and proceeds to step S19 otherwise. When the main subject type is other than a person, the control unit 70b stops the operations of the red-eye detection processing unit 70g and the red-eye reduction processing unit 70h, and thus these processes for the image data are not executed.

  Step S16: The red-eye detection processing unit 70g performs a red-eye detection process on the image data determined that the main subject is a person. For example, the red-eye detection processing unit 70g refers to the face position and region detected by the face recognition processing unit 70d, and detects a pupil located inside the face region. Specifically, since the luminance distribution around the pupil has a mountain shape as shown in FIG. 4, a portion where the luminance distribution is distributed in a mountain shape is specified. Then, it is determined whether the hue of the specified area belongs to the hue range corresponding to the red eye. If the hue belongs to the hue corresponding to the red eye, the area is set as the red eye area. In addition, you may make it detect red eyes by methods other than this.

  Step S17: The red-eye reduction processing unit 70h proceeds to step S18 when red-eye is detected by the red-eye detection processing unit 70g, and proceeds to step S19 in other cases.

  Step S18: The red-eye reduction processing unit 70h performs a red-eye reduction process on the red-eye area detected by the red-eye detection processing unit 70g. Specifically, the pixel data included in the red-eye area detected by the red-eye detection processing unit 70g is replaced with pixel data corresponding to the normal pupil color. The red eye may be corrected by other methods.

  Step S19: The correction processing unit 70i corrects the correction amount calculation processing unit 70f with respect to the image data on which the red-eye reduction processing is executed when the red-eye reduction processing is executed, and the original image data when the red-eye reduction processing is not executed. The correction process is performed based on the correction amount calculated by. As the correction processing, for example, when the main subject is a person, the hue of the image data in the corresponding region is converted so that the skin color of the person falls within the optimum range. In the case of a landscape, the hue of the corresponding area of the image data is converted so that green and blue are within the optimum range. Further, in the case of a night scene, if a color cast has occurred, it is corrected and a process for adjusting the contrast is performed. Then, the corrected image data is output to the printer engine controller 62, and the process ends.

  The printer engine controller 62 converts the image data for which the correction processing has been completed into CMYK color system image data, and then performs halftone processing, resolution conversion processing, and the like to obtain printing raster data. Then, the printer engine controller 62 drives the recording head 68 based on the printing raster data, discharges each color ink stored in the carriage 67 to the printing paper, and prints a color image corresponding to the image data. Print on paper.

  As described above, according to the printing apparatus 11 of the embodiment of the present invention, the red-eye detection process and the red-eye reduction process are executed only when the main subject is determined to be a person by the face recognition process, and otherwise Since these operations are stopped, for example, the processing time for these image data can be shortened by omitting the red-eye process for the image data whose main subject is a landscape or a night view. In the case of the printing apparatus 11, the waiting time from when printing is instructed to when printing is actually executed can be shortened. Furthermore, it is possible to prevent an unintended correction from being performed due to erroneous detection, for example, processing is performed on a portion of a landscape or night view that is not a red eye.

  In the above embodiment, since the red-eye detection process is performed on the face area detected by the face recognition process, the processing speed can be improved by narrowing the detection range. Further, by limiting the range, it is possible to prevent erroneous detection.

  Next, a printing apparatus according to a second embodiment of the present invention will be described. In the second embodiment, the configuration of the apparatus is the same as that shown in FIGS. 1 and 2, but part of the processing executed there is different. FIG. 5 shows processing executed in the second embodiment. In this process, parts corresponding to those in FIG. In the process of FIG. 5, the processes of steps S20 to S24 are newly added, and the other processes are the same as those of FIG. Below, it demonstrates focusing on a different part.

  Step S20: When the acquisition of the image data is completed in step S11, the control unit 70b displays a print setting menu on the LCD 15 and receives input of information from the user. Specifically, the control unit 70b displays a menu having a hierarchical structure as shown in FIG. 6 and receives selection of a predetermined item. In the example of this figure, M2 that does not execute automatic image quality correction and M3 that executes automatic image quality correction exist below the menu item automatic image quality correction M1. In addition, a standard M4, a person M5, a landscape M6, and a night view M7 exist below the M3 to be executed.

  Here, M2 not to be executed is selected when automatic correction processing for image data is not executed. M3 to be executed is selected when executing an automatic correction process for image data. The standard M4 is selected when the automatic correction process is executed and the standard correction process is executed. Since standard M4 is set as default, standard M4 is automatically selected when person M5, landscape M6, night view M7, etc. are not selected.

  The person M5 is selected when the main subject of the image data is a person, and correction processing is executed so that the skin color of the person is optimized. The landscape M6 is selected when the main subject of the image data is a landscape, and correction processing is executed so that the green color of a plant or a tree or the blue color of the sky or the sea is optimized. The night scene M7 is selected when the main subject of the image data is a night scene. For example, correction processing is executed so that a bright night scene is obtained by eliminating color cast or enhancing contrast. .

  Step S21: The control unit 70b acquires information corresponding to the selected menu item, determines whether or not to execute M2, landscape M6, or night view M7, and any of these is selected. If so, the process proceeds to step S12. Otherwise, the process proceeds to step S19. That is, when the landscape M6 and the night view M7 are selected, the main subject is a person other than a person, so that the face recognition process and the red-eye process need not be executed. Further, since face recognition processing and red-eye processing need not be executed even when M2 not to be executed is selected, face recognition processing and red-eye processing (red-eye detection processing and red-eye reduction processing) are skipped (operation is stopped). .

  Step S22: When a landscape or a night view is selected in step S20, the determination unit 70e refers to the information selected in step S20 to determine which of these is the main subject.

  Step S23: When a landscape or night view is selected in step S20, the correction amount calculation processing unit 70f obtains correction amounts for these image data. Then, the process proceeds to the correction process in step S19.

  If it is determined as N in step S21, face recognition processing is executed, and based on this, scene discrimination (step S13) and correction amount calculation processing (step S14) are executed. When the person M5 is selected, the scene determination 13 may be omitted.

  By the above processing, when M2, which is not executed, landscape M6, and night view M7 are selected in the menu shown in FIG. 6, it is a display indicating that no correction processing is necessary for M2 that is not executed, With regard to M6 and night view M7, it is considered that there is no person in the image data. Therefore, the processing can be speeded up by skipping the face recognition processing and red-eye processing (red-eye detection processing and red-eye reduction processing).

  Step S24: The control unit 70b refers to the size of the face area of the person specified by the face recognition processing unit 70d and the occurrence of the red-eye phenomenon when the distance between the imaging device and the person is longer than a predetermined distance. The process proceeds to step S16, and in other cases, the process proceeds to step S19 because the possibility of occurrence of the red-eye phenomenon is low.

  Here, the reason for determining the possibility of the occurrence of the red-eye phenomenon according to the distance to the subject is as follows. That is, as shown in FIG. 7A, when the digital camera 100 and the eyeball 110 of the subject are separated from each other to some extent, the light generated from the strobe 101 passes through the iris 111 of the eyeball 110 and is transmitted by the retina 112. The light is reflected, passes through the iris 111 again, and enters the lens 102 of the digital camera 100. For this reason, a red-eye phenomenon occurs in the captured image data.

  On the other hand, as shown in FIG. 7B, when the distance between the digital camera 100 and the eyeball of the subject is short, the light generated from the strobe 101 is blocked by the iris 111 and enters the lens 102 of the digital camera 100. Is no longer incident, so the red-eye phenomenon does not occur. In general, when the distance between the subject and the digital camera is less than 2 m, the probability that the red-eye phenomenon will occur is low. Therefore, in the present embodiment, the distance to the subject is estimated from the size of the person's face area, and when the estimated distance is less than 2 m, it is determined that the possibility of the red-eye phenomenon is low. Specifically, when the diameter of the face region is more than half of the length in the vertical direction of the image, it is determined that the possibility of occurrence of the red-eye phenomenon is low. When zoom is used, it may not correspond to the actual distance between the subject and the digital camera 100, but it is considered that zoom is used less frequently in portraits. Is estimated to be low. However, in the case of a user who frequently uses zoom, it may be possible to set so as not to execute the process of step S24.

  Therefore, when the distance to the subject is short (when the size of the face area in the image data is large) by interposing the determination process in step S24, the red-eye detection process is performed because the possibility of the occurrence of the red-eye phenomenon is low. Skip and speed up processing.

  The above embodiment is merely an example, and there are various other modified embodiments. For example, in the above embodiment, the memory card M is inserted into the card slot 18 of the main body and the image data is read out. For example, a digital camera (not shown) is connected to an I / F 55 or the like by a cable (not shown). The image data may be read out via the cable.

  In the above embodiment, the description has been given by taking the composite type printing apparatus as an example, but the present invention is applied to a normal printing apparatus (printing apparatus of a type connected to a personal computer). be able to. In addition, the present invention can be applied to a normal stand-alone printing apparatus that is not a composite type.

  In the above embodiment, the processing shown in FIGS. 3 and 5 is executed in the printing apparatus 11. However, for example, it can be executed in a host computer connected to the printing apparatus 11.

  The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the image processing apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory. Examples of the magnetic recording device include a hard disk device (HDD), a flexible disk (FD), and a magnetic tape. Examples of the optical disc include a DVD (Digital Versatile Disk), a DVD-RAM, a CD-ROM (Compact Disk ROM), and a CD-R (Recordable) / RW (ReWritable). Magneto-optical recording media include MO (Magneto-Optical disk).

  When distributing the program, for example, portable recording media such as a DVD and a CD-ROM in which the program is recorded are sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. In addition, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

It is a figure which shows the structural example of the printing apparatus which concerns on embodiment of this invention. It is a figure which shows the functional block implement | achieved in the printing apparatus shown in FIG. 3 is a flowchart executed in the printing apparatus shown in FIG. 1. It is a figure which shows the luminance distribution of the image data of a pupil. It is a flowchart performed in the 2nd Embodiment of this invention. It is a figure which shows an example of the menu displayed on LCD. It is a figure which shows the relationship between a camera and a to-be-photographed object.

Explanation of symbols

  11 Printing device, 70b Control unit (stopping unit), 70c Image acquisition unit (acquisition unit), 70d Face recognition processing unit (face recognition processing unit), 70e Discrimination unit (discrimination unit), 70g Red-eye detection processing unit (red-eye processing unit) Part of), 70h Red-eye reduction processing part (part of red-eye processing part)

Claims (7)

An acquisition means for acquiring image data to be processed;
Red-eye processing means for executing processing for reducing red-eye on the image data acquired by the acquisition means;
Discriminating means for discriminating the type of main subject of the image data acquired by the acquiring means;
Stopping means for stopping the operation of the red-eye processing means when the determining means determines that the main subject of the image data is other than a person;
An image processing apparatus comprising: Face recognition processing means for determining by face recognition processing whether or not a human face is included in the image data;
The determination means determines the main subject type of the image data based on the recognition result of the face recognition processing means,
The image processing apparatus according to claim 1, wherein the stopping unit stops the operation of the red-eye processing unit when the determining unit determines that the main subject of the image data is other than a person. .   The image processing apparatus according to claim 2, wherein the red-eye processing unit executes processing for reducing red eyes with reference to a human face area detected by the face recognition processing unit. The discriminating unit discriminates the type of main subject of the image data based on information input from the user,
2. The image according to claim 1, wherein the stopping unit stops the operation of the red-eye processing unit when the determining unit determines that the main subject is other than a person based on information input from a user. Processing equipment.   A printing apparatus comprising the image processing apparatus according to claim 1. Obtain image data to be processed,
Identify the main subject type of the acquired image data,
When it is determined that the main subject of the image data is a person, a process for reducing red eyes is executed. When it is determined that the object is not a person, red eyes are reduced for the image data. Do not execute the process of
An image processing method. Computer
Acquisition means for acquiring image data to be processed;
Red-eye processing means for executing processing for reducing red-eye on the image data acquired by the acquisition means;
Discrimination means for discriminating the type of main subject of the image data acquired by the acquisition means;
Stopping means for stopping the operation of the red-eye processing means when the determining means determines that the main subject of the image data is other than a person;
A computer-readable image processing program characterized in that it functions as a computer program.

Images