JP2005197809A - Image processing apparatus, imaging device, and control method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for accurately detecting the occurrence of a red eye and the position and size of the red eye on the basis of image data during light projection depending on the state of eyes changing with time during the light projection. <P>SOLUTION: While a projector 49 projects light for red-eye relaxation, an imaging element 14 executes photography by each prescribed time, and two newest imaged image data are stored and conserved in an image display memory 24 and a preceding image display memory 27. BY comparing R components of the two stored imaged image data, the position and the size of the red-eye are detected. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for suppressing a red-eye phenomenon at the time of photographing using a strobe in an apparatus such as an electronic still camera.

  As a technique for alleviating the red-eye phenomenon that occurs during shooting using a conventional strobe (flash), there is a technique in which flash is preliminarily irradiated before shooting to reduce the opening of a human pupil (Patent Document 1). This technique is intended to reduce the amount of reflected flash by the retina by reducing the pupil opening before imaging.

Further, in an electronic still camera, image data obtained by flash photography is compared with image data obtained by non-flash photography, and strobe light is reflected from the retina of the eye from image data obtained by flash photography. There is also a technique for detecting the red-eye location and size and correcting the image data (Patent Document 2).
Japanese Patent Publication No.58-48088 (page 3, Fig. 1 and Fig. 2) Japanese Patent No. 3114103 (6th page, FIG. 1)

  In Patent Document 1 having such a conventional red-eye mitigating means, the red-eye mitigating means permits photographing, assuming that a red-eye mitigating effect is obtained after a predetermined light emission. However, it is not known until the photo is taken whether the red-eye reduction effect is obtained.

  Further, in Patent Document 2, the image data obtained by flash photography and the image data obtained by non-flash photography are compared, and the red-eye location caused by the reflection of the strobe light from the retina of the eye from the image data obtained by flash photography. However, the shooting conditions of the two image data (one is non-flash emission and the other is flash emission) do not match. Further, the image at the time of flash emission is like an image 111 in FIG. 11, and the non-flash image after that becomes an image 113 in FIG. Since the target for red-eye correction is the image 111 at the time of flash emission, that is, an image with a large pupil, the correction result is likely to be unnatural as the correction target area is large as in the image 112. .

  Also, since the image when the flash is not emitted is not sufficiently bright, the position and size of the red eye cannot be accurately detected.

  The present invention has been made in view of such problems, and one is to provide a technique for accurately detecting the presence / absence of red-eye occurrence and the position and size of red-eye. In addition, another invention is intended to provide a technique for reflecting the detection result and performing natural red-eye correction while taking an image while relaxing the red eye.

In order to solve this problem, the image processing apparatus of the present invention has the following configuration. That is,
An image processing apparatus having an imaging unit and a light projecting unit,
A first image acquisition means for acquiring first captured image data by the imaging means during light projection by the light projecting means;
Second image acquisition in which second imaging image data is acquired by the imaging means during light projection by the light projecting means after a predetermined time after the first image acquisition means has acquired the first captured image data. Means,
Comparison means for comparing the first and second captured image data acquired by the first and second image acquisition means;
Detection means for detecting the position of the red eye and the size of the red eye based on the comparison result of the comparison means.

  According to the present invention, it is possible to accurately detect the occurrence, position, and size of red eyes based on image data being projected from the state of the eyes that changes over time during the projection.

  Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

<Description of device configuration>
FIG. 1 is a block diagram of a digital still camera according to the embodiment. In the figure, reference numeral 100 denotes an apparatus main body having the following configuration.

  10 is a photographing lens, 12 is a shutter having a diaphragm function, 14 is an image sensor (CCD or the like) that converts an optical image into an electrical signal, and 16 is an A / D converter that converts an analog signal output of the image sensor 14 into a digital signal. It is. A timing generation circuit 18 supplies a clock signal and various control signals to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50.

  An image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22. Further, the image processing circuit 20 performs predetermined calculation processing using the captured image data, and the system control circuit 50 controls the exposure control means 40 and the distance measurement control means 42 based on the obtained calculation result. TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-emission) processing are performed.

  Further, the image processing circuit 20 performs predetermined arithmetic processing using captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained arithmetic result.

  A memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32.

  The data of the A / D converter 16 is written into the image display memory 24 or the memory 30 via the image processing circuit 20 and the memory control circuit 22 or the data of the A / D converter 16 is directly passed through the memory control circuit 22. It is.

  Reference numeral 24 denotes an image display memory, 26 denotes a D / A converter, and 28 denotes an image display unit made up of a TFT type LCD or the like. Display image data written in the image display memory 24 is stored in the D / A converter 26. Via the image display unit 28. The display data written in the image display memory 24 does not write the data of all the pixels in the imaging area of the image sensor 14, but has a lower image quality than the image quality obtained by thinning out the image data of the image sensor 14. Image data.

  If the image data captured using the image display unit 28 is sequentially displayed, the electronic viewfinder function can be realized. Further, the image display unit 28 is provided with a backlight (not shown), and the display and backlight can be arbitrarily turned on / off according to an instruction from the system control circuit 50. When the display and backlight are turned off Thus, the power consumption of the apparatus 100 can be reduced.

  Reference numeral 27 denotes a previous image display memory which stores image data after the display in the image display memory 24 is finished. From the image data in the image display memory 24 and the previous image display memory 27, the presence / absence of the red eye and the size of the red eye in the image are detected (details will be described later).

  Reference numeral 30 denotes a memory for storing captured still images and moving images, and has a sufficient storage capacity to store a predetermined number of still images and a predetermined time of moving images. Thereby, even in the case of continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, it is possible to write a large amount of images to the memory 30 at high speed. The memory 30 can also be used as a work area for the system control circuit 50.

  A compression / decompression circuit 32 compresses and decompresses image data by adaptive discrete cosine transform (ADCT) or the like, reads an image stored in the memory 30, performs compression processing or decompression processing, and stores the processed data in the memory 30. Write to.

  Reference numeral 40 denotes an exposure control means for controlling the shutter 12 having a diaphragm function, and has a flash light control function in cooperation with the flash 48.

  Reference numeral 42 denotes a distance measurement control unit that controls focusing of the photographing lens 10, reference numeral 44 denotes a zoom control unit that controls zooming of the photographing lens 10, and reference numeral 46 denotes a barrier control unit that controls the operation of the protection unit 102 that protects the lens 10. It is.

  A flash (strobe) 48 has an AF auxiliary light projecting function and a flash light control function.

  The exposure control unit 40 and the distance measurement control unit 42 are controlled using the TTL method. Based on the calculation result obtained by calculating the captured image data by the image processing circuit 20, the system control circuit 50 performs the exposure control unit 40 and the distance measurement. Control is performed on the control means 42.

  An LED or lamp projector 49 emits light when the subject brightness is darker than a predetermined brightness. This light projecting device is located closer to or at the same distance than the flash 48 with respect to the photographing lens 10, and is configured so that the pupil is narrowed by projecting light to relieve red eyes, and a red eye phenomenon is likely to occur in an image. ing.

  Reference numeral 50 denotes a system control circuit that controls the entire apparatus 100, and reference numeral 52 denotes a memory that stores constants, variables, programs, and the like for operation of the system control circuit 50.

  Reference numeral 54 denotes a display unit such as a liquid crystal display device or a speaker that displays an operation state or a message using characters, images, sounds, and the like in accordance with execution of a program in the system control circuit 50. One or a plurality of locations are installed at positions where the user can easily see the image, and is configured by a combination of, for example, an LCD, an LED, and a sound generating element. In addition, the display unit 54 is partially installed in the optical viewfinder 104.

  Among the display contents of the display unit 54, what is displayed on the LCD or the like includes a mark for distinguishing single shot / continuous shooting display, a mark indicating that the self-timer display is being performed, a compression rate display, and a recording pixel number display. , Number of recorded images, number of remaining shots displayed, shutter speed display, aperture value display, exposure compensation display, flash on / off mark, red eye reduction on / off, macro shooting on / off Mark, buzzer setting display, clock battery level, battery level, error, multi-digit information display, recording medium 200 and 210 attachment / detachment status display, communication I / F operation display, date / time display, Etc.

  Further, among the display contents of the display unit 54, what is displayed in the optical viewfinder 104 includes in-focus display, camera shake warning display, flash charge display, shutter speed display, aperture value display, exposure correction display, and the like.

  Reference numeral 56 denotes an electrically erasable / recordable nonvolatile memory such as an EEPROMM.

  Reference numerals 60, 62, 64, 65, 66, 68 and 70 are operation means for inputting various operation instructions of the system control circuit 50, such as switches, dials, touch panels, pointing by line-of-sight detection, voice recognition devices, etc. Consists of a single or a plurality of combinations. These operating means will be specifically described as follows.

  Reference numeral 60 denotes a mode dial switch, which can switch and set various function modes such as power-off, automatic shooting mode, shooting mode, panoramic shooting mode, playback mode, multi-screen playback / erase mode, and PC connection mode.

  Reference numeral 62 denotes a shutter switch SW1, which is turned on during the operation of a shutter button (not shown) (detects the first stage of the shutter button), AF (auto focus) processing, AE (automatic exposure) processing, AWB (auto white balance). Instructs the start of operations such as processing and EF (flash pre-emission) processing.

  A shutter switch SW2 64 is turned on when the operation of the shutter button is completed (second stage of the shutter button = full press). When the shutter switch SW2 is turned on, exposure processing for writing the signal read from the image sensor 12 to the memory 30 via the A / D converter 16 and the memory control circuit 22, the image processing circuit 20 and the memory control circuit 22 The development processing using the calculation in the above, the image data is read from the memory 30, compressed by the compression / decompression circuit 32, and the start of a series of processing operations such as recording processing for writing the image data to the recording medium 200 or 210 is instructed.

  A strobe mode switch 65 sets a strobe light emission mode such as forced light emission mode / auto mode light emission / red-eye reduction auto mode light emission / light emission off.

  Reference numeral 66 denotes an image display ON / OFF switch which can be set to display / not display the image display unit 28. With this function, when photographing is performed using the optical viewfinder 104, it is possible to save power by cutting off the current supply to the image display unit including a TFT LCD or the like.

  Reference numeral 68 denotes a quick review ON / OFF switch, which sets a quick review function for automatically reproducing image data taken immediately after photographing. In the present embodiment, in particular, a function for setting a quick review function when the image display unit 28 is turned off is provided.

  Reference numeral 70 denotes an operation unit composed of various buttons, a touch panel, etc., a menu button, a set button, a macro button, a multi-screen playback page break button, a single shooting / continuous shooting / self-timer switching button, a menu moving + (plus) button, a menu moving. -(Minus) button, replay image move + (plus) button, replay image-(minus) button, shooting quality selection button, exposure compensation button, enlargement button to set display enlargement during red-eye reduction, date / time setting button, etc. There is.

  Reference numeral 80 denotes a power supply control unit, which includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, etc., and detects the presence / absence of a battery, the type of battery, the remaining battery level, and the detection result In addition, the DC-DC converter is controlled based on an instruction from the system control circuit 50, and a necessary voltage is supplied to each part including the recording medium for a necessary period.

  Reference numeral 82 denotes a device side connector, 84 denotes a power source side unit side connector, and 86 denotes a power source including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, NiMH battery or Li battery, an AC adapter, or the like.

  90 and 94 are interfaces with a recording medium such as a memory card or a hard disk, 92 and 96 are connectors for connecting to a recording medium such as a memory card or a hard disk, and 98 is a recording medium 200 or 210 attached to the connector 92 or 96. Recording medium attachment / detachment detecting means for detecting whether or not the recording medium is present.

  In the present embodiment, it is assumed that there are two systems of interfaces and connectors for attaching a recording medium. Of course, the interface and the connector for attaching the recording medium may have a single or a plurality of systems and any number of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard.

  The interface and the connector may be configured using a PCMCIA (trademark) card, a CF (compact flash (registered trademark)) card, or the like.

  Further, when the interfaces 90 and 94 and the connectors 92 and 96 are configured using a PCMCIA (trademark) card, a CF (compact flash (registered trademark)) card, or the like, a LAN card, modem card, USB Management of image data and image data attached to peripheral devices such as other computers and printers by connecting various communication cards such as cards, IEEE1394 cards, P1284 cards, SCSI cards, PHS, etc. You can transfer information.

  Reference numeral 102 denotes protection means that is a barrier that prevents the imaging unit from being soiled or damaged by covering the imaging unit including the lens 10 of the apparatus 100.

  Reference numeral 104 denotes an optical viewfinder, which can take an image using only the optical viewfinder without using the electronic viewfinder function of the image display unit 28. In the optical viewfinder 104, some functions of the display unit 54, for example, a focus display, a camera shake warning display, a flash charge display, a shutter speed display, an aperture value display, an exposure correction display, and the like are installed.

  A communication unit 110 has various communication functions such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication.

  Reference numeral 112 denotes a connector for connecting the apparatus 100 to other devices by the communication unit 110 or an antenna in the case of wireless communication, which is mounted on the surface of the apparatus 100.

  Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 composed of a semiconductor memory, a magnetic disk, and the like, an interface 204 with the apparatus 100, and a connector 206 for connecting with the apparatus 100. Reference numeral 210 denotes a recording medium such as a memory card or a hard disk. The recording medium 210 includes a recording unit 212 composed of a semiconductor memory, a magnetic disk, and the like, an interface 214 with the apparatus 100, and a connector 216 that connects to the apparatus 100.

  FIG. 10 shows the appearance of the apparatus (camera) in the embodiment. FIG. 1A is a front view of the camera, 301 is a camera body, 302 is an optical viewfinder, 303 is a lens barrel that includes a photographing lens for focusing and zooming, and 304 projects a lamp or LED. Thus, a red-eye projector 305 for detecting the red-eye position and reducing the red-eye, 305 is a flash light emitting unit.

  FIG. 5B is a view as seen from the rear of the camera. 306 is a button that is the above-described image display ON / OFF switch 66, 307 is a quick view button that is the above-mentioned quick view ON / OFF switch 68, and 308 is the above-mentioned. This is a mode dial showing the mode dial switch 60. Reference numeral 309 denotes an external LCD in the image display unit 28, and 310 denotes a strobe mode button which is a strobe mode SW65.

<Description of operation>
Hereinafter, the operation (processing) of the apparatus 100 according to the embodiment will be described with reference to FIGS.

  2 and 3 show a flowchart of a main routine of the apparatus 100 in the present embodiment. Hereinafter, the operation of the apparatus 100 will be described with reference to FIGS. 2 and 3.

  When power is turned on such as battery replacement, the system control circuit 50 initializes a flag, a control variable, and the like (step S101), and initializes the image display of the image display unit 28 to an OFF state (step S102).

  The system control circuit 50 determines the set position of the mode dial 60, and if the mode dial 60 is set to power OFF (step S103), the display of each display unit is changed to the end state, and the protection unit 102 The barrier is closed to protect the imaging unit, and necessary parameters, setting values, and setting modes including flags and control variables are recorded in the nonvolatile memory 56, and the power control unit 80 controls each part of the apparatus 100 including the image display unit 28. After performing a predetermined end process such as shutting off unnecessary power (step S105), the process returns to step S103.

  If it is determined in step S103 that the mode dial 60 is turned on and set to the shooting mode, the process proceeds to step S106. If it is determined that the mode dial 60 is set to another mode (for example, image browsing after being captured and stored, image selection / deletion, etc.), the system control circuit 50 responds to the selected mode. The process is executed (step S104), and when the process is finished, the process returns to step S103.

  When the process proceeds to step S106, the system control circuit 50 determines whether or not the remaining capacity or operation status of the power source 86 constituted by a battery or the like has a problem in the operation of the apparatus 100 by the power source control unit 80. If there is, a predetermined warning is displayed by image or sound using the display unit 54 (step S108), and the process returns to step S103.

  If it is determined that there is no problem with the power supply 86, the system control circuit 50 determines whether the operation state of the recording medium 200 or 210 has a problem with the operation of the apparatus 100, particularly with respect to the recording / reproducing operation of the image data with respect to the recording medium. (Step S107) If there is a problem, a predetermined warning is displayed by an image or sound using the display unit 54 (Step S108), and the process returns to Step S103.

  When it is determined that there is no problem in the operation state of the recording medium 200 or 210, the process proceeds to step S109, and the display unit 54 displays various setting states of the apparatus 100 using images and sounds. If the image display of the image display unit 28 is ON, the image display unit 28 is also used to display various setting states of the apparatus 100 using images and sounds.

  When the process proceeds to step S110, the system control circuit 50 checks the setting state of the quick review ON / OFF switch 68, and sets the quick review flag if the quick review ON is set (step S111). If it is set to OFF, the quick review flag is canceled (step S112). The state of the quick review flag is stored in the internal memory or the memory 52 of the system control circuit 50.

  Subsequently, the system control circuit 50 determines the setting state of the image display ON / OFF switch 66 (step S113). If the image display is set to ON, the image display flag is set (step S114), the image display of the image display unit 28 is set to the ON state (step S115), and the captured image data is sequentially displayed. The through display state is set (step S116), and the process proceeds to step S119.

  In the through display state, the data sequentially written in the image display memory 24 via the image sensor 12, the A / D converter 16, the image processing circuit 20, and the memory control circuit 22 are converted into the memory control circuit 22 and the D The electronic finder function is realized by sequentially displaying images on the image display unit 28 via the / A converter 26.

  On the other hand, if it is determined in step S113 that the image display ON / OFF switch 66 is set to image display OFF, the process proceeds to step S117 to cancel the image display flag and in step S118, the image display unit. The image display of 28 is set to the OFF state, and the process proceeds to step S119.

  When the image display is OFF, shooting is performed using the optical viewfinder 104 without using the electronic viewfinder function of the image display unit 28. In this case, it is possible to reduce power consumption of the image display unit 28, the D / A converter 26, and the like that consume a large amount of power. The state of the image display flag is stored in the internal memory of the system control circuit 50 or the memory 52.

  Now, when the process proceeds to step S119 (FIG. 3), it is determined whether or not the shutter switch SW1 is ON. That is, it is determined whether or not the shutter switch is half pressed. If it is determined that the button has not been pressed, the process returns to step S103 and the above process is repeated.

  If it is determined that the shutter switch SW1 has been turned on, the process proceeds from step S119 to step S120. The system control circuit 50 determines the state of the image display flag stored in the internal memory or the memory 52 (step S120). If the image display flag is set, the display state of the image display unit 28 is set to the freeze display state. (Step S121), the process proceeds to step S122.

  In the freeze display state, rewriting of the image data in the image display memory 24 via the image sensor 12, the A / D converter 16, the image processing circuit 20, and the memory control circuit 22 is prohibited, and the last written image data Is displayed by the image display unit 28 via the memory control circuit 22 and the D / A converter 26, so that it is displayed on the electronic viewfinder as a frozen image. If the image display flag has been canceled, the process of step S121 is skipped and the process proceeds to step S122.

  In step S122, the system control circuit 50 performs a distance measurement process to adjust the focal length of the photographing lens 10 to the subject, and also performs a light measurement process to determine an aperture value and a shutter opening time (shutter speed). In the photometric process, if the flash is set to auto, whether or not to perform flash is also set. Details of the distance measurement / photometry processing in step S122 will be described later with reference to FIG.

  If the photometry / ranging process is completed, the process proceeds to step S123, and the red-eye process is performed according to the flash mode setting state and the result of the photometry. Details of the red-eye process in step S123 will be described later with reference to FIG.

  When the red-eye process (step S123) is completed, the system control circuit 50 determines the state of the image display flag stored in the internal memory of the system control circuit 50 or the memory 52 (step S124), and the image display flag is set. If so, the display state of the image display unit 28 is set to the through display state (step S125), and the process proceeds to step S126. The through display state in step S125 is the same operation state as the through state in step S116 described above.

  Subsequently, in step S126, it is determined whether or not the shutter switch SW2 is turned on, that is, whether or not the shutter button is fully pressed. If not, it is determined in step S127 whether or not the shutter switch SW1 is also released. If the shutter SW1 is also released (OFF), the process returns to step S103.

  If it is determined that the shutter switch SW2 has been turned ON, the process proceeds to step S128, and it is determined whether or not the light projecting device 49 performs light projection according to the imaging conditions set in the previous step S123 (step S128). S128) If it is determined that the projection has been performed, the projection is stopped in step S129, and the process proceeds to step S130. If no light is projected, step S129 is skipped and the process proceeds to step S130.

  In step S130, the system control circuit 50 determines the state of the image display flag stored in the internal memory of the system control circuit 50 or the memory 52. If the image display flag is set, the image display is displayed in step S131. The fixed image display is displayed on the unit 28, and the process proceeds to step S132.

  In this fixed image display process, a predetermined mark (symbol) indicating that the process is being performed is displayed to the user while the image is captured, encoded, and recorded. . The mark is displayed by the image display unit 28 via the memory control circuit 22 and the D / A converter 26, thereby displaying a fixed color image on the electronic viewfinder.

  If it is determined in step S130 that the image display flag has been canceled, the process of step S131 is skipped and the process proceeds to step S132.

  When the processing proceeds to step S132, the system control circuit 50 passes through the image sensor 12, the A / D converter 16, the image processing circuit 20, the memory control circuit 22, or directly from the A / D converter. Then, the captured image data is written into the memory 30 (exposure processing). Further, the image data written in the memory 30 is read out and various processes are performed (development process) using the memory control circuit 22 and, if necessary, the image processing circuit 20. An imaging process is executed by the exposure process and the development process. Details of this photographing process (step S132) will be described later with reference to FIG.

  When the photographing process is completed, the system control circuit 50 proceeds to step S133 and determines the state of the image display flag stored in the internal memory of the system control circuit 50 or the memory 52. When it is determined that the image display flag is set, quick review display is performed (step S136). In this case, the image display unit 28 is always displayed as an electronic viewfinder even during shooting, and quick review display immediately after shooting is also performed.

  If it is determined in step S133 that the image display flag has been canceled, the state of the quick review flag stored in the internal memory of the system control circuit 50 or the memory 52 is determined (step S135). If the review flag has been set, the image display of the image display unit 28 is set to the ON state (step S135), and quick review display is performed (step S136).

  As described above, since the electronic viewfinder function is not required because power is taken or the optical viewfinder 104 is used for shooting, the quick review can be performed even if the image display of the image display unit 28 is set to OFF. If the quick review function is set by the switch, it is possible to automatically reproduce the captured image on the image display unit 28 immediately after shooting, providing a power saving and convenient function for confirming the captured image. It becomes possible.

  If it is determined in step S133 that the image display flag has been cancelled, and it is determined in step S134 that the quick review flag has also been cancelled, the process proceeds to step S137 while the image display unit 28 remains off. move on. In this case, the image display unit 28 remains off even after shooting, and no quick review display is performed. This is a usage method in which it is not necessary to check a photographed image immediately after photographing as in the case of continuing photographing using the optical viewfinder 104, and the power saving is emphasized without using the electronic finder function of the image display unit 28. .

  Now, when the processing proceeds to step S137, the system control circuit 50 reads the captured image data written in the memory 30, and performs various image processing using the memory control circuit 22 and, if necessary, the image processing circuit 20. Further, after performing an image compression process according to the mode set using the compression / decompression circuit 32, a recording process for writing image data to the recording medium 200 or 210 is executed (step S137). Details of this recording process (step S137) will be described later with reference to FIG.

  When the recording process ends, the process proceeds to step S138, and it is determined whether or not the shutter switch SW2 has been pressed. If it is determined that the shutter switch SW2 remains in the ON state, the process proceeds to step S139, and the system control circuit 50 determines the state of the continuous shooting flag stored in the internal memory of the system control circuit 50 or the memory 52 ( Step S139). If it is determined that the continuous shooting flag is set (ON), the flow returns to step S129 to perform continuous shooting, and the next shooting is performed.

  If it is determined in step S139 that the continuous shooting flag is not set, the process waits until it is determined in step S138 that the shutter switch SW2 has been released.

  When the recording process is completed, the shutter switch SW2 is released, or the shutter switch SW2 is released after the quick review display is continued by continuously pressing the shutter switch SW2 to confirm the captured image. Then, the process proceeds to step S140.

  In step S140, the system control circuit 50 waits for a predetermined time to elapse. In step S141, the system control circuit 50 determines whether the image display flag is set. If it is determined that the image display flag is set, the system control circuit 50 determines in step S142. Then, the display state of the image display unit 28 is set to the through display state, and the process proceeds to step S144. In this case, after confirming the captured image by the quick review display on the image display unit 28, it is possible to enter a through display state in which image data captured for the next imaging is sequentially displayed.

  If it is determined in step S141 that the image display flag has been cancelled, the image display of the image display unit 28 is set to the OFF state in step S142, and the process proceeds to step S144. In this case, after confirming the photographed image by the quick review display on the image display unit 28, the function of the image display unit 28 is stopped for power saving, and the image display unit 28 or D / A conversion with large power consumption is stopped. It is possible to reduce the power consumption of the device 26 and the like.

  In step S144, it is determined whether or not the shutter switch SW1 is in the ON state. If it is determined that the camera is in the ON state, the process returns to step S126 to prepare for the next shooting. If it is determined that the shutter switch SW1 has been opened once, the system control circuit 50 returns from step S144 to step S103 in order to finish a series of shooting operations.

  The main process in the embodiment has been described above. Next, each process of steps S122, S123, S132, and S137 in FIG. 3 will be described in detail.

<About ranging / photometry processing (step S122)>
FIG. 4 shows a detailed flowchart of the distance measurement / photometry process in step S122 of FIG. Hereinafter, description will be given with reference to FIG.

  The system control circuit 50 reads the charge signal from the image sensor 14 and sequentially reads the captured image data into the image processing circuit 20 via the A / D converter 16 (step S201). Using the sequentially read image data, the image processing circuit 20 is a predetermined used for TTL (through-the-lens) AE (automatic exposure) processing, EF (flash pre-flash) processing, and AF (autofocus) processing. Perform the operation.

  In each processing here, a specific portion of the total number of photographed pixels is extracted by extracting a necessary portion according to necessity and used for calculation. This makes it possible to perform optimum calculations for different modes such as the center-weighted mode, the average mode, and the evaluation mode in each of the TTL method AE, EF, AWB, and AF processes.

  Using the calculation result in the image processing circuit 20, the system control circuit 50 performs AE control using the exposure control means 40 (step S203) until it is determined that the exposure (AE) is appropriate (step S202).

  Further, using the measurement data obtained by the AE control, the system control circuit 50 determines whether or not the flash is necessary (step S204). If the flash is necessary, the flash flag is set and the flash 48 is charged (see FIG. Step S205). Note that step S204 is when the flash is automatic. If it is explicitly set not to drive the flash, step S204 is always determined as No.

  If it is determined in step S202 that the exposure (AE) is appropriate, the measurement data and / or setting parameters are stored in the internal memory or the memory 52 of the system control circuit 50.

  Using the calculation result in the image processing circuit 20 and the measurement data obtained by the AE control, the system control circuit 50 uses the image processing circuit 20 until the white balance (AWB) is determined to be appropriate (step S206). Color processing parameters are adjusted to perform AWB control (step S207).

  If it is determined in step S206 that the white balance (AWB) is appropriate, the measurement data and / or setting parameters are stored in the internal memory of the system control circuit 50 or the memory 52.

  This time, using the measurement data obtained by the AE control and the AWB control, the system control circuit 50 controls the distance measurement control means 42 until it is determined that the distance measurement (AF) is in focus (step S208). AF control is performed using this (step S209).

  If it is determined in step S208 that the distance measurement (AF) is in focus, the measurement data and / or setting parameters are stored in the internal memory or the memory 52 of the system control circuit 50, and this distance measurement / photometry processing routine (FIG. 3). Step S122) is terminated and the process returns to the main routine.

<About Red Eye Processing (Step S123)>
Next, details of the red-eye process in step S123 of FIG. 3 will be described according to the flowchart shown in FIG.

  First, in step S501, the system control circuit 50 determines whether or not the strobe setting state by the strobe mode switch 65 is set to “red-eye alleviation mode”. If it is determined that the result is not the red-eye reduction mode, the present process is terminated, the process returns to the main routine, and the process proceeds to step S124 in FIG.

  If it is determined that the red-eye reduction mode is set, it is determined whether the red-eye reduction is equal to or lower than a predetermined luminance that requires red-eye reduction based on the photometric result obtained in step S122 in FIG. If it is determined that the captured image has a luminance higher than the predetermined luminance, it is determined that the red-eye process is unnecessary, and the process returns to step S124 in FIG. On the other hand, if it is determined that the luminance is equal to or lower than the predetermined luminance, the process proceeds from step S502 to step S503.

  In step S503, the light projector 49 starts projecting light from the red-eye projector. Then, after the light projection is started, the image is captured and written into the image display memory 24 via the image sensor 12, the A / D converter 16, the image processing circuit 20, and the memory control circuit 22 (step S504), and the frozen image is displayed. Display is performed (step S505). Then, the data recorded in the image display memory 24 is transferred to the previous image display memory 25 (step S506).

  When this transfer is performed, the process proceeds to step S507 to start timing, and waits until it is determined in step S508 that the predetermined time (0.2 seconds in the embodiment) has elapsed. If it is determined that the predetermined time has elapsed, the image is captured again and written into the image display memory 24 (step S509). Then, the image data in the image display memory 24 and the image data stored in the previous image display memory 25 before a predetermined time (0.2 seconds before) are compared to determine whether or not red eyes have occurred (step S510).

  Here, the process of step S510 will be described in more detail.

  Since the captured image immediately after the start of light projection is stored in the red image display memory 27 in the previous image display memory 27, it becomes as shown by reference numeral 141 in FIG. The amount of light reaching the retina is the largest (the lightness of the red eye is the largest), and the red eye diameter is likely to be the largest. On the other hand, since the image stored in the image display memory 24 is an image after a predetermined time has elapsed since the start of red-eye projection, the pupil diameter of the subject in the image is small, resulting in a color change. Appear. This is as shown by reference numeral 142 in FIG. In step S510, the presence of red-eye is determined by comparing the image 141 with the image 142.

  That is, there is a difference in the pupil diameter between the image data in the image display memory 24 and the image data in the previous image display memory 27, so that there is a difference in brightness and diameter of the red eye, and a difference also appears in the image data. Therefore, the pixel in which the difference has occurred can be identified as the portion where the red eye is generated. Further, depending on the degree of red eye, a difference occurs in the brightness of the R component of the RGB data of the pupil image. Therefore, the occurrence of red eye can be detected by comparing R component data in the image data.

  Therefore, in the embodiment, the image data 141R in which the R component data is extracted from the image data 141 in FIG. 14 by the memory control 22 is generated, and the image data 142R in which the R component is extracted from the image data 142 is also generated. The image data 142R is subtracted from the image data 141R to generate a difference image 143R.

  The image 143 in FIG. 14 simply shows the difference between the image 141 and the image 142, but as can be seen from the image 143, there is a difference in the difference other than the pupil, or there is a possibility that the difference may occur. high. That is, the accuracy of identifying the position of the eyes is not good, and there is a possibility that the eyes are erroneously determined. On the other hand, in the case of the difference image 143R having only the R component, portions other than the red-eye are removed, so that it is possible to accurately detect the coordinate position and the size of the eyes.

  The processing in step S510 has been described above. However, if red-eye is not detected in this step, this processing is terminated and the processing returns to step S124 in FIG.

  If red eyes are detected, the data recorded in the image display memory 24 is rewritten in the previous image display memory 25 in step S511. In step S512, a predetermined image display is performed based on the image data captured in the previous step S509 and stored in the image memory 25. The details of this image display will be described later with reference to FIG.

  Thereafter, the process proceeds to step S513, and again starts counting a predetermined time (0.2 seconds in the embodiment), and waits until the predetermined time elapses in step S514. In step S515, the image is captured again, and the image is written in the image display memory 24. As a result, the latest captured image is stored in the image display memory 24, and the previous image is stored in the previous image display memory 25.

  In step S516, based on the coordinate position (coordinate position in the image) of the red-eye area detected in the previous step S510, the diameter of the red-eye area (the part where the R component is greater than the predetermined value) in the image display memory 24 is determined. It is determined whether or not the first predetermined size (first threshold value) has been reduced. The threshold value, which is the determination criterion, is a value indicating that photography is possible in a state where there is no sense of incongruity in a photograph in which red eyes have occurred even if the photograph is taken as it is.

  For example, the diameter is about ½ of the iris, or the diameter of the red eye is about 4 mm or less, as shown by the image 113 in FIG. The iris can be detected from the characteristics of the image around the red eye, white eye part (bright): iris part (dark) from the detection result of the red eye region described above. The diameter of the red eye can be predicted from the focal length and the distance measurement result (however, considering the simplicity of processing, the red eye diameter (or area) immediately after the start of red eye projection is not calculated as described above. May be used as a reference).

  If the red-eye diameter is smaller than the first threshold value, it is determined in step S517 whether or not the red-eye diameter is smaller than a second predetermined size (second threshold value). The second threshold value is a criterion for determining whether or not the red-eye correction process should be performed, and is also a criterion for ending the red-eye process at that time. In the embodiment, the second threshold value is 1/3 (1/9 in area) of the iris diameter (the initial diameter of red-eye detection).

  If NO in step S516, that is, if it is determined that there is a high possibility that a sense of discomfort will occur when red eyes are corrected, the process proceeds to step S519 to determine whether or not a predetermined number of images have been captured. If not, step S511 and subsequent steps are repeated. For this predetermined number of times, the red-eye alleviating effect is generally considered to be effective if the time is about 1.2 seconds. In the embodiment, since the captured image capture interval (the above-mentioned predetermined time) is about 0.2 seconds, it is 6 times, but in order to further improve the effect, 10 times, that is, the longest time of red-eye relaxation light projection is set. 2 seconds. That is, even when Step S516 is No, red-eye relaxation light projection is performed for a maximum of 2 seconds.

  If it is determined that the determination in step S516 is No for ten consecutive times, the process proceeds to step S521, and a mark (symbol image) as shown in FIG. Displayed on the image display unit 28, notifies the user that red-eye correction processing is to be performed, sets a correction flag indicating that red-eye correction is to be performed to ON in step S522, and ends this processing. This correction flag is set to OFF when the initial state and the red-eye correction process are completed.

  Further, although the determination in step S516 is Yes, that is, the red-eye diameter is ½ or less of the iris diameter, No is determined in step S517, that is, the red-eye diameter is larger than 1/3 of the iris diameter. In step S518, it is determined whether the shutter switch SW2 is ON. That is, it is determined whether or not the user has instructed to capture and record a still image. If it is determined that the shutter switch SW2 is OFF, the process proceeds to step S519, and the above process is repeated until it is determined that a predetermined number of images have been captured (up to 10 times). However, if it is determined in step S517 that the shutter switch SW2 has been turned ON, the process proceeds to step S521 to give priority to the user's request, and a mark for correction (FIG. 12) is displayed on the image display unit. 28, and the correction flag is set to ON in step S522.

  Note that there are two routes to step S521: a route when No is determined in step S516 and a case where No is determined in step S517. In the former case, the red-eye diameter remains relatively large, that is, when the correction is made, the image 112 in FIG. 11 is likely to be obtained, and in the latter case, the red-eye diameter can be imaged by correction without any sense of incongruity. That is, it indicates that it is not conspicuous like the image 114 of FIG. Therefore, the marks in FIG. 12 are displayed in distinction due to the difference between these routes. For example, the mark is displayed in red when the determination is No in step S516 and the process proceeds to step S521, and the mark is displayed in red when the determination is No in step S517 and the process proceeds to step S521.

  If the red-eye diameter is 1/3 or less of the iris diameter in step S517, the correction flag remains OFF and the mark in FIG. 12 indicating that red-eye correction is unnecessary is green. indicate. In this case, the image 115 shown in FIG. 11 is obtained.

  In the above description, the level of processing to be performed is changed by changing the color of the mark. However, a technique such as changing the shape of the mark may be adopted. Moreover, it may be displayed only when red-eye correction is performed, or a combination thereof may be used.

  Moreover, in the said embodiment, although the red eye diameter was 1/2 and 1/3 of the iris diameter, this is an example and this does not limit this invention. This is because when the processing is simplified, the diameter of the red eye at the initial stage of the red-eye processing may be used as a reference instead of using the diameter of the iris as a reference. Instead of obtaining the diameter, the degree of red-eye reduction may be determined by counting the number of pixels in which the difference value between the R components of the two images is greater than a predetermined value.

  As described above, the red-eye processing in the present embodiment starts light emission for red-eye reduction when the shutter button SW1 is turned ON, and continues for 0.2 seconds while this red-eye reduction light emission continues. By repeating imaging at intervals, the latest two image data are stored and held in the image display memory 24 and the previous image display memory 27, and the R component is compared to detect the position and size of the red eye. Therefore, since the two images were both taken in the same light projection environment, it was promised that the brightness between the two images was substantially the same, and the position of the red eye and the red eye It becomes possible to accurately detect a change in the size of the light from the start of light projection. Further, it is possible to inform the user whether or not to correct red-eye and the degree of influence of the correction when red-eye correction is performed.

  In the above description, the image display processing in steps S505 and S512 will be described with reference to the flowchart of FIG.

  First, in step S701, the system control circuit 50 checks the enlarged display setting state of the operation unit 70. If the enlarged display is set, the enlarged display area is set (step S702). The enlarged display area is set to an area including the red-eye occurrence portion detected by the above-described red-eye detection process. When this enlarged display area is set, the set area is displayed as a freeze image (step S703). FIG. 10C shows a display example in this case. If enlargement display is not set, a freeze image display of the entire image is performed (step S704). A display example in this case is shown in FIG.

  Here, the determination of the presence or absence of the red eye and the determination of the size of the red eye in the red eye processing in the embodiment will be described in more detail with reference to FIG.

  In the figure, 1901 indicates the R component of the image including the red-eye portion immediately after the start of red-eye relaxation projection, and 1902 indicates the R component of the image including the red-eye portion after Δt during the red-eye relaxation projection. (In this case, for the sake of convenience, a portion with a high R component is indicated by hatching).

  When the image 1902 is subtracted from the image 1901 by a subtracter (subtraction process) 1903, a difference image 1904 is obtained. Since the difference between the R components of the red-eye portion between the image 1901 and the image 1902 is small, a ring-shaped image like the image 1904 is obtained when the difference image is binarized with a predetermined threshold. As can be seen from the image 1904, the circular area indicated by the inner diameter r is the current red-eye area (which may be referred to as the pupil size). Since the red eyes appearing in the two images immediately after the start of the red-eye relaxation projection are different in size, the difference between the R components appears remarkably as shown in the figure, so that the position can be determined accurately. Further, as the projection is continued, the human pupil gradually becomes smaller but converges to a certain size. That is, the red-eye regions of two images that are temporally continuous become the same. In other words, the outer diameter of the ring-shaped image, which is the difference image in FIG. 19, becomes smaller and the thickness of the ring gradually becomes thinner. When the pupil reduction converges, the ring-shaped image cannot be detected. Accordingly, “r” at each stage is determined as the red-eye diameter (red-eye size), and the center position thereof is determined as the red-eye coordinate position. However, when the difference image cannot be detected when the red-eye contraction converges, the red-eye diameter and position at the immediately preceding stage are determined as the final position.

  In order to simplify the processing, the area (number of pixels) of the ring-shaped hatched portion in FIG. 19 is counted, and the counted number of pixels is compared with each threshold value in step S516 or step S517 in FIG. To do so.

<About the photographing process (step S132)>
Next, the photographing process in step S132 in FIG. 3 will be described with reference to the flowchart in FIG. Note that this processing is processing when it is determined in step S126 in FIG. 3 that the shutter switch SW2 is ON.

  The system control circuit 50 exposes the image sensor 10 by opening the shutter 12 having an aperture function according to the aperture value by the exposure control unit 40 according to the photometric data stored in the internal memory of the system control circuit 50 or the memory 52. Is started (steps S301 and S302).

  In step S303, it is determined whether or not the flash 48 is necessary based on the flash flag. If necessary, the flash (strobe) 48 is caused to emit light (step S304).

  In step S305, the system control circuit 50 waits for the end of exposure of the image sensor 12 according to the photometric data. If it is determined that the exposure is completed, the shutter 12 is closed (step S306), the charge signal is read from the image sensor 14, and the A / D converter 16, the image processing circuit 20, the memory control circuit 22 or A The captured image data is written in the memory 30 directly from the / D converter 16 via the memory control circuit 22 (step S307).

  In step S308, it is determined whether it is necessary to perform frame processing according to the set shooting mode. If it is determined that the processing is to be performed, the system control circuit 50 determines that the memory control circuit 22 If necessary, the image processing circuit 20 is used to read the image data written in the memory 30, perform vertical addition processing (step S 309), and color processing (step S 310) sequentially, and store them in the memory 30 again.

  Next, proceeding to step S311, it is determined whether or not red-eye correction is to be performed by examining whether or not the correction flag is ON in the red-eye processing described above. If the red-eye correction flag is ON, the process proceeds to step S312 and the red-eye correction is performed on the captured image stored in the memory 30. This red-eye correction process is a correction that lowers the brightness of the R component of the image data of the pupil part where the red eye is generated according to the coordinate position and size information of the red eye obtained in the previous red-eye process.

  When the process proceeds to step S313, the system control circuit 50 reads the image data from the memory 30, transfers the display image data to the image display memory 24 via the memory control circuit 22, and ends this process.

<Recording Process (Step S137)>
Next, the recording process in step S137 of FIG. 3 will be described with reference to the flowchart of FIG.

  First, in step S401, the system control circuit 50 controls the memory control circuit 22 and, if necessary, the image processing circuit 20, reads out the photographic image data written in the memory 30, and reads out the vertical / horizontal pixel ratio of the image sensor 14. The pixel squareization process for interpolating the signal to 1: 1 is performed and written in the memory 30 again.

  In step S402, the image data written in the memory 30 is read, and the compression / decompression circuit 32 controls the image compression processing according to the set mode. In step S403, the compression-encoded data (JPEG data is generally used) is transferred to a memory card, compact flash (registered trademark) card, or the like via the interface 90 or 94 and the connector 92 or 96. Writing compressed image data to the recording medium 200 or 210 is performed.

  When the writing to the recording medium is finished, the recording process (step S137) is finished, and the process returns to step S138 in FIG.

  As described above, according to the present embodiment, when the two-stage shutter button is in the first-stage pressed state, a preliminary operation for detecting the red-eye position and size while performing red-eye reduction projection is performed. Repeat imaging. Therefore, since it is possible to make the conditions for each preliminary imaging substantially the same, it is possible to accurately detect the transition of the position and size of the red eye. In addition, the red-eye relaxation projection causes the subject's pupil size to decrease, and finally determines whether or not to perform red-eye correction and affects the image quality to some extent when red-eye correction is performed. It is also possible to inform.

  Further, when the shutter button is pressed in the second stage, that is, when the shutter button is fully pressed, it is determined whether or not the red-eye correction obtained in the first stage is necessary, and the position and size of the correction target are determined only when necessary. The R component of the region based on is corrected. Accordingly, R component attenuation processing (correction processing) is performed only for the minimum necessary region. In other words, since the red-eye correction is not performed on an area larger than necessary, it is possible to record a natural image with no sense of incongruity.

  In addition, according to the present embodiment, red-eye mitigating light is emitted by the light projecting device 49, and the release is permitted by detecting that the red-eye mitigating effect is obtained by the red-eye detecting means 25. Further, by displaying the red-eye state on the image display 28, it is possible to take a picture while confirming the red-eye state. By performing such red-eye processing, it is possible to correct the red-eye of the captured image, and when the correction area becomes large, it is not possible to notify the fact in advance. It also eliminates giving a pleasant feeling to the user.

In the above embodiment, the latest two images are stored in the image display memory 24 and the previous image display memory 27 at the time of red-eye processing, but the previous image display memory 27 stores the image immediately after the start of red-eye processing. Data may be stored, and the image display memory 24 may sequentially store images captured at predetermined time intervals, compare them, and make a determination based on a reduction ratio with respect to the initial red eye size.
In the above embodiment, assuming that the size of the iris is 1, the red-eye diameter is 1/2, 1/3, or the like, and the state of red-eye relaxation is detected. In this case, the iris size is determined by the distance between the subject and the camera, but generally, the greater the distance between the camera and the subject, the worse the accuracy of distance measurement. Therefore, in the above embodiment, the function may be performed only when the distance is within a preset distance, or a warning mark or the like may be displayed when the distance is exceeded.

[Second Embodiment]
Next, a second embodiment according to the present invention will be described.

  FIG. 9 is a block diagram of a digital camera device according to the second embodiment. This is based on the configuration of FIG. 1 (configuration of the first embodiment). Accordingly, different configurations and feature points will be described below.

  The configuration of the second embodiment is that an image shift detection unit 29 is provided with respect to FIG. The image shift detection unit 29 detects the coordinate movement amount of the image due to camera shake by the photographer and the coordinate movement amount of the subject from the two images.

<About operation processing flow>
The red-eye process (step S123 in FIG. 3) in the first embodiment is performed in the order shown in FIG. 7, but the red-eye process in the second embodiment is processed according to the flowchart in FIG. . Hereinafter, a description will be given with reference to FIG.

  The system control circuit 50 checks whether or not the strobe setting state by the strobe mode switch 65 is the red-eye reduction mode (step S601). If this result is not the red-eye alleviation mode, the red-eye process is terminated and the process returns to step S124 in FIG. If the red-eye alleviation mode has been set, it is determined based on the photometric result performed in step S122 in FIG. 3 whether or not the brightness is below a predetermined luminance that requires red-eye reduction (step S602). Here, if the luminance is equal to or higher than the predetermined luminance, it is determined that the red-eye process is unnecessary, and the process returns to step S124 in FIG.

  If it is determined that the luminance is less than or equal to the predetermined luminance, the process proceeds to step S603, the light projecting device 49 is driven, and light projection from the red-eye mitigating projector 304 (see FIG. 10) is started. The following is the processing during this light projection.

  In step S604, the captured digital image data is written in the image display memory 24 via the image sensor 14, the image processing circuit 20, and the memory control circuit 22. Then, the frozen image is displayed (step S605). Then, the data recorded in the image display memory 24 is transferred to the previous image display memory 25 (step S606). Then, timing is started (step S607). In step S608, the process waits for a predetermined time (0.2 seconds in the embodiment) to elapse. When the predetermined time has elapsed, the image is captured again and the image is written in the image display memory 24 (step S609). Then, an image shift detection 29 based on the image data in the image display memory 24 and the image data in the previous image display memory 25 detects the image data in the previous image display memory 25 (predetermined by the camera shake and subject movement). A coordinate shift with respect to the image data before time) is detected (step S610). The configuration of the image shift detection unit 29 is to obtain the moving amount and direction of the moving body of each image by a motion vector detection method typified by a known matching method or density gradient method. The description is omitted.

  In step S611, based on the amount of deviation of the image data in the image display memory 24 from the image data in the previous image display memory 25, the pixel values of the image data in the image display memory 24 and the image data in the previous image display memory 25 are changed. The coordinates are matched and the two images are compared to determine the presence or absence of red eye.

  Here, the process of step S611 will be described in more detail.

  Since the captured image immediately after the start of light projection is stored in the red image display memory 27 in the previous image display memory 27, it becomes as shown by reference numeral 141 in FIG. There is a possibility that the degree of red eye will be the largest. On the other hand, since the image stored in the image display memory 24 is an image after a predetermined time has elapsed since the start of red-eye projection, the pupil diameter of the subject in the image is small, resulting in a color change. Appear. This is as shown by reference numeral 142 in FIG. In step S510, the presence of red-eye is determined by comparing the image 141 with the image 142.

  That is, there is a difference in the pupil diameter between the image data in the image display memory 24 and the image data in the previous image display memory 27, so that there is a difference in brightness and diameter of the red eye, and a difference also appears in the image data. Therefore, the pixel in which the difference has occurred can be identified as the portion where the red eye is generated. Further, depending on the degree of red eye, a difference occurs in the brightness of the R component of the RGB data of the pupil image. Therefore, the occurrence of red eye can be detected by comparing R component data in the image data.

  Therefore, in the embodiment, the image data 141R in which the R component data is extracted from the image data 141 in FIG. 14 by the memory control 22 is generated, and the image data 142R in which the R component is extracted from the image data 142 is also generated. The image data 142R is subtracted from the image data 141R to generate a difference image 143R. Since the images 141R and 142R are equal in that there is no difference in brightness of the entire image, in the difference image 143R, it is possible to generate only the difference in the R component of the pupil position. That is, based on the difference image 143R, the position of the coordinates on the image where the red eye is generated can be specified by the position having the predetermined value or more (the generation position of the red eye).

  The image 143 in FIG. 14 simply shows the difference between the image 141 and the image 142, but as can be seen from the image 143, there is a difference in the difference other than the pupil, or there is a possibility that the difference may occur. high. That is, the accuracy of identifying the position of the eyes is not good, and there is a possibility that the eyes are erroneously determined. On the other hand, in the case of the difference image 143R having only the R component, portions other than the red-eye are removed, so that it is possible to accurately detect the coordinate position and the size of the eyes. In addition, it is possible to make the processing as fast as the calculation of only the R component, not all the RGB components.

  However, since there is a time difference (0.2 seconds in the embodiment) between the image 141 and the image 142, the image data of the previous image display memory 27 is not the image data of the image display memory 24. Alternatively, it is conceivable that the subject is displaced. Therefore, when subtracting the image 142R from the image 141R, one of the images is relatively based on the amount of coordinate shift on the image of the camera shake or subject shift performed in the image shift detection process (step S610). Subtraction is performed after shifting, and the presence or absence of a red-eye area is determined. Instead of shifting the entire image, when comparing pixels of two images, an offset (deviation amount) is added to the coordinates of one pixel to be read.

  The processing in step S611 has been described above. If red-eye is not detected in the same step, this processing ends and the processing returns to step S124 in FIG.

  If it is determined that the red-eye phenomenon has occurred, the data recorded in the image display memory 24 is written in the previous image display memory 25 in step S612. In step S613, a predetermined image display is performed based on the image data stored in the previous image display memory 25.

  When the process proceeds to step S614, the time measurement is started again, and a predetermined time (0.2 seconds in the embodiment) elapses in step S615. Then, when the predetermined time has elapsed, in step S616, the image is captured again and written in the image display memory 24. In step S617, the image captured this time (stored in the image display memory 24) as in the previous step S610. ) Is detected from the image stored in the previous image display memory 25.

  When the shift amount is detected, the process proceeds to step S18, where the position and size of the red eye on the image stored in the image display memory 24 is determined, and whether the size is equal to or smaller than the first threshold value. Judging. If it is determined that it is greater than the first threshold, the above process is repeated until it is determined in step S621 that a predetermined number of images have been captured.

  If the size of the detected red-eye area is equal to or smaller than the first threshold value, whether or not it is equal to or smaller than the second threshold value (second threshold value <first threshold value) in step S619. Judging. If it is determined that the size is greater than or equal to the second threshold value, the process proceeds to step S620, where it is determined whether the shutter button SW2 is ON / OFF. If it is OFF, the process proceeds to step S621. In order to prioritize shooting / recording by the user, the process proceeds to step S623.

  In step S623, a mark indicating that red-eye correction is to be performed (see FIG. 12) is displayed. However, when it is determined No in step S618 and the process proceeds to step S623, the mark is displayed in red. When step S619 is determined as No and the process proceeds to step S623, the mark is displayed in yellow.

  In any case, the process proceeds to step S624, and the correction flag is set to ON to perform the red-eye correction process.

  On the other hand, if it is determined in step S619 that it is equal to or smaller than the second threshold value, it is determined that red-eye correction processing is unnecessary (the correction flag remains OFF), and the mark shown in FIG. indicate.

  In the above description, red-eye detection (red-eye) is detected based on the difference between the R components of the image data stored in the image display memory 24 and the previous image display memory 27, and the determination in steps S618 and S619. The processing is assumed to be the same as steps S516 and S517 in FIG. 7 in the first embodiment.

  The main part of the second embodiment has been described above, but it should be noted that the other parts are the same as those of the first embodiment. That is, in the second embodiment, the position of the red eye can be determined in consideration of the influence of camera shake or the like as compared with the first embodiment described above. High-precision red-eye correction without discomfort is possible.

[Third Embodiment]
Next, a third embodiment will be described. The apparatus configuration in the third embodiment is the same as in FIG. And, the characteristic feature is that the red-eye reduction effect priority mode (the first mode is the same as that in the first and second embodiments, hereinafter referred to as red-eye reduction mode 1). ) And the second mode red-eye alleviation speed priority mode (red-eye alleviation but simplification of the process and priority is given to the instruction of the shutter button SW2, hereinafter referred to as red-eye alleviation mode 2) It is a point. The selection of these two modes is determined by operating a switch provided on the operation unit 70.

<About operation processing flow>
The red-eye process (step S123 in FIG. 3) in the third embodiment will be described with reference to the flowchart in FIG.

  The system control circuit 50 checks whether or not the strobe setting state by the strobe mode switch 65 is the red-eye reduction mode (step S901). If this result is not the red-eye alleviation mode, the red-eye process is terminated and the process returns to step S124 in FIG. If the red-eye reduction mode is set, it is determined based on the photometric result performed in step S122 of FIG. 3 whether or not the brightness is below a predetermined luminance that requires red-eye reduction (step S902). Here, if the luminance is equal to or higher than the predetermined luminance, it is determined that the red-eye process is unnecessary, and the process returns to step S124 in FIG.

  If it is determined that the luminance is equal to or lower than the predetermined luminance, the process proceeds to step S903, the light projecting device 49 is driven, and light projection from the red-eye mitigating projector 304 (see FIG. 10) is started. The following is the processing during this light projection.

  In step S904, the captured digital image data is written in the image display memory 24 via the image sensor 14, the image processing circuit 20, and the memory control circuit 22. Then, the frozen image is displayed (step S905). Then, the data recorded in the image display memory 24 is transferred to the previous image display memory 25 (step S906). Thereafter, in order to take a timing for capturing the next image, time measurement is started (step S907). In step S908, the process waits for a predetermined time (0.2 seconds in the embodiment) to elapse. When the predetermined time has elapsed, the image is captured again and the image is written in the image display memory 24 (step S909). Then, image shift detection 29 based on the image data in the image display memory 24 and the image data in the previous image display memory 25, image data in the previous image display memory 27 (predetermined) of the image data in the image display memory 24 due to camera shake and subject movement. A coordinate shift with respect to the image data before time) is detected (step S910). The configuration of the image shift detection unit 29 is to obtain the moving amount and direction of the moving body of each image by a motion vector detection method typified by a known matching method or density gradient method. The description is omitted.

  In step S911, based on the amount of deviation of the image data in the image display memory 24 from the image data in the previous image display memory 25, the pixel values of the image data in the image display memory 24 and the image data in the previous image display memory 25 are changed. The coordinates are matched and the two images are compared to determine the presence or absence of occurrence of red eyes (coordinate position and size information is stored in the memory 30). Since the determination of the presence or absence of red eyes is the same as in the second embodiment, description thereof is omitted.

  If no red-eye is detected in step S911, the process is terminated and the process returns to step S124 in FIG.

  If it is determined in step S911 that the red-eye phenomenon has occurred, the process proceeds to step S912 to determine which of the red-eye reduction modes 1 and 2 is set. If it is determined that the red-eye reduction mode 2 is selected, the process proceeds to step S927 to determine whether or not the shutter button SW2 is turned on. If it is determined that the shutter button SW2 is ON, the process advances to step S926 to turn on a correction flag to unconditionally perform red-eye correction, and the process ends. If it is determined in step S927 that the shutter button SW2 is OFF, the process proceeds to step S913. Note that if it is determined in step S912 that the red-eye alleviation mode 1 is set, the process proceeds to step S913.

  In short, when the red-eye alleviation mode 2 is set, a process for determining whether or not the shutter button SW2 is ON is performed as appropriate, and when the user gives an instruction for imaging / recording, the red-eye process is immediately performed. Exit. In other words, when the shutter button SW2 is OFF, the operation is the same in modes 1 and 2.

  When the process proceeds to step S913, the data recorded in the image display memory 24 is written in the previous image display memory 25. In step S914, display based on the image data stored in the previous image display memory 25 is performed.

  When the process proceeds to step S915, the time measurement is started again, and it is waited for a predetermined time (0.2 seconds in the embodiment) to elapse in step S916. Then, when the predetermined time has elapsed, in step S917, the image is captured again and written in the image display memory 24. In step S918, the image picked up this time (stored in the image display memory 24) as in the previous step S910. ) Is detected from the image stored in the previous image display memory 25.

  When the shift amount is detected, the process proceeds to step S919, where the position and size of the red eye on the image stored in the image display memory 24 are determined by comparison with the previous image display memory 27, and the size is determined. It is determined whether or not it is equal to or less than a first threshold value. If it is determined that it is greater than the first threshold value, it is determined in step S922 which of red-eye alleviation modes 1 and 2 is set, and if it is determined that it is mode 1, the process proceeds to step S924. The above process is repeated until it is determined that a predetermined number of images have been captured (2 seconds have passed).

  If the size of the detected red-eye area is equal to or smaller than the first threshold value, whether or not it is equal to or smaller than the second threshold value (second threshold value <first threshold value) in step S920. Judging. If it is determined that the size is greater than or equal to the second threshold value, or if it is determined in step S922 that the red-eye reduction mode 2 is set, the process proceeds to step S923, and the shutter button SW2 is turned ON / OFF. Judge which one it is. If it is off, the process proceeds to step S924. If it is on, priority is given to shooting / recording by the user, and the process proceeds to step S925.

  In step S925, a mark for red-eye correction (see FIG. 12) is displayed. However, when it is determined No in step S919 and the process proceeds to step S925, the mark is displayed in red. When step S920 is determined as No and the process proceeds to step S925, the mark is displayed in yellow.

  In either case, the process proceeds to step S926, and the correction flag is set to ON to perform the red-eye correction process.

  On the other hand, if it is determined in step S921 that it is equal to or smaller than the second threshold value, it is determined that the red-eye correction process is unnecessary (the correction flag remains OFF), and the mark shown in FIG. indicate.

  In the above description, red-eye detection (red-eye) is detected based on the difference between the R components of the image data stored in the image display memory 24 and the previous image display memory 27, and the determinations in steps S919 and S920. The processing is assumed to be the same as steps S516 and S517 in FIG. 7 in the first embodiment.

  Although the main part of the third embodiment has been described above, according to the third embodiment, the following effects can be obtained in addition to the operational effects of the second embodiment.

  This mode was selected because the size of the red-eye area is greater than or equal to the first threshold and the user is allowed to preferentially capture and record an image / record instruction even before 2 seconds have elapsed. In this case, the possibility of missing a photo opportunity can be reduced.

[Fourth Embodiment]
A fourth embodiment will be described. The apparatus configuration in the fourth embodiment is the same as that in FIG. 9, and the description of the apparatus configuration is omitted.

  In the first to third embodiments, the red-eye reduction projection is performed before the flash photography. In the fourth embodiment, after a predetermined time has elapsed after the flash photography, the flash photography is further performed. The position of the red eye is detected from the comparison result of the respective images.

<About operation processing flow>
The processing procedure of the fourth embodiment will be described with reference to FIGS. 16 and 17 show the main processing in the fourth embodiment, and FIG. 18 shows the details of the photographing processing in step S1132 in FIG.

  In FIG. 16, when power is turned on such as battery replacement, the system control circuit 50 initializes flags, control variables, and the like (step S1101), and initializes the image display of the image display unit 28 to the OFF state (step S1102). .

  The system control circuit 50 determines the setting position of the mode dial 60, and if the mode dial 60 is set to power OFF (step S1103), the display of each display unit is changed to the end state, and the protection unit 102 The barrier is closed to protect the imaging unit, and necessary parameters, setting values, and setting modes including flags and control variables are recorded in the nonvolatile memory 56, and the power control unit 80 controls each part of the apparatus 100 including the image display unit 28. After performing a predetermined end process such as shutting off unnecessary power (step S1105), the process returns to step S1103.

  If it is determined in step S103 that the mode dial 60 is powered on and set to the shooting mode, the process advances to step S1106. If it is determined that the mode dial 60 is set to another mode (for example, image browsing after being captured and stored, image selection / deletion, etc.), the system control circuit 50 responds to the selected mode. The process is executed (step S1104). When the process is finished, the process returns to step S103.

  When the processing proceeds to step S1106, the system control circuit 50 determines whether or not the remaining capacity or operation status of the power source 86 configured by a battery or the like has a problem in the operation of the apparatus 100 by the power control unit 80, If there is, a predetermined warning is displayed by image or sound using the display unit 54 (step S1108), and the process returns to step S1103.

  If it is determined that there is no problem with the power supply 86, the system control circuit 50 determines whether the operation state of the recording medium 200 or 210 has a problem with the operation of the apparatus 100, particularly with respect to the recording / reproducing operation of the image data with respect to the recording medium. (Step S1107) If there is a problem, a predetermined warning is displayed by an image or sound using the display unit 54 (Step S1108), and the process returns to Step S1103.

  When it is determined that there is no problem in the operation state of the recording medium 200 or 210, the process proceeds to step S1109, and the display unit 54 is used to display various setting states of the apparatus 100 using images and sounds. If the image display of the image display unit 28 is ON, the image display unit 28 is also used to display various setting states of the apparatus 100 using images and sounds.

  When the process proceeds to step S1110, the system control circuit 50 checks the setting state of the quick review ON / OFF switch 68, and sets the quick review flag if the quick review ON is set (step S1111). If it is set to OFF, the quick review flag is canceled (step S1112). The state of the quick review flag is stored in the internal memory or the memory 52 of the system control circuit 50.

  Subsequently, the system control circuit 50 determines the setting state of the image display ON / OFF switch 66 (step S1113). If the image display is set to ON, the image display flag is set (step S1114), the image display of the image display unit 28 is set to the ON state (step S1115), and the captured image data is sequentially displayed. The through display state is set (step S1116), and the process proceeds to step S1119.

  In the through display state, the data sequentially written in the image display memory 24 via the image sensor 12, the A / D converter 16, the image processing circuit 20, and the memory control circuit 22 are converted into the memory control circuit 22 and the D The electronic finder function is realized by sequentially displaying images on the image display unit 28 via the / A converter 26.

  On the other hand, if it is determined in step S1113 that the image display ON / OFF switch 66 is set to image display OFF, the process proceeds to step S1117 to cancel the image display flag and in step S1118, the image display unit. The image display 28 is set to the OFF state, and the process proceeds to step S1119.

  When the image display is OFF, shooting is performed using the optical viewfinder 104 without using the electronic viewfinder function of the image display unit 28. In this case, it is possible to reduce power consumption of the image display unit 28, the D / A converter 26, and the like that consume a large amount of power. The state of the image display flag is stored in the internal memory of the system control circuit 50 or the memory 52.

  When the process proceeds to step S1119 (FIG. 17), it is determined whether the shutter switch SW1 is ON. That is, it is determined whether or not the shutter switch is half pressed. If it is determined that the button has not been pressed, the process returns to step S1103 and the above process is repeated.

  If it is determined that the shutter switch SW1 has been turned on, the process proceeds from step S1119 to step S1120. The system control circuit 50 determines the state of the image display flag stored in the internal memory or the memory 52 (step S1120). If the image display flag is set, the display state of the image display unit 28 is set to the freeze display state. (Step S1121), the process proceeds to step S1122.

  In the freeze display state, rewriting of the image data in the image display memory 24 via the image sensor 12, the A / D converter 16, the image processing circuit 20, and the memory control circuit 22 is prohibited, and the last written image data Is displayed by the image display unit 28 via the memory control circuit 22 and the D / A converter 26, so that it is displayed on the electronic viewfinder as a frozen image. If the image display flag has been canceled, the process of step S1121 is skipped and the process proceeds to step S1122.

  In step S1122, the system control circuit 50 performs a distance measurement process to adjust the focal length of the photographing lens 10 to the subject, and also performs a light measurement process to determine an aperture value and a shutter opening time (shutter speed). In the photometric process, if the flash is set to auto, whether or not to perform flash is also set. Note that the details of the distance measurement / photometry processing in step S1122 have already been described in the first embodiment, and thus description thereof will be omitted.

  If the photometry / ranging process is completed, the process proceeds to step S1124, and the system control circuit 50 determines the state of the image display flag stored in the internal memory of the system control circuit 50 or the memory 52. If the image display flag has been set, the display state of the image display unit 28 is set to the through display state (step S1125), and the process proceeds to step S1126. Note that the through display state in step S1125 is the same operation state as the through state in step S1116 described above.

  Subsequently, in step S1126, it is determined whether or not the shutter switch SW2 has been turned on, that is, whether or not the shutter button has been fully pressed. If not, it is determined in step S1127 whether or not the shutter switch SW1 is also released. If the shutter SW1 is also released (OFF), the process returns to step S1103.

  If it is determined that the shutter switch SW2 has been turned on, the process proceeds to step S1130. In step S1130, the system control circuit 50 determines the state of the image display flag stored in the internal memory of the system control circuit 50 or the memory 52. If the image display flag is set, the image is displayed in step S1131. The fixed image display is displayed on the display unit 28, and the process proceeds to step S1132.

  In this fixed image display process, a predetermined mark (symbol) indicating that the process is being performed is displayed to the user while the image is captured, encoded, and recorded. . The mark is displayed by the image display unit 28 via the memory control circuit 22 and the D / A converter 26, thereby displaying a fixed color image on the electronic viewfinder.

  If it is determined in step S1130 that the image display flag has been cancelled, the process in step S1131 is skipped and the photographing process in step S1132 is performed, which will be described later.

  When the process of step S1132 is completed, the system control circuit 50 proceeds to step S1133 and determines the state of the image display flag stored in the internal memory of the system control circuit 50 or the memory 52. If it is determined that the image display flag is set, a quick review display is performed (step S1136). In this case, the image display unit 28 is always displayed as an electronic viewfinder even during shooting, and quick review display immediately after shooting is also performed.

  If it is determined in step S1133 that the image display flag has been released, the state of the quick review flag stored in the internal memory of the system control circuit 50 or the memory 52 is determined (step S1135). If the review flag has been set, the image display of the image display unit 28 is set to the ON state (step S1135), and quick review display is performed (step S1136).

  As described above, since the electronic viewfinder function is not required because power is taken or the optical viewfinder 104 is used for shooting, the quick review can be performed even if the image display of the image display unit 28 is set to OFF. If the quick review function is set by the switch, it is possible to automatically reproduce the captured image on the image display unit 28 immediately after shooting, providing a power saving and convenient function for confirming the captured image. It becomes possible.

  If it is determined in step S1133 that the image display flag has been cancelled, and it is determined in step S1134 that the quick review flag has also been cancelled, the process proceeds to step S1137 while the image display unit 28 remains OFF. move on. In this case, the image display unit 28 remains off even after shooting, and no quick review display is performed. This is a usage method in which it is not necessary to check a photographed image immediately after photographing as in the case of continuing photographing using the optical viewfinder 104, and the power saving is emphasized without using the electronic finder function of the image display unit 28. .

  When the process proceeds to step S1137, the system control circuit 50 reads the captured image data written in the memory 30, and performs various image processing using the memory control circuit 22 and, if necessary, the image processing circuit 20. Further, after performing image compression processing according to the mode set using the compression / decompression circuit 32, recording processing for writing image data to the recording medium 200 or 210 is executed. Since this recording process is as described in the first embodiment, the description thereof is omitted.

  When the recording process ends, the process advances to step S1138 to determine whether the shutter switch SW2 has been pressed. If it is determined that the shutter switch SW2 remains in the ON state, the process proceeds to step S1139, and the system control circuit 50 determines the state of the continuous shooting flag stored in the internal memory of the system control circuit 50 or the memory 52 ( Step S1139). If it is determined that the continuous shooting flag is set (ON), the flow returns to step S1132 to perform continuous shooting, and the next shooting is performed.

  If it is determined in step S1139 that the continuous shooting flag is not set, the process waits until it is determined in step S1138 that the shutter switch SW2 has been released.

  When the recording process is completed, the shutter switch SW2 is released, or the shutter switch SW2 is released after the quick review display is continued by continuously pressing the shutter switch SW2 to confirm the captured image. Then, the process proceeds to step S1140.

  In step S1140, the system waits for a predetermined time to elapse. In step S1141, the system control circuit 50 determines whether the image display flag is set. If it is determined that the image display flag is set, the process proceeds to step S1142. The display state of the image display unit 28 is set to the through display state, and the process proceeds to step S1144. In this case, after confirming the captured image by the quick review display on the image display unit 28, it is possible to enter a through display state in which image data captured for the next imaging is sequentially displayed.

  If it is determined in step S1141 that the image display flag has been canceled, in step S1142, the image display of the image display unit 28 is set to an OFF state, and the process proceeds to step S1144. In this case, after confirming the photographed image by the quick review display on the image display unit 28, the function of the image display unit 28 is stopped for power saving, and the image display unit 28 or D / A conversion with large power consumption is stopped. It is possible to reduce the power consumption of the device 26 and the like.

  In step S1144, it is determined whether shutter switch SW1 is in the ON state. If it is determined that the camera is in the ON state, the process returns to step S1126 to prepare for the next shooting. If it is determined that the shutter switch SW1 has been opened once, the system control circuit 50 returns from step S1144 to step S1103 to complete a series of shooting operations.

  The main process in the embodiment has been described above. Next, the processing in step S1132 in FIG. 17 will be described with reference to the flowchart in FIG.

  The system control circuit 50 first opens the shutter 12 having an aperture function according to the aperture value by the exposure control unit 40 in accordance with the photometric data stored in the internal memory of the system control circuit 50 or the memory 52. The main exposure is started (steps S1301 and S1302).

  Next, in step S1303, it is determined whether or not the flash 48 is necessary based on the flash flag, and if necessary, the flash is emitted (step S1304). There are two types of conditions for flash emission: a case where the user explicitly instructs flash emission and a case where it is determined to be necessary by the photometric process. The process of step S1304 is executed when any one of the conditions is satisfied.

  In step S1305, the system control circuit 50 waits for the end of exposure of the image sensor 14 (end of charge accumulation of the image sensor 14) according to the photometric data. If it is determined that the exposure has ended, the shutter 12 is closed (step S1306), the charge signal is read from the image sensor 14, and the A / D converter 16, the image processing circuit 20, the memory control circuit 22 or the A / D The captured image data is written to the memory 30 directly from the D converter 16 via the memory control circuit 22 (step S1307).

  In step S1308, the system control circuit 50 checks whether the strobe is set by the strobe mode switch 65 and whether the red-eye reduction mode is set. If it is not the red-eye mode or if the flash is not emitted in step S1304, the processing of the following steps S1309 to S1316 is skipped, and the process proceeds to step S1317.

  If it is determined that the red-eye alleviation mode is set and it is determined in step S1304 that flash emission has been performed, the process proceeds to step S1309.

  In step S1309, timing is started, and in step S1310, the process waits for a predetermined time to elapse. If it is determined that the predetermined time has elapsed, the process proceeds to step S1311, where the previous photometry data is used again, and the exposure control unit 40 opens the shutter 12 having the aperture function according to the aperture value to expose the image sensor 14 ( Steps S1311, S1312), and the flash is also emitted (Step S1313). Then, the system control circuit 50 waits for the exposure of the image sensor 12 to end in accordance with the photometric data (step S1314), closes the shutter 12 (step S1315), reads the charge signal from the image sensor 14, and reads the A / D converter 16, The captured image data is written into the memory 30 via the image processing circuit 20, the memory control circuit 22, or directly from the A / D converter 16 via the memory control circuit 22 (step S1316). Accordingly, the memory 30 stores two captured image data.

  Next, proceeding to step S1317, it is determined whether or not frame processing needs to be performed in accordance with the set shooting mode. If it is determined that it is necessary, the system control circuit 50 reads out the image data written in the memory 30 using the memory control circuit 22 and, if necessary, the image processing circuit 20 in step S1318. Addition processing (step S1318) and color processing (step S1319) are sequentially performed.

  Next, in step S1320, the image data of the main exposure and the comparative exposure written in the memory 30 are compared to determine whether or not red-eye has occurred (step S1317). This image data is compared with the image data during the main exposure (steps S1302 to S1305) (corresponding to the image 141 in FIG. 14 with the largest pupil diameter) and a comparison after a predetermined time has elapsed after the main exposure. Image data at the time of exposure (corresponding to the image 142 in FIG. 14) is captured by the memory control 22 into the system control 50 and compared. The pupil diameter at the comparative exposure after a predetermined time has elapsed from the main exposure is reduced and changes in color from the pupil diameter at the main exposure. That is, in the image data of the main exposure and the comparative exposure, a difference occurs in the image data because of a difference in the pupil diameter. Therefore, the pixel in which the difference has occurred can be identified as the portion where the red eye is generated. Further, depending on the degree of red eye, a difference occurs in the brightness of the R component of the RGB data of the pupil image. Therefore, the occurrence of red eye can be detected by comparing R component data (images 141R and 142R in the figure) in the image data 141 and 142. Specifically, the image 142R, which is the R component of the image data 142, is subtracted from the image 141R, which is the R component of the image data 141, and the position and size of the red eye are determined based on the subtracted difference image 143R. judge. In addition, in the difference image 143R, when the pixel whose difference value is more than the predetermined value is not generated, or when the number of the pixels is less than the predetermined number, it is determined that the red eye is not detected.

  By comparing only the R component in this way, it is possible to obtain a detection result at a higher speed than when performing comparison detection with all image components (RGB data). If the red eye is not detected based on the detection result obtained in the red eye detection (step S1320), the image in the main exposure is transferred to the image display memory 24 and displayed (step S1322).

  When red eyes are detected, predetermined red-eye correction (step S1321) is performed on the main exposure image data based on the obtained coordinate position and size, and the processed image data is written in the memory 30. Then, the process proceeds to step S1322, and the corrected image data is displayed.

  Note that the comparative image becomes unnecessary at this point, and is therefore deleted from the memory 30 or set to be overwritten.

  When a series of processing is completed as described above, the photographing processing routine (step S1132) is terminated.

  As described above, according to the fourth embodiment, when there is an instruction to capture and record by flash (strobe) light emission, the main exposure by flash light emission at the timing according to the user instruction, and the By continuously performing exposure to obtain a comparative image by flash emission again, it becomes possible to accurately detect and correct the red eye under the substantially same environment of the shooting conditions of these two captured images. In addition, there is substantially no time lag between the user's instruction and the timing of actual shooting, and imaging and storage are possible.

  In the above embodiment, the digital still camera has been described as an example. However, the present invention can be applied to a digital video camera having a still image capturing function, a mobile phone having a camera function, and the like. The invention is not limited.

  Further, in the above-described embodiment, it is assumed that the light projection for performing red-eye detection and red-eye reduction is performed by using the LED 49 or the lamp as the light projecting device 49. However, the flash 48 may be used instead. In the case of the flash 48, the light is intermittently lit, but it may be sufficient to perform flash light emission during exposure in steps S504 and S515 in FIG.

  Further, the light emission at the comparison exposure in step S1313 is not limited to the flash light emission, and may be, for example, an LED or a lamp.

  Further, the first predetermined size and the second predetermined size for determining the size of the red eye are, in the embodiment, the first predetermined size is 1/2 or 1/3 of the iris diameter, etc. However, this determination level may be changed according to the focal length or the distance to the subject (ranging distance) instead of a fixed amount.

  Further, the brightness of the projection light of the projector may be changed brightly so that the red-eye alleviation effect can be sufficiently obtained according to the red-eye relaxation state.

  In addition, the imaging interval when red-eye reduction is performed, that is, the capturing of an image is set to 0.2 seconds. However, the predetermined time is not limited to 0.2 seconds. It may be arbitrarily set according to the respective capabilities of the camera, such as the light emission interval of the strobe capable of emitting light, the processing speed of image data, etc.

  In addition, when the predetermined time is 0.2 seconds and the predetermined number of times is 10 times, the light projection time is about 2 seconds. Therefore, this time (predetermined number of times) is not limited to about 2 seconds.

  Further, the image comparison 29 is not a unique image comparison means, and if the processing capability of the image comparison of the system control 50 is sufficiently high, and if the shift detection and the red-eye detection can be detected without a shooting time lag, the system control 50 The image comparison may be performed at.

  In the embodiment (especially the first and third embodiments), the target images to be compared are the image display memory 24 for storing the image subjected to the thinning process for displaying the captured image, the previous image display memory 27, and the like. did. This is desirable from the viewpoint of reducing the number of pixels when performing the comparison operation, but if the calculation speed is sufficient, it is desirable to perform a comparison operation on images that are not thinned out. This is because it is possible to determine the coordinates and size of the red eye with higher accuracy by the amount that is not thinned out.

It is a block block diagram of the digital still camera in 1st Embodiment. It is a flowchart of the main routine in 1st Embodiment. It is a flowchart of the main routine in 1st Embodiment. 5 is a flowchart of a distance measurement / photometry routine in the first embodiment. It is a flowchart of the imaging | photography routine in 1st Embodiment. It is a flowchart of the recording routine in 1st Embodiment. It is a flowchart of the red-eye processing routine in 1st Embodiment. It is a flowchart of the red-eye processing routine in 2nd Embodiment. It is a block block diagram of the digital still camera in 2nd Embodiment. It is a general-view figure of the device in an embodiment. It is a figure which shows the correction | amendment result about the generation | occurrence | production state of red eyes. It is a figure which shows an example of the red-eye mark displayed by embodiment. 5 is a flowchart of an image display routine in the embodiment. It is a figure for demonstrating the principle of the red eye detection in embodiment. It is a flowchart of the red-eye processing routine in 3rd Embodiment. It is a flowchart of the main routine in 4th Embodiment. It is a flowchart of the main routine in 4th Embodiment. It is a flowchart of the imaging | photography routine in 4th Embodiment. It is a figure for demonstrating the detection principle of a position and size of a red eye.

Claims (17)

An image processing apparatus having an imaging unit and a light projecting unit,
A first image acquisition means for acquiring first captured image data by the imaging means during light projection by the light projecting means;
Second image acquisition in which second imaging image data is acquired by the imaging means during light projection by the light projecting means after a predetermined time after the first image acquisition means has acquired the first captured image data. Means,
Comparison means for comparing the first and second captured image data acquired by the first and second image acquisition means;
An image processing apparatus comprising: a detection unit that detects a position of a red eye and a size of a red eye based on a comparison result of the comparison unit.   The comparison unit subtracts the red component data in the image data acquired by the second image acquisition unit from the red component data in the image data acquired by the first image acquisition unit. The image processing apparatus according to claim 1.   The image pickup apparatus further includes an image shift correction unit that corrects a shift amount of the second captured image data with respect to the first captured image data, and the comparison unit compares the captured images after correcting the image shift. Image processing device. An imaging device comprising an imaging means, a red-eye alleviating light projecting means, and a strobe means,
First control means for performing red-eye reduction by the red-eye reduction light projecting means, and second control means for performing imaging by the strobe means after control of the first control means,
The first control means includes
The red-eye relaxation light projection by the red-eye relaxation light projection unit is started, and preliminary imaging is performed by the imaging unit every predetermined time, and the latest first captured image data and the second before the predetermined time.
Storage holding means for storing and holding the captured image data of
Comparing means for comparing the first and second captured image data;
A red-eye judging means for judging the position and size of the red eye based on the comparison result by the comparing means;
Based on the determination result of the determination means, the process shifts to the second control means, if it is to be corrected, whether to correct the red eye, and whether to continue the red-eye reduction by the red-eye reduction floodlight means Processing determination means for determining whether or not,
The second control means includes
Imaging means for performing strobe emission by the strobe means and imaging with the imaging means;
And a correction unit that corrects image data of a red component in a region corresponding to the position and size of the red eye determined by the red eye determination unit when the processing determination unit determines to correct the red eye. apparatus.   The image processing apparatus according to claim 4, wherein the comparison unit subtracts red component data in the first captured image data from the second captured image data.   6. The imaging according to claim 4, wherein the processing determination unit uses red-eye relaxation as a continuation condition when the red-eye size exceeds a first threshold and the number of preliminary imagings is a predetermined number or less. apparatus.   The imaging apparatus according to claim 6, wherein the main imaging instruction is invalidated when the red-eye size exceeds the first threshold and the number of preliminary imaging is equal to or less than a predetermined number.   The processing determination means sets red-eye reduction as a further condition for continuing when the red-eye size exceeds a second threshold smaller than the first threshold, is less than the predetermined number of times, and there is no main imaging instruction. The imaging apparatus according to claim 6 or 7, wherein   The processing determination unit performs red-eye correction when the number of times of preliminary imaging reaches a predetermined number, or when the size of red-eye exceeds the second threshold value and there is a main imaging instruction. The imaging apparatus according to claim 8, wherein determination is made and a shift to the second control unit is permitted.   10. The imaging according to claim 9, wherein when the red-eye size is equal to or smaller than the second threshold, the processing determination unit permits the shift to the second control unit without correcting the red-eye. apparatus.   The imaging apparatus according to claim 10, further comprising notification means for notifying a result of the condition determination when it is determined to shift to the second control means.   12. The imaging apparatus according to claim 11, wherein the notification unit displays a mark having a different display form in accordance with each condition that has shifted to the second control unit. An imaging device comprising an imaging means, a red-eye alleviating light projecting means, and a strobe means,
A first image acquisition means for acquiring first captured image data by the imaging means during light projection by the strobe means;
After the predetermined time when the first captured image data is acquired by the first image acquiring unit, the second captured image data is acquired by the imaging unit during the light projection by the red-eye mitigating and projecting unit. Image acquisition means;
Comparison means for comparing the first and second captured image data acquired by the first and second image acquisition means;
Detection means for detecting the position of the red eye and the size of the red eye based on the comparison result of the comparison means;
An image pickup apparatus comprising: correction means for correcting image data of a red component in a region corresponding to the position and size of the red eye detected by the detection means of the first picked-up image data.   The comparison unit subtracts the red component data in the image data acquired by the second image acquisition unit from the red component data in the image data acquired by the first image acquisition unit. The imaging apparatus according to claim 13. A control method for an image processing apparatus having an imaging means and a light projecting means,
A first image acquisition step of acquiring first captured image data by the imaging means during light projection by the light projecting means;
Second image acquisition by which second imaging image data is acquired by the imaging means during light projection by the light projecting means after a predetermined time after acquiring the first imaging image data in the first image acquisition step. Process,
A comparison step of comparing the first and second captured image data acquired in the first and second image acquisition steps;
And a detection step of detecting the position of the red eye and the size of the red eye based on the comparison result of the comparison step. A method for controlling an imaging apparatus comprising an imaging means, a red-eye mitigating and projecting means, and a strobe means,
A first control step for performing red-eye reduction by the red-eye reduction light projecting unit, and a second control step for performing imaging by the strobe unit after the control of the first control step,
The first control step includes
The red-eye mitigation light projecting unit starts projecting red-eye mitigation light, and the imaging unit performs preliminary imaging every predetermined time, and the latest first captured image data and the second before the predetermined time. A storage holding step for storing captured image data;
A comparison step of comparing the first and second captured image data;
A red-eye determination step for determining the position and size of the red-eye based on the comparison result of the comparison step;
Based on the determination result of the determination step, whether to proceed to the second control step, whether to correct the red-eye if it is to be transferred, and whether to continue the red-eye reduction by the red-eye reduction flooding step A process determination step for determining whether or not,
The second control step includes
An imaging step of performing strobe light emission by the strobe means and taking an image with the imaging means;
A correction step of correcting red component image data in a region corresponding to the position and size of the red eye determined in the red eye determination step when it is determined in the processing determination step that red eye correction is to be performed. Device control method. A method for controlling an imaging apparatus comprising an imaging means, a red-eye mitigating and projecting means, and a strobe means,
A first image acquisition step of acquiring first captured image data by the imaging means during light projection by the strobe means;
After the predetermined time when the first captured image data is acquired by the first image acquiring unit, the second captured image data is acquired by the imaging unit during the light projection by the red-eye mitigating and projecting unit. An image acquisition process;
A comparison step of comparing the first and second captured image data acquired in the first and second image acquisition steps;
Based on the comparison result of the comparison step, a detection step of detecting the position of the red eye and the size of the red eye,
An image pickup apparatus control method comprising: a correction step of correcting the red component image data in a region corresponding to the position and size of the red eye detected in the detection step of the first picked-up image data.