JP2012231237A - Image recording apparatus and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image recording apparatus and an imaging apparatus capable of acquiring an image in which a subject turns the eyes in a direction desired by a user.SOLUTION: The image recording apparatus is provided with: a display which displays an image corresponding to the electronic image data of the subject; a face detector 230a which detects a face from the image; an eye region detector 230b which detects the regions of eyes and the regions of pupils from the range of the face detected by the face detector 230a; a pupil editor 230g performing editing including changing of the positions of the regions of the pupils in the regions of the eyes; and a recording part which records image data edited by the pupil editor 230g.

Description

  The present invention relates to an image recording apparatus and an imaging apparatus that record electronic image data obtained by imaging a subject.

  In recent years, imaging devices such as digital cameras and digital videos have been widely used. Recent imaging apparatuses are equipped with various functions not to miss the shooting timing such as eye-gaze detection of subjects and prevention of blinking, and are easily used by a wide range of users. As a technique related to such a function, for example, Patent Document 1 discloses an imaging apparatus that determines an imaging timing based on a change in the line of sight of a subject. Patent Document 2 discloses an imaging apparatus that controls an imaging operation by determining a camera line of a subject. Patent Document 3 discloses an imaging device that detects a face of a subject based on captured image data, and changes and displays the attribute of the area when there is a blinking face area.

  Further, as a technique related to the detection of the line of sight (line of sight), Patent Document 4 discloses a line-of-sight direction detection device that estimates the center of the eyeball and the center of the black eye from an image captured by a stereo camera and measures the line-of-sight direction of the driver. It is disclosed. Patent Document 5 discloses a camera provided with a function of detecting the line of sight of a photographer looking through a viewfinder.

JP 2010-206523 A JP 2010-199728 A JP 2008-131094 A JP 2004-254960 A JP 2004-12503 A

  However, when determining the imaging timing by detecting the line of sight of the subject, it is necessary to wait for the subject to look at the imaging apparatus. In addition, when taking a group photo of a plurality of subjects, particularly when the subject is a young child, it is very difficult to measure the timing when the lines of sight of all subjects face the imaging device. Therefore, a suitable shot may be missed while such an imaging timing is being measured.

  Furthermore, both of the techniques disclosed in Patent Documents 4 and 5 are techniques for detecting the direction of the line of sight directed by the operator (photographer), and look at a specific direction (for example, the direction of the imaging apparatus). There is no disclosure about obtaining an image of a directed subject.

  The present invention has been made in view of the above, and an object of the present invention is to provide an image recording apparatus that can acquire an image in which a subject has a line of sight in a direction desired by a user. It is another object of the present invention to provide an imaging apparatus that can perform imaging at a timing when a subject turns his / her eyes in a direction desired by a user.

  In order to solve the above-described problems and achieve the object, an image recording apparatus according to the present invention includes display means for displaying an image corresponding to electronic image data of a subject, and face detection for detecting a face from the image. And an eye area detecting means for detecting an eye area and a pupil area from the face range detected by the face detecting means, and an editing process including changing a position of the pupil area in the eye area And a recording means for recording image data edited by the pupil editing means.

  The image recording apparatus may further include eye line determination means for determining whether or not the eye line of the face is directed in a predetermined direction based on the position of the pupil area in the eye area.

  In the above-described image recording apparatus, the predetermined direction is a front direction, and the line-of-sight determination unit determines that the line of sight is facing the front when the pupil region is approximately in the center of the eye region. It is characterized by that.

  The image recording apparatus further includes notification means for notifying the determination result of the eye line determination means.

  In the image recording apparatus, the notification unit is a face detection frame synthesizing unit that has a display attribute determined according to a determination result by the eye line determination unit and combines a face detection frame surrounding the face with the image. It is characterized by.

  In the image recording apparatus, the face detection frame synthesizing unit synthesizes the face detection frame having two or more types of display attributes with the image.

  The image recording apparatus further includes a face selection unit that selects a face determined by the line-of-sight determination unit that the line of sight does not face a predetermined direction, and the pupil editing unit is selected by the face selection unit The pupil region in the face is edited.

  The image recording apparatus selects any of the operation input means for inputting a signal corresponding to an operation from the outside to the image recording apparatus and the face detected by the face detection means by the operation input means. Face selection means for selecting the selected face when a selection signal is input, and the pupil editing means edits the pupil region in the face selected by the face selection means. Features.

  The image recording apparatus further includes operation input means for inputting a signal corresponding to an operation from the outside to the image recording apparatus, and the pupil editing means is configured to input the eye based on the signal input by the operation input means. In this region, the pupil region is moved.

  In the image recording apparatus, the operation input unit is a touch panel provided on the display unit, and the pupil editing unit is configured to detect the pupil based on a trajectory of touching the pupil region with respect to the touch panel. The region is moved.

  In the image recording apparatus, the operation input unit is a touch panel provided on the display unit, and the pupil editing unit is based on a trajectory of a touch on a region other than the pupil region on the touch panel. The pupil region is moved.

  In the image recording apparatus, the pupil editing unit moves the other pupil region in conjunction with each other when a signal for moving the one pupil region is input by the operation input unit.

  In the image recording apparatus, the pupil editing unit moves the other pupil region in conjunction with the one pupil region, and then receives a signal for moving the other pupil region by the operation input unit. In the case of input, only the other pupil region is moved without interlocking the one pupil region.

  The image recording apparatus further includes a pupil position calculating unit that calculates a position of a pupil region when the eye line is directed in a predetermined direction in the eye region, and the pupil editing unit is detected by the eye region detecting unit. The pupil area is moved to a position calculated by the pupil position calculating means.

  In the image recording apparatus, the pupil editing unit expands or expands the pupil region when a gap with the eye region is formed above and below the pupil region after the pupil region is moved. Thus, the gap is interpolated.

  The image recording apparatus further includes an imaging unit that captures an image of a subject through a photographic lens to acquire electronic image data, and a release unit that issues a shooting instruction. The recording unit includes a shooting instruction by the release unit. Recording the image data corresponding to the image obtained by shooting according to the image data, and recording the image data corresponding to the image edited by the pupil editing unit with a file name different from the original image data. Features.

  An image pickup apparatus according to the present invention includes an image pickup unit that picks up an image of a subject through a shooting lens to acquire electronic image data, a release unit that issues a shooting instruction, and a display unit that displays an image corresponding to the image data And image processing means for generating a live view image that is continuously played back at a fixed minute time interval, recording means for recording an image obtained by shooting according to a shooting instruction by the release means, and A face detecting means for detecting a face from the image; an eye area detecting means for detecting an eye area and a pupil area from the range of the face detected by the face detecting means; and an area of the pupil area in the eye area. Based on the position, the eye determination unit that determines whether or not the eye of the face is facing a predetermined direction, and a warning display according to the determination result by the eye determination unit are combined with the image Characterized in that it comprises a warning display combining unit that.

  In the imaging apparatus, the predetermined direction is a front direction, and the line-of-sight determination unit determines that the line of sight is facing the front when the pupil area is approximately in the center of the eye area. It is characterized by.

  The imaging apparatus further includes a face detection frame synthesizing unit that has a display attribute determined according to a determination result by the eye determination unit, and synthesizes a face detection frame surrounding the face with the image.

  In the imaging apparatus, the face detection frame synthesizing unit synthesizes the face detection frame having two or more types of display attributes with the image.

  In the imaging apparatus, the warning display combining unit displays the number of the face detection frames having a predetermined attribute.

  The imaging apparatus further includes warning information adding means for adding warning information corresponding to a determination result by the eye line determination means to image data corresponding to the image.

  The imaging apparatus further includes image selection means for selecting an image displayed on the display means based on the warning information.

  In the imaging apparatus, the image selection unit selects an image to which the warning information is added.

  In the imaging apparatus, the image selection unit selects an image to which the warning information is not added.

  The imaging apparatus further includes pupil editing means for performing editing including changing the position of the pupil area in the eye area with respect to the image selected by the image selection means.

  According to the present invention, the eye region and the pupil region are detected from the image, and the editing is performed by changing the position of the pupil region in the eye region, so that the user can view the image of the subject in the desired direction. Can be easily obtained. In addition, according to the present invention, the line of sight is determined based on the position of the pupil region in the eye region for a live view image that is continuously played back at a fixed minute time interval, and the determination is made according to the determination result. Since the warning display is performed, the user can easily take an image at a timing when the subject turns his eyes in a desired direction.

FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 2 is a perspective view illustrating an appearance of the imaging apparatus illustrated in FIG. FIG. 3 is a block diagram showing a configuration of the eye line editing processing unit shown in FIG. FIG. 4 is a flowchart showing the operation of the imaging apparatus shown in FIG. FIG. 5 is a flowchart showing the reproduction operation of the imaging apparatus shown in FIG. FIG. 6 is a schematic diagram illustrating an example of an image displayed on the display unit. FIG. 7 is a diagram for explaining a method of determining a line of sight. FIG. 8 is a table showing display attributes of the face detection frame. FIG. 9 is a schematic diagram illustrating a display example of an image in which face detection frames are combined. FIG. 10 is a flowchart showing a manual editing operation for the line of sight. FIG. 11 is a diagram for explaining a manual line eye editing method. FIG. 12 is a diagram illustrating a pupil region editing method. FIG. 13 is a schematic diagram illustrating a display example of an edited image after editing the pupil region. FIG. 14 is a schematic diagram illustrating a display example of a line-of-sight editing menu. FIG. 15 is a flowchart showing an automatic or batch editing operation of the eyes. FIG. 16 is a schematic diagram illustrating a display example of an image in which pupil areas are collectively edited automatically. FIG. 17 is a schematic diagram illustrating a display example of an image in which the pupil region is collectively edited manually. FIG. 18 is a diagram for describing a pupil region editing method in Modification 1-1. FIG. 19 is a block diagram illustrating a configuration of a line-of-sight editing processing unit included in the imaging apparatus according to Embodiment 2. FIG. 20 is a flowchart illustrating a live view operation of the imaging apparatus according to the second embodiment. FIG. 21 is a block diagram illustrating a configuration of an image processing circuit included in an imaging apparatus according to Modification 2. FIG. 22 is a schematic diagram illustrating an example in which a list of images to which warning information is added is displayed. FIG. 23 is a schematic diagram showing an example of displaying a list of images to which no warning information is added.

Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as “embodiments”) will be described with reference to the drawings. The present invention is not limited to the embodiments described below. In the description of the drawings, the same parts are denoted by the same reference numerals.
In the present application, the horizontal direction means a direction in which two eyes of a face in an image are arranged.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a digital camera (hereinafter simply referred to as a camera) that is an image recording apparatus according to Embodiment 1 of the present invention. FIG. 2 is a perspective view showing the appearance of the camera shown in FIG. 1 from the back side.

  As shown in FIG. 2, the camera 1 includes an interchangeable lens 100 and a camera body 200 that are configured separately, and records an image obtained by capturing an object through the interchangeable lens 100 in the camera body 100. The interchangeable lens 100 is detachable from a mount opening (not shown) on the front surface of the camera body 200.

On the upper surface of the camera body 200, a release button 10 for instructing shooting, a power switch 20, and a mode dial 11 for setting a shooting mode are arranged.
The release button 10 is a two-stage switch in which the first release switch is turned on by half-pressing, and the second release switch is turned on by further pressing and pressing. When the first release switch is turned on, the camera 1 performs shooting preparation operations such as focus detection, focusing of the taking lens, and photometry of subject brightness. When the second release switch is turned on, the camera 1 is based on an output of an image sensor 221 described later. Then, a shooting operation for capturing the image data of the subject image is executed.

  The power switch 20 is linked to a power switch lever for controlling the power supply of the camera body 200 and the interchangeable lens 100. When the power switch 20 is turned on, power is supplied to the entire camera 1 and the camera 1 starts operating. On the other hand, when the power switch 20 is turned off, the camera 1 enters an operation state in a low current consumption mode (also referred to as a sleep mode). In this low current consumption mode, a sequence controller 227, which will be described later, is driven with a low current consumption, and power is not supplied except for circuits such as the atmospheric pressure sensor unit 215 and the switch detection circuit 253.

  On the back of the camera body 200, the display unit 26 and the touch panel 12 provided to overlap the display unit 26, the enlargement button 13 a, the reduction button 13 b, the playback button 14, the menu button 15, the eye button 16, and the edit button 17 are displayed. Further, an operation button group including a cross button 18 and a determination button 19 is arranged. The touch panel 12 and the operation button groups 13a to 19 correspond to operation input means for inputting a signal corresponding to a predetermined instruction or information in accordance with an external operation. In addition, an accessory port 29 serving as a connection portion for various accessories is provided on the back surface of the camera body 200.

  The display unit 26 is realized by a display panel such as a liquid crystal panel using an active element such as a TFT (Thin Film Transistor) or an organic EL (Electro Luminescence), and displays a live view of a subject image, a playback display of a recorded captured image, a menu screen. A variety of images and information such as display are displayed. In the first embodiment, the display unit 26 is disposed on the back surface of the camera body 200. However, the display unit 26 can be disposed on the back surface of the camera body 200 as long as it can be observed by the photographer. The live view display is a moving image (live view image) for observing a subject image displayed based on an image signal output from the image sensor 221.

  The touch panel 12 detects a position where an object such as a user's finger or a touch pen (stylus) contacts (touches) based on information displayed on the display unit 26, and performs an operation according to the touch position and the touch trajectory. Input the signal. In general, the touch panel includes a resistance film method, a capacitance method, an optical method, and the like. In the first embodiment, any type of touch panel may be applied.

  The enlargement button 13a is an operation button for inputting an instruction signal for enlarging the image displayed on the display unit 26, and the reduction button 13b is for inputting an instruction signal for reducing the image displayed on the display unit 26. This is an operation button.

The playback button 14 is an operation button for inputting an instruction signal for instructing transition to the playback mode and extraction from the playback mode.
The menu button 15 is an operation button for inputting an instruction signal for displaying a menu on the display unit 26.

  The line-of-sight button 16 instructs the shift to the automatic or batch edit mode for automatically or collectively editing the lines of the subject image in the image displayed on the display unit 26 and the extraction from the automatic or batch edit mode. An operation button for inputting an instruction signal.

  The edit button 17 is an operation button for operating the shift to the manual edit mode in which the user manually edits the line of sight of the subject image in the image displayed on the display unit 26 and the extraction from the manual edit mode. is there.

The cross button 18 includes an upper button 18a, a lower button 18b, a left button 18c, and a right button 18d, and is an operation button for inputting an instruction signal for moving a cursor, a pointer, and the like displayed on the display unit 26 up, down, left and right. is there.
The determination button 19 is an operation button for inputting an instruction signal for determining an item selected by the cross button 18 or the like.

  In addition to the touch panel 12 and the cross button 18, various pointing devices such as a track ball, a touch pad, and a pointing stick can be used as an operation member for operating the cursor and pointer displayed on the display unit 26. These pointing devices may be provided in the camera 1 or may be externally attached to the camera 1.

  The accessory port 29 is connected to various accessory parts such as a viewfinder provided with a viewfinder eyepiece and a viewfinder display monitor disposed at a position where the photographer can observe through the viewfinder eyepiece. The A display panel such as a liquid crystal or an organic EL is used as the in-finder display monitor.

  As shown in FIG. 1, when the interchangeable lens 100 is mounted on the camera body 200, both are electrically connected at the communication contact 300. An attachment / detachment detection switch 259 is provided in the vicinity of the mount opening in the camera body 200 to detect the mounting state of the interchangeable lens 100 and the camera body 200.

  In the interchangeable lens 100, a photographing optical system 101 including photographing lenses 101a and 101b for focus adjustment and focal length adjustment, and a diaphragm 103 for adjusting the aperture amount are arranged. Among these, the photographing optical system 101 is driven by the optical system driving mechanism 107 and its position is adjusted. The diaphragm 103 is driven by the diaphragm driving mechanism 109 to adjust its position. The optical position detection mechanism 105 detects the position of the photographing optical system 101 and acquires the focus adjustment lens position (focus position) and the focal length (zoom position) of the photographing optical system 101. A rotatable distance ring 112 is provided around the interchangeable lens 100. The rotation direction (operation direction) and the rotation amount (operation amount) of the distance ring 112 are detected by the encoder 110.

  Each of the optical position detection mechanism 105, the optical system drive mechanism 107, the aperture drive mechanism 109, and the encoder 110 is connected to a lens controller 111 connected to the camera body 200 via a communication contact 300. The lens controller 111 is realized by a CPU, for example, and controls focusing and zooming via drive control for the optical system drive mechanism 107, and controls the aperture value via drive control for the aperture drive mechanism 109.

  In addition, the lens controller 111 transmits lens information regarding the focus adjustment lens position and focal length acquired by the optical position detection mechanism 105 to the camera body 200. Specifically, the focal length information (in the case of a zoom lens) of the interchangeable lens 100 is stored in an electrically rewritable nonvolatile memory (not shown) built in the interchangeable lens 100 (or the lens controller 111). (Shortest focal length and longest focal length), open aperture value, minimum aperture value, shootable distance range (including the closest distance), lens color balance information, aberration information, information on autofocus (AF), mounted lens The lens information such as the type (macro lens, teleconverter, etc.) is stored. The lens information includes information related to the state of the interchangeable lens 100 such as the aperture state (open aperture state or aperture state) of the aperture 103 and the operation state (operation direction and operation amount) of the distance ring 112 in addition to the above information. Information regarding the operation status of the interchangeable lens 100 is also included, such as information about completion of aperture stop 103, completion of aperture opening by the drive mechanism 109, and completion of lens drive by the optical system drive mechanism 107. The lens controller 111 transmits the lens information stored in the nonvolatile memory to the camera body 200 via the communication contact 300.

  On the optical axis of the photographing optical system 101 in the camera body 200 (that is, on the photographing optical path), a focal plane type shutter 203 for controlling exposure time and shielding the image sensor 221 is disposed. The operation of the shutter 203 is controlled by the shutter drive mechanism 213. A light beam incident from the direction of the subject and corresponding to the subject image formed by the photographing optical system 101 is guided to the direction of the camera body 200 through the mount opening where the interchangeable lens 100 and the camera body 200 are coupled, and the shutter. The light enters the camera body 200 at the timing when 203 is opened.

  An image sensor unit 290 is disposed behind the shutter 203 (in the direction opposite to the photographing optical system 101). The image sensor unit 290 is a unit in which the dust filter 205, the infrared cut filter 209, the optical low-pass filter 210, and the image sensor 221 are arranged in this order from the photographing optical system 101 side, and are integrally configured. Is housed in a sealed package so as not to enter. Such an image sensor unit 290 is displaced two-dimensionally in the XY plane orthogonal to the optical axis of the photographing optical system 101 by the blur correction drive mechanism 292.

  The dustproof filter 205 prevents the dust generated in the mount opening of the camera body 200 and the inside of the camera body from adhering to the image sensor 221 and the optical element, so that the shadow of the dust is imprinted on the subject image and becomes an unsightly image. To do.

  A piezoelectric element 207 connected to the dustproof filter driving circuit 211 is fixed to the entire circumference or a part of the peripheral edge of the dustproof filter 205. The piezoelectric element 207 vibrates the dust filter 205 with ultrasonic waves of a predetermined frequency based on the drive signal of the dust filter drive circuit 211. As the dust filter 205 vibrates with ultrasonic waves, dust adhering to the front surface of the dust filter 205 is removed.

  The infrared cut filter 209 cuts an infrared light component from the light beam corresponding to the subject image. The optical low-pass filter 210 removes high frequency components from the subject light flux.

  The image sensor 221 photoelectrically converts the light beam formed by the photographing optical system 101 to generate an analog image signal. In the first embodiment, a two-dimensional solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) is used as the imaging device 221.

  The camera body 200 includes an image sensor driving circuit 223 that processes an analog image signal generated by the image sensor 221, a preprocessing circuit 225, a contrast AF circuit 226, an ASIC (Application Specific Integrated Circuit) 262, A shake correction drive mechanism 292, an actuator drive circuit 294, and a shake detection unit 297 are provided.

  The image sensor driving circuit 223 reads out an analog image signal generated in the image sensor 221 in accordance with a control signal output from the input / output circuit 239, amplifies the analog image signal, and performs analog-digital conversion (AD conversion) or the like. Perform signal processing.

The preprocessing circuit 225 performs various types of preprocessing such as pixel thinning processing for live view display on the image signal subjected to signal processing in the image sensor driving circuit 223.
The contrast AF circuit 226 extracts a high frequency component (contrast signal) of the image signal output from the preprocessing circuit 225. The range of the image from which the contrast signal is extracted is controlled by the sequence controller 227, and for example, extraction is performed on an area such as an AF target, the entire imaging screen, or an enlarged display area. The extracted contrast signal is output to the sequence controller 227.

  The ASIC 262 includes a data bus 261, a sequence controller (main body controller) 227 connected to the data bus 261, an image processing circuit 229, a compression / decompression circuit 231, a video signal output circuit 233, an SDRAM control circuit 237, and an input / output circuit 239. , A communication circuit 241, a recording medium control circuit 243, a flash memory control circuit 247, and a switch detection circuit 253. In FIG. 2, the contrast AF circuit 226 is provided outside the ASIC 262, but it may be provided in the ASIC 262.

  The sequence controller 227 is realized by a CPU, for example, and controls the operation of each unit of the camera body 200. Specifically, the sequence controller 227 controls the overall operation of the camera 1 according to a program stored in the flash memory 249. Further, as described above, the sequence controller 227 receives the contrast signal from the contrast AF circuit 226 and controls the automatic focus adjustment of the photographing optical system 101 via the lens controller 111.

  The image processing circuit 229 digitally amplifies (digital gain adjustment processing), color correction, and gamma (γ) for the digital image signal (image data) output from the preprocessing circuit 225 and input via the data bus 261. Various image processing such as correction, contrast correction, black and white / color mode processing, and live view display processing are performed to determine the luminance distribution and saturation of the image data. Further, the image processing circuit 229 includes a line-of-sight editing processing unit 230 that edits the line of sight of a face of a subject image such as a person or animal in an image corresponding to the image data. Note that such face detection and eye line editing may be performed using dedicated hardware.

  FIG. 3 is a block diagram showing a configuration of the eye line editing processing unit 230. The eye edit processing unit 230 includes a face detection unit 230a, an eye region detection unit 230b, a pupil position calculation unit 230c, a line of sight determination unit 230d, a face detection frame synthesis unit 230e, a face selection unit 230f, and a pupil editing unit 230g. Is provided.

  The face detection unit 230a determines whether a face is included in the image displayed on the display unit 26 using a known technique such as pattern matching processing. If a face is included, the position coordinates of the face range on the display unit 26 are acquired, and face range information representing the position coordinates is generated. At this time, when a plurality of faces are included in the image, face range information corresponding to each face is generated. The face detection unit 230a basically detects the face of a person, but may also detect the face of an animal such as a dog or cat.

  The eye region detection unit 230b detects an eye region (hereinafter referred to as an eye region) and a pupil region (hereinafter referred to as a pupil region) in the eye region based on the face range information. And the position coordinate in the display part 26 of the outline of an eye area | region and a pupil area | region is acquired, and eye area | region information showing these position coordinates is produced | generated.

  The pupil position calculation unit 230c calculates the position of the pupil region in the eye region when the eye line is directed in a predetermined direction based on the eye region information. Specifically, the center position of the eye area in the horizontal direction is calculated.

  Based on the position coordinates of the eye region and the pupil region detected by the eye region detection unit 230b and the position calculated by the pupil position calculation unit 230c, the eye line determination unit 230d moves the detected eye line of the face in a predetermined direction. It is determined whether or not it is suitable. In the first embodiment, it is determined whether or not the line of sight is facing the front (the direction of the camera 1). When the detected pupil area is substantially at the center of the eye area in the horizontal direction, the line of sight is the front. It is determined that it is suitable for.

  The face detection frame synthesis unit 230e synthesizes a face detection frame that surrounds each detected face with an image based on the face range information. At that time, the face detection frame compositing unit 230e has attributes (line thickness, color, type, blinking period, etc.) determined according to the determination result when the eye determination unit 230d performs the determination. The face detection frame displayed in the above is synthesized. Specifically, the face detection frame compositing unit 230e displays different colors and line types for the face determined to have the eye line facing the front and the face determined to have the eye line not facing the front. The detected face detection frames are synthesized. In this case, the face detection frame synthesis unit 230e corresponds to a notification unit that notifies the determination result of the eye line determination unit 230d.

The face selection unit 230f selects one or more faces in the image based on the determination result of the eye line determination unit 230d and / or the selection signal input by the touch panel 12 or the operation button groups 13a-19.
The pupil editing unit 230g performs a process of editing the pupil region for the face selected by the face selection unit 230f. This editing includes changing the position of the pupil region in the eye region.

  Note that such a line-of-sight editing processing unit 230 may be realized by dedicated hardware or by reading a line-of-sight editing processing program into the CPU.

  The compression / decompression circuit 231 compresses the image data stored in the SDRAM 238 in a format such as JPEG or TIFF. Note that image compression is not limited to JPEG or TIFF, and other compression methods may be applied.

  The video signal output circuit 233 is connected to the display unit 26 (and an in-finder display panel when the viewfinder is attached to the accessory port 29) via the display unit driving circuit 235. The video signal output circuit 233 converts the image data stored in the SDRAM 238 or the recording medium 245 into a video signal for display on the display unit 26 (and / or the liquid crystal monitor in the finder).

  The SDRAM 238 is connected to the data bus 261 via the SDRAM control circuit 237. The SDRAM 238 is a buffer memory that temporarily stores image data processed by the image processing circuit 229 or image data compressed by the compression / decompression circuit 231.

  The input / output circuit 239 is connected to the dust filter driving circuit 211, the shutter driving mechanism 213, and the image sensor driving circuit 223, and controls data input / output with each circuit such as the sequence controller 227 via the data bus 261.

  The communication circuit 241 is connected to the data bus 261 and is connected to the lens controller 111 via the communication contact 300. The communication circuit 241 performs communication such as data transmission / reception and control command transmission between the lens controller 111 and the sequence controller 227 and other circuits.

  The recording medium control circuit 243 is connected to the recording medium 245 and controls recording and reproduction of image data and the like on the recording medium 245. The recording medium 245 is configured so that a rewritable recording medium such as a compact flash (registered trademark), an SD memory card (registered trademark), or a memory stick (registered trademark) can be loaded. It is removable. In addition, a hard disk unit such as a microdrive (registered trademark) or a wireless communication unit may be connectable.

  The flash memory control circuit 247 is connected to a flash memory 249. The flash memory 249 is an electrically rewritable nonvolatile memory, and stores a program for controlling the operation of the camera, control adjustment values, and the like. The sequence controller 227 controls the digital camera in accordance with programs, adjustment values, etc. stored in the flash memory 249.

  The switch detection circuit 253 is connected to the mode dial 11, the touch panel 12, the operation button groups 13 a to 19, the power switch 20, and the attachment / detachment detection switch 259, and detects the switch state of each part, or is input from these parts. The input signal is received and transmitted to each unit via the data bus 261. Note that when the power switch 20 is turned off, any operation of the operation members other than the power switch 20 is not detected.

The blur correction drive mechanism 292 includes an actuator composed of a piezoelectric element drive motor or the like as a drive source, and displaces the image sensor unit 290 in a two-dimensional manner within an XY plane orthogonal to the optical axis of the imaging optical system 101, thereby generating an image. To prevent blurring.
The actuator drive circuit 294 drives the actuator in the shake correction drive mechanism 292.

  The blur detection unit 297 includes an angular velocity sensor 297a using a gyro and a blur detection unit 297 including an angular velocity detection circuit 297b. The angular velocity sensor 297a outputs a shake signal corresponding to the shake generated in the camera body 200. The angular velocity detection circuit 297b amplifies the blur signal output from the angular velocity sensor 297a, performs AD conversion, and outputs it to the sequence controller 227.

  The sequence controller 227 outputs a shake control signal for canceling the shake of the camera body 200 to the actuator drive circuit 294 based on the shake signal detected by the shake detection unit 297, and outputs a drive signal for the actuator in the shake correction drive mechanism 292. Let As a result, the actuator is driven, and the image sensor 221 moves in a direction orthogonal to the optical axis of the photographing optical system 101. As a result, the blur applied to the camera body 200 is canceled, and the deterioration of the image is prevented. Such a camera shake correction method is called an image sensor shift method.

  The operation of the camera according to Embodiment 1 will be described with reference to FIG. FIG. 4 is a flowchart showing the operation of the camera 1 after the power-on reset by the sequence controller 227. When a battery is loaded in the camera body 200 or an external power source is connected, the operation shown in FIG. 4 starts.

  First, in step S01, the camera 1 determines whether or not the power switch 20 is on. When the power switch 20 is on (step S01: Yes), power supply to each mechanism and each circuit in the camera body 200 and the interchangeable lens 100 is started (S02). On the other hand, when the power switch 20 is off (step S01: No), the camera 1 enters a sleep state that is a low power consumption state (step S12). This sleep state is canceled when the power switch 20 is turned off.

  In step S03, the camera 1 acquires lens information. Specifically, the sequence controller 227 receives the focal position information and focal length information of the photographing optical system 101 detected by the optical position detection mechanism 105 from the lens controller 111 via the communication circuit 241, and the closest focal length / Lens information including unique information of the interchangeable lens 100 such as a long focal length focal length, an open aperture value, an exposure correction value for each focal length, a peripheral light amount correction value (shading), a peripheral color correction value, and a distortion aberration correction value. get. At this time, information on accessories related to the lenses such as the intermediate ring and the teleconverter may be acquired including whether or not these accessories are attached.

  In step S04, the camera 1 sets an operation mode and parameters. Specifically, the camera body 200 determines the shooting condition based on information such as the shooting mode set by the mode dial 11, ISO sensitivity, shutter speed manually set using the operation button groups 13a to 19 and aperture value. Is read.

  In subsequent step S05, the camera 1 performs photometry and exposure amount calculation. More specifically, the camera 1 obtains subject brightness based on the output of the image sensor 221 and uses the subject brightness to calculate an exposure control value such as a shutter speed and an aperture value at which proper exposure is performed according to the shooting mode and shooting conditions. .

  In step S06, the camera 1 performs live view display. Specifically, the image processing circuit 229 takes in the image signal output from the image sensor 221 and performs predetermined image processing, whereby display image data that is continuously reproduced at a fixed minute time interval. Is generated. The display unit 26 displays an image (live view image) corresponding to the image data for display.

  In step S07, the camera 1 determines whether or not the playback switch is turned on. That is, it is determined whether or not a playback switch signal that is linked to the playback button 14 is detected. When the reproduction switch is turned on (step S07: Yes), the camera 1 reproduces and displays an image corresponding to the image data recorded on the recording medium 245 on the display unit 26 (step S20).

  On the other hand, when the playback switch is not turned on (step S07: No), or when the playback operation of step S20 ends, the camera 1 determines whether or not the menu switch is turned on (step S08). That is, it is determined whether or not a menu switch signal linked to the menu button 15 is detected.

  When the menu switch is turned on (step S08: Yes), the camera 1 performs menu setting (step S30). Specifically, the camera 1 displays various setting screens such as ISO sensitivity, white balance, manual focus (MF) mode, autofocus (AF) mode, and image stabilization mode in accordance with the menu switch being turned on. 26. The camera 1 sets various modes and parameters in accordance with an instruction signal input by touching such a screen (touch panel 12) or by operating the cross button 18 and the enter button 19, and the set mode is set. Let each part execute. For example, when the image stabilization mode is set, the image stabilization operation is executed during the live view operation and the shooting operation.

  On the other hand, when the menu switch is not turned on (step S08: No), or when the menu setting at step S30 is completed, the camera 1 determines whether or not the first release switch (1R switch) is turned on (step S09). ). Specifically, it is determined whether or not a first release switch signal due to half-pressing of the release button 10 is detected.

  When the first release switch is turned on (step S09: Yes), the camera 1 performs photographing (step S40). Specifically, for example, when the AF mode is set, the camera 1 focuses the photographing optical system 101 by contrast AF, and calculates an exposure control value that achieves proper exposure by photometry and exposure calculation. Then, when the second release switch is turned on by fully pressing the release button 10, an exposure operation is performed, and image data of a still image acquired by imaging is recorded on the recording medium 245.

  On the other hand, when the first release switch is not turned on (step S09: No), or when the photographing at step S40 is completed, the camera 1 determines whether or not the power switch 20 is turned on (step S10). If the power switch 20 is on (step S10: Yes), the operation returns to step S05. On the other hand, when the power switch 20 is turned off (step S10: No), the camera 1 enters the power off mode, and the power supply is stopped (step S11). Thereafter, the operation proceeds to step S12.

Next, the reproduction operation (step S20) of the camera 1 will be described with reference to FIG. FIG. 5 is a flowchart showing the playback operation of the camera 1.
First, in step S <b> 201, the camera 1 reads image data recorded on the recording medium 245. At this time, the camera 1 reads image data corresponding to the latest photographed image.

In subsequent step S <b> 202, the camera 1 displays an image corresponding to the read image data on the display unit 26. FIG. 6 is an example of an image displayed on the display unit 26. In the following, as an example, processing for the image G1 shown in FIG. 6 will be described. The image G1 includes a subject image having faces F _{01 to} F ₀₅ .
In step S203, the face detector 230a detects the faces F _{01 to} F ₀₅ from the image displayed on the display unit 26.

In step S204, the eye region detection unit 230b detects an eye region and a pupil region from the detected faces F _{01 to} F ₀₅ . In particular, the eye region detection unit 230b obtains the color information in the range of the face based on the face area information of each face F ₀₁ to F _05, and white (color of white eye) the boundary with the surrounding skin color This boundary is detected as the outline of the eye region. Subsequently, the eye region detection unit 230b detects a black (or brown or gray) region from the eye region, and uses the contour of the black region as the contour of the pupil region. The eye region detection unit 230b generates position coordinates of the contours of the eye region and the pupil region as eye region information.

In step S205, the eye line determination unit 230d determines whether or not the eye lines of the faces F _{01 to} F ₀₅ are facing the front based on the eye area information.
This eye-gaze determination method will be described with reference to FIG. FIG. 7 is an enlarged view showing a part of the subject image including the face F ₀₅ shown in FIG. Usually, in both the left and right eyes, when the pupil regions P _R and P _L are approximately in the center of the eye regions E _R and E _L , the eyes appear to face the front. Therefore, the pupil position calculating unit 230c, the distance L _{R (OUT} to the end of the outer center from the eye region E _R of the pupil region P _R of right as viewed from the subject image (left in the drawing sheet) (eye area side) _a) pupils and inner center from the eye region E _R of the right pupil region P _R (distance to the end of the inner corner side) L _{R (iN),} the left as viewed from the subject image (right in the drawing sheet) a region P _L the distance from the center to the inside edge of the eye region E _L of the _L L _(iN), the distance from the center of the left pupil region P _L to the outer edge of the eye region E _L L _{L (OUT )} And the evaluation value W is calculated using the following equation (1).

The eye line determination unit 230d determines the eye line based on the value of the evaluation value W and the following equation (2). That is, when the evaluation value W satisfies the following equation (2), it is determined that the line of sight faces the front.
0.8 ≦ W ≦ 1.2 (2)
When the evaluation value W is equal to 1.0, the pupil regions P _R and P _L are at the centers of the respective eye regions E _R and E _L , and the eye line is facing directly in front. Further, when the evaluation value W is within the range of the above formula (2), the pupil regions P _R and P _L are substantially in the center of the respective eye regions E _R and E _L , and the eye line is substantially facing the front. become.

  In step S206, the face detection frame synthesizing unit 230e generates a face detection frame surrounding each face range based on the face range information, and adds a display attribute according to the determination result in step S205 to synthesize the image. .

  FIG. 8 is a table showing correspondence between eye-gaze determination results and face detection frame display attributes. As shown in FIG. 8, the display attributes of the face detection frame include line thickness, line color, line type, frame shape, blinking cycle, and the like. As the thickness of the line, for example, a thin line, a thick line, or a middle line can be used. For a face that is determined to have the eye line deviated from the front (looking away), the face that the user wants to draw attention to. For example, a thick frame should be combined. As the color of the line, green, red, yellow, or the like can be used, and it is preferable to synthesize a red frame on the face that the user wants to attract attention. As the line type (1), a continuous line, a broken line, a one-dot chain line, or the like can be used. As another line type (2), a single line, a double line, a triple line, or the like can be used, and a frame (for example, a triple line frame) in which the number of lines to be overlapped is increased on the face that the user wants to attract attention. ). As the shape of the frame, a circle, a quadrilateral, a pentagon or more polygon (for example, a hexagon) or the like can be used. Frame). As the blinking cycle, lighting (not blinking), blinking at 4 Hz, blinking at 1 Hz, or the like can be applied.

  The face detection frame composition unit 230e has, for example, a fine line, green, continuous line, single line, circular shape, or lighting, or a display attribute that combines these for a face that is determined to have a line of sight. Combining face detection frames. Further, the face detection frame composition unit 230e has, for example, a bold line, a red line, a broken line, a triple line, a quadrangular shape, or blinking of 4 Hz, or a display attribute that combines these for a face determined to look away. Combining face detection frames. Furthermore, as will be described later, for the face after editing the line of sight, for example, a midline, yellow, alternate long and short dash line, double line, hexagonal shape, or blinking of 1 Hz, or a combination of these display attributes is provided. A face detection frame having the same is synthesized. In order to cope with various users (for example, color weakness), it is preferable to combine a plurality of types of display attributes.

FIG. 9 is a schematic diagram illustrating an image G1 in which face detection frames are combined. In FIG. 9, for example, solid lines (or solid lines and green, etc.) face detection frames a ₀₂ and a ₀₃ are combined with the subject images F ₀₂ and F ₀₃ for which it is determined that the eyes are facing the front. On the other hand, for the subject images F ₀₁ , F ₀₄ , F ₀₅ determined that the line of sight is deviated from the front (looking away), for example, a broken line (or broken line and red, etc.) face detection frame b ₀₁ , B ₀₄ , b ₀₅ are synthesized. Note that the camera 1 provides a warning display area M in the image G1, and when there is a face determined to be looking away, the warning display area M displays a message warning the user to that effect. good.

  In step S207, the camera 1 determines whether or not the cross button 18 has been operated (the operation signal has been input from the cross button 18). When the cross button 18 is operated (step S207: Yes), the camera 1 changes the image displayed on the display unit 26 (step S220). Specifically, when a signal is input from the upper button 18 a of the cross button 18, the camera 1 reads image data corresponding to the image 10 frames before from the recording medium 245. When a signal is input from the lower button 18b, image data corresponding to the image after 10 frames is read from the recording medium 245. When a signal is input from the left button 18c, image data corresponding to the previous frame image is read from the recording medium 245. Further, when a signal is input from the right button 18d, image data corresponding to the image after one frame is read from the recording medium 245.

  On the other hand, when the cross button 18 is not operated (step S207: No), the camera 1 determines whether the enlargement button 13a or the reduction button 13b is operated (step S208). When the enlarge button 13a or the reduce button 13b is operated (step S208: Yes), the camera 1 changes the display range of the image displayed on the display unit 26 (step S230). Specifically, when a signal is input from the enlarge button 13a, the camera 1 reduces the display range of the image displayed on the display unit 26 to a predetermined range in the center, and the predetermined range is displayed on the display unit 26. Zoom in. On the other hand, when a signal is input from the reduction button 13b, the camera 1 enlarges the range of the image displayed on the display unit 26 to a predetermined range on the outer peripheral side, and reduces the predetermined range on the display unit 26. At this time, if the image is already displayed on the display unit 26 at the same magnification, the camera 1 arranges a plurality of thumbnail images (for example, arranged in 2 rows, 2 columns, 3 rows, 3 columns, 4 rows, 4 columns, etc.). ) Display the index image to be displayed (see FIG. 22 described later). Furthermore, when the camera 1 detects a signal from the cross button 18, it is displayed on the display unit 26 according to the direction of the detected buttons (up button 18a, down button 18b, left button 18c, right button 18d). Move the image display range.

  When the enlarge button 13a or the reduce button 13b is not operated (step S208: No), the camera 1 determines whether the edit button 17 is operated (step S209). When the edit button 17 has been operated (step S209: Yes), the camera 1 manually edits the face line of the face detected from the displayed image (step S250).

  On the other hand, when the edit button 17 is not operated (step S209: No) and the eye button 16 is operated (step S210: Yes), the camera 1 automatically detects the face eye detected in the displayed image. Alternatively, batch editing is performed (step S270).

  Thereafter, when the playback button 14 is operated (step S211: Yes), the operation of the camera 1 returns to the main routine. On the other hand, when the playback button 14 is not operated (step S211: No), the operation of the camera 1 returns to step S202.

  Next, manual editing of the line of sight (step S250) will be described in detail with reference to FIG. FIG. 10 is a flowchart showing the manual editing operation of the eye line of the camera 1. In the following, processing for the image G1 shown in FIG. 9 will be described as an example.

First, in step S251, the camera 1 determines whether or not the touch panel 12 is touched by an object such as a user's finger or a touch pen. When the touch panel 12 is touched (step S251: Yes), the camera 1 determines whether any of the faces displayed in the image G1 is touched (step S252). Specifically, the eye line editing processing unit 230 determines in which range of the face detection frames a _{01 to} a ₀₅ the touched position coordinates are included.

  When any face is touched (step S252: Yes), the pupil editing unit 230g acquires the eye area information of the touched face (step S253). Specifically, the face selection unit 230f selects a face detection frame including the touched position coordinates, and the pupil editing unit 230g acquires eye area information corresponding to the selected face detection frame. At this time, the face detection frame composition unit 230e may change the display attribute of the selected face detection frame so that the user can recognize the currently selected face detection frame. For example, the line width of the selected face detection frame may be increased or blinked.

  In step S <b> 254, the camera 1 determines whether or not there is a touch movement by an object on the touch panel 12. When there is a touch movement (step S254: Yes), the pupil editing unit 230g edits the pupil region according to the touch trajectory (step S255).

As a specific example, a case where the object image F ₀₅ corresponding to the face detection frame a ₀₅ in the image G1 shown in FIG. 6 is selected. FIG. 11 is an enlarged view of the area of the face detection frame a ₀₅ . When editing the pupil region, it is preferable to enlarge the image in advance by operating the enlargement button 13a in step S208 (step S230).

As shown in FIG. 11, the pupil regions P _R and the left pupil region P _L of the right, respectively, since the end portion of the right eye area E _R and the left eye area E _L, the face F ₀₅ is Looking Seems to be not facing the front.

For such images, the user touches the object (right side of the pupil region P _R in FIG. 11) one of the pupil region by (own finger 11) 2, to move the position of a touch in the horizontal direction Perform the operation. In response to this, the pupil editing unit 230g moves the touched pupil region to a position corresponding to the touch trajectory. Specific image processing is as follows. First, the pupil editing unit 230g, as shown in FIG. 12 (a), fill initial pupil region P _R in the same color as the surrounding white. Subsequently, as shown in (b) of FIG. 12, the center coordinates movement of the touch is stopped, to draw a circle whose radius is the radius r of the original pupil region P _R, and the initial pupil region P _R Fill with the same color. This is the pupil area P _R after the movement. The movement range of the pupil region P _R is limited to the eye area E _R. Therefore, the pupil editing unit 230g, when the trajectory of the touch exceeds the eye area E _R, does not execute the process of moving the pupil region P _R. Alternatively, such a case may be disposed pupil region P _R to the end of the eye region E _R of the excess side.

Here, in general, the width h _c in the vicinity of the center of the eye region E _R, larger than the width h _e in the vicinity of the end portion. Therefore, the pupil region P _R that were in the end of the eye region E _R, is moved to the vicinity of the center left width d _1, gap gap _1, gap ₂ between the eye area E _R and below the pupil area P _R It will occur. Therefore, the pupil editing unit 230g is expanded to fit the pupil region P _R in the width h _c, or, by extending to draw around the pupil area P _R in the same color as the end region of the pupil area P _R , clearance gap ₁ between the eye area E _R, gap ₂ generates a pupil region P _R 'interpolated (see FIG. 12 (c)).

Furthermore, the pupil editing unit 230g includes the touched side pupil region P _L on the opposite side (left side in FIG. 11), is moved in the horizontal direction in conjunction with the pupil region P _R. At this time, the position of the pupil region P _L, the ratio of the distance from the center of the pupil region P _L to the opposite ends of the eye region E _L is the distance from the center C of the pupil region P _R 'to both ends of the eye region E _R L _1, is determined to be equal to the ratio of L _2. Thereby, the direction of the line of sight in the right eye and the left eye is aligned. In addition, when a gap with the eye area E _L is generated above and below the moved pupil area P _L , the pupil area P _L is enlarged or expanded to interpolate the gap with the eye area E _L. A region P _L ′ is generated. Thereby, the eyes of the face F ₀₅ can be directed to the front.

In step S <b> 256, the camera 1 displays the edited image edited by the line-of-sight editing processing unit 230 on the display unit 26. At this time, the face detection frame synthesis unit 230e changes the display attribute of the face detection frame synthesized with the edited face (see FIG. 8). FIG. 13 shows an image in which the pupil region of the face F ₀₅ is edited. In the image G2 shown in FIG. 13, a face detection frame c _{05 of,} for example, a one-dot chain line (or one-dot chain line and yellow, etc.) indicating that it is after editing is combined with the face F ₀₅ .

  In step S257, the camera 1 determines whether or not the enter button 19 has been operated. When the enter button 19 is operated (step S257: Yes), the camera 1 records the image data corresponding to the edited image on the recording medium 245 with a file name different from that of the image data corresponding to the original image (step S257). S258).

  On the other hand, when the touch panel 12 is not touched (step S251: No), or when an area other than the face on the touch panel 12 is touched (step S252: No), the edit button 17 is operated (step S261: Yes). ), The operation of the camera 1 returns to the main routine. On the other hand, when the edit button 17 is not operated (step S261: No), the camera 1 waits for the touch panel 12 to be touched.

  Further, when the movement of the touch is not detected (step S254: No), or when the determination button 19 is not operated (step S257: No), when the edit button 17 is operated (step S262: Yes), the camera 1 The operation returns to the main routine. In this case, the pupil position editing is discarded.

On the other hand, when the edit button 17 is not operated (step S262: No), the operation of the camera 1 returns to step S254. In this case, the user can edit the pupil position again by a touch operation on the touch panel 12. Pupil editing unit 230g is when performing pupil editing process after the first time, (in the above description, the right side of the pupil region P _R) 1 time touched side as pupil areas on the same side in a movement operation of a touch relative to In this case, the touched pupil area is moved according to the touch trajectory, and the other pupil area is also moved in conjunction with the touched locus. On the other hand, the pupil editing unit 230g responds to the touch trajectory when a touch movement operation is performed on the pupil region opposite to the first touched side (in the above description, the left pupil region P _L ). Only the pupil area touched is moved, and the other is not linked. Thereby, the user can finely adjust the position of the pupil region (pupil region P _L ) moved in conjunction with the touch operation on the other pupil region (pupil region P _R ).

  Next, the automatic or batch editing of the line of sight (step S270) will be described in detail with reference to FIGS. FIG. 14 is a display example of a menu screen for selecting automatic or batch editing of eyes. FIG. 15 is a flowchart showing an automatic or batch editing operation of the line of sight of the camera 1. In the following, processing for the image G1 shown in FIG. 9 will be described as an example.

In step S <b> 271, the camera 1 displays a line-of-sight editing menu for the displayed image on the display unit 26. The line-of-sight editing menu M ₁ shown in FIG. 14 includes an icon I ₁ for selecting a mode (individual automatic editing mode) for automatically editing the eyes of each face individually, and a mode for automatically editing the eyes of all faces ( An icon I ₂ for selecting (automatic edit mode), an icon I ₃ for selecting a mode (manual batch edit mode) for manually editing all face eyes at once, and a cancel icon I ₄ It is displayed. The user can select desired icons I _{1 to} I ₄ by operating the upper button 18 a, the lower button 18 b, and the determination button 19 or touching the touch panel 12 with respect to the eye line editing menu M _1. it can. As a result, a selection signal for selecting a user-desired eye edit mode is input.

When the camera 1 detects a selection signal for the individual automatic editing mode (step S272: Yes), the camera 1 subsequently determines whether or not the touch panel 12 has been touched (step S273). When the touch panel 12 is touched (step S273: Yes), the camera 1 further determines whether or not the face determined to be looking away is touched (step S274). Specifically, the eye line editing processing unit 230 determines in which range of the face detection frames b ₀₁ , b ₀₄ , and b ₀₅ the touched position coordinates are included.

  When the face determined to be looking away is touched (step S274: Yes), the eye line editing processing unit 230 automatically edits the pupil region of the touched face (step S275). Specifically, first, the face selection unit 230f selects the touched face, and the pupil position calculation unit 230c calculates the center coordinates of the eye area of the selected face. Subsequently, the pupil editing unit 230g moves the pupil region in the horizontal direction so that the center coordinate of the pupil region matches the center coordinate of the eye region. The specific image processing is the same as that described in step S254. Furthermore, when a gap is generated between the pupil region and the eye region, the pupil editing unit 230g interpolates the gap by expanding or expanding the pupil region (see FIG. 12C).

In step S276, the camera 1 displays the edited image edited by the pupil editing unit 230g on the display unit 26. At this time, the face detection frame synthesis unit 230e changes the display attribute of the face detection frame synthesized with the face after editing of the pupil region. Thereby, an image in which the pupil region of the face F ₀₅ is automatically edited is generated. At this time, for the face F ₀₅ , a face detection frame c _{05 of,} for example, a one-dot chain line (or one-dot chain line and yellow, etc.) indicating that it is after editing is synthesized as in FIG.

  In step S277, the camera 1 determines whether or not the enter button 19 has been operated. When the enter button 19 is operated (step S277: Yes), the camera 1 records the image data corresponding to the edited image on the recording medium 245 with a file name different from that of the image data corresponding to the original image (step S277). S278).

  On the other hand, when the touch panel 12 is not touched (step S273: No), when a face with a line of sight is not touched (step S274: No), or when the enter button 19 is not operated (step S277: No), the camera 1 Then, it is determined whether or not the edit button 17 has been operated (step S279). When the edit button 17 is operated (step S279: Yes), the operation of the camera 1 returns to the main routine. On the other hand, when the edit button 17 is not operated (step S279: No), the operation of the camera 1 returns to step S273.

  When the selection signal for the individual automatic editing mode is not detected in step S272 (step S272: No), the camera 1 determines whether or not the selection signal for the fully automatic editing mode is detected (step S280).

When the selection signal for the fully automatic editing mode is detected (step S280: Yes), the eye line editing processing unit 230 automatically edits the position of the pupil region for all the faces that are looking away (step S281). Specifically, first, the face selection unit 230f selects all the faces that are looking away (faces F ₀₁ , F ₀₄ , F ₀₅ in FIG. 9), and the pupil position calculation unit 230c selects each of the selected faces. The center coordinate of the eye area is calculated. Subsequently, the pupil editing unit 230g moves the pupil region in the horizontal direction so that the center coordinate of the pupil region matches the center coordinate of the eye region for each face. The specific image processing is the same as that described in step S254. Furthermore, when a gap is generated between the pupil region and the eye region, the pupil editing unit 230g interpolates the gap by expanding or expanding the pupil region (see FIG. 12C).

In step S282, the camera 1 displays the edited image edited by the pupil editing unit 230g on the display unit 26. At this time, the face detection frame synthesis unit 230e changes the display attribute of the face detection frame synthesized with the face after editing of the pupil region. FIG. 16 shows an image in which the pupil regions of the faces F ₀₁ , F ₀₄ and F ₀₅ are automatically edited. In the image G3 shown in FIG. 16, for the faces F ₀₁ , F ₀₄ , F ₀₅ , the face detection frames c ₀₁ , c ₀₄ , c of, for example, one-dot chain lines (or one-dot chain lines and yellow etc.) indicating that they are after editing. ₀₅ is synthesized.

  In step S283, the camera 1 determines whether or not the enter button 19 has been operated. When the determination button 19 is operated (step S283: Yes), the operation of the camera 1 proceeds to step S278. On the other hand, when the enter button 19 is not operated (step S283: No), the camera 1 subsequently determines whether or not the edit button 17 is operated (step S284). When the edit button 17 is operated (step S284: Yes), the operation of the camera 1 returns to the main routine. In this case, the editing of the pupil area is discarded. On the other hand, when the edit button 17 is not operated (step S284: No), the operation of the camera 1 returns to step S283.

  When the selection signal for the fully automatic editing mode is not detected in step S280 (step S280: No), the camera 1 enters the manual batch editing mode (step S285). Here, the manual batch edit mode is an edit in which the same editing is performed for all other faces when the user manually edits the pupil region for any of the faces detected in the image. It is a mode.

In this case, in step S286, the camera 1 determines whether or not the touch panel 12 has been touched. When the touch panel 12 is touched (step S286: Yes), the camera 1 determines whether any of the faces displayed in the image is touched (step S230). Specifically, the eye-line editing processing unit 230 determines whether the touched position coordinates are included in the range of the face detection frames b ₀₁ , a ₀₂ , a ₀₃ , b ₀₄ , and b ₀₅ .

  When any face is touched (step S287: Yes), the pupil editing unit 230g acquires the eye area information of the touched face (step S288). Specifically, the face selection unit 230f selects a face detection frame including the touched position coordinates, and the pupil editing unit 230g acquires eye area information corresponding to the selected face detection frame.

  In subsequent step S 289, the camera 1 determines whether or not there has been a touch movement on the touch panel 12. When there is a touch movement (step S289: Yes), the pupil editing unit 230g edits the pupil region according to the touch trajectory (step S290). Detailed processing in step S290 is the same as that in step S255.

In step S291, the pupil editing unit 230g further edits the pupil regions of all other faces in the same manner as the pupil region editing for the selected face. Specifically, the pupil editing unit 230g acquires distances L ₁ and L ₂ between the center of the edited pupil region and both ends of the eye region (see FIG. 12C), and the ratio L ₁ / L ₂ between the _two. Is calculated. Then, also in other faces, the pupil region is moved so that the ratio between the center of the pupil region and both ends of the eye region becomes the same ratio L ₁ / L ₂ . Furthermore, when a gap is generated between the pupil region and the eye region, the pupil editing unit 230g enlarges or expands the pupil region and interpolates the gap (see FIG. 12C).

In step S292, the camera 1 displays the edited image edited by the pupil editing unit 230g on the display unit 26. At this time, the face detection frame synthesis unit 230e changes the display attribute of the face detection frame synthesized with the face after editing of the pupil region. FIG. 17 shows an image edited so that the eyes of all the faces F _{01 to} F ₀₅ are directed rightward (leftward toward the paper surface). In the image G4 shown in FIG. 17, with respect to the face F ₀₁ to F _05, and the face detection frame c ₀₁ to c _05, for example, a chain line indicates a post-edit (or one-dot chain line and yellow, etc.) have been synthesized .

  In step S293, the camera 1 determines whether or not the enter button 19 has been operated. When the determination button 19 is operated (step S293: Yes), the operation of the camera 1 proceeds to step S278.

  On the other hand, when the touch panel 12 is not touched (step S286: No), or when there is no touch on the face (step S287: No), the camera 1 determines whether the edit button 17 has been operated (step S294). . When the edit button 17 is operated (step S294: Yes), the operation of the camera 1 returns to the main routine. On the other hand, when the edit button 17 is not operated (step S294: No), the camera 1 waits for the touch panel 12 to be touched.

  When there is no touch movement (step S289: No), or when the enter button 19 is not operated (step S293: No), the camera 1 determines whether the edit button 17 is operated (step S295). ). When the edit button 17 is operated (step S295: Yes), the operation of the camera 1 returns to the main routine. On the other hand, when the edit button 17 is not operated (step S295: No), the operation of the camera 1 returns to step S286.

  As described above, according to the first embodiment, since the pupil region is moved with respect to the eye region of the face detected from the image, it is possible to generate an image in which the line of sight of the subject image is in a desired direction. It becomes possible.

  Further, according to the first embodiment, in the mode in which the eye line is individually manually edited, the pupil area is moved to a desired position by a touch operation on the touch panel 12, so that the position of the pupil area can be finely adjusted. . In addition, since the pupil area on the side not touched is moved in conjunction with the pupil area on the side touched first, the left and right eyes can be easily aligned. Further, when moving the non-touched pupil area, the other pupil area is fixed, so that fine adjustment of the line of sight is possible.

  Further, according to the first embodiment, in the mode for automatically editing the eye line, the pupil area is arranged at the center of the eye area, so that the eye line can be directed to the front by a simple operation.

  In the first embodiment, the face to be edited is selected by touching the touch panel 12, but the face to be edited is selected by selecting the face detection frame by operating the cross button 18 and the decision button 19. You may do it.

(Modification 1-1)
In the first embodiment, when manually editing the line of sight, the pupil area is directly touched and moved with a finger, a touch pen, or the like, but the pupil area to be moved and the touch position do not necessarily match. Also good. For example, as shown in FIG. 18, a touch area _AT for editing a pupil area may be provided in an area other than the face F _{05 to be} edited (near the face F ₀₅ or a predetermined area on the touch panel 12). In this case, when the user touches the face to be edited (for example, the face F ₀₅ ) in advance and designates the editing position, and then performs a touch operation on the editing touch area _AT , the pupil editing unit 230g depending on the locus of the touch on the a _T, to move the pupil region in the face of the edited (face F ₀₅₎ (e.g., pupil area P _05). According to the first modification, the user can move the touch position while observing the pupil region P ₀₅ without being obstructed by his / her finger or the like, so that fine adjustment of the pupil region P ₀₅ is easily performed. be able to.

(Modification 1-2)
When manually editing the line of sight, the pupil region may be moved using an operation member other than the touch panel 12. For example, selection of a pupil region to be moved and input of a movement direction and a movement amount may be performed by a button operation on the cross button 18, or a pointing device such as a mouse or a touch pad is separately provided, and the pupil region is provided by the pointing device. May be selected and moved.

(Modification 1-3)
The line-of-sight editing processing unit 230 determines whether or not the edited image looks natural with respect to the pupil region manually edited by the user in the manual editing mode of the line of sight (step S250). May notify the user to that effect.

For example, in FIG. 7, if the centers C _R and C _L of the pupil regions P _R and P _L are both outside the centers of the respective eye regions E _R and E _L , the left and right eyes are separated from each other. It looks unnatural as if they were looking at each other. To prevent such an image is generated, Looking determining unit 230d, the pupil region P _R, the center of the P _L C _R, the distance L _{R (OUT} from C _L eye area E _R, until both ends of the E _{L )} , L _{R (IN)} , L _{L (IN)} , and L _{L (OUT)} are acquired, and determination is performed using the following equations (3) and (4).
L _{R (OUT)} / (L _{R (OUT)} + L _{R (IN)} ) <0.5 (3)
L _{L (OUT)} / (L _{L (OUT)} + L _{L (IN)} ) <0.5 (4)

  When the position of the pupil region after editing satisfies both equations (3) and (4), an unnatural image is obtained. Therefore, when editing is performed so as to satisfy the above expressions (3) and (4), the eye line editing processing unit 230 changes the display attribute of the face detection frame combined with the edited face, or displays a warning message. Or displayed on the display unit 26.

(Modification 1-4)
In Embodiment 1 described above, the eyes of the face in the image are directed to the front, but the eyes may be directed to a direction other than the front.
In this case, in the eye-gaze manual editing mode, the pupil region P _R (or P _L ) may be moved to a desired position by a touch operation on the face to be edited.

Further, in the automatic line-of-sight or batch edit mode, for example, in the image G1 shown in FIG. 9, a face with the line of sight facing the user's desired direction (for example, face F ₀₁ with the line of sight directed to the right) is selected by touch Then, the pupil region may be edited so that all other faces (for example, faces F _{02 to} F ₀₅ ) are directed in the same direction as the selected face. In this case, the ratio of the distance between the center of the pupil area and both ends of the eye area in the unselected face is equal to the ratio of the distance between the center of the pupil area and both ends of the eye area in the selected face. Edit the pupil area.

(Modification 1-5)
In the first embodiment, the display attribute of the face detection frame combined with the image is changed to notify the user of the eye gaze determination result. In addition to this, the eye gaze determination result is transmitted to the user by various means. It is good also as notifying. For example, a warning mark may be combined and displayed on a face that has been determined that the line of sight is not facing the front, or a warning display may be displayed that colors and impresses white eyes.

(Modification 1-6)
In the first embodiment, it is determined whether or not the line of sight of the subject image is facing the front based on the position of the pupil area in the eye area in the horizontal direction. The direction of the line of sight may be determined based on the position of the pupil region in the vertical direction. In this case, it is possible to determine not only looking away in the horizontal direction but also looking away in the vertical direction. Further, when editing the eye line looking away in the vertical direction, the pupil region may be moved to a substantially central position in the vertical direction of the eye region.

(Embodiment 2)
Next, a second embodiment of the present invention will be described.
FIG. 19 is a block diagram illustrating a configuration of a line-of-sight editing processing unit included in the imaging apparatus according to Embodiment 2. As illustrated in FIG. 19, the eye line editing processing unit 230-2 further includes a warning display combining unit 230h with respect to the eye line editing processing unit 230 illustrated in FIG.

  Note that the overall configuration of the imaging apparatus according to Embodiment 2 is the same as that of the camera 1 shown in FIGS. In addition, the overall operation and reproduction operation of the imaging apparatus according to Embodiment 2 are the same as those shown in FIGS.

FIG. 20 is a flowchart showing a live view display operation (step S06 in FIG. 4) of the imaging apparatus according to the second embodiment.
First, in step S301, the image processing circuit 229 takes in the image signal output from the image sensor 221.

In subsequent step S302, the image processing circuit 229 performs predetermined image processing on the captured image signal, and generates display image data that is continuously reproduced at a fixed minute time interval.
In step S303, the face detection unit 230a detects a face from each image subjected to image processing, and generates face range information corresponding to the detected face.

  When one or more faces are detected from the image as a result of the face detection (step S304: Yes), the eye area detection unit 230b detects an eye area and a pupil area from each face based on the face range information, Eye area information is generated (step S305).

  In step S306, the line-of-sight determination unit 230d determines whether the line of sight of each face is facing the front based on the eye area information. The details of the eye-gaze determination method are the same as in step S205.

  In step S307, the face detection frame synthesizing unit 230e generates a face detection frame (see FIG. 8) having a display attribute corresponding to the eye determination result, and synthesizes it with the image. The face detection frame synthesis unit 230e also functions as a warning display synthesis unit that synthesizes a warning display according to the determination result of the eye determination unit 230d with an image.

  As a result of the determination of the line of sight, if there is a face whose line of sight is deviated from the front (looking away) (step S308: Yes), the warning display combining unit 230h warns that there is a person looking away. A display is generated and combined with the image (step S309). At that time, the warning display composition unit 230h may include the number of people looking away in the warning display. Specifically, the warning display composition unit 230h creates a warning sentence such as “There are n people looking away” for display in the warning display area M shown in FIG.

In step S310, the display unit 26 displays an image in which the face display frame and the warning display are combined as a live view image. For example, in the case of the image G1 shown in FIG. 9, the face F _01, F _04, since the eyes of F ₀₅ is out from the front, the face detection frame b ₀₁ corresponding to these face _{F 01, F 04, F 05} , b ₀₄ and b ₀₅ are displayed by, for example, a broken line (or a broken line and a red color, etc.) indicating looking away. In the warning display area M at the upper right of the image G1, a warning that there is a person who is looking away is displayed. This warning includes the number n of people who are looking away (that is, the number of face detection frames having attributes corresponding to looking away; n = 3 in the image G1).

  On the other hand, when no face is detected from the image (step S304: No), or when a face looking away is not detected (step S308: No), the operation proceeds to step S310.

  In such a live view image, the presence / absence or content of warning display (for example, the number of people n described in the warning text in the message area M) changes every moment according to the state of the subject's eyes. Therefore, the user refers to the warning display, and the timing when the warning display is not displayed (that is, when there is no person who is out of sight) or when the number n of people who are looking away during the warning display decreases. Thus, the release button 10 can be operated to take an image. In the warning display, the display color of the warning text may be changed to red, green, or the like according to the number of people who are out of sight. For example, if the number of people out of line of sight is two or more, the display color is changed to red when the number is one, and green when the number is one, and the user can immediately grasp the content of the warning (number of people n) The release button 10 can be operated at an appropriate timing.

(Modification 2)
Next, Modification 2 will be described. FIG. 21 is a block diagram illustrating a configuration of an image processing circuit included in an imaging apparatus according to Modification 2.
As illustrated in FIG. 21, the image processing circuit 229-2 according to Modification 2 further includes a warning information addition unit 230-3 and an image selection unit 230-4 in addition to the eye line editing processing unit 230-2.

The warning information adding unit 230-3 adds warning information to the image data indicating that there is a person looking away when the second release switch is turned on at the timing when the warning is displayed on the live view image. To do.
The image selection unit 230-4 selects image data based on the warning information from the image data recorded on the recording medium 245 in accordance with operation inputs from the operation button groups 13a to 19-19.

  Specifically, the image selection unit 230-4 selects, for example, only image data to which warning information is added when reproducing and displaying a captured image. In response to this, the video signal output circuit 233 sequentially displays or lists images corresponding to the selected image data on the display unit 26. FIG. 22 is a schematic diagram illustrating an example in which images G11 to G14 to which warning information is added are displayed as a list (index display). At this time, in the images G11 to G14, a face detection frame synthesized with a face determined to be looking away may be displayed.

In this case, since the user can observe only the image that needs to be edited, the user can efficiently edit the eye. Alternatively, the user can easily select and delete only images that are out of line of sight.
In this case, since the eye area and the pupil area have already been detected for all the faces, the eye line editing process can be performed at high speed.

On the other hand, you may make it the image selection part 230-4 select only the image data to which the warning display is not added. In this case, only the images to which no warning information is added (that is, the person who is looking away is not detected) are sequentially displayed or displayed as a list on the display unit 26. FIG. 23 is a schematic diagram illustrating an example in which images G21 to G24 to which warning information is not added are displayed as a list.
In this case, the user can efficiently observe only an image that does not require line-of-sight editing.

  In the second embodiment and the second modification, it is determined whether or not the subject's line of sight is facing the front. However, by changing the calculation formula of the evaluation value W and the threshold value used for the determination, It can also be determined whether or not the line of sight is in a desired direction.

  In Embodiments 1 and 2 described above, the imaging apparatus 1 has been described as a digital camera. However, the imaging apparatus 1 is applied to various electronic devices having a photographing function such as a single-lens digital still camera, a digital video camera, and a camera-equipped mobile phone. be able to.

1 Imaging device (camera)
DESCRIPTION OF SYMBOLS 10 Release button 11 Mode dial 12 Touch panel 13a Enlarge button 13b Reduce button 14 Play button 15 Menu button 16 Eye button 17 Edit button 18 Cross button 18a Up button 18b Down button 18c Left button 18d Right button 19 Enter button 20 Power switch 26 Display part DESCRIPTION OF SYMBOLS 29 Accessory port 100 Interchangeable lens 101 Shooting optical system 101a, 101b Shooting lens 105 Optical position detection mechanism 107 Optical system drive mechanism 109 Aperture drive mechanism 110 Encoder 111 Lens controller 112 Distance ring 200 Camera body 203 Shutter 205 Dustproof filter 207 Piezoelectric element 209 Red Outer cut filter 210 Optical low-pass filter 211 Dust-proof filter drive circuit 213 Shutter drive mechanism 215 Barometric pressure sensor unit 221 Image sensor 223 Image sensor drive Path 225 Preprocessing circuit 226 Contrast AF circuit 227 Sequence controller 229 Image processing circuit 230, 230-2 Eye edit processing unit 230a Face detection unit 230b Eye region detection unit 230c Eye position calculation unit 230d Eye determination unit 230e Face detection frame synthesis unit 230f Face selection unit 230h Warning display synthesis unit 230g Pupil editing unit 230-3 Warning information addition unit 230-4 Image selection unit 231 Compression / decompression circuit 233 Video signal output circuit 235 Display unit drive circuit 237 SDRAM control circuit 239 Input / output circuit 241 Communication circuit 243 Recording medium control circuit 245 Recording medium 247 Flash memory control circuit 249 Flash memory 253 Switch detection circuit 259 Detachment detection switch 261 Data bus 290 Image sensor unit 292 Shake correction drive mechanism 94 actuator drive circuit 297 shake detection section 297a angular velocity sensor 297b angular velocity detecting circuit 300 communication contacts

Claims (26)

Display means for displaying an image corresponding to electronic image data of a subject;
Face detection means for detecting a face from the image;
Eye area detecting means for detecting an eye area and a pupil area from the range of the face detected by the face detecting means;
Pupil editing means for performing editing including changing the position of the pupil region in the eye region;
Recording means for recording image data edited by the pupil editing means;
An image recording apparatus comprising:   2. The image according to claim 1, further comprising eye line determination means for determining whether or not the eye line of the face faces a predetermined direction based on a position of the pupil area in the eye area. Recording device. The predetermined direction is a front direction;
The image recording apparatus according to claim 2, wherein the line of sight determination unit determines that the line of sight is facing the front when the pupil region is substantially in the center of the eye region.   The image recording apparatus according to claim 2, further comprising a notification unit that notifies the determination result of the eye line determination unit.   The said notification means is a face detection frame synthesizing means which has a display attribute determined according to a determination result by the eye line determination means, and combines a face detection frame surrounding the face with the image. 5. The image recording apparatus according to 4.   6. The image recording apparatus according to claim 5, wherein the face detection frame synthesizing unit synthesizes the face detection frame having two or more types of display attributes with the image. A face selection unit that selects a face determined by the line-of-sight determination unit that the line of sight is not facing a predetermined direction;
The image recording apparatus according to claim 2, wherein the pupil editing unit edits the pupil region of the face selected by the face selection unit. Operation input means for inputting a signal corresponding to an operation from the outside to the image recording apparatus;
Face selection means for selecting the selected face when a selection signal for selecting any of the faces detected by the face detection means is input by the operation input means;
Further comprising
The image recording apparatus according to claim 1, wherein the pupil editing unit edits an area of the pupil in the face selected by the face selection unit. It further comprises operation input means for inputting a signal corresponding to an operation from the outside to the image recording apparatus,
The image recording apparatus according to claim 1, wherein the pupil editing unit moves the pupil region in the eye region based on a signal input by the operation input unit. The operation input means is a touch panel provided on the display means,
The image recording apparatus according to claim 9, wherein the pupil editing unit moves the pupil region based on a trajectory of touching the pupil region with respect to the touch panel. The operation input means is a touch panel provided on the display means,
The image recording apparatus according to claim 9, wherein the pupil editing unit moves the pupil region based on a touch trajectory to a region other than the pupil region on the touch panel.   12. The pupil editing unit according to claim 9, wherein when the operation input unit receives a signal for moving one pupil region, the pupil editing unit moves the other pupil region in conjunction with each other. The image recording apparatus according to claim 1.   The pupil editing unit moves the other pupil region in conjunction with the one pupil region, and then receives a signal for moving the other pupil region by the operation input unit. 13. The image recording apparatus according to claim 12, wherein only the other pupil region is moved without interlocking the one pupil region. Pupil position calculating means for calculating the position of the pupil region when the eye line is directed in a predetermined direction in the eye region;
The image recording apparatus according to claim 1, wherein the pupil editing unit moves the pupil region detected by the eye region detection unit to a position calculated by the pupil position calculation unit.   When the pupil editing unit moves the pupil region and then creates a gap with the eye region above and below the pupil region, the pupil editing unit expands or expands the pupil region to reduce the gap. The image recording apparatus according to claim 12, wherein interpolation is performed. Imaging means for capturing an image of a subject via a photographing lens to obtain electronic image data;
Release means for instructing shooting,
Further comprising
The recording unit records image data corresponding to an image acquired by performing shooting in accordance with a shooting instruction by the release unit, and stores image data corresponding to the image edited by the pupil editing unit as an original image. The image recording apparatus according to claim 1, wherein the image recording apparatus records the file with a file name different from that of the data. Imaging means for capturing an image of a subject via a photographing lens to obtain electronic image data;
Release means for instructing shooting,
Display means for displaying an image corresponding to the image data;
Image processing means for generating a live view image that is continuously reproduced at a fixed minute time interval;
A recording means for recording an image obtained by photographing according to a photographing instruction by the release means;
Face detection means for detecting a face from the image;
Eye area detecting means for detecting an eye area and a pupil area from the range of the face detected by the face detecting means;
Line-of-sight determination means for determining whether the line of sight of the face is in a predetermined direction based on the position of the area of the pupil in the area of the eye;
Warning display combining means for combining the image with a warning display according to the determination result by the eye line determination means;
An imaging apparatus comprising: The predetermined direction is a front direction;
18. The imaging apparatus according to claim 17, wherein the eye line determination unit determines that the eye line is facing the front when the pupil region is substantially in the center of the eye region.   The warning display combining unit is a face detection frame combining unit that has a display attribute determined according to a determination result by the eye line determination unit, and combines a face detection frame surrounding the face with the image. The imaging device according to claim 17 or 18.   20. The imaging apparatus according to claim 19, wherein the face detection frame synthesizing unit synthesizes the face detection frame having two or more types of display attributes with the image.   21. The imaging apparatus according to claim 19, wherein the warning display combining unit displays the number of the face detection frames having a predetermined attribute.   The imaging apparatus according to any one of claims 17 to 21, further comprising warning information adding means for adding warning information according to a determination result by the eye line determination means to image data corresponding to the image. .   The imaging apparatus according to claim 22, further comprising image selection means for selecting an image to be displayed on the display means based on the warning information.   24. The imaging apparatus according to claim 23, wherein the image selection unit selects an image to which the warning information is added.   The imaging apparatus according to claim 23, wherein the image selection unit selects an image to which the warning information is not added.   26. The pupil editing means for performing editing including changing the position of the pupil area in the eye area on the image selected by the image selection means. The imaging apparatus of Claim 1.

Images