JP4888191B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus capable of detecting a face area of a subject from an image.

2. Description of the Related Art Conventionally, electronic cameras that detect the face of a subject in a shooting screen and perform AF (autofocus) in accordance with the face are known. For example, Patent Document 1 discloses an example of a configuration of an electronic camera having a face detection function.
JP 2004-317699 A

However, a conventional electronic camera having a face detection function performs AF and AE (automatic exposure) on the closest face when a plurality of faces are detected. Therefore, there is still room for improvement in that the user's intention may not be reflected in the shooting using the face detection function when the user takes a plurality of persons as main subjects.
The present invention is to solve the above-mentioned problems of the prior art. An object of the present invention is to provide means for further improving the convenience of a user who uses a face detection function in a scene where a plurality of main subjects are photographed.

An imaging apparatus according to a first invention includes an imaging unit, a face detection unit, an operation unit, a first control unit, and a second control unit. The imaging unit captures a subject and generates image data. The face detection unit detects the face area of the subject from the image. The operation unit accepts an operation from the user. The first control unit performs a grouping process for classifying each face region into one of at least two groups when the face detection unit detects a plurality of face regions. The second control unit executes a predetermined process based on the designated group designated by the above operation from among a plurality of groups. Further, the first control unit executes the grouping process based on the size of each face area in the same image data.

In a second aspect based on the first aspect, the second control unit performs focusing control based on the designated group, automatic exposure control based on the designated group, auto white balance control based on the designated group, and designation. At least one of generation of metadata regarding the face area in the group is executed.
According to a third aspect, in the first or second aspect , the operation unit receives a designation input of a face area to be a main subject from a user. Further, the first control unit executes the grouping process based on the position of the face area of the main subject.

In a fourth aspect based on any one of the first to third aspects, the first control unit executes grouping processing based on a plurality of algorithms. In addition, when there is a group change input from the operation unit, the first control unit presents the result of the grouping process obtained by an algorithm different from that before the change input to the user .

In a fifth aspect based on any one of the first to fourth aspects, the imaging apparatus further includes a monitor that displays a confirmation image indicating a correspondence relationship between each face region and the group.

  In the imaging apparatus of the present invention, a grouping process for classifying each face region into a plurality of groups is performed, and various processes are performed based on an arbitrary designated group, and a scene in which a plurality of main subjects are photographed Improve user convenience.

FIG. 1 is a block diagram showing the configuration of the electronic camera of this embodiment.
The electronic camera includes an imaging optical system 11, a lens driving unit 12, a diaphragm 13, a diaphragm driving unit 14, an imaging element 15, an AFE 16, an image processing unit 17, a buffer memory 18, and a recording I / F 19. , A monitor 20, a release button 21, an operation unit 22, a CPU 23, and a system bus 24.

Here, the image processing unit 17, the buffer memory 18, the recording I / F 19, the monitor 20 and the CPU 23 are connected to each other via a system bus 24. Further, the lens driving unit 12, the aperture driving unit 14, the release button 21, and the operation unit 22 are connected to the CPU 23, respectively.
The imaging optical system 11 includes a plurality of lens groups including a zoom lens and a focusing lens. The lens position of the focusing lens of the imaging optical system 11 is adjusted in the optical axis direction by the lens driving unit 12. For simplicity, the imaging optical system 11 is illustrated as a single lens in FIG.

The diaphragm 13 adjusts the amount of incident light from the imaging optical system 11. The aperture amount of the diaphragm 13 is adjusted by the diaphragm drive unit 14.
The imaging element 15 is disposed on the image space side of the imaging optical system 11. The light receiving elements are two-dimensionally arranged on the light receiving surface of the imaging element 15. The imaging element 15 generates an analog image signal by photoelectrically converting a subject image by a light beam that has passed through the imaging optical system 11. The output of the image sensor 15 is connected to the AFE 16.

  Here, in the photographing mode which is one of the operation modes of the electronic camera, the image sensor 15 captures a recorded image (main image) in response to the full pressing operation of the release button 21. Further, the imaging element 15 in the shooting mode captures a through image at predetermined intervals even during standby for shooting. The through image data is output by thinning out reading from the image sensor 15 and is used for image display on the monitor 20 and various arithmetic processes by the CPU 23.

The AFE 16 is an analog front end circuit that performs analog signal processing on the output of the image sensor 15. The AFE 16 performs correlated double sampling, image signal gain adjustment, and image signal A / D conversion. The output of the AFE 16 is connected to the image processing unit 17.
The image processing unit 17 performs various types of image processing (color interpolation processing, gradation conversion processing, contour enhancement processing, white balance adjustment, etc.) on the image signal of the main image or the through image. Note that the image processing unit 17 also executes compression processing or decompression processing of the data of the main image.

The buffer memory 18 temporarily records image data in the pre-process and post-process of image processing by the image processing unit 17.
A connector for connecting the recording medium 25 is formed in the recording I / F 19. The recording I / F 19 executes data writing / reading with respect to the recording medium 25 connected to the connector. The recording medium 25 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the recording medium 25.

The monitor 20 displays various images according to instructions from the CPU 23. Note that the configuration of the monitor 20 in the present embodiment may be any of an electronic viewfinder having an eyepiece and a liquid crystal display panel provided on the back surface of the housing.
On the monitor 20, a through image is displayed as a moving image under the control of the CPU 23 during shooting standby in the shooting mode. At this time, the CPU 23 can superimpose a display of various information necessary for photographing on the through image on the monitor 20 by an on-screen function.

The release button 21 receives from the user an instruction input for AF operation before photographing by a half-press operation and an instruction input for starting an imaging operation by a full-press operation.
The operation unit 22 includes, for example, a command dial, a cross-shaped cursor key, a determination button, a registration button, and the like. The operation unit 22 receives various inputs of the electronic camera from the user.

The CPU 23 is a processor that comprehensively controls each unit of the electronic camera. Here, the CPU 23 of the present embodiment functions as an imaging control unit 26, a face detection unit 27, and a grouping processing unit 28 by executing a program stored in a ROM (not shown).
The imaging control unit 26 executes a known contrast AF calculation, a known AE calculation, a white balance gain calculation during auto white balance, and the like based on the output of the through image. The imaging control unit 26 also generates metadata recorded in the header area of the image file in accordance with the Exif (Exchangeable image file format for digital still cameras) standard.

  The face detection unit 27 performs face detection processing on the through image data to detect a human face area included in the through image. This face detection process is performed by a known algorithm. As an example, the face detection unit 27 extracts feature points such as end points of eyebrows, eyes, nose, and lips from the image by a known feature point extraction process, and determines whether or not the face region is based on these feature points. judge. Alternatively, the face detection unit 27 may obtain a correlation coefficient between a face image prepared in advance and a determination target image, and may determine a face region when the correlation coefficient exceeds a certain threshold.

  When the face detection unit 27 detects a plurality of face regions, the grouping processing unit 28 can execute a grouping process for classifying each face region into a group. The grouping processing unit 28 can include a plurality of face regions in one group during the grouping process. Note that the shooting control unit 26 of the present embodiment can execute AF, AE, and auto white balance based on the above group.

Hereinafter, an operation example in the shooting mode of the electronic camera of the present embodiment will be described with reference to the flowchart of FIG. In the example of FIG. 2, description will be made on the assumption that the face detection function in the shooting mode is set to ON.
Step 101: The CPU 23 drives the image sensor 15 to start capturing a through image. Thereafter, the through image is sequentially generated by the image sensor 15 at predetermined intervals. Further, the CPU 23 displays the through image on the monitor 20 as a moving image. Therefore, the user can perform framing for determining the shooting composition based on the through image of the monitor 20.

  Step 102: The CPU 23 performs face detection processing on the data of the through image and detects a face area included in the through image. When the face area is detected in S102, the CPU 23 displays the position of the detected face area on the monitor 20. As an example, the CPU 23 superimposes and displays a rectangular frame at the position of the face area that can be detected in the through image by the on-screen function (see FIG. 3).

Step 103: The CPU 23 determines whether or not a face area is detected in the face detection process of S102. When the face area is detected (YES side), the CPU 23 proceeds to S104. On the other hand, when the face area is not detected (NO side), the CPU 23 proceeds to S116.
Step 104: The CPU 23 determines whether or not a plurality of face areas are detected in the face detection process of S102. When a plurality of face areas are detected (YES side), the CPU 23 proceeds to S105. On the other hand, when the detected face area is one (NO side), the CPU 23 proceeds to S113.

Step 105: The CPU 23 executes the grouping process of the face areas detected in S102, classifies each face area and generates a plurality of groups.
More specifically, the CPU 23 in S105 executes the grouping process according to either the following (1) or (2).
(1) First, the CPU 23 detects the size of each face area. Next, the CPU 23 sets an arbitrary one face area as a determination target. Then, the CPU 23 groups the face areas whose size difference is within the allowable range and whose distance on the screen is within the threshold with the determination target face area as a reference. On the other hand, when there is no face area that satisfies the above conditions, the CPU 23 generates a group including only the face area to be determined. Then, the CPU 23 changes the face area to be determined and repeats the above operation to group all face areas.

  (2) First, the CPU 23 receives a designation input of a face area to be a main subject from the user via the operation unit 22. Then, the CPU 23 groups together other face areas within a predetermined range from the face area of the main subject on the shooting screen. At this time, the CPU 23 may exclude the face areas having significantly different sizes from the grouping targets based on the size of the face area designated as the main subject. As a result, the CPU 23 can group the main subject and the faces of persons in the vicinity thereof together.

  Here, in the grouping process of S105, the CPU 23 can change the upper limit of the number of face regions included in one group in accordance with the user input. Alternatively, the CPU 23 may automatically change the upper limit of the number of face areas in one group according to the shooting conditions. As an example, the CPU 23 reduces the upper limit of the number of face areas in one group as the depth of field becomes shallower (that is, the more difficult it is to focus on a plurality of faces) according to the shooting angle of view and the aperture value.

Step 106: The CPU 23 automatically selects a designated group to be subjected to AF or the like from a plurality of groups generated by the grouping process.
For example, the CPU 23 selects a group including the largest face area or a group located at the center of the image as the designated group. Alternatively, when the grouping process is performed based on the position of the main subject designated by the user, the CPU 23 selects a group including the face area of the main subject as the designated group.

Step 107: The CPU 23 displays the result of the grouping process and the current designated group on the monitor 20. As an example, the CPU 23 superimposes and displays a group frame display indicating a group of face areas on the through image of the monitor together with a rectangular frame indicating the detected face area by using an on-screen function (see FIG. 4).
Here, in FIG. 4, the group frame display is indicated by a broken line. Note that the CPU 23 highlights the group frame indicating the designated group on the monitor 20 by changing the color and thickness of the other group frame.

Step 108: The CPU 23 determines whether or not the group reset input displayed in S107 has been received from the operation unit 22. If there is the above input (YES side), the CPU 23 proceeds to S109. On the other hand, if there is no input (NO side), the CPU 23 proceeds to S110.
Step 109: The CPU 23 executes the grouping process again with an algorithm different from that of S105. Thereafter, the CPU 23 returns to S106 and repeats the above operation.

For example, the CPU 23 in S109 changes the parameters relating to the number of face areas included in one group, the positional relationship and the size of the face areas to values different from those in S105. And CPU23 performs a grouping process again based on said parameter after a change.
Alternatively, when the grouping process is performed by the method (1) in S105, the CPU 23 in S109 executes the grouping process again by the method (2). Similarly, when the grouping process is performed by the method (2) in S105, the CPU 23 in S109 executes the grouping process again by the method (1).

Step 110: The CPU 23 determines whether or not a change input for the specified group is received from the operation unit 22. If there is an input for changing the designated group (YES side), the CPU 23 proceeds to S111. On the other hand, when there is no change input for the designated group (NO side), the CPU 23 proceeds to S112.
Step 111: The CPU 23 changes the designated group in accordance with a user operation.
For example, in the example of FIG. 4, the CPU 23 changes the group on the upper side of the screen to a designated group in accordance with a user operation. Thereby, the user can manually select a designated group from among a plurality of groups generated by the grouping process.

Step 112: The CPU 23 sets shooting conditions based on the designated group. The CPU 23 proceeds to S114 after the processing in S112.
First, the CPU 23 in S112 executes AF with the designated group as a reference, and brings all the face regions included in the designated group into a focused state. As an example, the CPU 23 obtains a focus evaluation value of each face area by contrast AF, and performs AF by regarding a lens position at which the focus evaluation values of all the face areas are equal to or greater than a threshold as a focus position. Note that the CPU 23 may control the aperture 13 as necessary to increase the depth of field so that all face areas are in focus. This makes it easy for the electronic camera to capture the main image while focusing on each face in the designated group.

Secondly, the CPU 23 in S112 performs AE calculation by spot metering based on the range of the designated group, and adjusts the exposure conditions (aperture value, exposure time, imaging sensitivity) of the main image. Thereby, the electronic camera can make the exposure of the face in the designated group appropriate in a scene with a large luminance difference, for example.
Thirdly, the CPU 23 in S112 executes auto white balance calculation based on the color information of the through image based on the designated group range, and determines the white balance gain to be applied to the main image. Thereby, the electronic camera can perform white balance on the basis of the faces in the designated group, for example, in a scene where the shooting screen is illuminated with a plurality of light sources.

Step 113: In this case, the CPU 23 executes AF control and AE calculation based on the detected face area. In this case, since only one face area is detected, the CPU 23 does not perform the grouping process.
Step 114: The CPU 23 determines whether or not a half-press operation of the release button 21 has been accepted. If the release button 21 is half pressed (YES side), the CPU 23 proceeds to S115. On the other hand, when the release button 21 is not half-pressed (NO side), the CPU 23 returns to S102 and repeats the above operation.

Step 115: In response to the half-pressing operation of the release button 21, the CPU 23 prohibits the focus adjustment control and performs AF lock. Thereafter, the CPU proceeds to S117.
Step 116: In this case, in response to a half-press operation of the release button 21 from the user, the CPU 23 executes AF and AE with the same algorithm (for example, center priority or closest priority) as in normal shooting. .

Step 117: The CPU 23 determines whether or not the release button 21 has received a full pressing operation. When the release button 21 is fully pressed (YES side), the CPU 23 proceeds to S119. On the other hand, when the release button 21 is not fully pressed (NO side), the CPU 23 proceeds to S118.
Step 118: The CPU 23 determines whether or not the half-press of the release button 21 is released. When the half-press of the release button 21 is released (YES side), the CPU 23 returns to S102 and repeats the above operation. On the other hand, when the half-press of the release button 21 is continued (NO side), the CPU 23 returns to S117 and repeats the above operation.

Step 119: The CPU 23 executes photographing processing for the main image. Specifically, the CPU 23 drives the image sensor 15 to capture the main image. Thereafter, the image processing unit 17 generates data of the main image.
Here, when the grouping process is performed, the image processing unit 17 may perform any one of the following image processing (1) to (6) on the data of the main image. . As a result, the electronic camera can generate a real image that is more faithful to the user's image.

(1) The image processing unit 17 changes the edge enhancement processing for the range of the designated group from another region. For example, the image processing unit 17 executes the contour enhancement process by applying a contour enhancement filter having a lower degree of contour enhancement than other regions to the range of the designated group. As a result, it is possible to obtain a preferable main image in which rough skin of a person as a main subject is not noticeable.
(2) The image processing unit 17 changes the tone correction characteristics of the designated group range from other regions. For example, the image processing unit 17 performs gradation correction by applying a gradation characteristic table having a gradation curve different from that of other areas to the range of the designated group of the main image. Specifically, a gradation characteristic table of a soft gradation curve in which contrast is relatively lower than the gradation curves of other areas is applied to the main subject area. As a result, it is possible to obtain a preferable main image having softness on the face of the main subject.

(3) The image processing unit 17 performs correction to increase the saturation of the range of the designated group more than other areas. For example, the image processing unit 17 sets the saturation parameter of the face area of the designated group relatively higher than other areas. As a result, it is possible to obtain a preferable main image in which the skin color dullness of the main subject person is reduced.
(4) The image processing unit 17 performs light amount correction processing for adjusting the gradation of the dark part or highlight part of the image based on the result of the image analysis for the range of the specified group. The above light amount correction processing is performed by, for example, a known technique disclosed in Japanese Patent Application Laid-Open No. 2006-5699. Specifically, the image processing unit 17 extracts a local fluctuation component from the main image and generates a local fluctuation image including the fluctuation component. Next, the image processing unit 17 obtains a modulation signal for modulating the main image into a local variation image and level-compresses the modulation signal. The image processing unit 17 then modulates the captured image with the level-compressed modulation signal to obtain a gradation corrected image. As a result, it is possible to obtain an image in a good state in which a gradation that is easily buried is visually restored while moderately suppressing a change in brightness of the correction range.

(5) The image processing unit 17 changes the noise reduction processing in the designated group range from another region. For example, the image processing unit 17 performs noise reduction processing using a median filter or the like for only the range of the specified group in the main image. As a result, the graininess of the face of the main subject can be reduced.
(6) The image processing unit 17 performs red-eye reduction correction by image processing only for the range of the designated group. The red-eye reduction correction is performed by a known technique. Specifically, the image processing unit 17 detects a red-eye area from the face area and performs correction to reduce the lightness or saturation of the detected red-eye area. As a result, red eyes of the main subject can be prevented.

Step 120: The CPU 23 generates an image file conforming to the Exif standard. At this time, the CPU 23 uses the work area of the buffer memory 18 to generate header data of the image file.
Here, when the grouping process is performed, the CPU 23 identifies the identification information indicating that the grouping process has been performed, the coordinate information indicating the position of the specified group, and the position of the face area included in the specified group. Record metadata such as information in the header. Note that the above metadata is recorded in the header using an Exif standard MakerNote tag. Then, the CPU 23 generates an image file by associating the main image data with the header data, and records the image file on the recording medium 25.

Here, the above metadata can be used for the following applications, for example. This greatly improves user convenience when using the image file of the main image after shooting.
(1) When playing back an image file with an electronic camera, the CPU 23 can search and sort images that have undergone grouping processing at the time of shooting based on the metadata.

(2) When trimming the person who is the main subject from the main image after shooting, the CPU 23 can grasp the position of the designated group and the position of the face area in the main image based on the metadata.
(3) When the image file is automatically reproduced by the electronic camera as a slide show, the CPU 23 can perform display with special effects based on the metadata. As an example, the CPU 23 can execute display such as zoom-up, zoom-down, image rotation, fade-in, and fade-out based on the position of the face in the designated group. Furthermore, the CPU 23 may divide and display the main image by the number of faces of the designated group in the above-described slide show display. For example, the CPU 23 performs the above-described divided display by means such as an enlarged display of the face of each main subject. Above, the description regarding FIG. 2 is complete | finished.

Hereinafter, the effect of this embodiment is demonstrated.
The electronic camera according to the present embodiment executes a grouping process for classifying each face area into a plurality of groups when a plurality of face areas are detected by the face detection process. Then, the electronic camera executes processing such as AF, AE, and auto white balance based on an arbitrary designated group. For this reason, in a scene where a plurality of persons are photographed as main subjects, all face areas in the designated group are the main processing targets, so that it is possible to easily obtain a main image in accordance with the user's intention. Particularly, since the electronic camera can change the designated group in accordance with the user's operation, for example, the designated group can be appropriately set even in a situation where a group of persons as a main subject is located behind the person in front.

  In addition, the electronic camera according to the present embodiment performs a grouping process based on the designation of a face area by the user and the detected face size. Therefore, the grouping of face areas is performed in a form close to the general user's intention. In addition, since the electronic camera can perform the grouping process again with a different algorithm in accordance with the user's operation, it is easy to realize the grouping process close to the user's intention.

(Supplementary items of the embodiment)
(1) In the above embodiment, the CPU 23 may perform the grouping process based on the subject distance. As an example, the CPU 23 drives the imaging device 15 while scanning the focusing lens in one direction to acquire a plurality of through images. Next, the CPU 23 acquires the contrast value of each face area in each through image by the same method as the AF calculation. The CPU 23 calculates the subject distance of each face area in the shooting screen based on the position of the focus lens that provides the highest contrast value for each face area. Then, the CPU 23 groups subjects whose subject distance difference is within a threshold value. At this time, if the position of the face area in the shooting screen is far, the CPU 23 may not group the face areas.

(2) When performing AF with reference to the designated group, the CPU 23 may execute AF with reference to any face area in the designated group. For example, the CPU 23 may perform AF on another face area in the designated group when the in-focus position cannot be detected in the face area initially targeted for focus detection.
(3) The CPU 23 may display each group by color-coding a rectangular frame (see FIG. 3) indicating the face area. The CPU 23 may superimpose a display indicating a group on the monitor 20 only when the user performs a predetermined operation on the operation unit 22. Further, after the grouping process, the CPU 23 may alternately display each group on the monitor 20 and select the group as the designated group when the user inputs a designation (these groups). In this case, illustration is omitted).

  (4) In the above embodiment, the example in which the face detection process and the grouping process are performed at the time of imaging with the electronic camera has been described. However, the present invention is not limited to the example of the above-described embodiment. For example, when an image file is played back with an electronic camera, face detection processing and grouping processing may be performed on the main image. Good. The present invention may also be implemented in an image processing apparatus such as a scanner or a computer that reads and reproduces image data.

  It should be noted that the present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

The block diagram which shows the structure of the electronic camera of this embodiment Flow chart for explaining an operation example in the shooting mode of the electronic camera Schematic diagram showing the display state of the through image during face detection Schematic diagram showing the display state of the through image after grouping processing

Explanation of symbols

DESCRIPTION OF SYMBOLS 15 ... Image sensor, 17 ... Image processing part, 20 ... Monitor, 22 ... Operation part, 23 ... CPU, 26 ... Shooting control part, 27 ... Face detection part, 28 ... Grouping process part

Claims (5)

An imaging unit that images a subject and generates image data;
A face detection unit for detecting a face area of the subject from the image;
An operation unit that receives operations from the user;
A first control unit that performs a grouping process for classifying each of the face regions into one of at least two or more groups when the face detection unit detects a plurality of the face regions;
A second control unit that executes a predetermined process based on a designated group designated by the operation among a plurality of the groups;
Equipped with a,
The imaging apparatus according to claim 1, wherein the first control unit executes the grouping process based on a size of each face area in the same image data . The imaging device according to claim 1,
The second control unit relates to focusing control based on the designated group, automatic exposure control based on the designated group, auto white balance control based on the designated group, and the face region in the designated group. An imaging apparatus that executes at least one of generation of metadata. In the imaging device according to claim 1 or 2,
The operation unit accepts designation input of the face area as a main subject from a user,
The imaging apparatus according to claim 1, wherein the first control unit executes the grouping process based on a position of the face area of the main subject. In the imaging device according to any one of claims 1 to 3,
The first control unit performs the grouping process based on a plurality of algorithms, and obtains the group using a different algorithm from that before the change input when the group change input is received from the operation unit. An image pickup apparatus that presents a result of the grouping process to a user. In the imaging device according to any one of claims 1 to 4,
An imaging apparatus, further comprising a monitor that displays a confirmation image indicating a correspondence relationship between each of the face regions and the group.

Images