JP5846420B2 - Image processing apparatus, image management method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image management method, and a program.

As a conventional technique, there is known a technique for classifying a person in an image by performing face detection again on an image in which face identification cannot be performed at the time of photographing or an image in which the face identification function is not used (for example, patents). Reference 1). The conventional technique performs face detection again for an image in which face identification cannot be performed or an image in which the face identification function is not used, and records the newly detected face person in association with an existing predetermined person. and it is intended to refer to recorded in association as a new person.

JP 2010-178160 A

  However, in the above-described prior art, once an image that has been mistakenly identified as a specific person is not classified again, the problem is that the image of the erroneously classified person is recorded as it is. was there.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus, an image management method, and a program capable of improving the accuracy of specifying an individual from a subject in an image.

In order to solve the above problems, the present invention provides an image acquisition means for acquiring an image, simple feature information for recognizing whether a subject in an image acquired by the image acquisition means is a specific individual, A first storage unit storing detailed feature information for recognizing whether or not a subject in the image acquired by the image acquisition unit is a specific individual; and the first storage unit stores the feature information. was based on the simple feature information, based the first recognition means for recognizing the specific individual from the subject, to the particular individual body that is recognized by the first recognition means in said acquired image Based on the detailed feature information corresponding to the specific individual read from the first storage means and the detailed feature information read by the read means, the first recognition means recognizes the feature information. Drawn An image processing apparatus characterized by the subject of the inner is characterized in that it comprises a second recognizing means for recognizing whether said particular individual.

  According to the present invention, the accuracy of recognizing a specific individual from a subject in an image can be improved.

It is a block diagram which shows the structure of the digital camera by embodiment of this invention. In the digital camera by this embodiment, it is a conceptual diagram which shows an example of the management table in which personal information is registered. It is a schematic diagram which shows an example of the screen at the time of setting personal information with the digital camera by this embodiment, or changing a setting content. It is a conceptual diagram which shows an example of simple face feature information and detailed face feature information in the digital camera by this embodiment. 6 is a flowchart for explaining an operation (setting process) of the digital camera according to the present embodiment. 5 is a flowchart for explaining an operation at the time of imaging in the digital camera according to the present embodiment. 5 is a flowchart for explaining an operation of image segmentation processing in the digital camera according to the present embodiment. FIG. 3 is a conceptual diagram illustrating an example of a desired image in the digital camera according to the present embodiment. It is a conceptual diagram which shows an example of the segmented image which is not desired.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. Configuration of Embodiment FIG. 1 is a block diagram showing a configuration of a digital camera according to an embodiment of the present invention. In the figure, the digital camera 1 includes an imaging unit 2, an imaging control unit 3, an image processing unit 4, a CPU 7, a RAM 8, a ROM 9, a recording unit 10, a display unit 11, and a key input unit 12.

  The imaging unit 2 includes a lens group such as a focus lens and a zoom lens, and an imaging device (CCD or the like), and adjusts autofocusing, aperture, zooming, and the like according to a control signal from the imaging control unit 3, and the lens group by the imaging device. The light of the subject projected via is converted into an electrical signal and output as an imaging signal. The imaging control unit 3 controls the imaging operation of the imaging unit 2 under the control of the CPU 7.

  The image processing unit 4 performs image processing (pixel interpolation processing, γ correction, generation of luminance and color difference signals, white balance processing, exposure correction processing, etc.) of image data captured by the imaging unit 2, and compression / decompression of image data (for example, , JPEG format, Motion-JPEG format or MPEG format compression / decompression) processing, and processing for combining a plurality of captured images.

  In particular, in the embodiment of the present invention, the image processing unit 4 includes a face detection unit 20, a face analysis unit 21, an identifier setting unit 22, and an image erasing unit 23. The face detection unit 20 detects a person's face from the through image. The face analysis unit 21 analyzes the face of a person in the captured image based on simple face feature information or detailed face feature information, and identifies the person. Here, in this embodiment, a person is specified by face detection, but an individual other than a person such as an animal may be specified.

  The identifier setting unit 22 classifies an undesired image in which a specific person is not captured based on the analysis result by the face analysis unit 21 performed using detailed facial feature information on the captured image. , An identifier is given to the undesired image. The image erasure unit 23 erases the divided undesired images in accordance with a user instruction.

  The CPU 7 is a one-chip microcomputer that controls each part of the digital camera 1. In particular, in the present embodiment, when the shutter is half-pressed and fully pressed, the CPU 7 captures and saves the through-image when a specific person is present as a subject, Control unit 4 and the like. In addition, the CPU 7 controls the image processing unit 4 to determine whether or not a specific person is captured with respect to the captured image in order to classify an undesired image from the plurality of captured images. To do.

  The RAM 8 is used as a buffer memory for temporarily storing image data picked up by the image pickup unit 2 and is used as a working memory for the CPU 7. The ROM 9 stores a program necessary for controlling each part of the digital camera 1 by the CPU 7 and data necessary for controlling each part. The CPU 7 performs processing according to this program.

  The recording unit 10 records the image data captured by the imaging unit 2. The recording unit 10 also includes a management table 24 for registering and setting a face image of a specific person, face feature information used for face recognition, and the like.

  The display unit 11 includes a color LCD and its driving circuit, and displays the subject imaged by the imaging unit 2 as a through image when in the imaging standby state, and is read out from the recording unit 10 when reproducing the recorded image. Display the expanded recorded image. The key input unit 12 includes a plurality of operation keys such as a shutter SW (switch), a zoom SW, a mode key, a SET key, and a cross key, and outputs an operation signal according to a user's key operation to the CPU 7.

  In this embodiment, in order to prevent missed shooting, in addition to full pressing, when a specific person registered in advance exists as a subject by recognizing a face against a through image in a half-pressed state of the shutter, It is designed to focus on a specific person's face and save it. However, at the time of imaging, face recognition is performed using simple face feature information in order to reduce processing. An analysis flag for designating a specific person is prepared and can be set for each person registered in advance.

  When the image is captured in this way, since a person is specified using simple face feature information, a person who is not the person may actually be captured. That is, an unnecessary image (an image in which a desired person is not captured) may be stored in the captured image. Therefore, in this embodiment, in order to easily categorize and delete unnecessary images, the captured image is reanalyzed using the detailed facial feature information of the specific person, and the specified person is the subject. If it does not exist, it is classified and deleted as necessary.

  Further, in this embodiment, when a plurality of pre-registered persons exist as subjects at the time of shooting, in order to specify which person is preferentially focused, a priority is set for each pre-registered person. It is possible to set.

  FIG. 2 is a conceptual diagram illustrating an example of a management table in which personal information is registered in the digital camera according to the present embodiment. In the management table 24, a registered person's face image, simple facial feature information used for face recognition for determining whether a specific person registered in advance exists as a subject, and a specific person exists in the captured image. Detailed face feature information used for face recognition for determining whether to perform, a priority for designating which person to focus on preferentially, and an analysis flag for designating a specific person are set.

Face image, time of registration, or a previously captured face image of the subject, the extraction of the characterizing face (and personally identifiable) facial feature information (simple facial feature information, and detailed facial feature information) It is used as a thumbnail when a specific subject is designated when registering personal information.

Simple face feature information is extracted and registered at the time when a face image is registered, and is used to specify a subject at the time of photographing. Detailed facial feature information is extracted and registered when a facial image is registered, and is used to determine whether a specific person exists in the captured image. The priorities are in the order of 1 <2 <3 <4 <5, and subjects with higher priorities are preferentially recognized as main subjects. Note that the priorities may overlap. Further, the priority of the subject not registered in the apparatus is determined as the lowest level “1” or lower. The analysis flag is used to designate a specific person, and is designated by “ON” and not designated by “OFF”.

  FIG. 3 is a schematic diagram illustrating an example of a screen when setting personal information or changing setting contents in the digital camera according to the present embodiment. When the user performs a predetermined operation to set personal information, a setting screen as shown in FIG. 3 is displayed. On the setting screen, the user sets a priority and an analysis flag (“ON” or “OFF”) indicating that the person performs face recognition.

  FIG. 4 is a conceptual diagram showing an example of simple face feature information and detailed face feature information in the digital camera according to the present embodiment. In the present embodiment, simple face feature information used at the time of photographing and detailed face feature information for reanalysis are registered for each person. Simple face feature information includes features of major parts such as a person's eyebrows, eyes, nose, and mouth. On the other hand, detailed facial feature information includes features such as a forehead and cheeks in addition to main parts such as a person's eyebrows, eyes, nose, and mouth.

B. Operation of Embodiment Next, the operation of the above-described embodiment will be described.
FIG. 5 is a flowchart for explaining the operation (setting process) of the digital camera according to the present embodiment. First, when the setting mode is designated by the user, as shown in FIG. 3, a setting screen for displaying a list of face data of previously registered persons is displayed, and a face is selected from the face data of previously registered persons. An analysis flag for designating a person to be recognized and an in-focus priority are notified (step S10). On the setting screen, the user sets an analysis flag and a priority for a person who is to perform face recognition.

  Next, it is determined whether or not all the focus priority and analysis flags are set (step S12). And when all are not set (NO of step S12), it returns to step S10. On the other hand, if all the settings have been made (YES in step S12), the setting contents are saved (step S14), and the process ends.

  FIG. 6 is a flowchart for explaining the operation at the time of imaging in the digital camera according to the present embodiment. When the shooting mode is designated by the user, first, the CPU 7 determines whether or not there is an imaging instruction including half-press and full-press of the shutter (step S20). If there is no imaging instruction (NO in step S20), the process waits until there is an imaging instruction.

  On the other hand, if there is an imaging instruction (YES in step S20), the face detection unit 20 and the face analysis unit 21 use the simple facial feature information to identify the person whose analysis flag is turned on for the through image. Identification (step S22), and the imaging controller 3 focuses on the face of a person with high priority among the identified persons (step S24), and identifies the captured image. The information of the person who has been added is added and recorded in the recording unit 10 (step S26). The person information includes the coordinates of the person, identification information (No. in FIG. 2), name, and the like.

  Next, it is determined whether or not imaging has ended (step S28). If imaging has not ended (NO in step S28), the process returns to step S20 and the above-described processing is repeated. On the other hand, when the imaging is finished (YES in step S28), the process is finished.

  If it is presumed that the person whose analysis flag is set to ON in advance will exist as a subject even if the shutter is half-pressed by the above imaging process, the person's information is added and the captured image is saved. Will be. Therefore, it is possible to prevent a mistake that a photo opportunity is missed.

  FIG. 7 is a flowchart for explaining the operation of the image segmentation process for segmenting / deleting an image not desired by the user from a stored image in the digital camera according to the present embodiment. is there. When the image segmentation mode is designated by the user, first, an image that is a candidate for segmentation is read from the recording unit 10 (step S30). Next, the face analysis unit 21 selects a face corresponding to a specific person in the image based on information (coordinates, identification information, name, etc.) of the specific person (one or more) associated with the read image. Then, re-analysis is performed using the corresponding detailed face feature information (step S32. After that, the face analysis unit 21 determines the person whose detailed face feature information does not match among the identified persons based on the reanalysis result. Is determined (step S34).

  If the detailed facial feature information matches the person among the identified persons (NO in step S34), the desired person has been imaged, and the process is terminated as it is. In this case, the image is not divided into images to be deleted.

  On the other hand, if there is a person whose detailed facial feature information does not match among the specified persons (YES in step S34), since the desired person has not been imaged, is the corresponding image displayed and classified? A message to the effect is displayed (step S36). Then, the user confirms the displayed image, determines whether or not to classify it as an image to be deleted, and gives an instruction.

  Here, in the present embodiment, as will be described later, if there is an instruction to delete a classified image, the corresponding image is deleted, but the present invention is not limited to this. For example, if there is no person whose detailed facial feature information matches among the specified persons, the person information added to the face image in S26 may be deleted from the associated image.

  The CPU 7 determines whether or not there is a classification instruction (step S38). If there is a classification instruction from the user (YES in step S38), the identifier setting unit 22 indicates an identifier indicating that the image is classified. (Step S40). In addition, you may make it move the said image to a specific folder instead of providing an identifier. On the other hand, when there is no classification instruction from the user (NO in step S38), the process proceeds without doing anything.

  Thereafter, it is determined whether or not there is a next segmentation candidate image (step S42). If there is a next segmentation candidate image (YES in step S42), the process returns to step S30 and becomes the target of the next segmentation candidate. The image is read and the above processing is repeated.

  On the other hand, if there is no next segmentation candidate image (NO in step S42), it is determined whether an instruction to delete the segmented image has been detected (step S44). The user designates an image to be deleted by checking a list of classified images. And when the deletion instruction | indication of the divided image is not detected (NO of step S44), the said process is complete | finished. On the other hand, when an instruction to delete the segmented image is detected (YES in step S44), the image erasing unit 23 deletes the image that has been instructed to delete (step S46), and the process ends.

  FIG. 8 is a conceptual diagram illustrating an example of a desired image in the digital camera according to the present embodiment. FIG. 9 is a conceptual diagram illustrating an example of a segmented image that is not desired. As shown in FIG. 8, in the captured image 100, the image is focused on the face of the person in the frames 101, 102, and 103, and the person identified by the person in the frame 102 (No in FIGS. 2 and 3). .3 person). In this case, since the face feature information in the frame 101 matches the detailed face feature information (data 103 in FIG. 2) of the person, the image is determined to be a desired image and is not divided into images to be deleted.

  On the other hand, as shown in FIG. 9, in the captured image 200, the image is focused on the face of the person in the frames 201, 202, and 203, and the person identified by the person in the frame 202 (FIG. 2). , No. 3 person in FIG. 3). However, the person in the frame 202 recognized by the simple face recognition information at the time of shooting is a different person. Therefore, the face feature information of the person in the frame 202 does not match the detailed face feature information of the person (data 103 in FIG. 2). Therefore, the image 200 is divided into images to be deleted. An image classified as a deletion target is deleted according to a user instruction.

  According to the present embodiment, as described above, an image that has not been picked up is re-analyzed using detailed facial feature information to classify an undesired image. Image management can be facilitated. This can also improve the accuracy of identifying an individual (person or animal) from the subject in the image.

As mentioned above, although several embodiment of this invention was described, this invention is not limited to these, The invention described in the claim, and its equal range are included. For example, in the above-described embodiment, the image processing apparatus to which the present invention is applied has been described as an example configured as the digital camera 1. However, the present invention is not particularly limited to this, and can be applied to general electronic devices. For example, the present invention can be widely applied to personal computers, portable navigation devices, portable game machines, and the like.
Below, the invention described in the claims of the present application is appended.

(Appendix 1)
The invention according to appendix 1 includes image acquisition means for acquiring an image, simple feature information and detailed feature information for recognizing whether a subject in the image acquired by the image acquisition means is a specific individual On the basis of simple feature information stored in the first storage means, and a first storage means for storing the information in association with identification information for identifying the specific individual First recognition means for recognizing the specific individual from the subject, and reading means for reading detailed feature information corresponding to the specific individual from the first storage means based on the identification information of the specific individual And second recognition means for recognizing whether the subject in the image recognized by the first recognition means is the specific individual based on the detailed feature information read by the reading means. To prepare An image processing apparatus characterized.

(Appendix 2)
According to the second aspect of the present invention, when the subject in the image recognized by the second recognition unit is not recognized as the specific individual, the first identifier is set for the image. The image processing apparatus according to appendix 1, further comprising setting means.

(Appendix 3)
The invention according to attachment 3 further comprises second storage means for storing the identification information of the specific individual recognized by the first recognition means in association with the image acquired by the image acquisition means, The reading means reads detailed feature information corresponding to the specific individual from the first storage means based on the identification information of the specific individual stored in association with the image by the second storage means. The image processing apparatus according to appendix 1 or 2, characterized by:

(Appendix 4)
In the invention according to appendix 4, the second identifier for identifying the object to be recognized by the first recognition unit is stored in the first storage unit in association with the specific individual. Second setting means for setting the feature information and the detailed feature information, and the first recognition means uses the second setting means to detect from the subject in the image acquired by the image acquisition means. 4. The image processing apparatus according to any one of appendices 1 to 3, wherein a specific individual set with a second identifier is recognized.

(Appendix 5)
The image processing apparatus according to any one of appendices 2 to 4, further comprising image erasing means for erasing an image in which the first identifier is set by the first setting means. It is.

(Appendix 6)
The invention according to appendix 6 includes notification means for informing to instruct whether or not to delete the image set with the first identifier by the first setting means, and the image notified by the notification means. Detecting means for detecting an instruction to erase, wherein the image erasing means has a first identifier set by the first setting means when an instruction to erase the image is detected by the detecting means The image processing apparatus according to appendix 5, wherein the image is erased.

(Appendix 7)
The invention according to appendix 7 is the image processing apparatus according to any one of appendices 1 to 6, further comprising an imaging unit, wherein the image acquisition unit acquires an image captured by the imaging unit. .

(Appendix 8)
The invention according to attachment 8 includes an image acquisition step of acquiring an image, simple feature information and detailed feature information for recognizing whether a subject in the image acquired by the image acquisition step is a specific individual Is stored in the storage means in association with identification information for identifying the specific individual, and based on the simple feature information, in the image acquired in the image acquisition step A first recognition step for recognizing the specific individual from a subject; a reading step for reading detailed feature information corresponding to the specific individual from the storage means based on the identification information of the specific individual; And a second recognition step for recognizing whether the subject in the image recognized in the first recognition step is the specific individual based on the detailed feature information that has been issued. It is an image managing method according to claim.

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Imaging part 3 Imaging control part 4 Image processing part 7 CPU
8 RAM
9 ROM
DESCRIPTION OF SYMBOLS 10 Recording part 11 Display part 12 Key input part 20 Face detection part 21 Face analysis part 22 Identifier setting part 23 Image erasing part 24 Management table

Claims (10)

Image acquisition means for acquiring images;
Simple feature information for recognizing whether the subject in the image acquired by the image acquisition unit is a specific individual and whether the subject in the image acquired by the image acquisition unit is a specific individual First storage means for storing detailed feature information for
First recognition means for recognizing the specific individual from a subject in the acquired image based on simple feature information stored in the first storage means;
Reading means for reading out detailed feature information corresponding to the specific individual from the first storage unit based on the specific individual recognized by the first recognizing unit;
Second recognition means for recognizing whether the subject in the image recognized by the first recognition means is the specific individual based on the detailed feature information read by the reading means. An image processing apparatus.   The image processing according to claim 1, wherein the first storage unit stores the simple feature information and the detailed feature information in association with identification information for identifying a specific individual. apparatus.   The apparatus further comprises first setting means for setting a first identifier for the image when the subject in the image recognized by the second recognition means is not recognized as the specific individual. The image processing apparatus according to claim 1 or 2. A second storage means for storing the identification information of the specific individual recognized by the first recognition means in association with the image acquired by the image acquisition means;
The reading unit reads detailed feature information corresponding to the specific individual from the first storage unit based on identification information of the specific individual stored in association with the image by the second storage unit. The image processing apparatus according to claim 2, wherein the image processing apparatus is an image processing apparatus. Simple feature information and detailed feature information stored in the first storage means in association with the specific individual, the second identifier for identifying the object to be recognized by the first recognition means And a second setting means for setting to
The first recognition unit recognizes a specific individual set with a second identifier by the second setting unit from a subject in an image acquired by the image acquisition unit. The image processing apparatus according to any one of 1 to 4. The image processing apparatus according to claim 3 , further comprising an image erasing unit that erases an image for which the first identifier is set by the first setting unit. An informing means for instructing whether or not to erase the image set with the first identifier by the first setting means;
Detection means for detecting an instruction to erase the image notified by the notification means,
The image erasing unit erases the image set with the first identifier by the first setting unit when an instruction to erase the image is detected by the detecting unit. The image processing apparatus described. Further comprising an imaging means,
The image processing apparatus according to claim 1, wherein the image acquisition unit acquires an image captured by the imaging unit. An image acquisition step of acquiring an image;
Simple feature information for recognizing whether the subject in the image acquired by the image acquisition step is a specific individual, and whether the subject in the image acquired by the image acquisition step is a specific individual A first storage control step for storing in the storage means detailed feature information for recognition;
A first recognition step for recognizing the specific individual from a subject in the image acquired in the image acquisition step based on the simple feature information;
Based on the specific individual recognized by the first recognition step, a reading step of reading detailed feature information corresponding to the specific individual from the storage unit;
A second recognition step for recognizing whether the subject in the image recognized in the first recognition step is the specific individual based on the read detailed feature information;
An image management method comprising: A computer of an image processing apparatus provided with an image acquisition means for acquiring an image and a storage means,
Simple feature information for recognizing whether the subject in the image is a specific individual, and detailed feature information for recognizing whether the subject in the image acquired by the image acquisition means is a specific individual First storage control means for storing the above in the storage means,
First recognition means for recognizing the specific individual from the subject in the image acquired by the image acquisition means based on the simple feature information stored in the first storage control means;
Based on the specific individual recognized by the first recognition unit, a reading unit that reads detailed feature information corresponding to the specific individual from the storage unit;
A second recognizing unit for recognizing whether the subject in the image recognized by the first recognizing unit is the specific individual based on the detailed feature information read by the reading unit;
A program characterized by functioning as

Images