JP2010178160A - Electronic camera and program for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic camera that also easily classifies images for which face identification is not carried out or face identification function is not applied when imaging the same. <P>SOLUTION: The electronic camera includes: an image sensor; a recording unit; a display; a face detection unit; a face identification unit; a first identification information appending unit; a face classification processing unit; a searching unit; an image display; a selecting condition display; and a selecting condition receiving unit. The face detection unit detects a face. The face identification unit extracts characteristic points thereof. The first identification information appending unit appends the identification information to the image data. The face classification processing unit classifies the image data based on the identification information. The searching unit searches the image data to which the identification information is not appended. The image display displays the image displayed on the display. The selecting condition display displays selecting conditions for selecting face figures. The selecting condition receiving unit receives selection or non-selection of the selecting conditions. The second identification information appending unit append the identification information in response to the selecting conditions to the image data of the displayed image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an electronic camera and an image processing program.

  2. Description of the Related Art Conventionally, an electronic camera equipped with a face identification function that extracts a facial feature point of a person from a photographed image and performs face identification has been disclosed (for example, see Patent Document 1).

  Also disclosed is an image correction device that reads an image file taken with an electronic camera, detects a face area of a person, performs face authentication, identifies each person, and corrects the image based on the identification result ( For example, see Patent Document 2).

  By the way, there is a case where it is desired to perform image processing on a specific person image after classifying the image showing the face for each person. However, the electronic camera disclosed in Patent Document 1 and the image correction apparatus disclosed in Patent Document 2 have a problem in handling when face identification cannot be performed even though the face area can be detected.

  Even if the user can identify the face by looking at the image, the image cannot be identified using the electronic camera, or the image is taken with priority given to other shooting modes, and the face is reflected. There is also a problem with handling images that do not use the face identification function.

  SUMMARY OF THE INVENTION In view of the above circumstances, an object of the present invention is to provide an electronic camera that can easily classify an image in which face identification cannot be performed at the time of shooting or an image in which a face identification function is not used.

  An electronic camera according to a first invention includes an imaging device, a recording unit, a display unit, a face detection unit, a face recognition unit, a first identification information addition unit, a face classification processing unit, a search unit, An image presenting unit, an image presenting unit, a selection condition presenting unit, and a selection condition receiving unit are provided. The imaging device generates image data by photoelectrically converting the subject image. The recording unit records image data. The display unit displays an image based on the image data. The face detection unit detects a face based on the image data. The face recognition unit extracts feature points representing the features of the face from the face. The first identification information adding unit adds identification information for identifying the face to the image data based on the feature points. The face classification processing unit classifies the image data based on the identification information. The search unit searches for image data to which no identification information is added from among the image data recorded in the recording unit. When the face detection unit detects a face based on the image data found by the search unit and without identification information added, the image presentation unit displays an image based on the image data on the display unit as a presentation image. The selection condition presenting unit causes the display unit to display a selection condition for selecting a person with a face based on the image data classified by the face classification processing unit for the presented image. The selection condition reception unit receives a selection condition selection or non-selection instruction by a user input. The second identification information adding unit adds identification information corresponding to the selection condition received by the selection condition receiving unit to the image data of the presented image.

  According to a second invention, in the first invention, when the selection condition accepting unit accepts a command for non-selection of the selection condition, the identification information is accepted based on a user input as to whether or not to add new identification information. When the reception unit and the identification information availability reception unit receive a command for adding new identification information, the reception unit and a new registration unit that adds new identification information to the image data of the presented image are further provided.

  According to a third aspect, in the first or second aspect, the recording unit includes a face classification folder that stores the image data classified by the face classification processing unit.

  According to a fourth aspect, in the third aspect, the image processing apparatus further includes a collective image processing unit that collectively performs image processing of a person specified by user input on the image data stored in the face classification folder.

  In a fifth aspect based on the third aspect, the display unit further includes a touch panel that performs user input on the display screen of the display unit.

  According to a sixth aspect, in the fifth aspect, the display control unit further controls a display mode of the display unit. When the face detection unit detects a face, the display control unit superimposes and displays a face detection frame indicating the face area on the presentation image. The selection condition presentation unit displays thumbnail images, which are reduced images for each person, as a selection condition on the display unit as a list based on the image data classified in the face classification folder. The second identification information adding unit associates the identification information of the same person by associating the face displayed in the face detection frame with the thumbnail image of the same person according to the operation of the touch panel by the user. And added to the image data of the presented image.

  In a seventh aspect based on the sixth aspect, the display control unit displays a face detection frame when the face recognition unit fails to extract the feature point even though the face detection unit detects a face. Change the aspect.

  In an eighth aspect based on the sixth aspect, when the selection condition presenting unit displays a list of thumbnail images, the selection condition presenting unit additionally displays a thumbnail image indicating a person in the newly created face classification folder.

  An image processing program according to a ninth invention is an image processing program for causing a computer to execute image processing, wherein an image data reading step, a face classification processing step, a search step, a face detection step, an image presentation step, , A selection condition presenting step, a selection condition receiving step, and an identification information adding step.

  In the image data reading step, image data recorded by the electronic camera is read. In the face classification processing step, the image data is classified based on identification information for identifying a face based on a feature point representing the feature of the face. The search step searches for image data to which no identification information is added from the image data read in the image data read step. In the face detection step, a face is detected based on the image data that is found in the search step and has no identification information added thereto. In the image presentation step, when a face is detected in the face detection step, an image based on the image data is displayed on the display unit as a presentation image. In the selection condition presenting step, a selection condition for selecting a person with a face based on the image data classified in the face classification processing step is displayed on the display unit for the presented image. In the selection condition receiving step, a selection condition selection or non-selection command is received by user input. In the identification information addition step, identification information corresponding to the selection condition received in the selection condition reception step is added to the image data of the presented image.

  According to the electronic camera of the present invention, it is possible to easily classify an image in which face identification cannot be performed at the time of photographing or an image in which the face identification function is not used.

1 is a block diagram illustrating a configuration of an electronic camera 1 according to a first embodiment. Block diagram showing the configuration of the CPU 23 Schematic representation of an Exif image file Flow chart showing photographing operation of electronic camera 1 Flow chart for checking presence / absence of identification information A flowchart for explaining identification information addition processing by the second identification information addition unit 23i. Subroutine of face registration processing shown in FIG. The figure explaining face registration processing The figure explaining face registration processing The figure explaining the face registration process in the case of newly registering The figure explaining the face registration process in the case of newly registering The figure explaining the face registration processing of multiple persons Flowchart representing batch image processing of image data in electronic camera 1 The figure explaining an example of the mode of the reproduction screen according to a person The figure explaining an example of mosaic processing The figure explaining an example of the thumbnail display after mosaic processing Conceptual diagram of bulk upload Diagram showing an example of batch digital blur processing Diagram showing an example of batch trimming processing Block diagram showing the configuration of the computer The flowchart showing the operation of the image processing program A of the second embodiment.

(First embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of an electronic camera 1 according to the first embodiment. As shown in FIG. 1, the electronic camera 1 includes a photographing lens 10, a lens driving unit 11, a diaphragm 12, a diaphragm driving unit 13, an image sensor 14, a timing generator (TG) 15, and an analog front end unit ( (Hereinafter referred to as “AFE”) 16, image signal processing unit 17, RAM (Random Access Memory) 18, ROM (Read Only Memory) 19, display control unit 20, display device 21, and operation unit 22. A CPU (Central Processing Unit) 23, a recording interface (recording I / F) 24, a bus 25, a recording medium 26, and a communication interface (communication I / F) 27. In FIG. 1, an image signal processing unit 17, a RAM 18, a ROM 19, a display control unit 20, a CPU 23, a recording interface (recording I / F) 24 and a communication interface (communication I / F) 27 are connected to each other via a bus 25. ing. The operation unit 22 is connected to the CPU 23.

  The photographing lens 10 includes a plurality of lens groups including a zoom lens and a focus lens. For the sake of simplicity, FIG. 1 shows the photographic lens 10 as a single lens. Each lens constituting the photographing lens 10 is moved in the optical axis direction by the lens driving unit 11. The input / output of the lens driving unit 11 is connected to the CPU 23.

  The diaphragm 12 adjusts the amount of incident light from the photographing lens 10. The opening amount of the diaphragm 12 is adjusted by the diaphragm driving unit 13. The input / output of the aperture driving unit 13 is connected to the CPU 23.

  The imaging element 14 generates image data by photoelectrically converting incident light from the photographing lens 10.

  Further, the timing generator (TG) 15 sends drive signals to each of the image sensor 14 and the AFE 16 according to instructions from the CPU 23, thereby controlling the drive timing of both.

  The AFE 16 is an analog front end circuit that performs signal processing on image data generated by the image sensor 14. The AFE 16 performs gain adjustment of image data, A / D conversion of an image signal, and the like. The image data output from the AFE 16 is input to the image signal processing unit 17.

  The image signal processing unit 17 temporarily stores the image data output from the AFE 16 in a frame memory provided in the RAM 18.

  The image signal processing unit 17 performs correction processing such as white balance correction processing on the image data stored in the frame memory.

  A connector for connecting the recording medium 26 is formed in the recording interface (recording I / F) 24. The recording interface (recording I / F) 24 accesses the recording medium 26 connected to the connector according to an instruction from the CPU 23 and performs image recording processing.

  The communication interface (communication I / F) 27 is a communication interface for realizing a wireless LAN (Local Area Network) that exchanges data by wireless communication, for example.

  The display device 21 includes a liquid crystal display monitor 21a. The liquid crystal display monitor 21a corresponds to an example of the display unit of the present invention. The display device 21 further includes a touch panel 21b that performs user input on the display screen of the liquid crystal display monitor 21a.

  The touch panel 21b is provided on the liquid crystal display monitor 21a, and employs, for example, a known resistive film method (pressure sensitive method). Thereby, the user can operate the electronic camera 1 by pressing the surface of the touch panel 21b with a finger.

  The display control unit 20 controls the display mode of the liquid crystal display monitor 21a. In addition, when the face detection unit 23a described later detects a face, the display control unit 20 superimposes and displays a face detection frame indicating a face area on the image.

  In addition, the display control unit 20 changes the display mode of the face detection frame when the face recognition unit 23b described later fails to extract a feature point even though the face detection unit 23a detects a face.

  The operation unit 22 receives an operation of a release button or a power button of the electronic camera 1 and issues a processing command to the CPU 23.

  FIG. 2 is a block diagram showing the configuration of the CPU 23. The CPU 23 is a processor that controls the electronic camera 1. The CPU 23 calculates a parameter of each process and controls each part of the electronic camera 1 by executing a sequence program stored in advance in the ROM 19. In addition, the CPU 23 of this embodiment includes a face detection unit 23a, a face recognition unit 23b, a first identification information addition unit 23c, a face classification processing unit 23d, a search unit 23e, an image presentation unit 23f, and a selection condition presentation unit 23g. Also, the selection condition receiving unit 23h, the second identification information adding unit 23i, the identification information availability unit 23k, the batch image processing unit 23l, and the communication processing unit 23m also function.

  The face detection unit 23a detects a face from a through image or a captured image displayed on the liquid crystal display monitor 21a. The face recognizing unit 23b extracts feature points (feature amounts) constituting the facial organ based on the detected face. The face detection unit 23a and the face recognition unit 23b are performed by a known technique. In this case, the face recognition unit 23b extracts facial feature points by a known feature point extraction process. Examples of feature points include eyebrows, eyes, nose, lips, facial contours, and the like. In the feature point extraction process, for example, the face color, brightness dispersion, intensity distribution, and the like can be detected as the feature amount.

  The first identification information adding unit 23c adds identification information for identifying the face to the image data based on the feature points. Here, an Exif (Exchangeable Image File Format) format is adopted as the image file (image data).

  FIG. 3 is a diagram schematically showing an Exif image file. In the first embodiment, the image file conforms to the file format of Exif (Exchangeable image file). This Exif is a typical file format of an electronic camera. Exif defines two types of formats, an uncompressed image file in the TIFF file (Tagged Image File Format) format and a compressed image file in the JPEG (Joint Photographic Experts Group) file format. FIG. 3 shows the structure of a conventionally known TIFF file.

  The TIFF file has a hierarchical structure starting with a TIFF header. In the TIFF file, an image file directory (IFD: Image File Directory) is arranged following the TIFF header. The image data is divided and recorded in blocks called strips or tiles. In the “Value of IFD area”, pointer values corresponding to the divided image data are stored for the number of strips and tiles.

  The IFD area includes a plurality of data blocks (IFD Entry). Here, in the first two bytes of “IFDEntry”, an identification number representing each additional information is arranged. This identification number is associated with specific additional information (image size, compression method type, resolution, etc.). In Exif, additional information that is not defined in normal TIFF can be defined. In this embodiment, identification information is added to the additional information. Thereby, the identification information for identifying each person can be recorded in association with the identification number. The identification information may be replaced with a person name or a person abbreviation by performing user input. The identification information may be managed in a sequence as a number for identifying a person's face.

  The face classification processing unit 23d classifies image data based on the identification information. The CPU 23 generates a face classification folder (face authentication folder) that stores the image data classified by the face classification processing unit 23 d in the recording medium 26. The CPU 23 records the image data to which the identification information is added on the recording medium 26.

  The search unit 23 e searches for image data to which no identification information is added, among the image data recorded on the recording medium 26.

  When the face detection unit 23a detects a face based on the image data to which the search unit 23e has found and the identification information is not added, the image presentation unit 23f uses the image based on the image data as a presentation image as a display image. 21a is displayed.

  The selection condition presentation unit 23g causes the liquid crystal display monitor 21a to display a selection condition for selecting a person with a face based on the image data classified by the face classification processing unit 23d for the presented image. Further, the selection condition presenting unit 23g displays a list of thumbnail images, which are reduced images for each person, as a selection condition on the liquid crystal display monitor 21a based on the image data classified in the face classification folder.

  The selection condition receiving unit 23g receives a selection condition selection or non-selection instruction by a user input.

  The second identification information adding unit 23i adds identification information corresponding to the selection condition received by the selection condition receiving unit 23g to the image data of the presented image. Further, the second identification information adding unit 23i associates the face displayed in the face detection frame with the thumbnail image of the same person as the face in accordance with the operation of the touch panel by the user. Identification information is added to the image data of the presentation image.

  When the selection condition acceptance unit 23h accepts a command for non-selection of the selection condition, the identification information availability acceptance unit 23j accepts a command as to whether or not to add new identification information by user input.

  When the selection condition accepting unit 23g accepts a selection condition non-selection command, the new registration unit 23k selects whether or not to add new identification information by user input, and selects the new identification information to be added. In this case, new identification information is added to the image data of the presented image.

  The batch image processing unit 231 performs batch image processing of a person specified by user input on the image data stored in the face classification folder. For example, the batch image processing unit 23l performs mosaic processing, digital blur processing, trimming processing, and the like (details will be described later). Note that providing a face classification folder facilitates batch image processing.

  The communication processing unit 23m transmits image data by wireless communication to, for example, an external server via the communication interface (communication I / F) 27.

  Next, an example of the operation of the electronic camera 1 in the first embodiment will be described.

  FIG. 4 is a flowchart showing the photographing operation of the electronic camera 1. Here, first, processing from the face detection function to the first identification information addition function will be described. In the following description, a case will be described in which the face detection function and the face recognition function are turned on, and the first identification information addition function is also turned on.

  Step S101: The CPU 23 starts acquiring a through image by driving the image sensor 14 via the timing generator (TG) 15. This through image is displayed on the liquid crystal display monitor 21a.

  Step S102: The CPU 23 determines whether or not it is half-pressed. When half-pressed (step S102: Yes), the process proceeds to step S103. On the other hand, if the release button is not half-pressed even after a predetermined time has elapsed (step S102: No), the power saving mode functions and this processing routine ends.

  Step S103: The face detection unit 23a of the CPU 23 performs the above-described face detection process to detect a face. At this time, the face detection unit 23 a acquires a color component and a luminance value based on image data generated by the image sensor 14. In this case, the face detection unit 23a detects the face by specifying a skin color component region or the like.

  Step S104: The CPU 23 determines whether or not a face has been detected in the face detection process of step S103. When the face detection unit 23a has detected a face (step S104: Yes), the process proceeds to face recognition processing (step S105).

  On the other hand, when the face detection unit 23a cannot detect the face (step S104: No), the process proceeds to step S106 as the normal shooting mode. In this case, when the full-press operation is performed (step S106: Yes), the process proceeds to the main image acquisition process (step S107). The processing after step S107 will be described later.

  Step S105: The face recognition unit 23b of the CPU 23 extracts feature points constituting the facial organ based on the detected face. For example, feature points such as eyes, nose, lips, and face contour are individually extracted from the face area of the face detection frame.

  Step S106: The CPU 23 determines whether or not the release button has been fully pressed. If the release button has not been fully pressed (step S106: No), the process returns to step S102. On the other hand, when the release button is fully pressed (step S106: Yes), the process proceeds to step S107.

  Step S107: Upon receiving the full-press operation from the operation unit 22, the CPU 23 drives the image sensor 14 via the timing generator (TG) 15 for acquiring the main image. The AFE 16 performs gain adjustment of the image signal and A / D conversion of the image signal. The image signal output from the AFE 16 is input to the image signal processing unit 17. The image signal processing unit 17 stores the main image data output from the AFE 16 in the frame memory. The image signal processing unit 17 performs correction processing such as white balance correction processing on the main image data stored in the frame memory.

  Step S108: The first identification information adding unit 23c of the CPU 23 adds the identification information of the person whose face is detected to the image data. Here, for the sake of simplicity, the first identification information adding unit 23c registers identification information for identifying a face for each face such as A, B, C... Based on the feature points. That is, the identification information is written in the header file of Exif. For the feature point, identification information is written in the header file of Exif as separate information. In addition, a person's name may be separately input for the identification information.

  In addition, the CPU 23 searches based on the feature points extracted by the face recognition unit 23b as to whether or not there is an image of a person having similar feature points in the already stored image data. When there is an image of a person having a similar feature point, the first identification information adding unit 23c writes the identification information (for example, A) corresponding to the feature point in the Exif header file.

  Step S109: The CPU 23 compresses the main image data and stores it in the face classification folder of the recording medium 26 based on the identification information.

Next, regarding images that could not be recognized at the time of shooting or images that did not use the face recognition function,
Processing of the second identification information addition function for adding identification information will be described. Here, first, the image data recorded on the recording medium 26 is read, and it is checked whether or not identification information is added.

  FIG. 5 is a flowchart for checking the presence or absence of identification information.

  Step S201: The search unit 23e of the CPU 23 reads one image data from the recording medium 26.

  Step S202: The CPU 23 refers to the image data and determines whether or not the first identification information adding unit 23c has added identification information. If the determination result is that there is no identification information (step S202: No), the processing routine proceeds to step S203. On the other hand, when the determination result includes identification information (step S202: Yes), the processing routine proceeds to step S204.

  Step S203: The CPU 23 registers image data with no identification information added to the list.

  Step S204: The CPU 23 determines whether or not the search unit 23e has searched all image data.

  When the search unit 23e has not searched for all image data (step S204: No), the processing routine returns to step S201 and reads the next image data.

  When the search unit 23e has searched all image data (step S204: Yes), this processing routine ends. By performing the processing from step S201 to step S204, image data to which no identification information is added is listed.

  Next, an identification information adding process by the second identification information adding unit 23i will be described.

  FIG. 6 is a flowchart for explaining identification information adding processing by the second identification information adding unit 23i.

  Step S301: The CPU 23 reads one piece of image data from the list created by the process of the flowchart of FIG.

  Step S302: The face detection unit 23a of the CPU 23 performs face detection processing.

  Step S303: If face detection is possible (step S303: Yes), the processing routine proceeds to step S304. When face detection cannot be performed (step S303: No), the processing routine proceeds to step S305.

  Step S304: The CPU 23 starts a subroutine for face registration processing, which will be described later.

  Step S305: The CPU 23 determines whether or not all image data in the list has been searched. When all the image data in the list has not been searched (step S303: No), the processing routine returns to step S301. On the other hand, when all the image data in the list has been searched (step S303: Yes), this processing routine ends.

  Next, a face registration subroutine will be described.

  FIG. 7 is a subroutine of the face registration process shown in FIG. 8 and 9 are diagrams illustrating the face registration process.

  Step S401: When the face detection unit 23a detects a face in step S303 in FIG. 6, the image presentation unit 23f of the CPU 23 displays an image based on the image data on the liquid crystal display monitor 21a as a presentation image.

  Step S402: The CPU 23 issues an instruction to the display control unit 20, and the display control unit 20 displays the face detection frame W. As shown in FIG. 8A, a face detection frame W is displayed for the person on the liquid crystal display monitor 21a. For convenience of explanation, the image file name of this image is “001” here.

  Step S403: The selection condition presentation unit 23g of the CPU 23 causes the liquid crystal display monitor 21a to display a selection condition for selecting a person with a face based on the image data classified by the face classification processing unit 23d for the presented image. As an example, a message “Does 001 correspond to any of registered persons A, B, and C?” Is displayed on the liquid crystal display monitor 21a (FIG. 8B). For convenience of explanation, it is assumed that the names of registered persons are displayed in A, B, and C. That is, it is assumed that a registered person name is associated with each identification information in advance by user input.

  Step S404: The selection condition accepting unit 23g of the CPU 23 accepts selection (A, B, or C: applicable) or non-selection (not applicable) of the selection condition on the touch panel 21b. If the user input command is “applicable” (step S404: Yes), the subroutine proceeds to step S405.

  Step S405: Based on the image data classified in the face classification folder, the selection condition presenting unit 23g uses the thumbnail image, which is a reduced image for each person, as the selection condition, as shown in FIG. To display a list. Then, as an example, the selection condition presenting unit 23g displays a message “Perform an association, please select any registered person” on the liquid crystal display monitor 21a (FIG. 9B).

  Step S406: The CPU 23 accepts selection of a thumbnail image (for example, selection of a thumbnail image of the registered person A) via the touch panel 21b.

  Step S407: When the selection condition receiving unit 23g receives a selection condition selection command, the second identification information adding unit 23i of the CPU 23 adds identification information corresponding to the selection condition to the image data of the presented image. That is, the second identification information adding unit 23i associates the face displayed in the face detection frame W with the thumbnail image of the same person A as that face. Specifically, the identification information of the registered person A displayed in the thumbnail image is added to the image data of the same person A displayed in the face detection frame. By doing so, the user can easily perform user input on the touch panel 21b. Furthermore, by selecting a thumbnail image, identification information can be added to an image for which face identification cannot be performed at the time of shooting or an image for which the face identification function is not used. Note that the thumbnail image display mode illustrated in FIG. 9B is merely an example, and is not limited to this display mode. Further, thumbnail images may be sequentially displayed according to the operation of the touch panel.

  Step S408: The face classification processing unit 23f classifies the image data based on the identification information of the registered person A. Then, the CPU 23 stores the image data in the face classification folder of the registered person A. Then, this subroutine ends, and the process returns to step S305 in FIG.

  Next, in the process of step S405, when the user input command is “not applicable” (step S405: No), the subsequent process will be described.

  10 and 11 are diagrams for explaining face registration processing in the case of new registration.

  In the process of step S403, as shown in FIG. 10A, for example, the display control unit 20 displays a message “Does the person of 002 correspond to any of registered persons A, B, and C?” Displayed on the liquid crystal display monitor 21a. If the user input command is “not applicable” (step S404: No), the subroutine proceeds to step S409.

  Note that the display control unit 20 may change the display mode of the face detection frame W when the face recognition unit 23b fails to extract a feature point even though the face detection unit 23a has detected a face. . For example, the display control unit 20 changes the color of the face detection frame W from green to red. Thereby, the user can know that the person 002 is an unregistered person.

  Step S409: As shown in FIG. 10B, the selection condition presenting unit 23g displays a message “Do you want to newly register?” On the liquid crystal display monitor 21a as an example. In the case of new registration, it means that there is no identification information of the same person added by the first identification information adding unit 23c. When the user input command is “new registration” (step S409: Yes), the subroutine proceeds to step S407.

  Step S407: The new registration unit 23k adds new identification information to the image data by user input via the touch panel 21b. As a result, for example, even if the face is not detected at the time of shooting and the identification information does not exist, the identification information can be added.

  Step S408: The face classification processing unit 23f classifies the image data based on the identification information of the person D. In this case, the face classification processing unit 23f newly generates a face classification folder for the person D. The CPU 23 stores the image data in the face classification folder. Then, this subroutine ends, and the process returns to step S305 in FIG.

  On the other hand, if the user input command is “not newly registered” in step S409 (step S409: No), this subroutine ends, and the process returns to step S305 in FIG.

  When a person D is newly registered, in the process of step S405, the selection condition presenting unit 23g displays a list of thumbnail images of the person D in the newly created face classification folder when displaying thumbnail images as a list. To be displayed (FIG. 11). As a result, the selection condition presenting unit 23g can display a list display of the latest thumbnail images on the liquid crystal display monitor 21a.

  As described above, the identification information adding process is performed by the second identification information adding unit 23i. The handling of images in which a plurality of persons are photographed will be described below.

FIG. 12 is a diagram illustrating face registration processing for a plurality of persons. FIG. 12A is an image in which three persons A, B, and C are photographed. A face detection frame W is displayed. At the time of shooting, an image that does not use the face identification function is used. In this state, as an example, when a thumbnail image of a registered person is displayed in step S405 of FIG. 7, the display is as shown in FIG.
When the persons A, B, and C are individually selected by user input, the image 003 is stored in the face classification folders of the registered persons A, B, and C. In consideration of the storage capacity, a so-called virtual folder format may be adopted in which a storage format that makes one image appear to be registered in three folders may be adopted.

  Further, as an example, if the persons A, B, and C are siblings, a folder of a plurality of persons such as a sibling folder and a family folder may be generated and stored so that the person A, B, and C are newly stored in sibling units. .

  In addition, as an example of the persons A, B, and C, when the face classification folder is created for the persons A and B and only the person C is not created, the user input from the touch panel 21b For the persons A and B, images may be stored in the respective face classification folders, and for the person C, a new face classification folder for the person C may be created and stored.

  Next, processing of the batch image processing unit 23l and the communication processing unit 23m will be described.

  FIG. 13 is a flowchart showing batch image processing of image data in the electronic camera 1. Here, processing that batches mosaic processing for image data with the same identification information added, and uploading images to an external image storage server for image data that has undergone mosaic processing First, the processing to be performed will be described.

  The flowchart shown in FIG. 13 starts when the user selects the mode of the reproduction screen for each person. By selecting the mode of the reproduction screen for each person, batch image processing of image data and image data upload can be performed.

  Step S501: When the user accepts a command for the reproduction screen for each person, the CPU 23 causes the liquid crystal display monitor 21a to display a menu for the mode for the reproduction screen for each person via the display control unit 20.

  FIG. 14 is a diagram illustrating an example of the mode of the playback screen for each person. As shown in FIG. 14A, the liquid crystal display monitor 21a displays a menu for the mode of the reproduction screen for each person.

  Step S502: Upon receiving a command indicating that the face classification folder a of the person A has been selected by user input, the CPU 23 reads an image showing the registered person A from the face classification folder a of the recording medium 26. The CPU 23 generates a thumbnail image of an image showing the registered person A.

  Step S503: The CPU 23 displays a thumbnail image showing the registered person A on the liquid crystal display monitor 21a via the display control unit 20. FIG. 15B shows an example of a thumbnail image showing the registered person A.

  Step S504: Upon receiving an image processing selection command by user input, the CPU 23 causes the liquid crystal display monitor 21a to display an image processing selection menu via the display control unit 20. For example, when a batch mosaic is selected by user input, a specific processing content of the batch mosaic is displayed in a menu. Here, it is possible to select two types by user input: (1) a case where the mosaic process is performed for the registered person A, and (2) a case where the mosaic process is performed for other than the registered person A. Note that this is an example, and for example, all registered persons in the image may be used. Further, the mosaic processing level can be set by user input. For example, the level may be set by a bar display (not shown). This mosaic process is performed by a known technique.

  Step S505: When the CPU 23 receives an instruction to execute a batch mosaic by a user input, the batch image processing unit 23l of the CPU 23 performs a mosaic process on the selected face area. In this way, the collective image processing unit 23l can collectively perform image processing on images for which face identification cannot be performed at the time of shooting and images for which the face identification function is not used.

  FIG. 15 is a diagram illustrating an example of mosaic processing. FIG. 15A shows an example when a mosaic process is performed on the registered person A. FIG. FIG. 15B shows an example when mosaic processing is performed for persons other than the registered person A.

  Step S506: The CPU 23 causes the liquid crystal display monitor 21a to display thumbnails on the liquid crystal display monitor 21a via the display control unit 20.

  FIG. 16 is a diagram illustrating an example of thumbnail display after mosaic processing. Here, for simplicity, two images after mosaic processing are displayed as thumbnail images. Here, for the thumbnail images displayed as a list, an image to be uploaded can be selected by user input. You may upload the original image of all thumbnail images.

  Step S507: Upon receiving a thumbnail image selection command by user input, the CPU 23 displays a batch upload menu screen on the liquid crystal display monitor 21a via the display control unit 20. Here, when the CPU 23 receives a batch upload selection command by user input, the processing routine proceeds to step S508.

  Step S508: The CPU 23 starts batch upload.

  FIG. 17 is a conceptual diagram of batch upload. The communication processing unit 23m of the CPU 23 first transmits image data to the external wireless LAN access point 100 by wireless communication via the communication interface (communication I / F) 27. Subsequently, the wireless LAN access point 100 uploads image data to the image management server 200 via the Internet.

  Then, when the communication processing unit 23m of the CPU 23 receives the upload completion command, the processing routine ends.

  Next, the batch digital blur process will be described. If the batch digital blurring process is selected in step S504, the process advances to step S509, and the batch image processing unit 23l performs the batch digital blurring process on the selected face area.

  FIG. 18 is a diagram illustrating an example of batch digital blurring processing. The digital blur processing is performed by a known technique. FIG. 18A is a diagram illustrating an example of a case where a person other than the registered person A is subjected to digital blurring processing. FIG. 18B is an example of a case where the digital blurring process is performed on the registered person A. In this way, the batch image processing unit 231 of the CPU 23 collectively performs digital blurring processing of the face area on the selected person. Since the subsequent processing is the same as the batch mosaic processing, description thereof is omitted. Note that, as the digital blurring process, the process of blurring the background as in the portrait mode may be performed at once.

  Next, the batch trimming process will be described. When the batch trimming process is selected in step S504, the process advances to step S510, and the batch image processing unit 23l of the CPU 23 performs the batch trimming process on the selected face area.

  FIG. 19 is a diagram illustrating an example of batch trimming processing. FIG. 19A shows a diagram in which a face to be trimmed is selected, and a face detection frame W is displayed. Here, the registered person A is selected. The collective image processing unit 23l of the CPU 23 can generate, for example, the image shown in FIG. 19B for the selected registered person A by a known trimming process.

  Here, the batch image processing unit 231 of the CPU 23 sequentially performs trimming processing on the image data of the registered person A registered in the face classification folder in a batch. Since the subsequent processing is the same as the batch mosaic processing, description thereof is omitted.

  As described above, according to the electronic camera 1 of the present embodiment, it is possible to easily classify an image in which face identification cannot be performed at the time of shooting or an image in which the face identification function is not used. For example, when the face of the subject is not facing forward, such as downward or diagonally, it is difficult for the face identification function to identify the face. However, with the electronic camera 1 of the present embodiment, it is possible to classify even images that are difficult to identify faces. Furthermore, it is possible to perform image processing in a lump without taking time and effort for the captured images.

  In this embodiment, the image data subjected to batch image processing is uploaded to the external image management server 200 by wireless communication. However, the present invention is not limited to wireless communication, and wired communication using a wired LAN may be performed. Alternatively, the USB cable may be connected from the electronic camera 1 to an external personal computer to transfer the image data subjected to batch image processing.

  In this embodiment, the batch-processed image data is uploaded to the external image management server 200. For example, the electronic camera 1 side performs batch print designation to the external printer by wireless communication or the like. Also good.

In this embodiment, the Exif file format is used. This provides versatility, but it may be applied to other types of files.
(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the second embodiment, the image data recorded by the electronic camera 1 is read out to the computer 3 shown in FIG. 20, and the image data of the image that could not be identified at the time of photographing or the image that did not use the face identification function was used. Add identification information. The image processing program which is one embodiment of the present invention is used for these processes.

This will be specifically described below. FIG. 20 is a block diagram illustrating a configuration of a computer. The computer 3 shown in FIG. 20 includes a computer CPU 30, a memory 31, a communication interface (I / F) 32, a keyboard 33, and a display monitor 34. The memory 31 also stores an image processing program that is an embodiment of the present invention.
In the second embodiment, the image processing program A shown in the dotted frame in FIG. 21 is incorporated in the computer CPU 30. That is, the computer CPU 30 incorporates the image processing program A which is an embodiment of the present invention, so that the image data reading unit 30a, the face classification processing unit 30b, the search unit 30c, the face detection unit 30d, An image presentation unit 30e, a selection condition presentation unit 30f, a selection condition reception unit 30g, and an identification information addition unit 30h are constructed.

  The image data reading unit 30a reads image data recorded by the electronic camera. The face classification processing unit 30b classifies image data based on identification information for identifying a face based on a feature point representing the feature of the face. The search unit 30c searches for image data to which no identification information is added from among the image data read by the image data reading unit 30a. The face detection unit 30d detects a face based on image data that is found by the search unit 30a and to which no identification information is added. When the face detection unit 30d detects a face, the image presentation unit 30e displays an image based on the image data on the display monitor 34 as a presentation image. The selection condition presenting unit 30f causes the display monitor 34 to display a selection condition for selecting a face person based on the image data classified by the face classification processing unit 30b for the presented image. The selection condition receiving unit 30g receives a selection condition selection or non-selection instruction by a user input. The identification information adding unit 30h adds identification information corresponding to the selection condition received by the selection condition receiving unit 30g to the image data of the presented image.

  Next, the operation of the image processing program A which is an embodiment of the present invention will be described. Here, it is assumed that an image recorded by the electronic camera 1 is stored in advance in the memory 31 via the communication interface (I / F) 32.

  FIG. 21 is a flowchart showing the operation of the image processing program A of the second embodiment. This flowchart starts when the activation of the image processing program A is received from the keyboard 33 shown in FIG. Further, it is assumed that image data recorded by the electronic camera is recorded in the memory 31 via the communication I / F 32 in advance.

  Step S601: The image data reading unit 30a reads the image data recorded by the electronic camera from the memory 31. Step S601 corresponds to an example of an image data reading step.

  Step S602: The face classification processing unit 30b classifies image data based on identification information for identifying a face based on a feature point representing the feature of the face. The face classification processing unit 30b generates a face classification folder for each identification information in the memory 31, and stores the image data to which the identification information is added. Step S602 corresponds to an example of a face classification processing step.

  Step S603: The CPU 30 executes a subroutine for checking whether or not there is identification information. This subroutine performs the same processing as the flowchart for checking the presence or absence of the identification information shown in FIG. Step S603 corresponds to an example of a search step.

  Step S604: The CPU 30 executes a subroutine for adding identification information. This subroutine performs the same process as the flowchart for explaining the identification information adding process by the second identification information adding unit 23i shown in FIG. In the program A, the processing of the second identification information adding unit 23i is performed by the identification information adding unit 30h. Further, the CPU 30 performs the same process as the face registration process subroutine shown in FIG. At this time, the CPU 30 may add the above-described new registration process.

  After the above steps, this processing routine is completed. Thereafter, although not shown in the flowchart, the CPU 30 performs processing similar to the batch image processing described in the first embodiment. Note that step S302 in FIG. 6 corresponds to an example of a face detection step. Step S401 in FIG. 7 corresponds to an example of an image presentation step. Step S403 in FIG. 7 corresponds to an example of a selection condition presenting step. Steps S404 to S406 in FIG. 7 correspond to an example of a selection condition receiving step. Step S407 in FIG. 7 corresponds to an example of an identification information adding step.

  Thereafter, the mosaic-processed image data is uploaded to the external blog server 300 by user input, for example.

As described above, according to the image processing program A of the present embodiment, it is possible to easily classify an image in which face identification cannot be performed at the time of shooting or an image in which the face identification function is not used. Furthermore, it is possible to perform image processing in a lump without taking time and effort for the captured images.
<Supplementary items of the embodiment>
In the present embodiment, mosaic processing, digital blurring processing, and trimming processing are taken up as predetermined batch image processing. However, these are examples, and are not limited to three processing. For example, red-eye image processing for correcting red eyes may be used. A plurality of image processes may be combined.

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 3 ... Computer, 23a, 30c ... Face detection part, 23b ... Face recognition part, 23c ... 1st identification information addition part, 23d, 30b ... Face classification Processing unit, 23e, 30c ... search unit, 23f, 30e ... image presentation unit, 23g, 30f ... selection condition presentation unit, 23h, 30g ... selection condition reception unit, 23i ... second Identification information adding unit, 23j: Identification information availability accepting unit, 23k: New registration unit, 23l: Batch image processing unit, 23m: Communication processing unit, 30a: Image data reading unit, 30h ... Identification information processing section

JP 2007-281760 A JP 2007-88993 A

Claims (9)

An image sensor that photoelectrically converts a subject image to generate image data;
A recording unit for recording the image data;
A display unit for displaying an image based on the image data;
A face detection unit for detecting a face based on the image data;
A face recognition unit for extracting feature points representing the features of the face from the face;
A first identification information adding unit for adding identification information for identifying the face to the image data based on the feature points;
A face classification processing unit that classifies the image data based on the identification information;
A search unit for searching for image data to which the identification information is not added among the image data recorded in the recording unit;
When the face detection unit detects a face based on image data found by the search unit and without the identification information added, an image presentation is displayed on the display unit as an image based on the image data. And
A selection condition presenting unit that causes the display unit to display a selection condition for selecting a person on the face based on the image data classified by the face classification processing unit with respect to the presented image;
A selection condition receiving unit for receiving a selection or non-selection instruction of the selection condition by a user input;
A second identification information adding unit for adding the identification information corresponding to the selection condition received by the selection condition receiving unit to image data of the presented image;
An electronic camera comprising: The electronic camera according to claim 1,
When the selection condition reception unit receives a non-selection command of the selection condition, an identification information availability reception unit that receives a command as to whether or not to add new identification information by user input,
The electronic apparatus further comprising: a new registration unit that adds new identification information to the image data of the presented image when the identification information availability receiving unit receives a command to add new identification information. camera. The electronic camera according to claim 1 or 2,
The electronic camera according to claim 1, wherein the recording unit includes a face classification folder for storing the image data classified by the face classification processing unit. The electronic camera according to claim 3.
An electronic camera, further comprising: a batch image processing unit that batch-processes image processing of a person designated by the user input with respect to the image data stored in the face classification folder. The electronic camera according to claim 3.
The display unit further includes a touch panel that performs the user input on a display screen of the display unit. The electronic camera according to claim 5,
A display control unit for controlling a display mode of the display unit;
When the face detection unit detects a face, the display control unit superimposes a face detection frame indicating the face area on the presented image,
The selection condition presenting unit displays a list of thumbnail images, which are reduced images for each person, as the selection condition on the display unit based on the image data classified in the face classification folder,
The second identification information adding unit associates the face displayed in the face detection frame with a thumbnail image of the same person as the face in response to an operation of the touch panel by a user. The identification information is added to the image data of the presented image. The electronic camera according to claim 6.
The display control unit changes a display mode of the face detection frame when the face recognition unit fails to extract the feature point even though the face detection unit detects a face. Electronic camera. The electronic camera according to claim 6.
The electronic camera according to claim 1, wherein when the thumbnail image is displayed as a list, the selection condition presenting unit additionally displays a thumbnail image indicating a person in the newly created face classification folder. An image processing program for causing a computer to execute image processing,
An image data reading step for reading image data recorded by the electronic camera;
A face classification processing step for classifying image data based on identification information for identifying the face based on a feature point representing a feature of the face;
A search step of searching for image data to which the identification information is not added among the image data read in the image data read step;
A face detection step for detecting a face based on the image data to which the identification information is not added, found in the search step;
An image presentation step of displaying an image based on the image data on the display unit as a presentation image when a face is detected in the face detection step;
A selection condition presenting step for causing the display unit to display a selection condition for selecting a person of the face based on the image data classified in the face classification processing step for the presented image;
A selection condition receiving step for receiving a selection or non-selection command of the selection condition by a user input;
An identification information adding step of adding the identification information corresponding to the selection condition received in the selection condition receiving step to the image data of the presented image;
An image processing program comprising:

Images