JP2012235273A - Imaging device, control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system which counts the number of photographs per target person by recognizing faces at photographing and prevents a user from intensively photographing a specific person.SOLUTION: An imaging device which photographs a subject by using face recognition is configured to: register information on a subject in association with a face image of the subject identified by the face recognition as identification information; store and manage the registered identification information on the subject in association with the number of images capturing the subject; detect a subject at photographing; determine whether identification information on the detected subject corresponds to the identification information on the subject already registered; display the number of images capturing the subject when it is determined that an identification image of the subject corresponds to the registered face image of the subject; and perform warning concerning the number of photographs of the subject according to the number of photographs of the subject.

Description

  The present invention relates to an image capturing method using an imaging apparatus. Specifically, an imaging unit that captures an image of a subject, a face detection unit that detects a face from the captured image, a person verification unit that compares the detected face, and counts the collated images The present invention relates to control of an imaging device including a measurement unit.

  Conventionally, a so-called “face recognition focus” function has been proposed that makes it possible to recognize a person's face at the time of shooting and automatically focus using a face recognition technique. Accordingly, it is possible to prevent blurring unintended by the photographer by effectively focusing on the face.

  For example, Patent Document 1 discloses a function of automatically performing imaging when a subject reaches the scheduled number of persons by using the face recognition technology and setting the scheduled number of persons to be photographed.

JP 2007-329602 A

  Patent Document 1 discloses a technique that can easily shoot a group photo or a specific person, but for example, when taking a photograph on a trip or the like, it may be desirable to shoot the participants evenly. is there. However, even if the photographer does not intend, only a specific person may be photographed in a biased manner.

  In addition, when going to an excursion at an elementary school, it is impossible to grasp how many students are in each photograph because the teacher shoots while taking action with multiple students. For this reason, there are problems that a large number of specific students are shown, or conversely, there are few.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a mechanism in which information of a target person is registered in advance, the number of shots for each target person is counted by recognizing a face at the time of shooting, and shooting is not biased to a specific person. And

That is, the imaging apparatus of the present invention registers the identification information by associating the subject information with the face image of the subject specified by the face recognition in an imaging apparatus that captures a subject using face recognition. Identification information registration means, storage management means for storing and managing the identification information of the subject registered by the identification information registration means, and the number of images in which the subject is photographed, and detecting the subject at the time of photographing A detection unit; a determination unit that determines whether or not the identification information of the subject detected by the detection unit matches the identification information of the subject registered by the identification information registration unit; and A number-of-photographs table that displays the number of images in which the subject is photographed when it is determined that the identification image of the subject matches the face image of the subject registered in the identification information registration unit And means, in accordance with the number of shots of the subject displayed on the display means, characterized in that it comprises a warning means for performing warning about the number of shots of the object.
The imaging apparatus according to the present invention further includes a number-of-photographing display unit that displays the number of images of the subject that is stored and managed by the storage management unit in a format in which the number of captured images can be identified for each subject. It is further provided with the feature.
Further, in the present invention, the determination unit includes a comparison unit that compares a value of the number of images in which the identification information of the subject detected by the detection unit is captured with a preset value of the minimum number of images to be captured, If the value of the number of images for which the identification information has been photographed by the comparing means is greater than the preset number of images to be photographed, the warning means has already photographed the minimum number of images. If the value of the number of shots of the identification information taken by the comparison means is smaller than the preset number of shots, the identification information reaches the minimum number of shots. This is characterized by warning the number of sheets necessary for the operation.
In addition, the present invention further includes a selection receiving unit that receives selection of an identification group that is a target for detecting a subject at the time of shooting and a period during which an image of identification information that belongs to the identification group is shot, and the detection unit includes: An image of identification information belonging to the identification group received by the selection receiving means is detected.
Further, the present invention further includes an imaging unit that captures an image, and when the subject registered by the identification information registration unit is captured in an image captured by the imaging unit, a portion of the subject whose face is recognized, The image processing apparatus further includes photographing result display means for displaying information for identifying the recognized subject.
In the present invention, the identification information registration unit registers information of an identification group for classifying the subject into a group as the identification information.

  According to the present invention, it is possible to provide a mechanism for registering information on a target person in advance, counting the number of shots for each target person by recognizing a face at the time of shooting, and preventing shooting with bias toward a specific person. Is possible.

It is a figure which shows the hardware constitutions of the imaging device which concerns on embodiment of this invention. It is a figure which shows the module of the program which operate | moves with the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the identification information data table which the imaging device which concerns on embodiment of this invention hold | maintains. It is a figure which shows an example of the face recognition information data table which the imaging device which concerns on embodiment of this invention hold | maintains. It is a figure which shows an example of the identification group data table which the imaging device which concerns on embodiment of this invention hold | maintains. It is a figure which shows an example of the identification information count data table which the imaging device which concerns on embodiment of this invention hold | maintains. It is a figure which shows an example of the image data table which the imaging device which concerns on embodiment of this invention hold | maintains. It is a figure which shows an example of the screen changed into the identification information registration mode displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the screen changed into the identification information registration screen displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the identification information registration screen displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the screen which registers the identification group displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the screen which selects the identification group displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of a screen when it corresponds with identification information at the time of imaging | photography displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the warning displayed according to the imaging | photography number of a to-be-photographed object at the time of imaging | photography displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the screen when not matching with identification information at the time of imaging | photography displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a figure which shows an example of the screen which displays the identification result below the minimum imaging | photography number displayed on the display part of the imaging device which concerns on embodiment of this invention. It is a flowchart performed at the time of registration of identification information with the imaging device which concerns on embodiment of this invention. It is a flowchart performed at the time of imaging | photography with the imaging device which concerns on embodiment of this invention, and an identification result determination.

  FIG. 1 is a block diagram showing a schematic configuration of a photographing apparatus according to the present invention. In FIG. 1, the photographing apparatus includes a so-called digital camera, a photographing lens 101, an image pickup element (hereinafter referred to as “CCD”) 102, a camera signal processing unit (hereinafter referred to as “ADC”) 103, image processing. Unit 104, controller 110, buffer memory 111, flash ROM 112, interface circuit (hereinafter referred to as “I / F circuit”) 113, card holder 114, recording medium 115, display driver 116, and operation unit 120.

  FIG. 1 is a diagram illustrating a configuration of an imaging apparatus to which the information addition apparatus according to the present embodiment is applied. The imaging apparatus includes a lens 101, an imaging device (CCD) 102, a camera signal processing unit 103, an image processing unit 104, a recording medium 115, a lens control unit 127, a controller 110, and an operation unit 120. Consists of.

  The lens 101 is a lens or the like, and includes an objective lens, a zoom lens, a focus lens, and the like. The zoom lens and the focus lens are driven in the optical axis direction by a driving mechanism (not shown).

  The imaging element 102 is configured by a CCD image sensor that forms an image of imaging light incident from the lens 101, converts the imaging light into an electrical signal (analog signal), and outputs the electrical signal.

  The camera signal processing unit (ADC) 103 has a function of performing signal processing such as digital conversion and white balance adjustment on the electric signal received from the image sensor 102 and converting it into a digital signal.

  The controller 110 is connected to the image processing unit 104, buffer memory 111, flash ROM 112, I / F circuit 113, display driver 116, sound driver 118, LED driver 128, and operation unit 120.

  The image processing unit 104 includes a preprocessing unit 105, a YC processing unit 106, an electronic zoom processing unit 107, a compression unit 108, and an expansion unit 109, and generates image data from a digital signal output from the camera signal processing unit 103. Have a function of performing various image processing.

  The pre-processing unit 105 has a function of performing white balance processing for adjusting the white balance of an image based on input image data and image gamma correction processing. The white balance process is a process of adjusting the color of an image so as to be close to the actual color, or adjusting the color to an appropriate color that matches a light source (such as a fluorescent lamp or sunlight). The gamma correction process is a process for adjusting the contrast of an image. Note that the pre-processing unit 105 can also perform image processing other than white balance processing and gamma correction processing.

  The YC processor 106 separates an image based on input image data into luminance information “Y”, luminance signal and blue color difference information “Cb”, and luminance signal and red color difference information “Cr”. Have The electronic zoom processing unit 107 has a function of trimming a part of an image (for example, a central part) with a predetermined size and enlarging the trimmed image to the size of the original image by signal processing. The electronic zoom processing unit 107 can, for example, cut out a central 1024 × 768 dot image from a captured 1600 × 1200 dot image and enlarge it to a size of 1600 × 1200 dot while performing data interpolation.

  The compression unit 108 has a function of compressing image data in a compression format such as a JPEG (Joint Photographic Expert Group) method. The decompression unit 109 has a function of decompressing compressed image data. For example, when compressing image data using the JPEG method, first, a discrete cosine transform process is performed (DCT process) that digitizes the ratio of high-frequency components and low-frequency components of image data. Next, a quantization process is performed for expressing the gradation or gradation of the image by a numerical value (quantization bit number). Finally, the image data is compressed by the Huffman encoding process.

  Specifically, the signal character string of the image data is segmented every certain bit, and a shorter code is given to a character string having a high appearance frequency. In the case of a system that records image data without performing compression processing, the compression unit 108 and the expansion unit 109 can be omitted. Further, the image data compression format is not limited to the JPEG format, and the same processing can be performed even in a GIF (Graphical Interchange Format) format.

  The buffer memory 111 temporarily stores image data when the image processing unit 104 performs image processing. The flash ROM 112 stores various setting information of the photographing apparatus and user authentication information described later. The I / F circuit 113 converts the image data output from the controller 110 into a data format that can be recorded on the recording medium 115. The I / F circuit 113 converts image data read from the recording medium 115 into a data format that can be processed by the controller 110.

  The card holder 114 includes a mechanism that allows the recording medium 115 that is a storage medium to be attached to and detached from the photographing apparatus, and includes an electrical contact that enables data communication with the recording medium 115. The card holder 114 has a structure corresponding to the type of storage medium used in the photographing apparatus. The recording medium 115 is a card-type storage medium that incorporates a semiconductor storage element such as a flash memory and is detachable from the card holder 114. The recording medium 115 can record image data captured by the imaging device.

  The display driver 116 converts the image data output from the controller 110 into a signal that can be displayed on the display 117. Specifically, the display driver 116 performs processing for converting digital image data output from the controller 110 into an analog image signal, and then converts the image size to a size suitable for the size of the displayable area of the display 117. Perform the process.

  The sound driver 118 performs a process of converting the audio data output from the controller 110 into a signal that can be sounded by the speaker 219. An LED (Light Emitting Diode) driver 128 controls the lamp 129 in accordance with a command output from the controller 110.

  The operation unit 120 includes a power switch 121, a mode dial 122, a shooting button 123, a cursor key 124, a zoom button 125, and a browse (playback) button 126. The operation unit 120 receives an operation input from the user and receives a signal corresponding to the operation content. Is output to the controller 110. The operation unit 120 includes operation buttons other than the illustrated operation buttons, but a description thereof is omitted.

  A mode dial (select button) 122 is a rotatable dial for switching the mode of the photographing operation. The user can select a plurality of shooting operation modes by operating the mode dial 122. In the embodiment of the present invention, the mode dial is described as a dial type mechanism. However, for example, a slide type button may be used instead of the dial. That is, the mechanism of the mode dial 122 is not limited to the dial, and it is essential to have a mechanism for switching the camera mode.

  The browsing button 126 is a button for switching to a browsing mode (playback mode) for browsing captured image data. When the browse button 126 is pressed, the shooting mode is changed to the browsing mode. Note that the method for switching to the viewing mode is not limited to pressing the viewing button 126, and may be a transition method using a touch panel (not shown) on the display 117.

  In the browsing mode, the controller 110 reads setting information and image data from the recording medium 115 attached to the card holder 114 via the I / F circuit 113. The setting information includes the number of images recorded on the currently loaded recording medium 115 and the capacity of the recorded image data. The read image data is, for example, image data or thumbnail data for one image. Note that the image data read from the recording medium 115 is compressed in a predetermined compression format as described above.

  The image data read from the recording medium 115 is input to the display driver 116 via the I / F circuit 113 and the controller 110. The display driver 116 displays the input image data on the display 117.

  The cursor key 124 is a key for the user to perform a predetermined instruction or function selection to the photographing apparatus. The user can perform operation inputs such as various setting information and user authentication information with respect to the photographing apparatus using the cursor key 124. The cursor key 124 can display a menu screen on the display 117 or select a predetermined function of the photographing apparatus. Further, the cursor key 124 can display the image data photographed by the photographing apparatus on the display 117 as a review. The review display is a function of causing the display 117 to display an image photographed by the photographing apparatus 200 immediately after that.

  Note that the review display is not limited to a configuration in which only one photographed image is displayed on the display 117, and a plurality of images including thumbnail images may be displayed. Further, the review display switching is not limited to the pressing of the cursor key 124 but may be switched using a general method such as pressing of another button. Further, the review display screen may be configured such that the review display image can be deleted or edited.

  The lens control unit 127 controls the lens 101 such as zoom, focus, and diaphragm. In addition, the photographing apparatus includes a photographing mode for capturing an image of a subject to obtain image data and a browsing (reproducing) mode for displaying the image data obtained in the photographing mode.

  Note that the sound can be recorded and reproduced in the same manner as the imaging apparatus by using a microphone or a speaker. Further, when recording video (moving image), normal audio recording is also performed at the same time, and video and audio are multiplexed in the compression unit 108 and the expansion unit 109.

  In addition, a photographing operation in the photographing apparatus 200 will be described with reference to FIG. First, the user operates the power switch 121 to turn on the photographing apparatus 200. Then, power is supplied to each circuit from a battery (not shown) built in the photographing apparatus 200, and activation processing of the photographing apparatus 200 (operation control for opening the lens barrier, reset processing of the microcomputer, etc.) is performed. The photographing apparatus 200 is configured such that when the power switch 121 is pressed when the power is turned off, the power is turned on and the photographing mode is automatically set. Further, when the viewing button is pressed when the power is turned off, the power is turned on and the viewing mode is automatically set.

  When the photographing apparatus 200 shifts to the photographing mode, an optical image enters the photographing apparatus through the lens 101 and is imaged on the image sensor (CCD) 102. The CCD 102 converts an incident optical image into an electric signal and outputs it to a camera signal processing unit (ADC) 103. The ADC 103 converts an input electric signal (analog signal) into a digital signal. A digital signal output from the ADC 103 is input to the image processing unit 104.

  A preprocessing unit 105 in the image processing unit 104 generates image data based on an input digital signal, and performs white balance processing, gamma correction processing, and the like. In the YC processing unit 106 in the image processing unit 104, the image data is separated into the luminance signal Y and the color difference signals Cr and Cb, and processing for reducing the information amount of the color difference signals Cr and Cb is performed. The information amount reduction processing of the color difference signals Cr and Cb is, for example, “4: 2: 2 downsampling processing” for thinning out color information in the main scanning direction of an image, or “401 downsampling for thinning out color information in the vertical and horizontal directions of an image. Processing ". When image processing is performed in the preprocessing unit 105 and the YC processing unit 106, image data is temporarily stored in the buffer memory 111, and image processing is performed while reading the image data stored in the buffer memory 111 as needed. Is called.

  Image data (uncompressed) output from the image processing unit 104 is input to the controller 110. The controller 110 outputs the image data output from the image processing unit 104 to the display driver 116. The display driver 116 converts input image data (digital signal) into an analog image signal, and adjusts the size of the image based on the analog image signal to a size that can be displayed on the display 117.

  Further, the display driver 116 controls the display 117 to display an image. At this time, the image displayed on the display 117 is an image (through image) generated by continuously performing signal processing in the CCD 102, the ADC 103, and the image processing unit 104.

  When the user operates the shooting button 123 at a desired timing while the through image is displayed on the display 117, the controller 110 outputs a control signal to the image processing unit 104. When a control signal is input to the image processing unit 104, the compression unit 108 stores the image data subjected to image processing in the preprocessing unit 105 and the YC processing unit 106 in the buffer memory 111, and performs compression processing.

  Specifically, discrete cosine transform processing (DCT processing) that digitizes the ratio of high-frequency components and low-frequency components of image data, and quantization that expresses the gradation and gradation of an image with the number of quantization bits Processing, Huffman coding processing for dividing a signal character string of image data into fixed bits and giving a shorter code to a character string having a high appearance frequency is performed. The compressed image data is displayed on the display 117 via the controller 110 and the display driver 116. The compressed image data is recorded on the recording medium 115 via the card holder 114.

If the zoom button 125 is operated before the shooting button 123 is operated, the controller 110 can execute an optical zoom process or an electronic zoom process to enlarge or reduce the size of the image. When the shooting button 123 is operated after the zoom button 125 is operated, the image data of the enlarged or reduced image is recorded on the recording medium 115.
The hardware configuration and the imaging function of the imaging apparatus that implements the present invention have been described above with reference to FIG.

  Next, the process of this invention is demonstrated using FIGS. First, FIG. 2 is a diagram showing a module of a program that operates in the photographing apparatus according to the embodiment of the present invention.

  The identification information registration module 201 has a function of recognizing a person's face image from a photographed image, generates identification information from the recognized face image, and converts the identification information data, face recognition information data, and identification group data into Has the function to register. The identification information matching module 202 has a function of recognizing a person's face image from a photographed image, and has a function of matching whether the recognized face image matches the feature registered in the identification information. After the collation, an imaging number counting module 203 described later is executed.

  The photographing number counting module 203 has a function of recording the number of photographed images in the identification information count data table based on the result collated by the identification information collating module 202. The identification result display module 204 has a function of referring to the identification information count data, the identification information data, the face recognition information data, and the identification group data, and displaying a list of identification numbers in units of identification IDs.

  FIG. 3 is a diagram illustrating an example of the identification information data table held by the imaging apparatus according to the embodiment of the present invention. In the identification information data table, one record is created for each person to be identified.

  The identification ID 301 of the identification information data table sets a character string that is unique within the identification information data table. As the name 302 of the identification information data table, an arbitrary alphanumeric kana is set for each identification ID. The set character is displayed when the recognition result is displayed on the display. The identification group 303 is a group for classifying identification information in the present embodiment, which is set on the identification information registration screen of FIG. 3 is set through an identification information registration screen shown in FIG. 10 to be described later.

  FIG. 4 is a diagram illustrating an example of a face recognition information data table held by the photographing apparatus according to the embodiment of the present invention. In the face recognition information data table, one record is created for each identification ID 401 in the identification information data table. The identification ID 401 sets a character string that is unique in the face recognition information data table. The image name 402 is set by the file name of the face image registered as identification information on the identification information registration screen shown in FIG.

  FIG. 5 is a diagram illustrating an example of an identification group data table held by the imaging apparatus according to the embodiment of the present invention. The identification group data table is data obtained by grouping subject identification information into units that the photographer wants to measure. In this figure, it is classified into two groups, “A group” and “B group”.

  The identification group ID 501 sets a character string that is unique in the identification group data table. The group name 502 sets a group name with an arbitrary character string for the identification group ID 501. The set characters are displayed on the display when selecting the identification group. An identification group registration and selection method will be described later with reference to FIG.

  The identification ID 503 sets the value of the identification ID 301 of the identification information data table that is the target of the identification group. Separate multiple settings with a comma. Further, here, the value of the identification ID 301 is set as an example, but it is of course possible to use the value of the name 302 of the identification information data table.

  The minimum number of shots 504 is set to the minimum number of shots that indicates how many photos of subjects belonging to each group should be taken. In the present embodiment, the minimum number of shots is set in each identification group in FIG. In addition to this method, for example, a method of setting the minimum number of shots for each subject on the identification information registration screen of FIG. 10 is also possible. In this case, the value of the minimum number of shots is also added to the identification information data table of FIG. 3 for each identification information (subject).

  FIG. 6 is a diagram illustrating an example of the identification information count data table held by the imaging apparatus according to the embodiment of the present invention. The identification information count data table creates a record for each identification ID belonging to the identification group selected as the identification target.

  The identification ID 601 sets the identification ID set in the identification ID 503 of the identification group data selected as the identification target. Further, the identification ID 601 in this figure has the same value as the identification ID 301 in FIG.

  The initial value of the count 602 is set to 0 for each identification ID 601, and a numerical value as a result of collation is set by the shooting number counting module 303.

  FIG. 7 is a diagram illustrating an example of an image data table held by the photographing apparatus according to the embodiment of the present invention. The image data table creates a record for each captured image after capturing. The image file 701 sets the name of the captured image file.

  The identification ID 702 is set to the identification ID when the identification information collating module 202 matches the image file name 701 as a result of collation. That is, information of a person who can be identified as a result of face recognition is set in one image. If there are a plurality of identification IDs as a result of identification, all the identification IDs are set by separating them with a single-byte comma.

  3 to 7 have the same item name (for example, “Identification ID” 301 in FIG. 3, “Identification ID” 601 in FIG. 6 and “Identification ID” in FIG. 7). 702, etc.). From this, it can be seen that the data tables of FIGS. 3 to 7 are each configured as a relational database. These data tables are set at each step of the flowchart to be described next.

  Next, a setting process for photographing and a photographing process according to the present invention will be described with reference to FIGS. 17 and 18. FIG. 17 is a flowchart executed at the time of registration of identification information by the imaging apparatus according to the embodiment of the present invention.

  First, in order to register identification information, in step S <b> 1701, the imaging apparatus 100 accepts a “face recognition registration” button press on the mode selection screen. Alternatively, when the setting mode is selected with the mode dial 122 and the setting screen (not shown) is displayed, the item selection of “identification information registration” is accepted. At this time, the mode selection screen shown in FIG. 8 is displayed on the display unit of the imaging apparatus.

  FIG. 8 is a diagram illustrating an example of a screen for changing to an identification information registration mode for registering identification information displayed on the display unit of the imaging apparatus according to the embodiment of the present invention.

  The “select shooting mode” button 801 is a button that receives a press when performing normal shooting (starting “shooting mode”). The “select count shooting mode” button 802 uses the identification information (face recognition information) registered in the imaging apparatus of the present invention to perform shooting while collating a person (activates “count shooting mode”). ) Is a button that accepts pressing in this case.

  The “select identification information registration mode” button 803 is a button that receives a press when registering identification information (starting “identification information registration mode”) in step S1701. When the pressing of the “identification information registration mode” button 803 is accepted, the imaging apparatus 100 further displays an “identification information registration content selection screen” shown in FIG.

  FIG. 9 is a diagram illustrating an example of a screen that is displayed on the display unit of the imaging apparatus according to the embodiment of the present invention and is changed to an identification information registration screen for registering identification information.

  The “register identification group” button 901 is selected when a new identification group is created. An identification group is a group category of subjects that are selected simultaneously when registering identification information (face recognition information). For example, when the present invention is considered as an imaging device used in a group trip, the passengers on the first car and the second car of the bus are classified by the identification group registration function of FIG. It becomes possible to manage the number of shots of (a group travel participant).

  An identification information registration button 902 is selected when setting identification information (subject information) for a captured image.

  Upon accepting pressing of the “identification group registration button” 901, the imaging apparatus 100 displays an “identification group registration screen (not shown)” on the display unit and accepts registration of the group name.

  Returning to the description of FIG. 17, when “identification information registration mode” is selected, the face of a person to be registered as identification information is photographed in step S1702. The image captured in step S1702 is set in the face recognition image 402 of the face recognition information data table in FIG. 4, and a unique character string is set and registered in the face recognition information ID 401.

  If it is desired to register the identification information of a plurality of persons, in step S1703, confirmation is made as to whether or not there are other target persons. If there are other target persons to be registered, the process returns to step S1702 to execute photographing. Then, steps S1702 to S1703 are repeated.

  When it is determined that the shooting of all the subjects has been completed (in the confirmation screen after shooting (not shown)), in response to the question of whether there are other subjects, select the “No (all shots)” button. If accepted, the process proceeds to step S1704, and the imaging apparatus 100 registers the identification information data table of FIG. 3 based on the captured image. At this time, the identification information registration screen of FIG. 10 is displayed on the display 117 of the imaging apparatus 100.

  Here, the identification information registration screen of FIG. 10 will be described. FIG. 10 is a diagram showing an example of an “identification information registration screen” for registering identification information displayed on the display unit of the imaging apparatus according to the embodiment of the present invention.

  The captured image 1001 displays a face image of a person whose identification information is registered. The face image displayed at this time is one obtained by automatically trimming a part of the face image recognized in step S1702 of FIG. 17 and stored. Further, it may be an image in a state where trimming is not performed (as it is at the time of photographing). The identification ID label 1002 displays an identification ID assigned to the identification information so as to be unique. The character string set by the identification ID label 1002 may be a character string automatically assigned by the imaging apparatus 100 or may be arbitrarily set by the user.

  The name label 1003 is a place where an arbitrary character string set for the identification information is displayed. When setting this item, a software keyboard is displayed on the display unit of the imaging apparatus 100 to accept setting of a name. The identification group drop-down list 1004 displays which identification group the identification information belongs to. The identification group drop-down list 1004 is selected and selected from the displayed identification groups. The identification group will be described in detail separately in the UI of FIG.

  A registration button 1005 is a button for registering the contents of the identification ID 1002, the name label 1003, and the identification group drop-down list 1004 set on the identification information registration screen as identification information. A cancel button 1006 is a button for canceling the contents set in the name label 1003 and the identification group drop-down list 1004, and registration of identification information is not executed. If the user presses the cancel button 1006, the image registered in step S1702 is automatically deleted from the recording medium.

FIG. 11 is a diagram showing an example of a screen for registering an identification group displayed on the display unit of the photographing apparatus according to the embodiment of the present invention. The identification group registration screen of FIG. 11 is displayed upon accepting the pressing of the identification group information registration button 901 of FIG.
The identification group name 1101 displays the registered identification group name.

  The minimum number of shots 1102 displays the minimum number of shots for each piece of identification information (subject) registered in the identification group. A minimum photograph number increase / decrease button 1103 increases the value displayed in the minimum number of shots 1102 by 1 when a button labeled “+” is pressed, and is displayed in the minimum number of shots 1102 when a button labeled “−” is pressed. Decrease the current value by one.

  A registration button 1104 registers the contents set in the identification group name 1101 and the minimum number of shots 1102 as identification group information.

  A cancel button 1105 cancels the contents set in the identification group name 1101 and the minimum number of shots 1102, and returns to the identification information registration content selection screen in FIG. 9 without registering the identification group information.

  Subsequently, returning to the description of FIG. 17, in step S1704, the imaging apparatus 100 displays the identification information registration screen of FIG. At this time, the information of “identification ID 401” in the face recognition information data table of FIG. 4 set at the time of shooting in the above-described step S1702 is used as a key, and the same as “identification ID 301” of the identification information data table of FIG. Set the value. The name 302 sets the content displayed on the name label 1003, and the image name 403 is set to the image taken in step S1702. The identification information data table in FIG. 3 is generated by the registration process in step S1704.

  Next, FIG. 18 is a flowchart executed at the time of shooting by the imaging device according to the embodiment of the present invention and at the time of determination of the identification result.

  First, in order to shoot while collating a person using identification information, in step S1801, the imaging apparatus 100 displays a mode selection screen in FIG. 8 and accepts a “face recognition registration” button press. Alternatively, when the setting mode is selected with the mode dial 122 and a setting screen (not shown) is displayed, an item selection 802 of “count photographing mode” is accepted. At this time, the mode selection screen shown in FIG. 8 is displayed on the display unit of the imaging apparatus. If the mode selection screen is not displayed, it is confirmed that the mode dial 122 is set to “count shooting mode”, and if it is set, the process proceeds to step S1802.

  In step S1802, the imaging apparatus 100 accepts selection of an identification group to be imaged in order to execute the “count imaging mode”. The identification group selection screen shown in FIG. 12 is displayed at this time.

  FIG. 12 is a diagram showing an example of a screen for selecting an identification group when photographing is performed while collating a person using identification information displayed on the display unit of the photographing apparatus according to the embodiment of the present invention. It is. Group buttons 1201 and 1202 are buttons for selecting an identification group to be photographed when photographing, and the selected identification group is set as a collation target. In this embodiment, FIG. 9 shows that “A group” and “B group” are set as identification groups in FIG. 9 and the identification group setting screen (not shown).

  Further, by selecting a period during which the identification information belonging to the identification group is captured, it is possible to determine how many images have been captured within the selected period. For example, when taking a group trip or the like, a minimum of five images per day must be included in the image, and the shooting period is set to the current day (for example, from April 1, 2011 to April 1, 2011). If it is set, it is possible to extract the number of pieces of imaged identification information on the shooting date.

  In step S1802, when the selection of the identification group of the subject to be photographed and the period during which the subject (identification information) was photographed is accepted (when the button 1201 or 1202 is pressed), the identification group data table of FIG. An identification information count data table (FIG. 6) is created based on the value of the identification ID 503 set (belonging to) the identification group that has received the selection.

When the selection of the identification group and the setting of the shooting period of the subject belonging to the identification group are completed, the imaging apparatus 100 enters a standby state in the shooting state, and starts shooting in step S1803. When the shutter press is accepted and pressed halfway, the focus function of the imaging device is activated.
Face recognition is performed by detecting the subject captured by the lens.

  In step S1804, the identification information matching module 202 of the imaging apparatus 100 has identification information that matches the portion of the captured image that can be recognized by the face recognition information stored in the face recognition information data table of FIG. Determine whether or not. If the identification information is stored as a result of the determination, the process proceeds to step S1806. If no identification information is stored, the process advances to step S1805.

  In step S1806, the imaging apparatus 100 displays the identification information image 1301 and the identification subject 1302 on the display unit (display 117). The screen example displayed at this time is a captured image confirmation screen when the face recognition information in FIG. 13 matches.

  FIG. 13 is stored in the face recognition information data table at the time of shooting when shooting is performed while collating a person using identification information displayed on the display unit of the shooting apparatus according to the embodiment of the present invention. It is a figure (an imaging | photography number display screen) which shows an example when it corresponds with the identification information of the to-be-photographed object. The captured image is displayed on the display 117 of the imaging apparatus 100, and the face recognition information whose matching result matches the identification information stored in the identification information matching module 202 at the time of shooting is displayed in an identifiable manner as indicated by 1302. (For example, a face-recognized part is surrounded by a thick dotted line). Further, as indicated by reference numeral 1301, the matching face recognition information displays the name and the number of images in which the identification information is photographed within the period selected in step S1802, together with the face image (identification information). According to this display method, it is possible to instantly confirm who has been identified as a subject and how many images of each subject (identification information) have been taken.

  In step S1805, the imaging apparatus displays a non-identified subject on the display unit (display 117). The screen example displayed at this time is a captured image confirmation screen when the face recognition information in FIG. 13 does not match.

  FIG. 15 shows a screen displayed when the image is not matched with the identification information at the time of photographing when photographing while collating a person using the identification information displayed on the display unit of the photographing apparatus according to the embodiment of the present invention. It is a figure which shows an example.

  When the identification information collation module 302 does not match the identification information with the collation result at the time of shooting, the non-identification subject 1401 is surrounded by a dotted line and displayed. Also, “ZZZZZ” is set in the identification ID 601 as the non-identification data in the identification information count data table, and a value obtained by adding the total number of the non-identification subjects 1501 to the count 602 is set and registered.

  Returning to the description of FIG. 18, in step S1807, the imaging apparatus 100 acquires the number of images of the recognized subject (identification information) captured within the period set in step S1802, and determines whether the minimum number of images has been reached. I do.

  As a method, an image is extracted from the image data table of FIG. 7 using the identification ID of identification information already identified as a subject as a key. For example, if a person with identification ID “00002” is identified as a subject in the current shooting, an image including s-tree-specific information “00002” is extracted from the image data table of FIG.

  Next, an image captured within the period set in the previous step S1802 is extracted from the extracted images. If “from April 1, 2011 to April 4, 2011” is set in step S1802, “IM00001.jpg” and “IM00003.jpg” are extracted. It is determined whether or not the number of captured images extracted here is larger than the value of the minimum number of captured images 504 set in FIG. If the number of shot images has reached the minimum number of shots (YES), the process proceeds to step S1808, and if not (NO), the process proceeds to step S1809.

  In step S1808, the imaging apparatus 100 issues a warning that the identification information identified as the subject has already reached the minimum number of shots. For example, in FIG. 14, “Identified Maruko” has already been photographed in seven images, and in 1403, a warning message “Identified Maruko has already been photographed in the minimum number” is displayed. Yes.

  In step S1809, the imaging apparatus 100 issues a warning that the identification information identified as the subject has not reached the minimum number of shots. For example, in FIG. 14, “identified Kanda” has been photographed in only one image, so a warning message “Please shoot another four, Mr. Kanda is identified” is displayed. At this time, it is possible to store a lot of identification information in an image with a better balance by displaying a specific instruction of “remaining ○”.

  In step S1810, the imaging apparatus 100 determines whether or not the number of captured images for all detected subjects (identification information) has been confirmed. That is, it is determined whether or not any warning has been given to all detected subjects (identification information) in step S1808 or step S1809. If warnings for all subjects have been completed (YES), the process proceeds to step S1811. If the warning for all the subjects has not ended (NO), the process returns to step S1807.

  In step S1811, the imaging apparatus 100 receives an instruction to fully press the shutter, and captures an image. The photographed image is stored in the image data table (FIG. 7), and the number of times the identification information that has been identified as the subject of the image is photographed is counted up in the identification information count data table in FIG.

  If it is determined in step S1812 whether or not to shoot the person using the identification information (specifically, it is determined whether a menu switching instruction or a power-off instruction has been received), and if shooting continues, step S1803 is executed. The process of S1811 is repeated. When not continuing, it progresses to step S1813 and a photography result is displayed.

  In step S1813, the identification information count data table, the identification information data table, and the face recognition information data table are referred to, and the identification result list display screen shown in FIG. 16 displaying the number of shots for each subject is displayed.

  FIG. 16 shows an example of a screen for displaying an identification result when photographing is performed while collating a person using identification information displayed on the display unit of the photographing apparatus according to the embodiment of the present invention. FIG.

  The name 1601 of the identification information is the same as the name displayed in 1301 of FIG. 13 when the name of the identification information photographed on the image is displayed and the subject is identified on the photographing screen. The identification information image 1602 is an image for each identification information set in FIG. The number of shots 1603 indicates the number of shots whose identification information has been shot so far. The shooting state display text 1604 indicates the difference between the number of shots 1603 and the minimum number of shots set for each identification group (or each identification information), and when the minimum number of shots has already been reached. Displays “Minimum number of shots already taken” is displayed. If the minimum number of shots has not been reached, “The number of shots left until the minimum number of shots.” Is displayed.

  Above, description of the process performed with the imaging device of this invention is finished. In the embodiment of the present invention, if the display unit (display) 6 of the imaging apparatus has a touch panel function, the instruction reception from the user is not the operation by the cursor key of the operation input unit. Instead, the control can be executed by directly accepting the pressing to the display unit (display) 6.

  As described above, according to the present invention, there is provided a mechanism for registering information on a target person in advance, counting the number of shots for each target person by recognizing the face at the time of shooting, and preventing shooting with bias toward a specific person. It becomes possible to do.

  Specifically, the imaging apparatus according to the present invention, in an imaging apparatus that captures a subject using face recognition, associates the information on the subject and the face image of the subject specified by the face recognition as identification information. The identification information registration means to be registered (steps S1701 to S1704 in FIG. 17), the identification information of the subject registered by the identification information registration means, and the number of images in which the subject is photographed are linked and stored and managed. Storage management means (FIGS. 3, 4, 5, 6, and 7), detection means for detecting a subject at the time of shooting (step S1803 in FIG. 18), and identification information of the subject detected by the detection means Determining means (step S1804 in FIG. 18) for determining whether or not the identification information coincides with the identification information of the subject registered by the identification information registering means; Is determined to match the face image of the subject registered by the identification information registration means, the number-of-photographing display means for displaying the number of images taken of the subject (steps S1808 and S1809 in FIG. 18). And warning means (steps S1808 and S1809 in FIG. 18) for giving a warning regarding the number of shots of the subject in accordance with the number of shots of the subject displayed by the display means.

  As described above, the program in the present invention is a program that can be executed by a computer as an imaging apparatus in accordance with the processing of the flowcharts shown in FIGS. 17 and 18, and the storage medium of the present invention uses the processing method in FIGS. A program executable by the imaging apparatus is stored. Note that the program in the present invention may be a separate independent program for each processing method of each apparatus shown in FIGS.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by executing the reading. In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the imaging apparatus, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer is actually executed based on the instructions of the program. It goes without saying that a case where the function of the above-described embodiment is realized by performing a part or all of the process and the process is included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention. In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

  In addition, each process realized by the above-described software can be realized as firmware or hardware configuration, and each process can be realized as each means. The technical scope of the present invention is realized by such firmware or hardware configuration. Is also included.

100 Imaging device 101 Optical system (lens)
102 Image sensor (CCD)
103 Camera Signal Processing Unit 110 System Controller 117 Display 120 Operation Unit

Claims (7)

In an imaging device that captures a subject using face recognition,
Identification information registration means for associating and registering the subject information and the face image of the subject specified by the face recognition as identification information;
Storage management means for storing and managing the identification information of the subject registered by the identification information registration means and the number of images in which the subject is captured;
Detection means for detecting a subject at the time of shooting;
Determination means for determining whether or not the identification information of the subject detected by the detection means matches the identification information of the subject registered by the identification information registration means;
When the determination unit determines that the identification image of the subject matches the face image of the subject registered by the identification information registration unit, the number-of-photographing display unit displays the number of images taken of the subject When,
An image pickup apparatus comprising: warning means for performing a warning regarding the number of shots of the subject according to the number of shots of the subject displayed by the shot number display means. The determination unit includes a comparison unit that compares a value of the number of images in which the identification information of the subject detected by the detection unit is captured with a preset value of the minimum number of images to be captured,
If the value of the number of images for which the identification information has been photographed by the comparing means is greater than the preset number of images to be photographed, the warning means has already photographed the minimum number of images. If the value of the number of shots of the identification information taken by the comparison means is smaller than the preset number of shots, the identification information reaches the minimum number of shots. The image pickup apparatus according to claim 1, wherein a warning is issued for a number of sheets required to perform the operation. A selection receiving means for receiving selection of an identification group to be subjected to detection of a subject at the time of shooting and a period in which an image of identification information belonging to the identification group was shot;
The imaging device according to claim 1, wherein the detection unit detects an image of identification information belonging to the identification group received by the selection reception unit. It further comprises imaging means for taking an image,
When the subject registered by the identification information registration unit is photographed in the image captured by the imaging unit, a photographing result display for displaying a part where the subject is face-recognized and information for identifying the recognized subject The imaging apparatus according to claim 1, further comprising means.   The imaging apparatus according to claim 1, wherein the identification information registration unit registers information of an identification group that classifies the subject into a group as the identification information. A control method for an imaging apparatus that includes storage management means for storing and managing subject identification information and the number of images in which the subject is photographed, and photographing the subject using face recognition,
An identification information registration step in which identification information registration means of the imaging device associates and registers the subject information and the face image of the subject specified by the face recognition as identification information;
A detection step in which the detection means of the imaging device detects a subject at the time of shooting;
A determination step of determining whether or not the identification information of the subject detected in the detection step matches the identification information of the subject registered in the identification information registration step;
When the number-of-photographing number display unit of the imaging apparatus determines that the identification image of the subject matches the face image of the subject registered by the identification information registration unit in the determination step, the subject is captured A number of shots display step for displaying the number of images;
A control method, wherein the warning means of the imaging apparatus includes a warning step of issuing a warning regarding the number of shots of the subject in accordance with the number of shots of the subject displayed in the shooting number display step. A program for controlling an image pickup apparatus for photographing a subject using face recognition, comprising storage management means for storing and managing subject identification information and the number of images taken of the subject.
The imaging device;
Identification information registration means for associating and registering the subject information and the face image of the subject specified by the face recognition as identification information;
Detection means for detecting a subject at the time of shooting;
Determination means for determining whether or not the identification information of the subject detected by the detection means matches the identification information of the subject registered by the identification information registration means;
When the determination unit determines that the identification image of the subject matches the face image of the subject registered by the identification information registration unit, the number-of-photographing display unit displays the number of images taken of the subject When,
A computer-readable and executable program that functions as warning means for issuing a warning regarding the number of shots of the subject in accordance with the number of shots of the subject displayed by the shot number display means.

Images