JP5016999B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program for detecting and recognizing a human face.

  Currently, a technique for recognizing a specific individual from a person in a moving image is known (see Patent Document 1 and Patent Document 2). According to Patent Document 1, by installing a security camera in an automatic ticket vending machine at a station and performing subject recognition (recognition of a specific individual), it is possible to prevent passengers who are not desirable in terms of security. Further, according to Patent Document 2, an entertainment property is improved by mounting a subject recognition function on a robot or the like.

By the way, in order to perform subject recognition, it is generally necessary to compare a large number of pre-registered subjects (person's face) with subjects (person's face) detected in the captured image. is there. Therefore, a relatively high processing capability is required to perform subject recognition. Therefore, as disclosed in Patent Literature 1 and Patent Literature 2, in order to perform subject recognition in a moving image in real time, an imaging device such as a security camera needs to have abundant calculation resources.
JP 2003-187352 A Japanese Patent Laid-Open No. 2003-271958

  However, when abundant computing resources are installed in the imaging apparatus, various problems such as an increase in cost, an increase in power consumption, and an increase in the size of the imaging apparatus occur. Such a problem is particularly serious in an imaging apparatus such as a digital camera that is desired to be downsized. If subject recognition is performed by an imaging apparatus having only limited computing resources, it takes a relatively long time from image acquisition to completion of recognition processing. Therefore, a specific individual cannot be tracked in real time and is not practical.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a technique capable of executing subject recognition processing at high speed while suppressing an increase in processing load in an imaging apparatus.

In order to solve the above-described problem, the first aspect of the present invention provides an imaging unit that captures an optical image of a subject to acquire a captured image, a detection unit that detects a human face in the captured image, and a person by the detection unit Main recognition means for recognizing an individual with respect to a person's face detected by the detection means, using feature information relating to characteristics of the individual's face acquired from the storage unit over a longer processing time than detection of A first providing means for providing identification information to each face of the person detected in the first captured image, and a person detected in the captured image acquired later in two captured images acquired before and after in time The difference between the face and the position of any of the faces of the persons to whom identification information has been assigned in the captured image obtained earlier is less than the threshold, and the difference in size is less than the threshold Small And a process of the main recognizing unit for the face of the person detected in the first captured image, and a second providing unit for adding the identification information given to the one of the faces to a person's face satisfying either of them. While the processing of the main recognition unit is performed on the face of the person detected in the second captured image acquired after completion, the second assigning unit is applied to the captured image acquired continuously in time. The identification information is repeatedly given by the same as the individual recognized by the main recognition unit in the first captured image among the faces of the person detected in the third captured image acquired after the second captured image. a face identification information is assigned, to provide an imaging apparatus characterized by comprising an auxiliary recognizing means for recognizing a personal said main recognition means recognized in the first captured image

According to a second aspect of the present invention, the imaging means captures an optical image of a subject to acquire a captured image, the detection means detects a human face in the captured image, and a main recognition means. However, the main recognition step of recognizing an individual with respect to the face of the person detected in the detection step using feature information related to the characteristics of the individual face acquired from the storage unit over a longer processing time than the detection step When the first applying means comprises a first step of applying the identification information on each face of the person detected in the first image, the second application means, the two imaging obtained by temporally preceding In the image, the difference between the face of one of the faces of the person who has been given identification information in the captured image acquired earlier among the faces of the person detected in the captured image acquired later is equal to or less than the threshold value. And size A second providing step of adding identification information given to any one of the faces satisfying at least one of the difference being equal to or less than a threshold value, and an auxiliary recognition unit in the first captured image; While the main recognition process is being performed on the detected human face in the second captured image acquired after the completion of the main recognition process on the detected human face, it is acquired continuously in time. In the first captured image among the human faces detected in the third captured image acquired after the second captured image, the identification information is repeatedly applied to the captured image by the second providing unit. recognizing auxiliary recognize faces said main recognition personalization same identification information recognized by the process is given, as a person who is recognized by the main recognition process in the first image To provide an imaging method, characterized by comprising: a degree, the.

The third present invention, a computer, an imaging means for acquiring a captured image by capturing an optical image of an object, detecting means for detecting a human face in the captured image, the detection of the face of a person by the detecting means The main recognition means for recognizing the person's face detected by the detecting means using the feature information relating to the characteristics of the person's face acquired from the storage unit over a long processing time, detected in the first captured image First assigning means for assigning identification information to each of the human faces obtained, among the two picked-up images acquired before and after the time, of the human faces detected in the captured image acquired later, At least one of the difference between the face and the position of any face of the person to whom identification information is given in the acquired captured image is equal to or less than the threshold, and the difference in size is equal to or less than the threshold After the completion of the processing of the main recognition means for the second giving means for giving the identification information given to any of the faces to the face of the person satisfying the condition, and the face of the person detected in the first captured image While the process of the main recognition unit is performed on the face of the person detected in the acquired second captured image, the second providing unit identifies the captured images acquired continuously in time. By repeatedly giving information, the same identification as the individual recognized by the main recognition unit in the first captured image among the faces of persons detected in the third captured image acquired after the second captured image A program for causing a face to which information is given to function as auxiliary recognition means for recognizing an individual recognized by the main recognition means in the first captured image is provided.

  Other features of the present invention will become more apparent from the accompanying drawings and the following description of the best mode for carrying out the invention.

  With the above configuration, according to the present invention, it is possible to execute subject recognition processing at high speed while suppressing an increase in processing load in the imaging apparatus.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The individual embodiments described below will help to understand various concepts from the superordinate concept to the subordinate concept of the present invention.

  The technical scope of the present invention is determined by the claims, and is not limited by the following individual embodiments. In addition, not all combinations of features described in the embodiments are essential to the present invention.

[First Embodiment]
<Configuration of imaging device>
An embodiment in which an imaging apparatus of the present invention is applied to a digital camera will be described.

  FIG. 1 is a block diagram illustrating a functional configuration of the digital camera 100 according to the first embodiment.

  Reference numeral 101 denotes an optical system including a zoom lens, a focus lens, and a stop, 102 denotes a mechanical shutter, 103 denotes an image sensor, and 104 denotes a CDS circuit that performs analog signal processing. Reference numeral 105 denotes an A / D converter that converts an analog signal into a digital signal. Reference numeral 106 denotes a timing signal generation circuit that generates signals for operating the image sensor 103, the CDS circuit 104, and the A / D converter 105. .

  Reference numeral 107 denotes a drive circuit that drives the optical system 101, the mechanical shutter 102, and the image sensor 103, and reference numeral 108 denotes a signal processing circuit that performs necessary signal processing on the captured image data. The signal processing circuit 108 includes a face recognition circuit 122 and a face detection circuit 126.

  The face detection circuit 126 detects a human face in the captured image represented by the digital signal (that is, image data) output from the A / D converter 105. The face recognition circuit 122 performs recognition processing (processing for recognizing an individual) for the face of the person detected by the face detection circuit 126. In order to shorten the time required for the face detection process, the face detection circuit 126 may detect a human face in the captured image represented by the image data resized by the signal processing circuit 108.

  A known face detection technique can be used for face detection in the present embodiment.

  As a known face detection technique, a method based on learning using a neural network or the like, template matching is used to search a part having a characteristic shape of eyes, nose, mouth, etc. from an image, and if the degree of similarity is high, it is regarded as a face There are methods. In addition, many other methods have been proposed, such as a method that detects image feature amounts such as skin color and eye shape and uses statistical analysis. In general, a plurality of these methods are combined to improve the accuracy of face detection.

  Specific examples include a face detection method using wavelet transform and image feature amount described in JP-A-2002-251380.

  A known face recognition technique can also be used for face recognition in the present embodiment. For example, the face recognition circuit 122 performs face recognition processing by acquiring feature information related to personal facial features from a storage unit such as the ROM 115 and comparing it with the facial feature information detected by the face detection circuit 126. be able to.

  An image memory 109 stores image data signal-processed by the signal processing circuit 108, results of face recognition processing and face detection processing by the face recognition circuit 122 and the face detection circuit 126, and the like. Reference numeral 110 denotes a recording medium such as a memory card that can be removed from the digital camera 100, and reference numeral 111 denotes a recording circuit that records the image data processed by the signal processing circuit 108 on the recording medium 110.

  Reference numeral 112 denotes an image display device such as a liquid crystal display that displays image data, and reference numeral 113 denotes a display circuit that displays the image data signal-processed by the signal processing circuit 108 on the image display device 112. Reference numeral 114 denotes a system control unit that controls the entire digital camera 100. Reference numeral 115 denotes a nonvolatile memory (ROM) that stores a program executed by the system control unit 114, control data such as parameters and tables used when the program is executed, and correction data such as a scratch address.

  A volatile memory (RAM) 116 temporarily stores the program, control data, and correction data stored in the ROM 115 and is used as a work area when the system control unit 114 executes the program. Reference numeral 117 denotes a strobe.

  The system control unit 114 includes a face recognition operation control unit 123 that controls the operation of the face recognition circuit 122. The optical system 101 includes an IS (Image Stabilizer) lens that performs camera shake correction. A shake detection unit 124 includes a vibration gyro and the like, and can detect a shake of the digital camera 100. The drive circuit 107 performs camera shake correction by driving the IS lens based on information given from the shake detection unit 124.

  Reference numeral 118 denotes an operation unit that is used by the user to set shooting conditions, select a shooting mode, and the like. Reference numeral 121 denotes a main switch (main SW), which is a switch for turning on the power of the digital camera 100. Reference numeral 119 denotes a switch (SW1) that is turned on when a shutter button (not shown) is half-pressed. When the SW1 119 is turned on, the system control unit 114 performs a standby operation for photographing. Reference numeral 120 denotes a switch (SW2) that is turned on when a shutter button (not shown) is fully pressed. When the SW2 120 is turned on, the system control unit 114 performs a photographing operation of recording the image data output from the A / D converter 105 on the recording medium 110 through processing by the signal processing circuit 108 or the like.

  A zoom key 125 is used by the user to control the zoom of the optical system 101.

  Hereinafter, a photographing operation using the mechanical shutter 102 using the digital camera 100 configured as described above will be described. Prior to the shooting operation, when the operation of the system control unit 114 is started (for example, when the digital camera 100 is turned on), the system control unit 114 transfers necessary programs, control data, and correction data from the ROM 115 to the RAM 116. To remember. In addition to these programs and data, the system control unit 114 can also transfer and store additional programs and data from the ROM 115 to the RAM 116 as necessary. Further, the system control unit 114 may directly read and use data in the ROM 115.

  First, the optical system 101 drives an aperture and a lens according to a control signal from the system control unit 114 to form an optical image of a subject set to an appropriate brightness on the image sensor 103. Next, the mechanical shutter 102 is driven by the control signal from the system control unit 114 so as to shield the image sensor 103 in accordance with the operation of the image sensor 103 so that the exposure time is appropriate. At this time, when the image sensor 103 has an electronic shutter function, the exposure time may be adjusted in combination with the mechanical shutter 102.

  The image sensor 103 is driven by a drive pulse based on an operation pulse generated by the timing signal generation circuit 106 controlled by the system control unit 114. The image sensor 103 converts an optical image of a subject into an electrical signal by photoelectric conversion and outputs it as an analog image signal. From the analog image signal output from the image sensor 103, the clock synchronization noise is removed by the CDS circuit 104 by the operation pulse generated by the timing signal generation circuit 106 controlled by the system control unit 114. Then, it is converted into a digital image signal (that is, image data) by the A / D converter 105.

  Next, the signal processing circuit 108 controlled by the system control unit 114 performs color conversion processing, white balance processing, gamma correction processing, resolution conversion processing, image compression processing, and the like on the image data. The image memory 109 is used for temporarily storing the image data being processed by the signal processing circuit 108, storing the image data subjected to signal processing, and temporarily storing the result of the face recognition processing. It is done.

  The image data processed by the signal processing circuit 108 and the image data stored in the image memory 109 are converted into data suitable for the recording medium 110 (for example, file system data having a hierarchical structure) by the recording circuit 111. It is recorded on the recording medium 110. Further, the image data is subjected to resolution conversion processing by the signal processing circuit 108 and then converted into a signal suitable for the image display device 112 (for example, NTSC or PAL analog signal) by the display circuit 113 and the image display device. 112.

  Here, the system control unit 114 outputs the image data output from the A / D converter 105 to the image memory 109 and the recording circuit 111 without performing signal processing on the signal processing circuit 108 using the control signal. Can also be instructed. Further, the signal processing circuit 108, when requested by the system control unit 114, information of image data generated during the signal processing (for example, the spatial frequency of the image, the average value of the designated area, the data amount of the compressed image) Etc.) is output to the system control unit 114. Alternatively, the signal processing circuit 108 outputs information extracted from these pieces of information to the system control unit 114. Further, the recording circuit 111 outputs information such as the type and free capacity of the recording medium 110 to the system control unit 114 when requested by the system control unit 114.

  Next, the reproduction operation of image data recorded on the recording medium 110 will be described. In response to a control signal from the system control unit 114, the recording circuit 111 reads image data from the recording medium 110. When the image data is compressed, the signal processing circuit 108 performs an image expansion process according to a control signal from the system control unit 114 and stores it in the image memory 109. The image data stored in the image memory 109 is subjected to resolution conversion processing by the signal processing circuit 108, converted to a signal suitable for the image display device 112 by the display circuit 113, and displayed on the image display device 112. .

<Face detection processing and face recognition processing>
FIG. 2 is a flowchart showing a flow of photographing processing using the digital camera 100. The processing in this flowchart is executed when the digital camera 100 is turned on and the digital camera 100 is set via the operation unit 118 or the like to perform face recognition processing. Although not shown, while the processing of the flowchart of FIG. 2 is being executed, the A / D converter 105 and the signal processing circuit 108 continuously capture an optical image of the subject to obtain image data. Is generated. The image data used in the following steps is image data generated in this way.

  In S201, the face detection circuit 126 detects a human face in the captured image represented by the image data output from the A / D converter 105 (face detection processing). The face detection circuit 126 temporarily stores face information of the detected face in the image memory 109. The face information includes at least one of the position and size of the face in the captured image. If no face is detected, information indicating that is stored in the image memory 109.

  In step S <b> 202, the system control unit 114 determines whether the face recognition circuit 122 is executing a process for recognizing an individual (face recognition process) for the human face detected by the face detection circuit 126. This determination is performed when the system control unit 114 inquires of the face recognition operation control unit 123 about the state of the face recognition circuit 122.

  The fact that the face recognition circuit 122 is executing the face recognition process means that the face recognition process started before has not been completed yet. In an apparatus with relatively limited computation resources, the face recognition process requires a longer processing time than the face detection process, and thus the face recognition process cannot be performed every time the face detection process is performed. Therefore, determination in S202 is necessary.

  In S202, if the face recognition circuit 122 is executing the face recognition process, the process proceeds to S204, and if not, the process proceeds to S203.

  In S203, the face recognition operation control unit 123 causes the face recognition circuit 122 to start face recognition processing for the face detected in S201. In the face recognition processing, for example, it is determined whether or not a person registered in advance in the ROM 115 is included in the face detected in S201. In S203, the face detection circuit 126 gives identification information to each face (face of the face recognition process) detected in S201 in accordance with the control of the face recognition operation control unit 123 (first giving means). Here, the identification information does not specify an individual, but indicates whether or not the face detected at a certain timing is the same as the face detected at another timing. Next, the process proceeds to S204.

  In S204, the face detection circuit 126 performs comparison processing. The details of the comparison process will be described later with reference to FIG. 3. Briefly, the face detection circuit 126 compares the face detected in S201 with the previously detected face and detects them at two timings. It is determined whether or not the given faces are the same person.

  More specifically, it is assumed that face detection processing is performed on the first captured image acquired at a predetermined timing, and face recognition processing is performed together. Then, the face detection circuit 126 is a first predetermined one of the faces detected in the second captured image acquired after the processing of the face recognition circuit 122 is completed from among the faces detected in the first captured image. A face satisfying the above relationship is searched (first search means). The face detection circuit 126 recognizes a face in the second captured image corresponding to the face found as a result of the search as an individual recognized by the face recognition circuit 122 (main recognition unit) in the first captured image. Recognize (auxiliary recognition means).

  That is, the face detection circuit 126 does not perform face recognition processing by the face recognition circuit 122 having a long processing time for faces satisfying a first relationship that is predetermined in the first captured image and the second captured image. An individual can be recognized based on the result of the previous face recognition process.

  In step S205, the system control unit 114 controls the display circuit 113 to cause the image display device 112 to display information about the individual recognized by the face recognition circuit 122 or the individual recognized as a result of the comparison processing in step S204. The personal information is information such as the name of a specific individual, for example, and is stored in the ROM 115 together with feature information indicating facial features.

  In step S206, the system control unit 114 determines whether the SW1 119 is ON. If it is ON, the process proceeds to S207, and if not, the process returns to S201.

  In step S207, the system control unit 114 performs shooting standby operations such as AF processing and AE processing using the drive circuit 107, the signal processing circuit 108, and the like. Specifically, the signal processing circuit 108 calculates the subject brightness from the image data output from the A / D converter 105. Next, based on the calculation result, the drive circuit 107 adjusts the exposure time by controlling the diaphragm of the optical system 101, the mechanical shutter 102, and the like. The signal processing circuit 108 detects the focus of the subject from the image data output from the A / D converter 105, and the drive circuit 107 drives the zoom lens of the optical system 101 based on the detection result.

  Here, AF processing and AE processing may be performed based on personal information recognized by face recognition processing or comparison processing. For example, when the photographer's family is included in the subject, the drive circuit 107 drives the optical system 101 and the like so that the focus and exposure with respect to the family are appropriate in preference to other persons. Therefore, the system control unit 114 can set shooting conditions at the time of shooting (when a captured image is acquired) based on an individual recognized by the face detection circuit 126 as auxiliary recognition means.

  In step S208, the system control unit 114 determines whether the SW2 120 is ON. If it is ON, the process proceeds to S210, and if not, the process proceeds to S209.

  In step S209, the system control unit 114 determines whether the SW1 119 remains ON. If it remains ON, the process returns to S208, and if not ON, the process returns to S201.

  In S210, the system control unit 114 performs a shooting operation and returns to S209.

  As described in the above steps, face detection processing, face recognition processing, and comparison processing are performed, and display processing for the image display device 112, photographing operation, and the like are performed based on the results of these processing.

<Details of the comparison process>
Next, a detailed flow of the comparison process in S204 of FIG. 2 will be described with reference to FIG. In this flowchart, the timing at which the face detection circuit 126 performs the face detection process in S201 is time t = n. Then, the face detection circuit 126 compares the detected face with the face detected at the previous timing (t = n−k), and determines whether or not they are the same. Here, k is an interval for performing face detection processing.

  The timing chart of FIG. B. Face detection processing in S201; Face recognition processing in S203, C.I. The time relationship of the comparison process in S204 is represented.

  FIG. 5 is a diagram illustrating faces detected in the captured image at times t = n−k and t = n. “A”, “B”, and “C” are assigned as identification information to the three faces detected at time t = n−k.

  First, in S301 of FIG. 3, the face detection circuit 126 selects one face Cn among the faces detected at t = n.

  In S302, the face detection circuit 126 selects one face Cn-k among the faces detected at t = n−k.

  In step S303, the face detection circuit 126 compares the position of the face Cn (face center coordinates) with the position of the face Cn-k. Assume that the position of the face Cn is (xd, yd) and the position of the face Cn-k is (xa, ya). If the differences ΔX = | xd−xa | and ΔY = | yd−ya | are less than or equal to a predetermined threshold (ThX, ThY or less), respectively (ΔX ≦ ThX, ΔY ≦ ThY), the process proceeds to S304. If not, the process proceeds to S306.

  In S304, the face detection circuit 126 compares the size of the face Cn with the size of the face Cn-k. If the size of the face Cn is Fd and the size of the face Cn−k is Fa, the difference between the two (ΔF = | Fd−Fa |) is less than or equal to a predetermined threshold (ΔF ≦ ThF), the process proceeds to S305. If not, the process proceeds to S306.

  In S305, the face detection circuit 126 determines that the face Cn and the face Cn-k are the same person because the face Cn and the face Cn-k are close to each other, and gives the same identification information to the face Cn as the face Cn-k ( (2nd provision control means). Next, the process proceeds to S308.

  On the other hand, in S306, the face detection circuit 126 determines that the face Cn and the face Cn-k are different people, and proceeds to S307.

  In step S307, the face detection circuit 126 determines whether all the faces detected at t = n−k have been selected as the face Cn−k. If all are selected, the process proceeds to S308, and if not, the process proceeds to S302. Proceeding from S307 to S308 instead of S305 means that a face determined to be the same person as the face Cn was not found at t = n−k. In this case, identification information is not given to the face Cn. Further, the face detection circuit 126 may be configured not to select a face to which no identification information is assigned at t = n−k as a comparison target face Cn−k.

  In S308, the face detection circuit 126 determines whether all the faces detected at t = n have been selected as the face Cn. If all are selected, the process proceeds to S205. Otherwise, the process returns to S301, and the same processing is repeated for the other faces detected at t = n. In the example of FIG. 5, the three faces detected at t−n are selected as Cn and compared with the three faces detected at t = n−k, respectively.

  By repeating these processes at a predetermined timing (time interval k), the same person can be tracked at the time interval k.

  FIG. 4 shows a series of these flows in time series. Face recognition processing: A (n) is performed based on the face detection result: F (n−k) obtained in S201. Even during face recognition processing, face detection processing is performed at time interval k. A comparison process is also performed. For the face determined to be the same person as the face detected at F (n−k) by this comparison processing, the previous face recognition processing: A () until t = n + 9k at which the result of A (n) is obtained. n-9k) results are assigned.

  Thereby, even during face recognition by the face recognition circuit 122, face recognition (auxiliary recognition) is performed by the face detection circuit 126, and an individual of the detected face is specified. Further, even for a face detected after the result of A (n) is obtained, an individual can be recognized based on the result of A (n) even during the process of A (n + 9k).

  The above process is summarized and described as follows. However, the scope of the present invention is not limited to the following description.

  Two captured images acquired continuously in time (for example, a captured image acquired at time t = n−k (captured image acquired earlier)) and the next timing (time t = n) Consider a generated captured image (captured image acquired later)). The face detection circuit 126 satisfies a predetermined second relationship with any one of the human faces detected in the captured image acquired later from among the human faces detected in the previously acquired captured image. A person's face is searched (second search means). The face detection circuit 126 is, for example, a face whose difference from face information detected in a captured image acquired later from a face detected in a captured image acquired earlier is equal to or less than a threshold value. A face having information is searched as a face satisfying a predetermined second relationship. The face information can include at least one of the position and size of the face in the captured image.

  The face detection circuit 126 identifies the face of the person in the captured image acquired later corresponding to the face of the person found in the previously acquired captured image, which is assigned to the face of the discovered person. Information is given (second giving means).

  The face detection circuit 126 is preliminarily selected from any of the faces detected in the second captured image acquired after the completion of the processing of the face recognition circuit 122 related to the first captured image from among the human faces detected in the first captured image. A face that satisfies the defined first relationship is searched (first search means). The face detection circuit 126 recognizes the face in the second captured image corresponding to the face found in the first captured image by the face recognition circuit 122 (main recognition unit) in the first captured image. Recognize as an individual (auxiliary recognition means).

  Here, the faces satisfying the predetermined first relationship are searched as follows. After the acquisition of the first captured image, until the second captured image is reached, the above-described search by the second search means and the provision by the second provision means are repeated for the two captured images obtained in succession in time. It is. As a result, the face in the first captured image to which the same identification information as the identification information given to the face in the second captured image is assigned is a face that satisfies a predetermined first relationship.

  Further, the face detection circuit 126 may decrease the above-described predetermined threshold (ThF) as the number of faces detected in the captured image acquired earlier or in the captured image acquired later increases. . Since the threshold value becomes smaller as the number of faces is more dense, the possibility that the face of another person is erroneously detected as the face of the same person is suppressed.

  In the example of FIG. 4, when the face recognition process by the face recognition circuit 122 is completed, the next face recognition process is started. However, when the face detection circuit 126 can recognize an individual as a result of the comparison process for all detected faces, the system control unit 114 may prohibit the execution of the recognition process by the face recognition circuit 122. This is expected to save power consumption.

  As described above, according to the present embodiment, during the face recognition processing by the face recognition circuit 122, the face detection circuit 126 determines that the detected face is the face to be previously recognized by the face recognition circuit 122 in advance. Whether the first relationship is satisfied is determined. When the predetermined first relationship is satisfied, the face detection circuit 126 recognizes the detected face as an individual recognized by the previous face recognition process by the face recognition circuit 122.

  Accordingly, it is possible to execute subject recognition processing at high speed while suppressing an increase in processing load in the imaging apparatus. In other words, even with a small apparatus with relatively limited computing resources such as the digital camera 100, the detected face individual can be recognized at a practical frequency.

[Other Embodiments]
In order to realize the functions of the above-described embodiments, a recording medium in which a program code of software embodying each function is recorded may be provided to the system or apparatus. The functions of the above-described embodiments are realized by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-described embodiments, and the recording medium on which the program code is recorded constitutes the present invention. As a recording medium for supplying such a program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, or the like can be used. Alternatively, a CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM, or the like can be used.

  The configuration for realizing the functions of the above-described embodiments is not limited to executing the program code read by the computer. Including the case where the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. It is.

  Further, the program code read from the recording medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments may be realized by the processing. Is included.

It is a block diagram which shows the function structure of the digital camera which concerns on 1st Embodiment. It is a flowchart which shows the flow of the imaging | photography process using the digital camera which concerns on 1st Embodiment. It is a flowchart which shows the detailed flow of the comparison process in S204 of FIG. A. B. Face detection processing in S201; Face recognition processing in S203, C.I. It is a timing chart showing the time relation of the comparison processing in S204. It is a figure which shows the face detected in the captured image at the time t = n-k and t = n.

Claims (7)

Imaging means for capturing an optical image of a subject to obtain a captured image;
Detecting means for detecting a human face in the captured image;
An individual is recognized for the person's face detected by the detecting means using characteristic information relating to the characteristics of the person's face acquired from the storage unit over a longer processing time than the detection of the person's face by the detecting means. Main recognition means;
First assigning means for assigning identification information to each face of the person detected in the first captured image ;
Among the two captured images acquired before and after the time, whichever face of the person is given the identification information in the previously acquired captured image among the human faces detected in the captured image acquired later The identification information given to any one of the faces is given to a person's face that satisfies at least one of the difference between the face and the position being equal to or smaller than the threshold and the difference in size being equal to or smaller than the threshold. Second granting means,
While the process of the main recognition unit is being performed on the face of the person detected in the second captured image obtained after the completion of the process of the main recognition unit on the face of the person detected in the first captured image, By repeatedly applying the identification information by the second assigning unit to the captured images acquired continuously in time, the person detected in the third captured image acquired after the second captured image is displayed. Auxiliary recognition means for recognizing a face of the face to which the same identification information as the individual recognized by the main recognition means in the first captured image is given as an individual recognized by the main recognition means in the first captured image When,
An imaging apparatus comprising: The imaging apparatus according to claim 1 , wherein the second assigning unit decreases the threshold as the number of human faces detected in the previously acquired captured image increases. The imaging apparatus according to claim 1 , wherein the second assigning unit decreases the threshold value as the number of human faces detected in the captured image acquired later increases. The apparatus further comprises prohibition means for prohibiting execution of recognition processing by the main recognition means when an individual is recognized by the auxiliary recognition means for all human faces detected in the third captured image. Item 4. The imaging device according to any one of Items 1 to 3 . Based on the individual auxiliary recognition means recognized in the third captured image, any of claims 1 to 4 wherein the imaging means, characterized by further comprising setting means for setting a photographing condition used at the time of acquisition of the captured image The imaging apparatus of Claim 1. An imaging step in which the imaging means captures an optical image of a subject to acquire a captured image;
A detecting step in which the detecting means detects a human face in the captured image;
The main recognition means recognizes the person with respect to the face of the person detected in the detection step by using characteristic information regarding the characteristics of the face of the person acquired from the storage unit over a longer processing time than the detection step. The main recognition process;
A first assigning step in which first assigning means assigns identification information to each face of the person detected in the first captured image ;
In the two captured images acquired before and after the time, the second assigning unit is provided with identification information in the captured image acquired earlier among the faces of the person detected in the captured image acquired later. A face of a person who satisfies at least one of the difference between the face and the position of any one of the face of the person being equal to or smaller than the threshold and the difference of the size being equal to or smaller than the threshold is given to any of the faces. A second providing step for providing the identification information,
While the auxiliary recognition means is performing the main recognition process for the face of the person detected in the second captured image obtained after the completion of the main recognition process for the face of the person detected in the first captured image Is detected in the third captured image acquired after the second captured image by repeatedly applying the identification information by the second providing unit to the captured images acquired continuously in time. Recognize the face of the person's face to which the same identification information as that of the individual recognized by the main recognition step in the first captured image is given as the individual recognized by the main recognition step in the first captured image. An auxiliary recognition process to perform,
An imaging method comprising: Computer
An imaging means for capturing an optical image of a subject to obtain a captured image;
Detecting means for detecting a human face in the captured image;
An individual is recognized for the person's face detected by the detecting means using characteristic information relating to the characteristics of the person's face acquired from the storage unit over a longer processing time than the detection of the person's face by the detecting means. Main recognition means,
First assigning means for assigning identification information to each face of the person detected in the first captured image ;
Among the two captured images acquired before and after the time, whichever face of the person is given the identification information in the previously acquired captured image among the human faces detected in the captured image acquired later The identification information given to any one of the faces is given to a person's face that satisfies at least one of the difference between the face and the position being equal to or smaller than the threshold and the difference in size being equal to or smaller than the threshold. Second granting means, and
While the process of the main recognition unit is being performed on the face of the person detected in the second captured image obtained after the completion of the process of the main recognition unit on the face of the person detected in the first captured image, By repeatedly applying the identification information by the second assigning unit to the captured images acquired continuously in time, the person detected in the third captured image acquired after the second captured image is displayed. Auxiliary recognition means for recognizing a face of the face to which the same identification information as the individual recognized by the main recognition means in the first captured image is given as an individual recognized by the main recognition means in the first captured image ,
Program to function as.

Images