JP2012137861A - Video processing device and video processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce power consumption while maintaining accuracy in face authentication.SOLUTION: A video processing device 100 includes: a face detection unit 150 for detecting a face from obtained video data; a detection number assignment unit 152 for assigning the same detection number as that of a face detected in a first frame to a face detected in a second frame at a position within a predetermined distance from the face detected in the first frame between two frames, and for assigning a new detection number to a face determined not to be at a position within the predetermined distance; and a face authentication unit 154 for performing face authentication processing for determining in each frame whether or not the detected face is an authenticated face that is the face of the same person as a person having a registered face. When the detected face is the authenticated face, the face authentication unit does not perform face authentication processing using the registered face while the same detection number as a detection number assigned to the authenticated face is continuously assigned in the subsequent frames, and regards the face with the same number assigned thereto as the authenticated face.

Description

  The present invention relates to a video processing apparatus and a video processing method for performing authentication processing of a face in video data.

  2. Description of the Related Art Conventionally, an imaging device such as a camcorder or a digital camera has a face authentication function that detects a human face from video data generated by the imaging device and identifies a pre-registered face among the detected faces. Are known. Specifically, a technique has been proposed in which a face region is cut out from video data generated by an imaging device, a matching rate with a registered face is derived, and the face is determined to be a registered face (for example, And prior literature 1).

  In order to enable this face authentication function continuously, a method of performing face authentication on continuous video data every time and updating the authentication result can be considered. At this time, the authentication result varies depending on the position and orientation of the face. Therefore, in order to suppress variations in authentication results, the detected face is tracked, the face being tracked is compared with the registered face registered to identify the face, and the registered face is A technique has been proposed in which the accuracy of the face authentication function is improved by updating the detected face (for example, prior document 2).

Japanese Patent Laid-Open No. 2002-333651 JP 2008-257425 A

  By the way, in recent years, an imaging apparatus is desired to be used for a long time by battery drive. However, when the face authentication function is executed using the techniques of Patent Documents 1 and 2 described above, the amount of image processing associated with face authentication increases and power consumption increases. It was intense.

  Therefore, in view of such problems, the present invention has an object to provide a video processing apparatus and a video processing method capable of suppressing power consumption while maintaining the accuracy of face authentication.

  In order to solve the above problems, an image processing apparatus of the present invention includes an information holding unit that holds in advance face image information indicating features of a registered face that is a registered face, a video acquisition unit that acquires video data, A face detection unit that detects a face from the acquired video data and a face between two frames at a position within a predetermined distance from the face detected in the first frame, which is the previous frame, in the subsequent frame It is determined every frame whether or not a face is detected in a certain second frame, and the same detection number as the face detected in the first frame is assigned to a face detected at a position within a predetermined distance. Then, for a face that is not determined to be at a position within a predetermined distance, a detection number assigning unit that assigns a new detection number, and a face that is detected based on the face image information is a registered face. Whether the authentication face is the face of the same person or not A face authentication unit that performs a face authentication process for determining whether the detected face is an authentication face, the detection number that is the same as the detection number assigned to the authentication face However, the face authentication process using the registered face corresponding to the authentication face is not performed while the face is continuously assigned, and the face to which the same detection number is assigned is used as the authentication face.

  After a detection number identical to the detection number assigned to the face detected in the first frame is no longer assigned in the second frame, it reaches a predetermined number of times or is again within a predetermined distance An erasure interpolation unit that assigns the same detection number to a region corresponding to the face detected in the first frame until a face is detected may be further provided.

  In order to solve the above-described problem, the video processing method of the present invention pre-stores face image information indicating the characteristics of a registered face, which is a registered face, acquires video data, and extracts a face from the acquired video data. Whether or not a face is detected in the second frame, which is the subsequent frame, at a position within a predetermined distance from the face detected in the first frame, which is the previous frame, between the two frames The face detected at a position within a predetermined distance is given the same detection number as the face detected at the first frame, and is at a position within a predetermined distance. A new detection number is assigned to a face that is not determined to be a face, and a face that determines whether the detected face is an authentication face that is the face of the same person as the registered face based on the face image information is determined every frame When authentication processing is performed and the detected face is an authentication face While the detection number that is the same as the detection number assigned to the authentication face continues to be assigned in subsequent frames, the same detection number is assigned without performing the face authentication process using the registered face corresponding to the authentication face. The detected face is used as an authentication face.

  As described above, according to the present invention, it is possible to suppress power consumption while maintaining the accuracy of face authentication.

It is the functional block diagram which showed the schematic structure of the video processing apparatus. It is a flowchart which shows the flow of a process of a video processing method. It is a flowchart which shows the flow of a process of a video processing method. It is a flowchart which shows the flow of a process of a video processing method. It is a flowchart which shows the flow of a process of a video processing method.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating the understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

(Video processing apparatus 100)
For example, the video processing apparatus 100 captures a subject and generates video data. At this time, the face of the subject in the generated video data is detected (face detection function), the face is identified by performing face authentication (face authentication function), and the face is focused. Various processing is done. In the following embodiment, among the functions of the video processing apparatus 100, the face detection function and the face authentication function will be described in detail.

  FIG. 1 is a functional block diagram showing a schematic configuration of the video processing apparatus 100. The video processing apparatus 100 includes an imaging lens 108, a CMOS sensor 110, a camera controller 112, a video memory 114, a memory controller 116, a memory selector 118, a resize controller 120, and a VGA (Video Graphics Array) selector 122. QVGA (Quarter VGA) controller 124, first memory 126, VGA controller 128, second memory 130, face detection controller 132, third memory 134, face authentication controller 136, and fourth memory 138 And the CPU 140.

  Here, the camera controller 112, the memory controller 116, and the memory selector 118 function as a video acquisition unit, and acquire video data.

  Specifically, in the recording mode for recording video data generated by imaging a subject, the CMOS sensor 110 photoelectrically converts the light collected by the imaging lens 108, and the video data input from the CMOS sensor 110 is converted into a camera. The data is output to the memory selector 118 via the controller 112 and sent to the resize controller 120. In the reproduction mode for reproducing the recorded video data, the video data output from the video memory 114 is sent to the resize controller 120 via the memory controller 116 and the memory selector 118. Thus, the memory selector 118 switches the video data output to the resize controller 120 between the recording mode and the playback mode.

  The resizing controller 120 adjusts gain, offset, and the like, and further performs processing such as electronic zoom and digital effect according to user input through an operation unit (not shown), and sends the processed video data to the VGA selector 122.

  When performing the face detection, the VGA selector 122 outputs the video data output from the resize controller 120 to the QVGA controller 124, and the QVGA controller 124 uses the first memory 126, which is a work memory, to input the input video. The data is subjected to QVGA conversion to generate face detection video data.

  In addition, when performing face authentication, the VGA selector 122 outputs the video data output from the resize controller 120 to the VGA controller 128, and the VGA controller 128 uses the second memory 130, which is a work memory, to perform face authentication. Video data is generated.

  The face detection controller 132 receives face image data for face detection newly generated by the QVGA controller 124, performs face detection in accordance with a control command output by the face detection unit of the CPU 140, which will be described later, and outputs a detection result. The data is stored in the third memory 134 which is a work memory.

  The fourth memory 138 functions as an information holding unit and holds in advance face image information indicating the characteristics of a registered face, which is a registered face, in response to a user input for registering a face. Then, the face authentication controller 136 detects the detection acquired from the face detection controller 132 in response to a control command output from the face detection unit when the video data for face authentication is newly generated by the VGA controller 128. Face authentication is executed based on the result and the face image information held in the fourth memory 138, and the authentication result is stored in the fourth memory 138 which is a work area.

  The video processing apparatus 100 continuously performs the face detection and face authentication operations described above for each frame of video data. Hereinafter, the CPU 140 that controls the operations of face detection and face authentication will be described in detail.

  The CPU 140 manages and controls the entire video processing apparatus 100 by a semiconductor integrated circuit including a central processing unit, a ROM storing programs, a RAM as a work area, and the like. The CPU 140 also functions as a face detection unit 150, a detection number assigning unit 152, a face authentication unit 154, and an erasure interpolation unit 156.

  The face detection unit 150 outputs a control command to cause the face detection controller 132 to detect a face from the acquired video data.

  The detection number assigning unit 152 acquires the center coordinates and the like of the detected face from the face detection unit 150. Then, the detection number assigning unit 152 temporally moves to a position within a predetermined distance from the face detected in the first frame, which is a temporally previous frame, between two consecutive frames of video data. Whether the face is detected in the second frame, which is a later frame, is determined every frame, and the face detected at a position within a predetermined distance is the same as the face detected in the first frame And a new detection number is assigned to a face that is not determined to be at a position within a predetermined distance.

  Since the frame period of the video data is a short time such as 1/60 seconds, for example, the position of the face in the video data often does not vary greatly during that period. Therefore, there is a high possibility that faces detected at positions within a predetermined distance between two consecutive frames are the faces of the same person. The detection number assigning unit 152 tracks the same person's face across a plurality of frames by assigning the same detection number to faces detected at positions within a predetermined distance between two frames. Is possible.

  In the present embodiment, the detection number assigning unit 152 acquires information on the detected face, such as the face size, the face direction, and the face rotation direction, from the face detection unit 150 in addition to the center coordinates. Then, when a face is detected at a position within a predetermined distance from the face detected in the first frame in the second frame, the detection number assigning unit 152 determines the size of the face for the detected face. Whether or not they are the same person is determined based on whether or not the difference (absolute value or ratio) is within a predetermined range and whether or not the angle difference between the face orientations is within a predetermined angle range.

  The face authentication unit 154 acquires the detection result stored in the third memory 134 through the face detection controller 132, and based on the face image information indicating the feature of the registered face stored in the fourth memory 138, the face detection controller The face authentication controller 136 performs a face authentication process for determining whether or not the face detected by 132 is an authentication face that is the face of the same person as the registered face. The face authentication controller 136 derives an authentication score indicating the degree of similarity of the detected face with respect to the registered face, and determines that the face is an authentication face when the authentication score is equal to or greater than a predetermined value.

  However, the face authentication unit 154 of this embodiment does not necessarily perform the face authentication process every frame. That is, when the face authentication unit 154 determines that the detected face is an authentication face, the face authentication unit 154 stores the detection number assigned to the authentication face in association with the registered face corresponding to the authentication face. Even in the frame, while the detection number assigned to the detected face is the same as the held detection number (while the same detection number is continuously given), the face using the registered face corresponding to the authentication face Faces with the same detection number are regarded as authentication faces without performing the authentication process.

  When an authenticated face that has been authenticated as the face of the same person as the registered face can be tracked thereafter, the face that can be tracked is likely to be the face of the same person as the registered face. Therefore, when there is a face that can be tracked after authentication, the video processing apparatus 100 according to the present embodiment does not perform face authentication processing using the authentication face in subsequent frames. With this configuration, the video processing apparatus 100 can suppress power consumption while maintaining the accuracy of face authentication.

  The erasure interpolation unit 156 reaches a predetermined number of times after the detection number identical to the detection number assigned to the face detected in the first frame is no longer assigned in the second frame, or determines again in advance. Until the face is detected at a position within the specified distance, the same detection number is assigned to the area corresponding to the face detected in the first frame (disappearance interpolation process).

  For example, there may be a case where the face cannot be detected temporarily due to camera shake of the photographer or light adjustment to the subject. At this time, when face tracking is finished, even if the same face as the authentication face is detected again after several frames at that position, face authentication processing must be performed again. With the configuration including the erasure interpolation unit 156, the video processing apparatus 100 avoids a situation where the tracking of the authentication face ends due to a temporary factor, and the frequency of the face authentication process regarding the face for which detection is immediately resumed. Therefore, power consumption can be suppressed.

  Furthermore, the erasure interpolation unit 156 further determines that the number of times that the same detection number as that of the detection number is continuously given to the face before disappearance is a predetermined number or more as a condition for performing the erasure interpolation process. May be added. By limiting the number of detections assigned to a detected face continuously for a predetermined number of times or more, erasure interpolation processing is performed on all faces with a low detection frequency and a short detection time, thereby increasing the processing load. Thus, the accuracy of the erasure interpolation process can be improved.

  As described above, the video processing apparatus 100 according to the present embodiment can suppress power consumption while maintaining the accuracy of face authentication.

(Video processing method)
Furthermore, a video processing method using the video processing apparatus 100 described above is also provided. 2 to 5 are flowcharts showing the flow of processing of the video processing method. In the flowchart of the video processing method of the present embodiment, the case where there is one registered face has been described as an example, but a plurality of registered faces may be targeted. In this case, the face authentication unit 154 performs face authentication processing for each registered face, or regards a face assigned with the same detection number as the detection number assigned to the authentication face as an authentication face.

  In FIG. 2, each functional unit functioning as the video acquisition unit described above performs imaging in the recording mode and playback of the video data in the playback mode (S200), and the video acquisition unit acquires the video data (frame) (S202). ). Then, face detection and disappearance interpolation processing is performed on the acquired video data (S204). The detection interpolation step S204 will be described in detail with reference to FIG.

  In FIG. 3, the face detection unit 150 outputs a control command to the face detection controller 132 to detect a face from the acquired video data, and the detection number assigning unit 152 includes the number of detected faces and the center of the face. The coordinates, face size, face orientation, rotation direction, etc. are acquired (S230). Subsequently, the detection number assigning unit 152 determines whether or not there is an unprocessed face among the detected faces, that is, a face to which no detection number has been assigned yet (S232). When there is an unprocessed face (YES in S232), the detection number assigning unit 152 is located within a predetermined distance from the face detected in the first frame for one of the unprocessed faces. It is determined whether or not (S234).

  When one of the unprocessed faces is located within a predetermined distance (YES in S234), the detection number assigning unit 152 and the first frame face and the first frame located within the predetermined distance It is determined whether or not the difference in face size is within a predetermined range for the unprocessed faces of two frames (S236).

  If the difference in size of the face is within a predetermined range (YES in S236), the detection number assigning unit 152 is located within the predetermined distance and the face of the first frame and the second frame For the unprocessed face, it is determined whether or not the angle difference of the face direction (for example, the direction of the line of sight) is within a predetermined angle range (S238).

  When the angle is within the predetermined angle range (YES in S238), the detection number assigning unit 152 is detected in the first frame on the unprocessed face of the second frame located within the predetermined distance. The same detection number as that of the face is assigned (S240).

  Further, it is not determined that one of the unprocessed faces is at a position within a predetermined distance (NO in S234), or the face size difference is not within a predetermined range (S236). If the angle difference between the face orientations is not within a predetermined angle range (NO in S238), the detection number assigning unit 152 sets a new detection number for the unprocessed face in the second frame. (S242).

  In the unprocessed detected face determination step S232, when there is no unprocessed face among the detected faces (NO in S232), the detection number assigning unit 152 performs erasure interpolation (S244), and the detection interpolation step S204 is performed. finish. The erasure interpolation step S244 will be described in detail with reference to FIG.

  In FIG. 4, the erasure interpolation unit 156 does not assign the same detection number as the detection number assigned to the face detected in the first frame or the previous frame in the second frame, that is, the lost detection number. In addition, it is determined whether or not there is an unprocessed one, that is, one that has not yet undergone erasure interpolation (S260). When there is an unprocessed detection number that has disappeared (YES in S260), the disappearance interpolation unit 156 reads the history of the same detection number for any of the undetected and unprocessed detection numbers (S262). The erasure interpolation unit 156 determines whether or not the number of times that the same detection number as the detection number is continuously given to the face is a predetermined number (for example, 10 times) or more (S264).

  If it is less than the predetermined number (NO in S264), the erasure interpolation is not performed for the detected number, and the process returns to the erasure detected number determination step S260. When the number is equal to or greater than the predetermined number (YES in S264), it is determined whether or not the erasure interpolation has been performed by the first frame, and the number of times has reached a predetermined upper limit number (for example, five times). (S266). If it has arrived (NO in S266), the erasure interpolation is not performed for the detected number, and the process returns to the erasure detected number determination step S260.

  When the number is less than the upper limit number (YES in S266), the erasure interpolation unit 156 gives the same detection number to the area corresponding to the face detected in the first frame or earlier frame to which the detection number is assigned. An erasure interpolation process is executed (S268). Thereafter, the process returns to the disappearance detection number determination step S260.

  In erasure detection number determination step S260, if there is no erasure detection number and there is no unprocessed one (NO in S260), erasure interpolation step S244 is terminated. As described above, the continuity of the detection numbers is further improved by performing the erasure interpolation process.

  Returning to FIG. 2, the CPU 140 determines whether or not there is a user input instructing registration of face image information (S206). When there is a user input (YES in S206), the CPU 140 stores, for example, face image information indicating facial features detected near the position designated by the user through a touch panel provided on a display unit (not shown) in the fourth memory. It is held at 138 (S208).

  When there is no user input instructing registration of face image information (NO in S206), the face authentication unit 154 determines whether or not a detection number is already associated with the registered face in the first frame (S210). When there is a detection number associated with the registered face (YES in S210), the face authentication unit 154 gives the same detection number as the detection number associated with the registered face to the face detected in the second frame. It is determined whether or not there is (S212). If not assigned (NO in S212), or if there is no detection number associated with the registered face (NO in S210), the face authentication unit 154 acquires the detection result stored in the third memory 134, The face authentication controller 136 determines whether the face detected by the face detection controller 132 is an authentication face that is the face of the same person as the face having the characteristics indicated in the face image information (S214).

  When the same detection number as the detection number associated with the registered face is assigned to the face detected in the second frame (YES in S212), the face authentication unit 154 causes the face authentication controller 136 to perform face authentication. The face to which the same detection number is assigned without considering the process is regarded as an authentication face (S216).

  Next, the video processing method will be described with reference to FIG. 5 focusing on the start and stop timings of the face authentication circuit (face authentication controller 136). When the video acquisition unit acquires video data (YES in S280), the same processing as in the detection interpolation step S204 described above is performed. Then, the face authentication unit 154 determines whether or not the face authentication circuit is activated (S282). If not activated (NO in S282), the face authenticating unit 154 starts supplying the CLK (CLocK) signal to the face authenticating circuit and activates it (S284).

  When the face authentication circuit is activated (YES in S282) or after the face authentication circuit activation step S284, the face authentication unit 154 acquires the detection result stored in the third memory 134 through the face detection controller 132, The face authentication controller 136 determines whether the face detected by the face detection controller 132 is an authentication face (S286). Then, the face authentication unit 154 determines whether there is an authentication face (S288). If there is no authentication face (NO in S286), if the detection number associated with the registered face is held in the fourth memory 138, it is discarded (S290), and the process returns to the video data acquisition step S280.

  When there is an authentication face (YES in S288), the face authentication unit 154 associates (links) the detection number of the authentication face with the corresponding registered face and causes the fourth memory 138 to hold it (S292). The face authentication unit 154 stops the face authentication circuit by stopping the CLK signal supplied to the face authentication circuit that performs face authentication (S294).

  Subsequently, when the next video data (frame) is acquired (YES in S296), the same processing as that in the detection interpolation step S204 described above is performed, and the face authentication unit 154 is the same as the detection number associated with the registered face. It is determined whether the detection number is assigned to the face detected in the second frame (S298). When the same detection number is not given to the face detected in the second frame (NO in S298), the process returns to the face authentication circuit activation determination step S282.

  When the same detection number is assigned (YES in S298), the face authentication unit 154 does not perform processing on the face authentication controller 136 and regards the face assigned the same detection number as an authentication face (S300). ). Then, the process returns to the video data acquisition step S296. In this way, it is possible to determine whether or not the detected face is an authentication face while stopping the face authentication circuit.

  As described above, according to the video processing method using the video processing apparatus 100, it is possible to suppress power consumption while maintaining the accuracy of face authentication.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this embodiment. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Is done.

  Note that each step in the video processing method of the present specification does not necessarily have to be processed in time series in the order described in the flowchart, and may include parallel or subroutine processing.

  INDUSTRIAL APPLICABILITY The present invention can be used for a video processing apparatus and a video processing method that perform face authentication processing in video data.

100 ... Video processing device 112 ... Camera controller (video acquisition unit)
116 ... Memory controller (video acquisition unit)
118 ... Memory selector (video acquisition unit)
138 ... 4th memory (information holding part)
150 ... face detection unit 152 ... detection number assigning unit 154 ... face authentication unit 156 ... erasure interpolation unit

Claims (3)

An information holding unit for holding in advance face image information indicating the characteristics of a registered face, which is a registered face;
A video acquisition unit for acquiring video data;
A face detection unit for detecting a face from the acquired video data;
Whether or not a face is detected in the second frame, which is the subsequent frame, at a position within a predetermined distance from the face detected in the first frame, which is the previous frame, between the two frames. Frame detection is performed, and the same detection number as the face detected in the first frame is assigned to the face detected at the position within the predetermined distance, and the face is at the position within the predetermined distance. A detection number giving unit for assigning a new detection number to a face that has not been determined,
A face authentication unit that performs face authentication processing for determining whether or not each detected face is an authentication face that is the face of the same person as the registered face based on the face image information;
With
When the detected face is the authentication face, the face authentication unit applies the same detection number as the detection number assigned to the authentication face to the authentication face while continuing to be assigned in subsequent frames. An image processing apparatus, wherein a face assigned with the same detection number is set as the authentication face without performing a face authentication process using the corresponding registered face.   After a detection number that is the same as the detection number assigned to the face detected in the first frame is no longer assigned in the second frame, a predetermined number of times is reached, or again the predetermined distance The erasure interpolation unit that assigns the same detection number to a region corresponding to the face detected in the first frame until a face is detected at a position within the first frame. Video processing device. Pre-stores face image information indicating the characteristics of a registered face, which is a registered face,
Get video data,
Detecting a face from the acquired video data,
Whether or not a face is detected in the second frame, which is the subsequent frame, at a position within a predetermined distance from the face detected in the first frame, which is the previous frame, between the two frames. Frame detection is performed, and the same detection number as the face detected in the first frame is assigned to the face detected at the position within the predetermined distance, and the face is at the position within the predetermined distance. A new detection number is assigned to faces that are not determined as
Based on the face image information, a face authentication process is performed for determining whether each detected face is an authentication face that is the face of the same person as the registered face,
When the detected face is the authentication face, the registered face corresponding to the authentication face is detected while the same detection number as the detection number assigned to the authentication face is continuously assigned in subsequent frames. An image processing method, wherein the face to which the same detection number is assigned is set as the authentication face without performing the used face authentication process.

Images