WO2012057665A1 - A face data acquirer, end user video conference device, server, method, computer program and computer program product for extracting face data - Google Patents
A face data acquirer, end user video conference device, server, method, computer program and computer program product for extracting face data Download PDFInfo
- Publication number
- WO2012057665A1 WO2012057665A1 PCT/SE2010/051174 SE2010051174W WO2012057665A1 WO 2012057665 A1 WO2012057665 A1 WO 2012057665A1 SE 2010051174 W SE2010051174 W SE 2010051174W WO 2012057665 A1 WO2012057665 A1 WO 2012057665A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- face
- image
- face data
- video conference
- extracted
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
- G06F16/784—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content the detected or recognised objects being people
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/28—Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/772—Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/50—Maintenance of biometric data or enrolment thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
Definitions
- the invention relates to extracting face data for
- face recognition can be used to identify people in snap shots or in video conferences.
- An object of the invention is to automate the collection of face data to be used for face recognition.
- a first aspect is a face data acquirer comprising: an image capture module arranged to capture an image from a video stream of a video conference; a face detection module arranged to determine a subset of the image, the subset representing a face; an identity acquisition module arranged to acquire an identity of a video
- the conference participant coupled to the face represented by the subset of the image; and a face extraction module arranged to extract face data from the subset of the image and to determine whether to store the extracted face data for subsequent face recognition.
- the face data is automatically associated with an identity and can be stored for subsequent face recognition. This allows face data to be collected without user involvement and effort.
- face data can be updated over time, reflecting a change in appearance of the face.
- the image analysis module may be arranged to determine to store the extracted face when an accuracy of face
- detection is determined to be improved when the extracted face data is stored compared to refraining from storing the extracted face data. In other words, only if the extracted face data contributes to an improved face detection accuracy is the extracted face data stored. This prevents storing new extracted face data that in effect does not contribute to an improved face
- the image analysis module may be arranged to determine to store the extracted face data when a measurement of difference of the extracted face data compared to a set of previously stored extracted face data for the same identity is greater than a threshold value. In other words, if the current face is sufficiently different from the previously stored faces, it should be stored. The reason for this is that later face recognition is more accurate if it is based on face data which is captured from images of varying angles, lighting conditions, etc.
- the image analysis module may be arranged to determine to store the extracted face data when a measurement of difference of lighting or contrast conditions in the image compared to a set of previously stored extracted face data for the same identity is greater than a
- the image analysis module may be arranged to determine to store the extracted face data when a measurement of difference of lighting or contrast conditions in the subset of the image compared to a set of previously stored extracted face data for the same identity is greater than a threshold value. In other words, when the lighting or contrast of the face is sufficiently
- Fig 1 is a schematic diagram illustrating a video
- End user video conference devices lOa-c are connected with each other via a network 14, such as a local area network or a wide area network, e.g. Internet.
- a network 14 such as a local area network or a wide area network, e.g. Internet.
- Each end user video conference device comprises a respective camera 12a-c and a respective display lla-c. While the cameras 12a-c are shown as embedded in respective end user video conference device lOa-c, the cameras 12a-c could equally well be separate devices connected to the respective end user video conference device lOa-c.
- the end user video conference devices lOa-c are connected with each other via a network 14, such as a local area network or a wide area network, e.g. Internet.
- Each end user video conference device comprises a respective camera 12a-c and a respective display lla-c. While the cameras 12a-c are shown as embedded in respective end user video conference device lOa-c, the cameras 12a-c
- conference devices lOa-c can for example be laptop or desktop computers, mobile phones, smart phones and/or personal digital assistants (PDAs) .
- PDAs personal digital assistants
- Each end user video conference device lOa-c has a
- a server 16 is also connected to the network.
- the server can be configured to oversee any video conferences, facilitating setup and teardowns, providing directory services, etc.
- the server 16 can be any suitable computer capable of performing the tasks as described herein.
- the server can be a computer running an
- the server 16 comprises a processor 18 (e.g. a CPU), primary memory 19, such a RAM and a storage device 17 providing persistent storage, such as magnetic storage, solid state storage, optical storage or any combination of these.
- the server also comprises an I/O interface 15 for connecting to the network 14, and thereby providing connectivity to the end user video conference devices lOa-c.
- two or more end user video conference devices lOa-c can set up video conferences with each other.
- video conferences with two or more simultaneous parties can be set up in this system.
- a face data acquirer is implemented in one or more of the end user video conference devices lOa-c and/or the server 16, as will be explained in further detail below.
- the face data acquirer obtains face data from the video conference for use in subsequent face recognition.
- Fig 2 is a flow chart illustrating a method of
- the original source of the image is a digital camera or similar used for the video conference.
- a subset of the image is determined.
- the subset represents a face.
- the subset may for example be a closed loop encompassing the face in the image or a rectangular subset of the image .
- conference software in the end user video conference device or server to get the participant information.
- it is the identity of the user which is connected to the end user video conference device with the camera that generates the image which is applicable.
- face data is extracted from the subset of the image.
- This face data is of a format which could later be used for face recognition.
- the face data could simply be a lower resolution image of the subset of the image, as is explained in more detail below.
- the extract face data step 23 is only performed if it is determined that the face data should be stored. Now a number of examples will be presented of how it is determined whether or not to store the face data. The examples may be implemented exclusively or in any
- the face data is stored when a measurement of difference of the extracted face data compared to a set of previously stored extracted face data for the same identity is greater than a threshold value. In other words, if the current face is
- the reason for this is that later face recognition is more accurate if it is based on face data which is captured from images of varying angles, lighting conditions, etc.
- the set of previously stored faces can be all previously stored data of faces or a subset of these images.
- the face data is stored when a measurement of difference of lighting or contrast
- the captured image is an image with one or more lost packets, the quality of the image is quite low, whereby the image is not ideal to be used for subsequent face recognition.
- Such a difference between faces can for instance be calculated in the following way.
- the relevant portions of the image i.e., the part of the image containing the face
- a face detection system such as the one described by Viola and Jones [1] .
- Face data can then be extracted using this rectangular frame by sampling the image to a fixed size, for example 24x24 pixels, and converting to grey scale.
- This 24x24 pixel grey scale image now represents extracted face data and can now be compared to similarly subsampled versions of the previously obtained face images for the same individual.
- the comparison involves summing the squared difference of the grey scale value in each of the 24x24 pixels.
- SSD difference
- the 24x24 pixel image is further processed before the SSD value is calculated.
- the average value of the grey scale value is calculated and removed from the image. This is done to exclude new images that only differ from previous images in light intensity, not in light direction.
- the average is removed, but then the standard deviation of the grey scale value is also calculated, and the image is divided by this factor so that the resulting image standard deviation is constant, for instance equal to 1.0. This is done to exclude new images that only differ from previous images in contrast, but not in light direction.
- the 24x24 pixel is smoothed by a Gaussian filter before the SSD value is calculated. This is done to make the method less dependent upon small shifts in the face detection frame .
- the method continues to a store face data step 28.
- the method continues to a conditional continue face data acquisition step 30.
- conditional continue face data acquisition step 30 it is determined whether to continue face data acquisition or not.
- One reason not to continue can for example be that the video conference has ended.
- Another reason not to continue can be that a sufficient amount of face data is stored.
- face data sets are stored, e.g. from about 50 to 100 images, additional face data is unlikely to provide a base for better face recognition.
- the larger number of face data sets to analyse during face recognition the larger computation is required, increasing resource requirements and/or face recognition times.
- the method continues to the extract face data step 23. Otherwise, the method eventually continues to a determine identity of a face step 32.
- the actual face recognition is performed.
- the face recognition is performed using the previously stored face data.
- a number of sets of face data are required for the face recognition to be acceptably accurate. For example, at the time of filing this application, it is desired to have at least about five sets of face data obtained from the same number of captured images, for the face recognition to be acceptably accurate. This step is performed at a later stage than the previous steps.
- this step is performed in parallel with the previous steps. Moreover, this step can be performed in the same or a different physical device as for the previous steps.
- the determine identity of a face step 32 is repeated. After this step, the method ends.
- Fig 7 is a schematic diagram showing an image 76 captured from a video conference.
- image 76 there is a subset 75 of the image which corresponds to a face.
- the subset can be defined by an outline following the face, or it can optionally be a rectangular subset which envelopes the face.
- the subset 75 of the image can be used to extract face data which can later be used for face recognition.
- Fig 3 is a schematic diagram showing functional modules of a face data acquirer included in end user video conference device and/or server of Fig 1.
- the face data acquirer is a device or software package capable of extracting face data from a video conference image and to associate the face data with an identity, such that the face data can later be used in face recognition.
- the modules can be implemented using software such as a computer program executing in the end user video
- All modules depend on an execution environment (not shown) which utilises a controller 18, 60 and optionally a computer program product 17/62 and a I/O interface 15/64.
- a face extraction module 46 is arranged to extract face data from the subset 75 of the image 76 and to determine whether to store the extracted face data for subsequent face recognition.
- This module corresponds to step 23 of Fig 2.
- Fig 4 schematically shows an embodiment of one or more of an end user video conference device 10.
- the end user video conference device corresponds to any of the end user video conference devices lOa-c of Fig 1, and the components of Fig 4 correspond, where applicable, to components of the end user video conference devices lOa-c of Fig 1 without the end user video conference device identification suffix letter (a, b or c) .
- Fig 5a is a schematic diagram showing a configuration for face data acquisition according to a first embodiment.
- the face data acquirer 40 and the storage module 41 are comprised in the end user video conference device 10.
- the server 16 comprises the storage device 42, which stores the face data which the face data acquirer obtains. The decision whether to store or discard the face data is made in the storage module 41.
- This configuration allows the end user video conference device 10 to acquire face data, but the face data is stored centrally in the server 16 to allow access by more interested entities.
- conference device 10 has local access to the face data for later face recognition by the optional face
- the end user video conference device Furthermore, the end user video conference device
Abstract
It is presented a face data acquirer comprising: an image capture module arranged to capture an image from a video stream of a video conference; a face detection module arranged to determine a subset of the image, the subset representing a face; an identity acquisition module arranged to acquire an identity of a video conference participant coupled to the face represented by the subset of the image; and a face extraction module arranged to extract face data from the subset of the image and to determine whether to store the extracted face data for subsequent face recognition. A corresponding end user video conference device, server, method, computer program and computer program product are also provided.
Description
A FACE DATA ACQUIRER, END USER VIDEO CONFERENCE DEVICE, SERVER, METHOD, COMPUTER PROGRAM AND COMPUTER PROGRAM
PRODUCT FOR EXTRACTING FACE DATA
TECHNICAL FIELD
The invention relates to extracting face data for
subsequent for subsequent face recognition.
BACKGROUND
Face recognition is continuously receiving increased attention. This is due to improved recognition
technologies and increased use of digital imaging. For example, face recognition can be used to identify people in snap shots or in video conferences.
However, in order to be able to identify faces,
sufficient face data must be available. Typically at least a handful images are required for acceptable accuracy of face recognition.
In the prior art, users have had to indicate the identity of faces in images manually to obtain face data until face data from a sufficient number of faces are obtained. This is a tedious and cumbersome process which requires time and effort from the user.
Furthermore, even if a sufficient amount of face data is stored, people may change their appearance by choice or e.g. by aging, whereby the face data may become obsolete. There is no known way to automatically update face data over time
SUMMARY
An object of the invention is to automate the collection of face data to be used for face recognition.
A first aspect is a face data acquirer comprising: an image capture module arranged to capture an image from a video stream of a video conference; a face detection module arranged to determine a subset of the image, the subset representing a face; an identity acquisition module arranged to acquire an identity of a video
conference participant coupled to the face represented by the subset of the image; and a face extraction module arranged to extract face data from the subset of the image and to determine whether to store the extracted face data for subsequent face recognition. By using image data from a video conference, the face data is automatically associated with an identity and can be stored for subsequent face recognition. This allows face data to be collected without user involvement and effort. Moreover, face data can be updated over time, reflecting a change in appearance of the face.
The image analysis module may be arranged to determine to store the extracted face when an accuracy of face
detection is determined to be improved when the extracted face data is stored compared to refraining from storing the extracted face data. In other words, only if the extracted face data contributes to an improved face detection accuracy is the extracted face data stored. This prevents storing new extracted face data that in effect does not contribute to an improved face
recognition.
The image analysis module may be arranged to determine to store the extracted face data when a measurement of difference of the extracted face data compared to a set of previously stored extracted face data for the same identity is greater than a threshold value. In other words, if the current face is sufficiently different from the previously stored faces, it should be stored. The reason for this is that later face recognition is more accurate if it is based on face data which is captured from images of varying angles, lighting conditions, etc.
The image analysis module may be arranged to determine to store the extracted face data when a measurement of difference of lighting or contrast conditions in the image compared to a set of previously stored extracted face data for the same identity is greater than a
threshold value. In other words, when the lighting or contrast of the whole image is sufficiently different from the previously stored images, the face data is stored . The image analysis module may be arranged to determine to store the extracted face data when a measurement of difference of lighting or contrast conditions in the subset of the image compared to a set of previously stored extracted face data for the same identity is greater than a threshold value. In other words, when the lighting or contrast of the face is sufficiently
different from the previously stored images, the face data is stored.
The image analysis module may be arranged to determine to refrain from storing the extracted face data when it is determined that the face represented in the subset of the
image is looking away from a camera used to obtain the image. Data of a face looking away is unlikely to
contribute to better face recognition
The image analysis module may be arranged to determine to refrain from storing the extracted face data when it is determined that the quality of the image is low, e.g. due to packet losses. Video conferences often use packet based communication, e.g. UDP, due to a desire to reduce latency in communication. Since such packet based
communication is often used, packet losses do
occasionally occur. If the captured image is an image with one or more lost packets, the quality of the image is quite low, whereby the image is not ideal to be used for subsequent face recognition. A second aspect is an end user video conference device comprising the face data acquirer according to the first aspect .
The end user video conference device may comprise a storage module arranged to send a message to store the extracted face data in a storage device. In other words, the end user video conference device instructs another entity, such as a server, to store the face data.
The end user video conference device may comprise a storage device arranged to store the extracted face data. In other words, the face data can be stored locally in the end user video conference device.
The end user video conference device may comprise a face recognition device arranged to determine an identity of a face represented in a face recognition image using the
extracted face data stored in the storage device. This allows local face recognition to be performed.
The end user video conference device may further comprise a face signature generator arranged to generate a face signature for the identity using the extracted face data in the storage device, wherein the face signature is a set of data being sufficient to allow a face recognition device to recognise a face coupled to the identity in a face recognition image. A third aspect is a server comprising the face data acquirer according to the first aspect.
The server may comprise a storage device for storing the extracted face data for subsequent face recognition.
The server may comprise a face recognition device
arranged to determine an identity of a face represented in a face recognition image using the extracted face data stored in the storage device.
A fourth aspect is a method performed in a face data acquirer comprising: capturing an image from a video stream of a video conference; determining a subset of the image, the subset representing a face; acquiring an identity of a video conference participant coupled to the face represented by the subset of the image; extracting face data from the subset of the image; and determining whether to store the extracted face data for subsequent face recognition.
The method may further comprise effecting storage of the extracted face data in a storage device.
The method may further comprise determining an identity of a face represented in a face recognition image using the extracted face data stored in the storage device.
A fifth aspect is a computer program for a face
acquisition device, the computer program comprising computer program code which, when run on the face
acquisition device, causes the face acquirer to perform the steps of: capturing an image from a video stream of a video conference; determining a subset of the image representing a face; acquiring an identity of a video conference participant coupled to the face represented by the subset of the image; extracting face data from the subset of the image; and determining whether to store the extracted face data for subsequent face recognition. A sixth aspect is a computer program product comprising a computer program according to the fifth aspect and a computer readable means on which the computer program is stored .
It is to be noted that any feature of the first, second, third, fourth and fifth aspects may, where appropriate, be applied to any other of these aspects.
Generally, all terms used in the application are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a/an/the element, apparatus, component, means, step, etc." are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method
disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention is now described, by way of example, with reference to the accompanying drawings, in which:
Fig 1 is a schematic diagram illustrating a video
conference system where embodiments can be applied,
Fig 2 is a flow chart illustrating a method of
embodiments described herein,
Fig 3 is a schematic diagram showing functional modules of a face data acquirer included in end user video conference device and/or server of Fig 1,
Fig 4 schematically shows an embodiment of one or more of the end user video conference device of Fig 1,
Figs 5a-c are a schematic diagram showing various
configurations for face data acquisition,
Fig 6 shows one example of a computer program product comprising computer readable means, and
Fig 7 is a schematic diagram showing an image captured from a video conference.
DETAILED DESCRIPTION
The invention will now be described more fully
hereinafter with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many
different forms and should not be construed as limited to the embodiments set forth herein; rather, these
embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout the description.
Fig 1 is a schematic diagram illustrating a video
conference system where embodiments can be used. End user video conference devices lOa-c are connected with each other via a network 14, such as a local area network or a wide area network, e.g. Internet. Each end user video conference device comprises a respective camera 12a-c and a respective display lla-c. While the cameras 12a-c are shown as embedded in respective end user video conference device lOa-c, the cameras 12a-c could equally well be separate devices connected to the respective end user video conference device lOa-c. The end user video
conference devices lOa-c can for example be laptop or desktop computers, mobile phones, smart phones and/or personal digital assistants (PDAs) . Each end user video conference device lOa-c has a
respective connection 13a-c to the network 14.
A server 16 is also connected to the network. The server can be configured to oversee any video conferences, facilitating setup and teardowns, providing directory services, etc. The server 16 can be any suitable computer capable of performing the tasks as described herein. For example, the server can be a computer running an
operating system such as Linux, Microsoft Windows, Apple Mac OS, UNIX variants, etc. The server 16 comprises a processor 18 (e.g. a CPU), primary memory 19, such a RAM and a storage device 17 providing persistent storage,
such as magnetic storage, solid state storage, optical storage or any combination of these. The server also comprises an I/O interface 15 for connecting to the network 14, and thereby providing connectivity to the end user video conference devices lOa-c.
Using the system of Fig 1, two or more end user video conference devices lOa-c can set up video conferences with each other. In other words, video conferences with two or more simultaneous parties can be set up in this system. A face data acquirer is implemented in one or more of the end user video conference devices lOa-c and/or the server 16, as will be explained in further detail below. The face data acquirer obtains face data from the video conference for use in subsequent face recognition.
Fig 2 is a flow chart illustrating a method of
embodiments described herein. The method is performed in a face recognition device at a time when a video
conference is in progress. It is to be noted that this process can occur for one or more of the video conference participants simultaneously.
In an initial capture image step 20, an image from a video stream of the video conference is captured. The image is represented by a matrix of pixels, as known in the art per se, and is encoded using any suitable
encoding scheme. The original source of the image is a digital camera or similar used for the video conference.
In a determine subset step 21, a subset of the image is determined. In this case, the subset represents a face. The subset may for example be a closed loop encompassing
the face in the image or a rectangular subset of the image .
In the acquire identity step 22, an identity of a video conference participant coupled to the face represented by the subset of the image is acquired. This can for example be done by retrieving information from the video
conference software in the end user video conference device or server to get the participant information. In other words, it is the identity of the user which is connected to the end user video conference device with the camera that generates the image which is applicable.
In an extract face data step 23, face data is extracted from the subset of the image. This face data is of a format which could later be used for face recognition. For example, the face data could simply be a lower resolution image of the subset of the image, as is explained in more detail below.
In a conditional store face data step 26, it is
determined whether to store the extracted face data or whether the data should be discarded. Optionally, the extract face data step 23 is only performed if it is determined that the face data should be stored. Now a number of examples will be presented of how it is determined whether or not to store the face data. The examples may be implemented exclusively or in any
combination of two or more examples.
In a first example, the face data is stored when a measurement of difference of the extracted face data compared to a set of previously stored extracted face data for the same identity is greater than a threshold
value. In other words, if the current face is
sufficiently different from a set of previously stored faces, it should be stored. The reason for this is that later face recognition is more accurate if it is based on face data which is captured from images of varying angles, lighting conditions, etc. The set of previously stored faces can be all previously stored data of faces or a subset of these images.
In a second example, the face data is stored when a measurement of difference of lighting or contrast
conditions in the image compared to a set of previously stored extracted face data for the same identity is greater than a threshold value. In other words, when the lighting or contrast of the whole image is sufficiently different from the previously stored images, the face data is stored.
In a third example, the face data is stored when a measurement of difference of lighting or contrast
conditions in the subset of the image compared to a set of previously stored extracted face data for the same identity is greater than a threshold value. In other words, when the lighting or contrast of the face data is sufficiently different from the previously stored images, the face data is stored. In a fourth example, the face data is discarded when it is determined that the face represented in the subset of the image is looking away from a camera used to obtain the image. Data of a face looking away is unlikely to contribute to better face recognition.
In a fifth example, the face data is discarded when it is determined that the quality of the image is low, e.g. due to packet losses. Video conferences often use packet based communication, e.g. UDP, due to a desire to reduce latency in communication. Since such packet based
communication is often used, packet losses do
occasionally occur. If the captured image is an image with one or more lost packets, the quality of the image is quite low, whereby the image is not ideal to be used for subsequent face recognition.
In a sixth example, the face data is discarded when it is determined that the system would perform worse if the extracted face data is included, compared to not
including it. This can be done by evaluating the
performance with and without the added face data item.
For instance, assume that there are five images of user X in the system already: A, B, C, D and E. The system can then determine its current performance by removing A from the system and try to recognise A using B, C, D and E. Then it can remove B and try to recognize B using A, C, D and E and so forth. Assume all images but C and E are correctly recognized. The accuracy is then 3 out of 5 or 60 per cent. Now assume we have a set of new extracted face data F available. We can now evaluate how the system would perform if all images A-F were available. Again, A would be taken out, and the system would try to recognize A using B-F. Then B would be taken out, and so forth. Perhaps the system would now be able to recognize all images except E. Then the performance would be 5 out of 6, or 83 per cent. Since this accuracy is higher than the accuracy without the new extracted face data, which was 60 per cent, it is determined that image F was beneficial
for the purpose of identifying person X. However, if the accuracy would have decreased or stayed the same, extracted face data F would be rejected.
Such a difference between faces can for instance be calculated in the following way. First, the relevant portions of the image (i.e., the part of the image containing the face) must be identified. This is done using a face detection system, such as the one described by Viola and Jones [1] . Such a face detection system typically first determines a subset of an image using a rectangular frame around the area believed by the method to be the face. Face data can then be extracted using this rectangular frame by sampling the image to a fixed size, for example 24x24 pixels, and converting to grey scale. This 24x24 pixel grey scale image now represents extracted face data and can now be compared to similarly subsampled versions of the previously obtained face images for the same individual. In one embodiment, the comparison involves summing the squared difference of the grey scale value in each of the 24x24 pixels. The
resulting sum can be used as a measure of how similar the two images are. In this manner the summed squared
difference (SSD) value can be calculated between the new face image and all previous images. If the SSD value is larger than a predetermined threshold for all previous images, the face image is selected. Thus, if the smallest SSD value is larger than a certain threshold, the face image is selected.
In an alternate embodiment, the 24x24 pixel image is further processed before the SSD value is calculated. In this case, the average value of the grey scale value is calculated and removed from the image. This is done to
exclude new images that only differ from previous images in light intensity, not in light direction. In yet another embodiment, the average is removed, but then the standard deviation of the grey scale value is also calculated, and the image is divided by this factor so that the resulting image standard deviation is constant, for instance equal to 1.0. This is done to exclude new images that only differ from previous images in contrast, but not in light direction. In yet another embodiment the 24x24 pixel is smoothed by a Gaussian filter before the SSD value is calculated. This is done to make the method less dependent upon small shifts in the face detection frame .
While the face detection method described by Viola and Jones [1] typically works well, the frame around the face is sometimes not accurately determined. For instance, sometimes the frame is too far to the left, whereby the face in the 24x24 image will be too far to the right, and not in the centre of the image. This will result in an SSD value that is quite high even though the faces may be very similar. Therefore, in another embodiment of the invention, an extra step called face normalization is inserted between the face detection and the subsampling to the 24x24 pixel image. This face normalization can be done using the method described by Cootes et al . [2] in subsection 3.3 and shown in image 12. By setting the vector b_s in equation (5) to zero, the shape can be normalized to a neutral shape. In this neutral shape, the left eye is always in the same position in the image, the mouth is always in the same position in the image etc. This shape free image can now be subsampled to 24x24 pixels for comparison.
In yet another embodiment, the SSD value is also compared to an upper threshold. If the SSD value is higher than this upper threshold, the face image is not selected. This is to avoid selecting images that arise from errors in the face-detection step (such as described in [1]) or in the face normalization step (such as described in
[2]) . As an example, if the face-detection step
malfunctions and selects part of the background as the face, then hopefully the SSD value will be higher than if a regular face was found. In this way, by comparing against the second threshold, some false face candidates can be removed.
If it is determined that the face data is to be stored, the method continues to a store face data step 28.
Otherwise, if the face data is to be discarded, the method continues to a conditional continue face data acquisition step 30.
In the store face data step 28, the face data is stored in a storage device in the end user video conference device and/or the server.
In the conditional continue face data acquisition step 30, it is determined whether to continue face data acquisition or not. One reason not to continue can for example be that the video conference has ended. Another reason not to continue can be that a sufficient amount of face data is stored. When a large number of face data sets are stored, e.g. from about 50 to 100 images, additional face data is unlikely to provide a base for better face recognition. Furthermore, the larger number of face data sets to analyse during face recognition, the
larger computation is required, increasing resource requirements and/or face recognition times.
If it is determined to continue the face data
acquisition, the method continues to the extract face data step 23. Otherwise, the method eventually continues to a determine identity of a face step 32.
In the determine identity of a face step 32, the actual face recognition is performed. The face recognition is performed using the previously stored face data.
Typically, a number of sets of face data are required for the face recognition to be acceptably accurate. For example, at the time of filing this application, it is desired to have at least about five sets of face data obtained from the same number of captured images, for the face recognition to be acceptably accurate. This step is performed at a later stage than the previous steps.
Optionally, this step is performed in parallel with the previous steps. Moreover, this step can be performed in the same or a different physical device as for the previous steps.
Optionally, the determine identity of a face step 32 is repeated. After this step, the method ends.
Fig 7 is a schematic diagram showing an image 76 captured from a video conference. Within the image 76 there is a subset 75 of the image which corresponds to a face. The subset can be defined by an outline following the face, or it can optionally be a rectangular subset which envelopes the face. The subset 75 of the image can be used to extract face data which can later be used for face recognition.
Fig 3 is a schematic diagram showing functional modules of a face data acquirer included in end user video conference device and/or server of Fig 1. The face data acquirer is a device or software package capable of extracting face data from a video conference image and to associate the face data with an identity, such that the face data can later be used in face recognition.
The modules can be implemented using software such as a computer program executing in the end user video
conference device and/or server. All modules depend on an execution environment (not shown) which utilises a controller 18, 60 and optionally a computer program product 17/62 and a I/O interface 15/64.
An image capture module 43 is arranged to capture an image, such as the image 76 shown in Fig 7, from a video stream of a video conference. This module corresponds to step 20 of Fig 2.
A face detection module 44 is arranged to determine the subset 75 of the image 76 (Fig 7), where the subset represents a face. This module corresponds to step 21 of the method disclosed in Fig 2.
An identity acquisition module 45 is arranged to acquire an identity of a video conference participant coupled to the face represented by the subset 75 of the image 76. This module corresponds to step 22 of Fig 2.
A face extraction module 46 is arranged to extract face data from the subset 75 of the image 76 and to determine whether to store the extracted face data for subsequent face recognition. This module corresponds to step 23 of Fig 2.
Fig 4 schematically shows an embodiment of one or more of an end user video conference device 10. The end user video conference device corresponds to any of the end user video conference devices lOa-c of Fig 1, and the components of Fig 4 correspond, where applicable, to components of the end user video conference devices lOa-c of Fig 1 without the end user video conference device identification suffix letter (a, b or c) . A controller 60 is provided using any suitable central processing unit (CPU) , microcontroller, digital signal processor (DSP) , etc., capable of executing software instructions stored in a computer program product 62, e.g. in the form of a memory. The computer program product 62 can be a memory or any combination of read and write memory (RAM) and read only memory (ROM) . The memory also comprises
persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, or solid state memory or even remotely mounted memory, which can also be used as a storage device to store face data. An input/output interface 64 is provided to allow the end user video conference device 10 to interact with other components, such as the server 16 or other end user video conference devices, e.g. to set up or conduct a video conference. The I/O interface 64 can for example be a network interface such as an interface for wireless networks (e.g. any of the IEEE 801. llx standards) or wired networks (e.g. Ethernet). Alternatively or
additionally, the I/O interface 64 can be an interface to a mobile (cellular) network. A camera 12 is provided for capturing images that can be used in a video conference. The camera can be embedded in
the end user video conference device 10 or external and connected to the end user video conference device 10.
A display 11 is provided in the end user video conference device to allow the user to view other participants of the video conference. Additionally, input devices (not shown) such as a keyboard and/or pointing device (e.g. mouse, track pad or touch screen) are provided to allow user input.
Fig 5a is a schematic diagram showing a configuration for face data acquisition according to a first embodiment. In this embodiment, the face data acquirer 40 and the storage module 41 are comprised in the end user video conference device 10. The server 16 comprises the storage device 42, which stores the face data which the face data acquirer obtains. The decision whether to store or discard the face data is made in the storage module 41. This configuration allows the end user video conference device 10 to acquire face data, but the face data is stored centrally in the server 16 to allow access by more interested entities.
Fig 5b is a schematic diagram showing a configuration for face data acquisition according to a second embodiment. In this embodiment, the face data acquirer 40 and the storage device 42 are both located in the server 16. This reduces the requirements on the end user video conference device to acquire images from the video stream of the video conference. Optionally, the face data acquirer 40 can be delegated by the server 16 to another node in the network which is involved in the video conference, such as an intermediate video conferencing routing node between end user video conference devices lOa-c.
Furthermore, the server 16 comprises an optional face recognition device 48. This can be used to identify someone from a captured image of an unidentified face.
Fig 5c is a schematic diagram showing a configuration for face data acquisition according to a third embodiment. In this embodiment, the end user video conference device 10 comprises both the face data acquirer 40 and the storage device 42. This implies that the end user video
conference device 10 has local access to the face data for later face recognition by the optional face
recognition device 48.
Furthermore, the end user video conference device
optionally comprises a face signature generator 47. The face signature generator 47 is arranged to generate a face signature for the identity using the extracted face data in the storage device 42. The face signature is a set of data being sufficient to allow a face recognition device to recognise a face coupled to the identity in a face recognition image. This requires a sufficient number of sets of face data for the identity to generate the signature for. This signature can for example be attached to e-mails to facilitate face recognition by third parties. If several files are make up the signature, the files can be combined in the signature, e.g. by
concatenation and/or compression.
Fig 6 shows one example of a computer program product 70 comprising computer readable means. On this computer readable means a computer program 71 can be stored, which computer program can cause a controller to execute a method according to embodiments described herein. In this example, the computer program product is an optical disc,
such as a CD (compact disc) or a DVD (digital versatile disc) or a Blu-Ray disc. As explained above, the computer program product could also be embodied as a memory of a device, such as memory 62 of the end user video
conference device 10 or memory 17 of the server 16. While the computer program 71 is here schematically shown as a track on the depicted optical disk, the computer program can be stored in any way which is suitable for the computer program product. The invention has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other
embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.
References :
[1] Viola, Jones: Robust Real-time Object Detection, IJCV 2001
[2] T.F. Cootes, G. J. Edwards, and C. J. Taylor. Active appearance models. ECCV, 2:484-498, 1998
Claims
1. A face data acquirer (40) comprising:
an image capture module (43) arranged to capture an image (76) from a video stream of a video conference; a face detection module (44) arranged to determine a subset (75) of the image (76), the subset representing a face ;
an identity acquisition module (45) arranged to acquire an identity of a video conference participant coupled to the face represented by the subset (75) of the image (76) ; and
a face extraction module (46) arranged to extract face data from the subset (75) of the image (76) and to determine whether to store the extracted face data for subsequent face recognition.
2. The face data acquirer (40) according to claim 1, wherein the image analysis module (46) is arranged to determine to store the extracted face when an accuracy of face detection is determined to be improved when the extracted face data is stored compared to refraining from storing the extracted face data.
3. The face data acquirer (40) according to claim 1 or 2, wherein the image analysis module (46) is arranged to determine to store the extracted face data when a
measurement of difference of the extracted face data compared to a set of previously stored extracted face data for the same identity is greater than a threshold value .
4. The face data acquirer (40) according to any one of the preceding claims, wherein the image analysis module (46) is arranged to determine to store the extracted face data when a measurement of difference of lighting or contrast conditions in the image compared to a set of previously stored extracted face data for the same identity is greater than a threshold value.
5. The face data acquirer (40) according to any one of claims 1 to 4, wherein the image analysis module (46) is arranged to determine to store the extracted face data when a measurement of difference of lighting or contrast conditions in the subset of the image compared to a set of previously stored extracted face data for the same identity is greater than a threshold value.
6. The face data acquirer (40) according to any one of claims 1 to 5, wherein the image analysis module (46) is arranged to determine to refrain from storing the extracted face data when it is determined that the face represented in the subset of the image is looking away from a camera used to obtain the image.
7. The face data acquirer (40) according to any one of claims 1 to 6, wherein the image analysis module (46) is arranged to determine to refrain from storing the extracted face data when it is determined that the quality of the image (76) is low, e.g. due to packet losses . 8. An end user video conference device (10, 10a, 10b,
10c) comprising the face data acquirer (40) according to any one of claims 1 to 7.
The end user video conference device (10, 10a, 10b, according to claim 8, comprising a storage module (41) arranged to send a message to store the extracted face data in a storage device (42) .
10. The end user video conference device (10, 10a, 10b, 10c) according to claim 8, comprising a storage device (42) arranged to store the extracted face data.
11. The end user video conference device (10, 10a, 10b, 10c) according to any one of claims 8 to 10, comprising a face recognition device (48) arranged to determine an identity of a face represented in a face recognition image using the extracted face data stored in the storage device ( 42 ) .
12. The end user video conference device (10, 10a, 10b, 10c) according to any one of claims 8 to 11, further comprising a face signature generator (47) arranged to generate a face signature for the identity using the extracted face data in the storage device (42), wherein the face signature is a set of data being sufficient to allow a face recognition device to recognise a face coupled to the identity in a face recognition image.
13. A server (16) comprising the face data acquirer (40) according to any one of claims 1 to 7.
14. The server (16) according to claim 13, wherein the server ( 16) comprises a storage device (17, 42) for storing the extracted face data for subsequent face recognition.
15. The server (16) according to claim 14, comprising a face recognition device (48) arranged to determine an identity of a face represented in a face recognition image using the extracted face data stored in the storage device ( 42 ) .
16. A method performed in a face data acquirer
comprising :
capturing (20) an image from a video stream of a video conference;
determining (21) a subset (75) of the image (76), the subset (75) representing a face;
acquiring (22) an identity of a video conference participant coupled to the face represented by the subset (75) of the image (76);
extracting (23) face data from the subset of the image; and
determining (26) whether to store the extracted face data for subsequent face recognition.
17. The method according to claim 16, further comprising effecting (28) storage of the extracted face data in a storage device.
18. The method according to claim 16, further comprising: determining (32) an identity of a face represented in a face recognition image using the extracted face data stored in the storage device.
19. A computer program (71) for a face acquisition device, the computer program (71) comprising computer program code which, when run on the face acquisition device, causes the face acquirer to perform the steps of: capturing (20) an image from a video stream of a video conference;
determining (21) a subset of the image representing a face; acquiring (22) an identity of a video conference participant coupled to the face represented by the subset of the image;
extracting (23) face data from the subset of the image; and
determining (26) whether to store the extracted face data for subsequent face recognition.
20. A computer program product (70) comprising a computer program (71) according to claim 19 and a computer
readable means on which the computer program (71) is stored .
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/824,822 US8965067B2 (en) | 2010-10-28 | 2010-10-28 | Face data acquirer, end user video conference device, server, method, computer program and computer program product for extracting face data |
PCT/SE2010/051174 WO2012057665A1 (en) | 2010-10-28 | 2010-10-28 | A face data acquirer, end user video conference device, server, method, computer program and computer program product for extracting face data |
EP10859039.9A EP2633474A4 (en) | 2010-10-28 | 2010-10-28 | A face data acquirer, end user video conference device, server, method, computer program and computer program product for extracting face data |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/SE2010/051174 WO2012057665A1 (en) | 2010-10-28 | 2010-10-28 | A face data acquirer, end user video conference device, server, method, computer program and computer program product for extracting face data |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012057665A1 true WO2012057665A1 (en) | 2012-05-03 |
Family
ID=45994162
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/SE2010/051174 WO2012057665A1 (en) | 2010-10-28 | 2010-10-28 | A face data acquirer, end user video conference device, server, method, computer program and computer program product for extracting face data |
Country Status (3)
Country | Link |
---|---|
US (1) | US8965067B2 (en) |
EP (1) | EP2633474A4 (en) |
WO (1) | WO2012057665A1 (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103136357A (en) * | 2013-03-06 | 2013-06-05 | 广东欧珀移动通信有限公司 | Method, device and mobile equipment for positioning webpages by face recognition technology |
CN103400106A (en) * | 2012-06-22 | 2013-11-20 | 微软公司 | Self learning face recognition using depth based tracking for database generation and update |
EP2869546A1 (en) * | 2013-10-31 | 2015-05-06 | Alcatel Lucent | Method and system for providing access to auxiliary information |
US9300916B1 (en) | 2015-02-10 | 2016-03-29 | International Business Machines Corporation | Natural gazes during online video conversations |
WO2017004464A1 (en) | 2015-06-30 | 2017-01-05 | Nec Corporation Of America | Facial recognition system |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5740972B2 (en) * | 2010-09-30 | 2015-07-01 | ソニー株式会社 | Information processing apparatus and information processing method |
WO2013191680A1 (en) * | 2012-06-19 | 2013-12-27 | Empire Technology Development, Llc | Group nature verification for providing group incentives |
GB201410285D0 (en) * | 2014-06-10 | 2014-07-23 | Appeartome Ltd | Augmented reality apparatus and method |
CN105611209A (en) * | 2015-12-23 | 2016-05-25 | 天津天地伟业数码科技有限公司 | Human face retrieving and snapshooting method based on NVR (Network Video Recorder) video file |
US10148649B2 (en) * | 2016-05-18 | 2018-12-04 | Vercrio, Inc. | Automated scalable identity-proofing and authentication process |
US11843597B2 (en) * | 2016-05-18 | 2023-12-12 | Vercrio, Inc. | Automated scalable identity-proofing and authentication process |
KR20200136961A (en) * | 2018-03-23 | 2020-12-08 | 아비질론 코포레이션 | Methods and systems for interacting with users to facilitate image search for interests |
CN108540817B (en) * | 2018-05-08 | 2021-04-20 | 成都市喜爱科技有限公司 | Video data processing method, device, server and computer readable storage medium |
CN109302628B (en) * | 2018-10-24 | 2021-03-23 | 广州虎牙科技有限公司 | Live broadcast-based face processing method, device, equipment and storage medium |
US11386562B2 (en) | 2018-12-28 | 2022-07-12 | Cyberlink Corp. | Systems and methods for foreground and background processing of content in a live video |
CN110414798A (en) * | 2019-07-03 | 2019-11-05 | 天津市多智信息科技有限公司 | A kind of meeting signature system and method based on video human face identification |
CA3133908A1 (en) * | 2020-10-20 | 2022-04-20 | Sean David Paul Rutherford | Method and system for capturing student images |
CN114302098A (en) * | 2021-12-29 | 2022-04-08 | 北京德为智慧科技有限公司 | Display method and conference system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2008045153A2 (en) * | 2006-10-11 | 2008-04-17 | Cisco Technology, Inc. | Interaction based on facial recognition of conference participants |
EP1998286A1 (en) * | 2006-03-15 | 2008-12-03 | Omron Corporation | Face-image registration device, face-image registration method, face-image registration program, and recording medium |
US20100149305A1 (en) * | 2008-12-15 | 2010-06-17 | Tandberg Telecom As | Device and method for automatic participant identification in a recorded multimedia stream |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8218829B2 (en) * | 2001-08-20 | 2012-07-10 | Polycom, Inc. | System and method for using biometrics technology in conferencing |
JP4303602B2 (en) * | 2004-01-09 | 2009-07-29 | 本田技研工業株式会社 | Facial image acquisition system |
US7734067B2 (en) * | 2004-12-07 | 2010-06-08 | Electronics And Telecommunications Research Institute | User recognition system and method thereof |
JP4636190B2 (en) * | 2009-03-13 | 2011-02-23 | オムロン株式会社 | Face collation device, electronic device, face collation device control method, and face collation device control program |
-
2010
- 2010-10-28 US US13/824,822 patent/US8965067B2/en active Active
- 2010-10-28 WO PCT/SE2010/051174 patent/WO2012057665A1/en active Application Filing
- 2010-10-28 EP EP10859039.9A patent/EP2633474A4/en not_active Ceased
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1998286A1 (en) * | 2006-03-15 | 2008-12-03 | Omron Corporation | Face-image registration device, face-image registration method, face-image registration program, and recording medium |
WO2008045153A2 (en) * | 2006-10-11 | 2008-04-17 | Cisco Technology, Inc. | Interaction based on facial recognition of conference participants |
US20100149305A1 (en) * | 2008-12-15 | 2010-06-17 | Tandberg Telecom As | Device and method for automatic participant identification in a recorded multimedia stream |
Non-Patent Citations (1)
Title |
---|
See also references of EP2633474A4 * |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103400106A (en) * | 2012-06-22 | 2013-11-20 | 微软公司 | Self learning face recognition using depth based tracking for database generation and update |
WO2013192253A1 (en) * | 2012-06-22 | 2013-12-27 | Microsoft Corporation | Self learning face recognition using depth based tracking for database generation and update |
US8855369B2 (en) | 2012-06-22 | 2014-10-07 | Microsoft Corporation | Self learning face recognition using depth based tracking for database generation and update |
US9317762B2 (en) | 2012-06-22 | 2016-04-19 | Microsoft Technology Licensing, Llc | Face recognition using depth based tracking |
CN103136357A (en) * | 2013-03-06 | 2013-06-05 | 广东欧珀移动通信有限公司 | Method, device and mobile equipment for positioning webpages by face recognition technology |
EP2869546A1 (en) * | 2013-10-31 | 2015-05-06 | Alcatel Lucent | Method and system for providing access to auxiliary information |
WO2015063055A1 (en) * | 2013-10-31 | 2015-05-07 | Alcatel Lucent | Method and system for providing access to auxiliary information |
US9626003B2 (en) | 2015-02-10 | 2017-04-18 | International Business Machines Corporation | Natural gazes during online video conversations |
US9300916B1 (en) | 2015-02-10 | 2016-03-29 | International Business Machines Corporation | Natural gazes during online video conversations |
WO2017004464A1 (en) | 2015-06-30 | 2017-01-05 | Nec Corporation Of America | Facial recognition system |
CN108027883A (en) * | 2015-06-30 | 2018-05-11 | 美国日本电气公司 | Facial-recognition security systems |
JP2018525723A (en) * | 2015-06-30 | 2018-09-06 | エヌイーシー コーポレイション オブ アメリカ | Face recognition system |
EP3317815A4 (en) * | 2015-06-30 | 2019-02-20 | NEC Corporation of America | Facial recognition system |
AU2016287726B2 (en) * | 2015-06-30 | 2019-12-12 | Nec Corporation Of America | Facial recognition system |
US10657362B2 (en) | 2015-06-30 | 2020-05-19 | Nec Corporation Of America | Facial recognition system |
US11501566B2 (en) | 2015-06-30 | 2022-11-15 | Nec Corporation Of America | Facial recognition system |
Also Published As
Publication number | Publication date |
---|---|
EP2633474A4 (en) | 2017-03-22 |
US20130177219A1 (en) | 2013-07-11 |
US8965067B2 (en) | 2015-02-24 |
EP2633474A1 (en) | 2013-09-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8965067B2 (en) | Face data acquirer, end user video conference device, server, method, computer program and computer program product for extracting face data | |
CN109858371B (en) | Face recognition method and device | |
KR102354692B1 (en) | Rule-based video importance analysis | |
CN109284729B (en) | Method, device and medium for acquiring face recognition model training data based on video | |
US9251588B2 (en) | Methods, apparatuses and computer program products for performing accurate pose estimation of objects | |
US20170111532A1 (en) | Real-time processing of video streams captured using mobile devices | |
US10089534B2 (en) | Extracting high quality images from a video | |
US9531999B2 (en) | Real-time smart display detection system | |
CN106575280B (en) | System and method for analyzing user-associated images to produce non-user generated labels and utilizing the generated labels | |
US10205875B2 (en) | Image enhancement and repair using sample data from other images | |
US11710347B2 (en) | Information processing apparatus, information processing method, and program | |
JP6026680B1 (en) | Method and system for performing image identification | |
CN110765134A (en) | File establishing method, equipment and storage medium | |
WO2014074959A1 (en) | Real-time face detection using pixel pairs | |
CN110505513A (en) | A kind of video interception method, apparatus, electronic equipment and storage medium | |
CN109711287B (en) | Face acquisition method and related product | |
US20230359699A1 (en) | Information processing apparatus, information processing method, and non-transitory computer-readable storage medium | |
US10282633B2 (en) | Cross-asset media analysis and processing | |
CN115499677A (en) | Audio and video synchronization detection method and device based on live broadcast | |
JP2012033054A (en) | Device and method for producing face image sample, and program | |
CN114973293A (en) | Similarity judgment method, key frame extraction method, device, medium and equipment | |
EP2336975A1 (en) | Apparatus and method for image registration in portable terminal | |
US11893791B2 (en) | Pre-processing image frames based on camera statistics | |
JP2014229092A (en) | Image processing device, image processing method and program therefor | |
CN111507289A (en) | Video matching method, computer device and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10859039 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010859039 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13824822 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |