CN105917360A - Face detection and recognition - Google Patents

Abstract

The present invention provides, in at least one aspect, a method and a system. The method and the system detect at least one face in at least one digital image, determine and store area co-ordinates of a location of the at least one detected face in the at least one digital image, apply at least one transformation to the at least one detected face to create at least one portrait of the at least one detected face, rotate the at least one portrait at least until the at least one portrait is shown in a vertical orientation and a pair of eyes of the at least one face shown in the at least one portrait are positioned on a horizontal plane; and store the rotated at least one portrait.

Description

Face detection and identification

Cross-Reference to Related Applications

This application claims that on November 12nd, 2013 submits to, entitled " FACE DETECTION AND RECOGNITION " U.S. Patent Application Serial Number 14/078, the priority of 071, entire contents is merged by entirety accordingly by quoting.

Technical field

The present invention relates to use face detection and the face identified and portrait extracts and application.

Background technology

Social networks currently exists as a kind of means using website to connect individuality.For " social networks " PCMAG.COM^TMEncyclopedia exists defined below:

According to family, work or like the group being attracted personnel together.First this term is taught by J.A.Barnes Awarding and create in nineteen fifties, the size of social networks is defined as the group of about 100 to 150 personnel by him.

PCMAG.COM for " social network site "^TMEncyclopedia exists defined below:

The website of virtual community is provided to personnel that are interested in particular topic or that " saunter " the most together.Member passes through language Sound, chat, instant message, video conference exchange with blog, and service and generally provide a kind of to member and contact other members' The mode of friend.Such website is also used as the medium contacted face-to-face." social network sites " is for " virtual community " 21st century term, uses a group of people that the Internet exchanges each other with everything feelings about anything.

Friendster (www.friendster.com) is first social network sites, and it was introduced in 2002, and MySpace (www.myspace.com) is introduced after 1 year.By two friends, MySpace becomes the most popular, and And issue after 2 years at MySpace, his father company Intermix by News Corporation with 5.8 hundred million dollars of purchases.

Facebook (www.facebook.com) occurred in 2004, and it is initially for university students, but welcomes later Everyone.It is TagWorld (www.tagworld.com) and Tagged (www.tagged.com) after Facebook. TagWorld introduces the instrument for creating more personalized webpage, and Tagged introduces for having similar interests Teenager builds the concept of wrestling doubles (tag team).

Social network sites like when the Internet the first portal website when middle 1990s is announced to the world splendidly that Sample competition attention.Expect to have a lot of modification.

A lot of social network sites allow user upload and share photo.Some social network sites also comprise labelling photo to identify photograph Feature (the such as FACEBOOK of the name (facial) in sheet^TMThere is provided this feature).Based on user investigation, most surveyees refer to Go out labelling and make great efforts to be manual and the most time-consuming.It addition, for privacy purposes, a lot of users are not by its all digital photos Upload to share website.Uploading into thousand sheets photo the most time-consumingly and is bandwidth intensive.Therefore, although social network sites User can have 10000 digital photos on its local computer, but they only upload 100 with its friend Share.This is based on the user investigation carried out in JIUYUE, 2007 by Applied Recognition company.

There is also permission registration user's upload digital photo and digital video and store it on website for sharing The website of purpose.These websites are exclusively used in this purpose.The example of these websites includes FLICKR^TMAnd PHOTOBUCKET^TM.This The defect of a little websites is, is manual and time-consuming for identifying all photograph tags of friend；PHOTOBUCKET^TMDo not permit Permitted personnel to be marked in photo.At FLICKR^TMIn the case of, if an average pictures comprises two personnel, then it can Every photo cost labelling personnel within 10 to 15 seconds, can be carried out.When this time is multiplied by 100 or 1000, it becomes too time-consuming and puts down All individual is by not execution flag.

Along with digital camera merging in most modern cell phones, occur in that the quick growth of photograph taking equipment now. It practice, compared with all dedicated digital cameras sums, more polyphaser is sold via mobile phone now.Which results in is in uses tricks The propagation of the number of the digital picture being uploaded on calculation machine and store.Due to the investigation based on us of the average number of digital picture More than 1000, so hand labeled and correctly organize the image of a large amount of personnel to take very much energy.

One company RIYA^TM(www.riya.com) create for identify in digital photos face based on website Product.This product relates to the download of software module, and this software module is for long-range RIYA^TMServer based on website Upload on the local computer of user, identify the face in photo before these photos, these faces in this server with its Its face compares to find out coupling.This product is prototype, and the automatic photograph being therefore not based on identifying shares spy Levy.Its also not used for allow users to correct any automatic face detection and recognition methods in occur inevitable The feature of mistake.

Before 2007, use the text relevant to photo or date (such as title, exercise question, description, date created Deng) rather than picture material itself carry out picture search.Since then, having had a large amount of company to start should consumer photograph Introduce face detection with middle and identify, including: in 2008, Google introduced facial recognition in Picasa；In 2009, use Apply in the Polar Rose of Flickr；In 2010, Apple purchased Polar Rose and introduces this spy in iPhoto Levy；In 2010, Microsoft introduced facial recognition in its Photo Gallery product；In 2010, Facebook drew Enter face detection；And in 2010, Sony Ericsson is integrated facial recognition software in its photo library.

Summary of the invention

In one aspect of the invention, it is provided that a kind of side for identifying one or more face in digital picture Method, the method is characterized in that: (a) generates or many of the one or more candidate regions corresponding to one or more faces Individual face coordinate；B () is based on face Coordinate generation eyes coordinates (eye coordinate)；C () uses by face coordinate and eye One or more projection pictures of eyeball coordinate definition detect each face；And (d) by each projection picture with one or more Known projection image compares, and which provides the optimal phase mated for defining between projection picture with known projection image Seemingly spend thresholding.

In another aspect of this invention, it is provided that the side of a kind of digital picture for the one or more face of shared description Method, the method is characterized in that: multiple terminals are linked to computer network by (a), and each terminal is with individual It is associated；B digital picture is linked at least one terminal in terminal by ()；C () makes terminal In at least one terminal can initiate facial recognition routine on the digital image, facial recognition routine produce its face The list of the one or more individuals being depicted in digital picture, at least one individual in these people be above-mentioned individuality it One；And (d) make at least one terminal in terminal can initiate for one or more individuals The terminal being associated spreads the shared routine of digital picture.

In another aspect of this invention, it is provided that the side of a kind of digital picture for the one or more face of shared description Method, the method is characterized in that: (a) multiple terminals are linked to computer network and by multiple cloud services with based on The digital picture stored in the data storage bank of cloud and metadata link；B digital picture is linked in terminal by () At least one terminal；C () makes at least one terminal in terminal to send out on the digital image Playing facial recognition routine, facial recognition routine produces the row of one or more individuals that its face is depicted in digital picture Table, at least one individual in these people is one of above-mentioned individuality；And (d) makes at least one meter in terminal Calculation machine terminal can initiate the shared example for spreading digital picture to the terminal being associated with one or more individuals Journey.

In another aspect of this invention, it is provided that the side of a kind of digital picture for the one or more face of shared description Method, the method is characterized in that: multiple terminals are linked to computer network by (a), and each terminal is with individual It is associated, and by multiple cloud services and the number of storage on each terminal and in data storage bank based on cloud Word image and metadata link；B digital picture is linked at least one terminal in terminal by ()；C () makes At least one terminal obtained in terminal can initiate facial recognition routine, facial recognition example on the digital image Journey produces the list of one or more individuals that its face is depicted in digital picture, at least one individual in these people It it is one of above-mentioned individuality；And (d) makes at least one terminal in terminal can initiate for one The terminal that individual or multiple individuals are associated spreads the shared routine of digital picture.

In another aspect of this invention, it is provided that the side of a kind of digital picture for the one or more face of shared description Method, the method is characterized in that: (a) is by multiple cameras or the intelligence of the camera with shooting picture or shooting video image Telephone link, and multiple terminals are linked to computer network, each terminal is associated with individuality, and By multiple cloud services and the digital picture stored on each terminal and in data storage bank based on cloud and unit Data link；B digital picture is linked at least one terminal in terminal by ()；(c) make camera or The smart phone with camera can initiate facial recognition routine on the digital image, and facial recognition routine produces its face and counting The list of the one or more individuals being depicted in word image, at least one individual in these people is one of above-mentioned individuality； And (d) makes at least one terminal in terminal can initiate for relevant to one or more individuals The terminal of connection spreads the shared routine of digital picture.

Other aspect in the present invention, it is provided that a kind of for identify one or more face in digital picture be System, the system is characterized in that: (a) corresponds to one or more of one or more candidate regions of one or more faces Portion's coordinate；(b) eyes coordinates based on face Coordinate generation；(c) by face coordinate and eyes coordinates define one or more Projection picture；And (d) optimal mates for define between each projection picture with one or more known projection images Similarity thresholding, this most preferably coupling determines the identity corresponding to each face in one or more faces.

In still another aspect of the invention, it is provided that a kind of digital picture for the one or more face of shared description be System, the system is characterized in that: (a) links to multiple terminals of computer network, and each terminal is with individual It is associated；The digital picture of (b) operable at least one terminal to link in terminal；C () is operable With the facial recognition routine initiated by least one terminal in terminal, facial recognition routine produces its face The list of the one or more individuals being depicted in digital picture, at least one individual in these people be above-mentioned individuality it One；And (d) operable with by terminal at least one terminal initiate shared routine, share routine to The terminal being associated with one or more individuals spreads digital picture.

In still another aspect of the invention, it is provided that a kind of method for providing safe targeted ads, the spy of the method Levy and be: (a) follows the tracks of the one or more demographic attributes being associated with the individuality registered to computer program；(b) from The advertisement pointer being associated with the one or more advertisements oriented based on one or more demographic attributes is retrieved in first source List；C () retrieves one or more advertisements from the second source；D () deletes one or more demographic attributes from the first source； And (e) presents advertisement to individuality.

In still another aspect of the invention, it is provided that a kind of for identify one or more face in digital picture be System, the system is characterized in that: (a) corresponds to one or more of one or more candidate regions of one or more faces Portion's coordinate；(b) eyes coordinates based on face Coordinate generation；(c) by face coordinate and eyes coordinates define one or more Projection picture；And (d) optimal mates for define between each projection picture with one or more known projection images Similarity thresholding, this most preferably coupling determines the identity corresponding to each face in one or more faces.

In still another aspect of the invention, it is provided that a kind of digital picture for the one or more face of shared description be System, the system is characterized in that: (a) links to multiple terminals of computer network, and each terminal is with individual It is associated；The digital picture of (b) operable at least one terminal to link in terminal；C () is operable With the facial recognition routine initiated by least one terminal in terminal, facial recognition routine produces its face The list of the one or more individuals being depicted in digital picture, at least one individual in these people be above-mentioned individuality it One；And (d) operable with by terminal at least one terminal initiate shared routine, share routine to The terminal being associated with one or more individuals spreads digital picture.

In still another aspect of the invention, it is provided that a kind of for providing the system of safe targeted ads, the spy of this system Levy and be: one or more demographic attributes that (a) is associated with the individuality registered to computer program；B () is operable To provide the row of the advertisement pointer being associated with the one or more advertisements oriented based on one or more demographic attributes First source of table；C () is operable to provide the second source of one or more advertisements；D () is for deleting one or many from the first source The device of individual demographic attributes；And (e) for presenting the device of advertisement to individuality.

In still another aspect of the invention, it is provided that a kind of computer journey identifying one or more face in digital picture Sequence product, this computer program is characterised by: (a) includes the computer-readable medium of software instruction；And (b) is used for Enabling a computer to perform the software instruction of scheduled operation, this scheduled operation includes step: (i) generates corresponding to one or many One or more face coordinates of one or more candidate regions of individual face；(ii) based on face Coordinate generation eyes coordinates； (iii) the one or more projection pictures defined by face coordinate and eyes coordinates are used to detect each face；And (iv) incites somebody to action Each projection picture, compared with one or more known projection images, which provides for defining projection picture and known throwing The similarity thresholding of the optimal coupling between shadow image.

In another aspect of this invention, it is provided that a kind of based on the digital picture of the one or more face of shared description Calculation machine program product, this computer program is characterised by: (a) includes the computer-readable medium of software instruction；And B (), for enabling a computer to perform the software instruction of scheduled operation, this scheduled operation includes step: (i) is by multiple calculating Machine final link is connected to computer network, and each terminal is associated with individuality；(ii) digital picture is linked to computer At least one terminal in terminal；(iii) at least one terminal in terminal is made to count Initiating facial recognition routine on word image, this facial recognition routine produces or many that its face is depicted in digital picture The list of individual, at least one individual in these people is one of above-mentioned individuality；And (iv) makes in terminal At least one terminal can initiate for spreading numeral to the terminal that is associated with one or more individuals The shared routine of image.

In still another aspect of the invention, it is provided that a kind of for providing the computer program of safe targeted ads, This computer program is characterised by: (a) includes the computer-readable medium of software instruction；And (b) is used for making to succeed in one's scheme Calculation machine is able to carry out the software instruction of scheduled operation, and this scheduled operation includes step: (i) follows the tracks of and to computer program registration One or more demographic attributes that individuality is associated；(ii) from the first source retrieval and based on one or more demographics The list of the advertisement pointer that one or more advertisements of attribute orientation are associated；(iii) one or more from the second source retrieval Advertisement；(iv) one or more demographic attributes are deleted from the first source；And (v) presents advertisement to individuality.

The present invention makes it possible to capture the portrait of the personnel that its face is positioned in image at its other aspect.If at figure Face being detected in Xiang, computer program the most provided herein captures the XY coordinate of the position of face and is recorded in data In storehouse.Computer program may be configured to regulate the size of facial zone so that capture and create portrait (or the breviary of individual Figure).It addition, computer program provided herein can be automatically adjusted the rotating with for eyes portrait in the horizontal plane of face Render facial positions vertically.

The present invention makes it possible for captured at its other aspect and is stored in the people that its face is positioned in image Portrait in the data base of member.Computer program can select portrait and the advertisement of portrait with product image is merged be used for Consumer shows.Computer program may be configured to the size regulating portrait and wants with the size of coupling advertisement or product image Ask.

The present invention makes it possible to hide the personnel that its face is positioned in image at its other aspect.Computer program is counting The XY coordinate of the position of repressed face is wanted according to inquiry in storehouse.The user of computer program can select to be used for covering existing The image in region, portion.Computer program is by stacked with selected image (at single figure above the face selected individual In Xiang or comprise this individual all images in) and show amended image to consumer.This move can be for Amusement purpose is carried out or is carried out hiding negative memory.Computer program may be configured to regulate portrait size with The size requirement of the facial size to be hidden selected by coupling.Computer program can by amendment storage in the metadata or For good and all apply change to original photo.The edge of (displacement) image overlapped can be adjusted to matching color, intensity, bright Degree, texture and other characteristics are to be mixed in original image and the most more attractive.

Another aspect of the present invention makes it possible to show the face from photo capture or portrait according to history order.Computer Program makes it possible to from image capturing portrait.Computer is always according to fileinfo, the day of EXIF (metadata) information trace photo Phase, or the user of computer program can specify older nonnumeric photo alternatively scanning required for day of image Phase.Computer program uses the date data from the photo relevant to the portrait from such photo capture to arrange according to the date The display of portrait.Computer program can use the face that photograph deformation technology generates and display is individual to develop in time subsequently Animation.

Another aspect of the present invention makes the face from photo capture or portrait can act as game or the content of exercise.Meter Calculation machine program makes it possible to from image capturing portrait.Computer program also safeguards the relation between the personnel in photo and personnel Data base.Computer program makes user's (" player ") of calling program and photo and from such by using such data The portrait of photo capture is relevant thus arranges above-mentioned photo or the display of portrait of player based on game or exercise customization.This The customization of sample can be simple as the lantern slide of photo based on relation.Customization can also be posture, such as having Posture very useful for the personnel of Alzheimer or Dementia, wherein can present individual's photo to Alzheimer patient With relation as game or exercise.This process commonly referred to photo is missed old times or old friends treatment.Use the photo and portrait extracted from photo The computer program of index retrieve such photo and portrait based on data base querying and the image of selection be embedded into In game and exercise.

The present invention is in the establishment of the convenient family tree as the index for photo of its other aspect.Computer program is counting According to storehouse being inquired about all and given user-dependent portrait and being then presented to user.The user of computer program is then Select on portrait and the specific node that is dragged in family tree to add new relation.When new relatives' portrait is dragged by user When being pulled on a spectrogram, computer program will dynamically multiply additional relationships.Once all portraits of given family are added to Family tree, then it completes and subsequently becomes and shows for consumer and the dynamic index of verification film.It is alternatively possible to use phase (self-defining) relation that friend is linked together and illustrates between different types of friend by same tree format.Example can To be colleague, senior middle school friend, college friends etc..

The present invention makes it possible for organizing portrait at its other aspect and is used as the index of photo.Anyone tissue is the most right This is interested.Generally, tissue is according to safeguarding group photo over year.Such as, to have the display in each season athletic for exercise group Group photo.Boy legion, orchestra, class's photo, club etc. generally use identical method.Computer program makes it possible to The synthesis identity of the object in enough use group photos and the above-mentioned photo index accomplishing the individual photo comprised in group.This makes Can be based on year/season and organized and construct photo by group membership and group membership's subset and show.

Another aspect of the present invention makes it possible to show face or the portrait of the missing child that FBI wants most and finds The similar demand of people.Computer program shows this to the user helping the computer program to find above-mentioned individual seeking user A little images.Computer also delivers the face signature of the above-mentioned individual found and is signed and it by above-mentioned face to user's request License that individual database compares and report potential coupling.If there is potential coupling, then will present such to user Potential coupling, and user can notify such potential coupling to corresponding authoritative institution alternatively.This is so that organize energy Enough determine the recently or currently position of individuality.It is it is also possible that tissue can determine the name of individuality, because the name of individuality Can be listed in known the list of user one or more in.

According to an aspect of the present invention, it is provided that a kind of method performed by least one computer, the method includes: At least one digital picture detects at least one face；Determine and store at least one detected face at least one number The area coordinate of the position in word image；Apply what at least one conversion was detected with establishment at least one detected face At least one portrait of at least one face；Rotate at least one portrait and be illustrated in vertical side at least up at least one portrait The pair of eyes of at least one face being illustrated on position and at least one portrait is positioned in the horizontal plane；And deposit At least one portrait that storage is rotated.

According to an aspect of the present invention, it is provided that one includes at least one system for computer, at least one computer Including at least one processor and non-Transient calculation machine readable memory, non-Transient calculation machine readable memory includes that process refers to Order, processes instruction and makes computer when being performed by least one processor: detect at least one at least one digital picture Individual face；Determine and store the area coordinate of at least one detected face position at least one digital picture；To At least one detected face applies at least one conversion to create at least one portrait of at least one detected face； Rotate at least one portrait to be illustrated in vertical orientation at least up at least one portrait and quilt at least one portrait The pair of eyes of at least one face illustrated is positioned in the horizontal plane；And at least one portrait that storage is rotated.

According to an aspect of the present invention, it is provided that a kind of method performed by least one computer, at least one calculates Machine includes data base or is connected with data base interface, and data base includes shown in multiple portrait, each portrait and corresponding portrait The individual of identification be associated, the method includes: display be associated to user at least one identified individual corresponding portrait； Show that at least one personal relationship presents to the virtual of user；According to the user's input received, in the personal relationship of display At least one personal relationship distributes at least one portrait in the portrait of display；And personal relationship's distribution is stored in data base In.

According to an aspect of the present invention, it is provided that a kind of method performed by least one computer, at least one calculates Machine includes that portrait and the data base of digital picture or the data base interface with portrait and digital picture are connected, each portrait and phase Answering the individual of identification shown in portrait to be associated, the method includes: at least one digital picture in display digit image；Hand over Fork quotes at least one digital picture of display and the data base of portrait to create shown at least one digital picture extremely A few list having identified individual；The subset of the list index at least digital picture of individual has been identified according at least one；With And input according to user, from least one digital picture of display, select at least one individual identifying in individual, display At least one digital picture in the digital picture correspondingly indexed.

According to an aspect of the present invention, it is provided that a kind of method performed by least one computer, at least one calculates Machine includes the data base of portrait or is connected with the data base interface of portrait, each portrait and the identification shown in corresponding portrait Individual is associated, and the method includes: receive at least one unidentified portrait；By at least one unidentified portrait face signature with Identify known to user that the face signature of the portrait of individual compares；According to the positive result compared, prompting user knot certainly The confirmation of fruit；And according to confirmation so that at least one unidentified portrait and user confirm at least one identified individual phase Associate and at least one unidentified portrait is stored in data base.

According to an aspect of the present invention, it is provided that a kind of method performed by least one computer, at least one calculates Machine includes the data base of digital picture and identifies that at least one shown in respective digital image has identified the name of individual Respective meta-data or with a data base of digital picture and identify that at least one shown in respective digital image has identified The respective meta-data interface of the name of people connects, and the method includes: select to identify one of individual from data base；Determine and know Not individual occurs in the total counting of digital picture therein；Selectively identify in the digital picture of individual at tool At least one digital picture shown in each identified individual, determine that tool has selectively identified that the corresponding of individual identifies Individual occurs in the counting of the sum of digital picture therein；And display visual representation, visual representation includes: primary nodal point, Primary nodal point represents identifying individual and having identified that individual occurs in the mutually accrued of the sum of digital picture therein of selection Number；Selectively identified shown at least one digital picture in the digital picture of individual for tool each has identified The secondary nodal point of people, secondary nodal point represents that tool has selectively identified at least one digital picture in the digital picture of individual Illustrate has identified individual accordingly, and each respective nodes also includes that tool has selectively identified that the corresponding of individual has identified that individual goes out The visual representation of the corresponding counts of the sum of present digital picture therein.

According to an aspect of the present invention, it is provided that a kind of method performed by least one computer, at least one calculates Machine includes the data base of digital picture and identifies that at least one shown in respective digital image has identified the name of individual Respective meta-data or with a data base of digital picture and identify that at least one shown in respective digital image has identified The respective meta-data interface of the name of people connects, and the method includes: select to identify one of individual from data base；Determine numeral Shown at least one digital picture in image at least one first identified individual together with select identification individual；With And display visual representation, visual representation includes: for each first the first hierarchy node having identified individual, the first hierarchy node Represent select identify individual and corresponding first shown in the digital picture of at least one in digital picture be together Identify individual；And having identified the second individual hierarchy node for each first, the second level node table shows nonoptional Identify that corresponding first shown at least one digital picture in the digital picture of individual has identified individual.

According to an aspect of the present invention, it is provided that a kind of method performed by least one computer, at least one calculates Machine includes the data base of portrait or is connected with the data base interface of portrait, each portrait and the identification shown in corresponding portrait Individual is associated, and the method includes: select to identify one of individual from data base；It is at least partially based on and each corresponding portrait The day metadata being associated is for the individual of the identification associated plurality of portrait sequence selected；Display virtual representation, virtual Expression includes: timeline；And multiple portrait according to corresponding sequence along the layout of timeline.

According to an aspect of the present invention, it is provided that a kind of method performed by least one computer, at least one calculates Machine include digital picture data base and identify corresponding data image date respective meta-data or with digital picture The respective meta-data interface on the date of data base and identification corresponding data image connects, and multiple digital pictures illustrate at least one Corresponding unidentified individual (" Unidentified digital picture "), the method includes: by corresponding day metadata to Unidentified number Word image is classified；Clustering token accordingly to the distribution of each Unidentified digital picture, wherein distribution includes: according to Unidentified number The subset of word image is each to be illustrated public unidentified individual this determines, each Unidentified digital picture in subset is divided Join and public cluster token accordingly；By corresponding cluster token, Unidentified digital picture is grouped；Receive new digital picture And identifying the respective meta-data on the date of the newest digital picture, new digital picture includes new unidentified individual；Press At least one of at least one corresponding unidentified individual performing new unidentified individual and multiple digital picture according to order compares, Wherein for often organizing Unidentified digital picture, only perform the single of unidentified individual with the most public new unidentified individual Relatively；According to comparing the digital picture distribution cluster token performed to new, produce corresponding in each group of unidentified digital picture Organize the determination of the public new unidentified individual of unidentified digital picture.

According to an aspect of the present invention, it is provided that the identification of the unidentified individual in a kind of digital picture advising receiving Method, the method is performed by least one computer, and at least one computer includes portrait and the number of the digital picture being associated Be connected according to storehouse or with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait Identifying that individual is associated, the method comprises determining that at least two shown in one of digital picture has identified corresponding Xiao of individual As meeting comparison threshold with the portrait of the unidentified individual determined according to the digital picture received；Advise identifying unidentified individual It is confirmed as at least two and has identified for having individual is most closely mated being associated accordingly of portrait of unidentified individual What at least two of portrait had identified in individual corresponding has identified individual；And get rid of advise subsequently being identified as unidentified individual At least two from any other digital picture in digital picture has identified that remaining in individual has identified individual.

According to an aspect of the present invention, it is provided that the identification of the unidentified individual in a kind of digital picture advising receiving Method, the method is performed by least one computer, and at least one computer includes portrait and the number of the digital picture being associated Be connected according to storehouse or with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait Identifying that individual is associated, the method has comprised determining that at least one shown at least one digital picture in digital picture Identify that the corresponding portrait of individual meets comparison threshold with the portrait of the unidentified individual determined according to the digital picture received；According to The digital picture that the metadata on the date of the day metadata being associated with the digital picture included corresponding to and receive is associated In at least one digital picture of determination, it is proposed that unidentified individual is identified as at least one and has identified individual.

According to an aspect of the present invention, it is provided that the identification of the unidentified individual in a kind of digital picture advising receiving Method, the method is performed by least one computer, and at least one computer includes portrait and the number of the digital picture being associated Be connected according to storehouse or with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait Identifying that individual is associated, the method includes: include that at least one being associated with the group of definition identifies according to the picture received Individual determines this, at least one the corresponding portrait having identified individual and basis that the group being at least partially based on and defining is associated The portrait of the unidentified individual that the digital picture received determines meets comparison threshold, and this determines the knowledge carrying out the unidentified individual of suggestion Not.

According to an aspect of the present invention, it is provided that the identification of the unidentified individual in a kind of digital picture advising receiving Method, the method is performed by least one computer, and at least one computer includes portrait and the number of the digital picture being associated Be connected according to storehouse or with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait Identifying that individual is associated, the method has comprised determining that at least one shown at least one digital picture in digital picture Identify that the corresponding portrait of individual meets comparison threshold with the portrait of the unidentified individual determined according to the digital picture received；According to To date of day metadata that the digital picture included corresponding to and receive is associated and and the digital picture of reception relevant At least one of determination in the digital picture that the metadata of the position in the preset distance thresholding of the location metadata of connection is associated Individual digital picture, it is proposed that unidentified individual is identified as at least one and has identified individual.

According to an aspect of the present invention, it is provided that the identification of the unidentified individual in a kind of digital picture advising receiving Method, the method is performed by least one computer, and at least one computer includes portrait and the number of the digital picture being associated Be connected according to storehouse or with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait Identifying that individual is associated, the method has comprised determining that at least one shown at least one digital picture in digital picture Identify that the corresponding portrait of individual meets comparison threshold with the portrait of the unidentified individual determined according to the digital picture received；At least The metadata that the digital picture being based partially on and receive is associated, determines that the digital picture of reception is associated with event；According to At least one digital picture of determination in the digital picture that event is associated, it is proposed that unidentified individual is identified as at least one Identify individual.

According to an aspect of the present invention, it is provided that the identification of the unidentified individual in a kind of digital picture advising receiving Method, the method is performed by least one computer, and at least one computer includes portrait and the number of the digital picture being associated Be connected according to storehouse or with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait Identifying that individual is associated, the method has comprised determining that at least one shown at least one digital picture in digital picture Identify that the corresponding portrait of individual meets comparison threshold with the portrait of the unidentified individual determined according to the digital picture received；From number At least one digital picture of determination in word image is extracted non-portrait visual information；According to the non-portrait visual information extracted This determines to meet comparison threshold with the non-portrait visual information from the digital picture received, it is proposed that unidentified individual identified Individual has been identified at least one.

In this, before at least one embodiment of the present invention is explained in detail, it will be appreciated that the present invention should at it The structure detail and the parts that are not limited in following description be given or illustrate in accompanying drawing by aspect are arranged.The present invention can have Other embodiments and can being practiced and carried out in various manners.In addition, it will be appreciated that word used herein and term For purposes of illustration, be not construed as restrictive.

Accompanying drawing explanation

Now it is only used as example and embodiment is described in reference to the drawings, in the accompanying drawings:

Fig. 1 illustrates the particular implementation of the system of the present invention of the social networking service merging the fixed direction allocation performing photo Example.

The system of diagram in the further pictorial image of Fig. 21, wherein user adds new digitized map from various equipment in time Picture.

Fig. 3 illustrate the present invention its on the one hand in for generating the face " label with the signature of known individual compared with Name " face recognition method.

Fig. 4 illustrates the face detection in the facial database in the storage device of link local computer system, eyes inspection Survey and the result of facial recognition.

Fig. 5 illustrates how peer-group can grow in time and evolve into the known individual list organized.

Fig. 6 illustrate correct can detect due to automatic face, eye detection and face identification step produce mistake dive In method.

Fig. 7 illustrates the automatic selectivity providing photo between the user in identical peer-group of the present invention and spreads System and method.

Fig. 8 illustrates so that can browse through the photo and graphical user circle of facial database managed by computer program The example embodiment in face.

Fig. 9 shows that the face-image of known individual is plus for the Boolean operator narrowed in the field of photo.

Figure 10 illustrates the optional advertisement display capabilities provided by GUI.

Figure 11 illustrates present invention type of face detection method based on texture in its one side.

Figure 12 illustrates the method for eye detection in an aspect of of the present present invention.

Figure 13 illustrates the face recognition method in an aspect of of the present present invention.

Figure 13 a illustrates the face recognition method in an aspect of of the present present invention.

The method of the eyes in Figure 14 A, 14B and 14C diagram isolation photo.

Figure 15 illustrates the example arrangement of the system of the present invention.

Figure 16 diagram is for allowing users to the interface of the identity of the face of appearance in confirmation image.

Figure 17 illustrates user and may be used for deleting the device of the false positive face detection in image.

Figure 18 illustrates user and may be used for reorientating corresponding to image for the purpose increasing detection algorithm accuracy In the device of eyes coordinates of detection of face.

Figure 19 illustrates the process of the face in hand labeled image.

Figure 20 illustrates the frame extracting video and the video scanning method that these frames perform face detection.

Figure 21 illustrates an aspect of of the present present invention, and wherein remote web browser or mobile device are able to access that agency service Device, to provide the connection of the function of the present invention.

Figure 22 illustrates the deletion of false positive face detection mistake.

Figure 23 illustrates an aspect of of the present present invention, in the image of the xy coordinate that its septum reset is positioned at the position of capture face, its Capture face is provided and in terms of size and rotation, regulates the image of face to produce portrait (actual size or thumbnail) Method.

Figure 24 diagram photo after being conditioned in terms of size and rotation replicates the image of face and is embedded into In advertisement information or in product photo.

Figure 25 illustrates use image and covers photo on the face of object to hide negative memory.

Figure 26 illustrates according to history order from the display of the face of photo capture.

Figure 27 illustrates the display of the face/portrait from photo capture and embeds it in game and exercise.

Figure 28 illustrates the establishment of family tree and is used as the index of photo.

Group photo is used as the index of photo by Figure 29 diagram.

Figure 30 illustrates the display of the delivery of the face signature of missing child and wanted criminal.

Figure 31 illustrates the coupling of face signature to find the individual seeming alike.

Figure 32 illustrates the workflow creating customization advertisement.

Figure 33 illustrates the workflow creating customed product.

Figure 34 illustrates the workflow hiding negative memory.

Figure 35 illustrates establishment customized games and the workflow of exercise.

Figure 36 illustrates the workflow creating family tree.

Figure 37 illustrates the workflow using photo as index.

Figure 38 illustrates the workflow finding missing individual.

Figure 39 illustrates the workflow finding similar appearance.

Figure 40-45 diagram is according to the work using metadata to enhancing facial recognition result of various aspects of the invention Stream.

Figure 46 diagram according to an aspect of the present invention show the graph of a relation of frequency identifying that individual occurs together.

Figure 47 diagram according to an aspect of the present invention with hierarchical lists form display photos.

Figure 48 illustrate represent the level shown in Figure 48 piece photo together.

The display according to an aspect of the present invention of Figure 49 diagram is for identifying the timeline of the facial pictures of individual.

Figure 50 diagram according to an aspect of the present invention by use cluster identify the face in photo.

Figure 51-53 diagram face according to an aspect of the present invention is grouped and cluster.

In the accompanying drawings, embodiments of the invention are illustrated by example.It should be expressly understood that specification and drawings only goes out In the purpose that explanation and help understand, and it is not intended to the restriction as the scope of the present invention.

Detailed description of the invention

General introduction

On the one hand the present invention provides a kind of at it and makes it possible to automatically distribute with to operate computer system on network many The networked computer framework of the image that individuality is relevant.

The present invention its on the other hand provide one be operable so that each individuality can be provided for herein Share the computer program that the networked computer architecture interface of the information including image connects.Computer program makes the individuality can Upload image, including the image of the description of the face with one or more individual.Computer program can perform face detection Technology is to detect one or more face in the picture, and this can cause the generation of one or more face signature, each face Sign corresponding to one of face.Then computer program can access data base, and wherein data base is individual with known by face signature The list link of people, each known individual is associated with one or more face signatures.Each detected face signature is permissible It is provided to the individuality that corresponding known individual is associated, or its septum reset signature is not associated with any known individual, should Information can be provided by individuality.The device confirming face signature with known person-to-person incidence relation can be provided to individuality.

The present invention provides a kind of face described in based on image to generate the new of face signature at its another aspect Grain husk method.Additionally provide for relevant at the one or more face signatures making the face signature identified to link to data base The device of error rate is reduced during connection.

The present invention makes it possible to the automatic selectivity distribution realizing describing the image of face at its other aspect.If The face detected in image with and the individual that is connected of networked computer architecture interface provided herein be associated, carry the most herein The computer program of confession can automatically transmit image to the computer of individual and present for individual.It should be noted that, term " shines Sheet " and " image " use used interchangeably herein.

The present invention is operable in the case of its another aspect provides a kind of networked computer framework provided in this article Novel advertising method.

Networked computer framework

On the one hand the present invention provides a kind of making it possible to automatically to distribute and the multiple single operation calculating on network at it The networked computer framework of the image that machine system is relevant.Fig. 1 illustrates the example embodiment of the networked computer framework of the present invention. Multiple individualities are each can be connected to the Internet (11) by the operable terminal to access the Internet (11).The Internet (11) numerical data making it possible to send and receive the equipment from Internet connection, the equipment of each Internet connection are connected Can be as presented below exercisable.

On the other hand the present invention provides one to be operable so that at it, and each individuality can be with networking meter provided herein The computer program that calculation machine architecture interface connects.Figure 15 illustrates the example arrangement of the system of the present invention.The user (13) of the present invention Can register to its computer system (15), download and computer program is installed.

On the one hand computer program can allow user (13) to invite at it and set up the pass of other users with the present invention System.Computer program at it on the other hand from the aggregate requests ad of advertisement network service device (23), advertisement network service device (23) Deliver ad to computer program to show to user (13) for the various piece at graphic user interface (GUI).Computer program Can be to other users or to third party website transmission photo and the metadata relevant to these photos, unit at its another aspect Data can be with being similar to stenography (25) (such as FLICKR^TMAnd FACEBOOK^TM) mode be stored in EXIF or similar literary composition In part stem or be embedded in inside jpg or similar image file format.These third party websites (25) can be answered by issuing Encourage the use of its website with routine interface (API), with realize from client software or other network be applied to it The connection of service.

The composition making it possible to realize the computer program of the present invention may include that

The process engine that can run on an operating system as context process.It can scan user uploaded or Copy the new numeral of regulation file locally and/or remotely or in cloud computer (photo of these files is monitored) to Photo, or it can detect new photo automatically when removable media (such as storage card) is inserted into computer.In inspection When measuring new photo, face detection, eye detection and face identification step can be performed, as described more fully below. Result can be stored in data base, all data bases as described more fully below.Can be far it should be noted that, process engine Performing on journey computer or cloud service computer, wherein the computer program of the present invention is provided as service, and it is permissible Use software i.e. service model.

Navigation Picture can be provided a user with, by identifying that known individual trains oneself of application, editor and correction engine Move result, create based on search criteria and amendment photograph album, interpolation peer group members and the energy to third party website transmission photograph album The GUI of power, each such action is described more fully below.

May be located on the computer of user or data base's (such as SQL data in remote computer or cloud computer Storehouse), data base can comprise face detection described below, eye detection and the result of face identification step.Data base also may be used To comprise the relation between metadata and known individual and the face-image being associated of photo and individual.

GUI, process engine and data base can be arranged on one or more equipment.Figure 21 illustrates a side of the present invention Face, wherein makes remote web browser or mobile device (83) be able to access that proxy server (81), to provide the present invention The connection of function.GUI can be arranged in mobile device (83) (such as PDA or mobile phone), and perform face detection and know , and to the remote engine run on the desktop computer or notebook of website, server or user do not index to and fro and transmit Information.In such embodiment, PDA or mobile phone can be provided with for perform facial recognition, picture browsing instrument with And for uploading the instrument of the image using the camera capture being incorporated on equipment.Uploading can be according to retouching more fully below The general upload procedure stated performs.

The step being used computer program to perform by user may include that

Regulation is for monitoring the file of new digital photos.

The name and the metadata that are associated by the face identified with find in digital photos train application.

Correct the mistake that application produces；Both false positive and false negative.

Creating photograph album (set of photo) by regulation search criteria, search criteria includes scope of data, known individual Boolean combination (selecting via face), EXIF label and universal tag.Alternatively, user can be by each photo or photo Group be dragged and dropped into photograph album.

Once creating photograph album, user then can specify that various output intent option, including:

Third party website, such as Flickr^TMAnd Facebook^TM。

Format slide, such as Microsoft^TM Powerpoint^TMFile.

Document format, such as Adobe^TM PDF^TMFile.

Invite other application users to add peer-group, it is stipulated that the option relevant with these relations in peer-group, and connect Inviting of peer-group is added by for another user.

Networked computer framework can also include the one or more servers realizing technique described herein.Such as, originally Face detection and recognition methods that literary composition provides can be realized by server.Can be arranged on one or more server system Server can include server program, and server program realizes the function of server, including in the following or many :

Date of there is the list realizing e-mail address, being associated and the data base's of the association of other ad datas Registrar.Registrar can present interface (such as webpage) to user and be used for collecting log-on data and then by this One data write into Databasce.Then the device of user installation computer program described above can be given, such as by arranging There is the URL for downloading client software.

Ad match server can accept the CIPHERING REQUEST of the demographic information comprising user.This server is all right Accept the ad from gray multiple forms (such as .JPG .GIF .SWF etc.).For each ad submitted to application, also The target demographic information of ad can be recorded.Can be based on target demographic data by the stock of ad request with ad Join.The pointer (unique ID#'s) for making ad and requesting client Software match can be returned.

Ad delivery server can accept to cover the CIPHERING REQUEST of the pointer of ad.Application can be found and be quoted by pointer Ad and this ad is returned to requesting client software.

Peer-group

Referring again to Fig. 1, the first user (13) of the present invention can be from website downloading computer program or be provided use In any other seating means installing computer program.First user (13) can install computer program, and it can be in fortune Row operating system (such as MICROSOFT^TM WINDOWS^TM、APPLE^TMOr LINUX^TMOperating system) notebook or desk-top calculating Realize the system of the present invention in machine system (15), or program can be at browser (such as Microsoft^TM Internet Explorer^TM、Mozilla^TM Firefox^TM、Google^TM Chrome^TM、Apple^TM Safari^TMOr mobile browser) interior fortune Row and therefore program can perform in browser and in backend network servers.

First user (13) can be by present invention provide that its e-mail address defines the friend in peer-group (17).Computer program can send invitation email subsequently, and its request friend (17) is also downloaded or realizes computer journey The installation of sequence.After mounting, the corresponding computer program of friend (17) can present addition by first user to friend (17) (13) the pending request of the peer-group started.May require that it can is that friend (17) approval of the second user (17) is to equity now The connection of group.Once ratify, first user (13) and the second user (17) computer program run now can be according to this The image sharing method that literary composition describes exchanges photo and the metadata relevant with these photos and known individual.

Peer-group can be reciprocity by inviting more personnel (19,21) to add by first user (13) or the second user (17) Group extends.Second user (17) can also create the new peer-group that first user (13) is not its part, and individually Extend this peer-group." a lot of to a lot " relation can be there is between personnel and peer-group.Therefore, first user (13) can be The member of multiple peer-group and the second user (17) can be the members of multiple peer-group.This makes it possible to form based on equity Member's relation shares photo with other users easily.

As described more fully below, on the one hand the present invention achieves known list at it.Known individual is permissible It is added to one or more peer-group alternatively, as described above.

Peer-group can realize sharing of the photo metadata relevant with photo and known individual.GUI can realize using Family to the establishment of peer-group, revise and delete.GUI can also realize the face-image of known individual or thumbnail to existing equity Association in group, such as, pass through to allow users to pull face-image or breviary on the region at interface representing peer-group Figure, such as territory or icon.User can distribute the representative face image being associated with its name, and this face-image is total to Enjoy and distribute to occur in all of peer-group contacts list.

Computer program can generate when adding friend to peer-group every time and invite.As an alternative, user can be subject to Inviter be not known list part in the case of manually add the metadata by inviter to peer-group.Can be via Email is invited to being sent to individuality by inviter.For installing the computer of the present invention in its computer system The individuality of program, Email, once be received, can be pointed out and be accepted the invitation by inviter.Alternatively, it is desirable to individual at meter In calculation machine program, input code is to accept the invitation.

If friend not yet installs computer program or the cloud service reality less than the present invention in its computer system Execute in mode and register, then Email is once received and can include that the link of the installation and activation downloading or realize service is posted Storage, and the instruction for installing computer program on the computer systems can be provided.Computer program is successfully being installed Afterwards, the present invention can be presented to new user, and new user step as described above can receive addition peer-group Invitation.

Once accepted the invitation by inviter, then can will be added to peer-group by inviter.Can be at networked computer frame On structure, distribution is updated so that and can update corresponding peer-group in the computer program of each individual being associated with peer-group Information.

According to face detection technique described below, peer-group can realize automatic on networked computer framework of information Selectivity spreads.The most also distribution technology it is described more fully with.

Face detection

On the one hand the present invention provides the novelty side for generating face signature based on the face described in the picture at it Method.Fig. 3 is shown in the face recognition method according to the present invention of an aspect of of the present present invention.User (13) can be to operable with reality The computer system (15) of the execution of existing computer program provides image.Computer program can via add new file time or The operating system notice that person generates when revising existing file monitors the file being associated with computer system (15).Sending out During existing (28) new images, new images can be ranked (29) for process.

The face in (31) photo, above-mentioned multiple skills can be positioned by using any one technology in multiple technology Art includes: generate face coordinate, and face coordinate can be defined on the bounding box of facial surroundings；Determine that eyes are sat based on face coordinate Mark；And based on face coordinate and eyes coordinates and by use face signature technology for face create face signature (33).Face signature technology can be principal component analysis (PCA), and it is known to a person skilled in the art.Face can be signed Compared with known face signature (34), and can automatically and optionally spread photo to other users (36).Under Face provides the further detail below of the aspects of the invention.

In Fig. 4 facial database of being linked in the storage device of local computer system of diagram or as an alternative can be Face detection, eye detection and the knot of facial recognition techniques in facial database in the storage device of cloud computer system Really.Result can be the coordinate of the object being associated.In the case of face detects, coordinate can define by original photo On upper left, upper right, sit down and bottom right pixel position is to define the profile of face (37), it is referred to as bounding box.In eye detection In the case of, coordinate can represent the pupil position (35) of left eye and right eye.In the case of facial recognition, result can be face Portion's signature (42).

The graphic user interface (GUI) of the present invention can show face (37) and eyes (39) on each image (35). As described more fully below, on the one hand the present invention provides the list of known individual at it.If corresponding to detected face Face signature (42) in portion is associated with the individual in known list, then GUI can use on image or around figure Pictograph number represents such incidence relation to user.Otherwise, GUI can use on image or around another graphical symbol to User represents and there is not such incidence relation.In the example that Fig. 4 describes, it is known that face can be known with check mark (41) Not, unknown face can identify with symbol " X " (43).

Originally, system may be unaware that all faces, until user " trains " identification of the present invention face.Training method is permissible Relate to user.By click on around face optical bounds frame on Anywhere and individual name (or represent individual Icon) on pull face, user can use mouse or other input equipments by facial recognition for belonging to via the GUI of the present invention Unique individual.As an alternative, user can pull the icon representing this individual in target face.In another alternative, Yong Huke With click on optical bounds frame Anywhere and select the function for identifying not previously known face, this is so that use Family can input the data relevant to this individual, and such as name, e-mail address and other details, it may be collectively referred to as correspondence Metadata in individual.This training step can perform once for each known individual.The signature created for face is right After can realize the comparison of all unknown face signature in facial database and the individual identified.It is described more fully below For the method compared and for face detection, eye detection and both methods of facial recognition.

The present invention is most possible with single by unknown face sequence is allowed users to identification at its other aspect The group of the detected face that individuality is associated is to facilitate the optimum training stage.It is, for example possible to use sign class based on face Like face cluster algorithm together.Similarity can be based on some aspect of its face signature, even if face does not has and face Individuality in data base is associated.Therefore, user can by face clustering recognition for belong to specific known individual and thus Most preferably perform training method described above.

The present invention facilitates optimal instruction at its other aspect by the more than one face signature being associated with individual Practice, thus by supporting multiple postures of individual and improving accuracy and solve owing to age, glasses or face become The change of the individual human face changing (such as being caused) by beard or beard and produce.

The facial incidence relation with known individual

Figure 16 diagram is for allowing users to confirm the interface of the identity of face occurred in the picture.Face in image Portion signature can with and known individual face signature similarity thresholding in.In such a case, it is possible to detected It is associated between facial and known individual.The method that a kind of comparison facial signature is described more fully below.

If there is incidence relation between facial and known individual, then can create face signature in data base with known Person-to-person other incidence relation.Can be by each face being previously identified for each known individual and by system The each new face of reason compares.Check relevant to specific known individual facial time, can show and be generated by the present invention Any suspicious coupling, and user can be asked to confirm, and coupling is correct.

Development over time, when the number of identified face increases, the most facial and correct the whole of individual mates Accuracy can increase, because being usually present a lot of different views of individual, each view has new face.According to provided herein is The method of the facial signature of comparison, therefore false-positive number generally increases over time.

Fig. 5 illustrates peer-group how when the list of known individual grows can increase over time and develop.Known The list of people (101) increases when user is with cooperation of the present invention, because user can continue so that the unknown is facial and known individual It is associated.

Spread

On the other hand the present invention provides one to be operable so that at it, and each individuality can be with networking meter provided herein Calculation machine architecture interface connects the computer program for shared image.Fig. 2 also illustrates the system of the present invention.User (13) is permissible Capture digital picture and periodically it copied to computer system from one or more vision facilitiess storage system (27) (15).User (13) can be with allocating computer system by using GUI as described above to computer system import folders Title carry out the new images of the particular file folder in monitoring computer system (15).

According to the novel method for facial recognition provided by the present invention, on the one hand the present invention achieves in equity at it The automatic selectivity of the user being depicted in the picture to its face between group spreads.This is described more fully below.

Inspection optimization

On the one hand the present invention provides a kind of for optimizing the new of face detection according to other aspects of the invention at it Grain husk technology.Fig. 6 illustrate correct be likely to be due to automatic face detection, eye detection and face identification step produce mistake potential Method.Present invention use relates to user and corrects these the wrongest novel manner by GUI.

Figure 17 illustrates user and may be used for deleting the means of the false positive face detection in image.At face detection and eyes During detection-phase, it is understood that there may be false positive mistake.In face detection technique, these mistakes can determine that face exists but real Occur when not having face on border on digital photo images.In order to correct these mistakes, GUI is so that user can be by making The upper all-moving surface in region (it can represent) is being deleted by paper basket or other representative icon with mouse or other input equipments Portion (103) (bounding box on its septum reset image identifies), by press while highlighted face lower keyboard deletion key, Or by selecting the menu option (105) of the deletion corresponding to face to delete false positive mistake.

Figure 18 illustrates user and may be used for reorientating in the picture for the purpose of the accuracy increasing detection algorithm Means corresponding to the detected eyes coordinates of face.In face detection with during the eye detection stage, eye position coordinate (107) there may be error.The method of the present invention may determine that eye pupil position and visually shows eye on image Eyeball coordinate (107), but the coordinate generated is not likely to be fabulous, because they are probably approximation (example in some cases Face as with sunglasses).GUI is so that user can be such as by using mouse or other input equipments to move expression The icon (109) of eye position manually reorientates eyes coordinates (107).In this way, can increase the present invention's Accuracy and performance, because eyes coordinates (107) is generally used for generating the face signature of face.The change of eyes coordinates (107) Therefore can generate the change of face signature, the change of face signature can be relevant to other known faces by signature with appreciable impact Connection.Except the use of eyes coordinates, the method for the present invention can use edge detecting technology to make face template and quilt in image The face alignment of detection.Alignment methods provides and confirms that face is to reduce false-positive other method.This technology can be by making Facial top/bottom and left/right are directed at and generate the digital measurement of trust and perform with alignment template.

Figure 22 illustrates the detection of false positive face detection error.During the facial recognition stage, it is understood that there may be with face and The false positive mistake that the incorrect association of known individual is relevant.If face signature is mated and it by the present invention with known individual It not correct relation, be then likely to occur false positive mistake.GUI is so that user can be correct known individual (113) Face-image (thumbnail) on pull face-image (111) (thumbnail).Therefore the present invention can revise the chain in data base Road is to illustrate facial and known person-to-person new relation.Old relation can also be deleted.

The error of another kind is false negative.Can also have the two kinds of situations being classified as error of the second kind, it is in figure 6 It is illustrated:

1) if system detects face in the case of there is face (47) actually the most in the picture, then GUI can Enable a user to use mouse or other input equipment to pull the bounding box at facial surroundings, and then by using table Show that the icon of eye position places two eye positions.Then system can use the information being manually entered to generate face label Name and perform face recognition method provided herein.As an alternative, system is so that user can be manual with face by label It is associated without regard to face detection or identification process.It is right that this hand labeled method can be used for other in labelling image As with animal (such as house pet), the human face's detection method employed in it is not resulted in the automatic inspection of above-mentioned object or animal Survey.

2) also likely to be present error of the second kind, but wherein system detects that face face signature is not signed with known face , even if there is the face of known individual in coupling.This can for face signature with for any other of this unique individual Face signature between difference the most different in the case of occur.In this case, system can not automatically detect relation and Face can keep unknown.In this case, GUI is so that user can pull on the face-image of known individual (49) face-image.By carrying out this operation, face-image can be linked to the known individual in data base by system.Pass through From this help of user, system has another face signature now, and it will be used for face signature with new and the unknown Future compares.This can improve the accuracy of the present invention.Figure 19 also illustrates the process of the face in hand labeled image.

The automatic selectivity of image spreads

The system and method that Fig. 7 diagram spreads for providing the automatic selectivity of photo between the user of the present invention.When When identifying known individual facial of the member as peer-group in photo (53), photo (53) can be ranked with Transmitted to peer group members by the Internet (11) according to secured fashion.Such as, first user (13) can be to its department of computer science System (15) uploads the photo of the face of the known individual comprised as the second user (17), and the second user (17) is also equity composition Member.In this case, when system determines facial match, photo (53) can be ranked for transmission.In transmission Before, photo (53) can be reduced to less version and metadata can be included in digital photo files.Size Corresponding reduction can optimize the use of bandwidth.In cloud embodiment, can physically transmit photo or can be by file institute Have the right to expand to include peer group members.Or can not transmit or replicate file.

When second user (17) accesses the computer program in its computer system (15) next time, can receive illustrate former The image of the reduction of beginning photo and the confirmation request of metadata being associated, and notify to go to via e-mail the Two users, the adnexa asked during wherein thumbnail is also included in email body or as delivery confirmation.Can point out Second user (17) they whether want the copy of photo (55) in its computer system (51).If the second user (17) is certainly Ground response, then system can pass through the Internet from the computer system (15) of first user to the computer system of the second user (55) the whole image metadata together with photo (55) and the known face from photo and signature are copied.

Another novel aspect of the present invention uses existing social networking service and network photo storage site Photo is shared with peer group members.The present invention can be to objective network or service transmission single photo based on cloud or photo Group.The photo of transmission can be included the metadata relevant with the personnel in photo from the present invention.Such as, social networks Website FACEBOOK^TMOffer instrument, this instrument upload pictures, shared photo and hand labeled photo are to represent which friend is being shone In sheet.Labeling process is manual and the most time-consuming.The present invention can make such upload procedure automatization and eliminate for The needs of hand labeled photo thus provide obvious benefit for social network user.The present invention can be suitable for other societies equally Hand over network service and network photo storage site.As described above, user can also by the Internet directly from It is associated with the mobile device upload pictures of camera.

The another novel aspect of the present invention is, can load the initial column of known individual from the social network account of user Table, with the most time-consuming when configuring system provided by the present invention.

Fig. 8 illustrates so that can browse through the example of the GUI of the photo by computer system management and facial database Embodiment.In this example, user interface is shown as having the optional banner advertisements (57) can sold to advertiser. User can select photo (123) from photograph album (125), and GUI can show known corresponding to find in photo (123) The thumbnail (121) of individual (117).Check box (119) can be used for representing depositing of known individual (117) in photo (123) ?.

The example described in Fig. 9 illustrates that the face-image of known individual is (all plus the check box for applying boolean search As selected to AND, OR and NOT that the name of known individual is associated or the metadata relevant with image).The novelty of the present invention Feature is the thumbnail of the face by allowing users to click on known individual (59) and opens for by check box Each face apply boolean operation (61) come with visual manner select photo ability.On the one hand this of GUI make it possible to Compare total photo and the various search criterias of facial database applying by combination and wave filter creates photograph album.In the present invention Further embodiment in, user can to unique individual's distribution diagram picture (real, manually generate or computer generate ), and this replacing image cover find wherein in the part of each image of this unique individual.

The search criteria provided by computer program may include that

File selects (65), the folder location of expression photograph image on computer memory device or filename Claim；

Known individual (67), it is provided that the selection of the boolean operation (AND, OR or NOT) being associated with the face of known individual；

Date range (69), it is provided that correspond to the choosing of the date range of the photograph image being included in specific photograph album Select；

EXIF data, based on being generally attached to, by camera, the letter that the standard photographs of photo is correlated with during photo acquirement process Breath provides selection approach.

The alternative method selecting the photo for photograph album can provide a user with use GUI dragging nominally in photograph album (63) Put each photo or photo group or the ability that photo is associated with photograph album (63).

Photograph album (63) can be the one group of photo being saved under the title of photograph album as logic entity.User can be via GUI specifies that it wants to various target file type or position (71) transmission photograph album, including lantern slide, MICROSOFT^TM POWERPOINT^TMOr other present computer program, ADOBE^TM PDF^TMOr other document files, network sharing site (such as FLICKR^TMOr FACEBOOK^TM) or third party's print service.

Advertising method

The present invention provides the novelty cooperated with networked computer framework provided herein at its optional non-limiting aspect Advertising method.

Figure 10 illustrates the optional advertisement display capabilities provided by GUI.This is the novel feature of the present invention, and its offer is used for Based on user's demographic information targeting advertising simultaneously and the safety method safeguarding privacy of user.GUI can be to new user (13) prompting bottom line can include the demographic information of sex, age and position data.This information can be local It is stored in the computer system (15) running computer program.Can periodically send to network server (73) Return the request of the list of ad pointer.Request can comprise the encrypted transmission of the Demographic data of user.Request can also The certificate sent by registrar (77) is used to come signed.This can verify the verity of request in rear step.Based on The server (73) of network can carry out, based on its particular demographic information, with request user (13), ad is mated this Process, ad is associated with target demographic information.Can be to the row of computer system (15) return pointer of request user Table (i.e. to the reference of the ad being present on different ad server) and pass it to computer program.Computer program is then Another request downloading the ad quoted by pointer can be sent to different network servers (75).Network clothes Business device (37) then can abandon Demographic data to protect the privacy of each user.

By separate from the server (75) of actual delivery ad perform ad matching process network server (73, 77), rather than by the individual Demographic data network server (73,77) upper storage user, have with user The personal information closed is the safest.Ad delivery server (75) can store the letter relevant with the ad for purpose of presenting the bill Breath, can not include personal information in these data.This is in a secure manner to any network for population in use statistical data Browser or the novel embodiment of software program service ad.

Other embodiment

The non-limiting optional aspect of another exemplary in the present invention, another ability of the present invention is so that computer journey Sequence can receive digital facial image or signature from central authorities' registrar server.Such as, individual (such as missing child or wanted circular are found Violate) tissue can put up individuality face data.Select share its facial database these users can download data with Realize automatically comparing of face data and its facial database.If sent out between target individual and the known individual of specific user Now mate, then can alert tissue.This is so that tissue can determine the title of individuality, because the title of individuality can be listed in In one or more in known the list of user.

The another embodiment that the present invention provides has similar facial characteristics with itself so that individuality can find Other people.Such application such as may be used for individual and finds its twins.In the present embodiment, user can submit to and include The photo of its face-image, the present invention can generate face signature according to it.Then can by facial characteristics with previously uploaded The face signature of other individualities compares.Based on predetermined similarity thresholding, can be to one of individuality of its facial match or complete Similar face is reported in portion.System according to the present embodiment can provide the means contacted for user's approval with other people.Coupling Individuality it has been specified that can select to send message to another people in the case of its approval to such contact.Can use Similar embodiment is as a part for appointment service based on form fit personnel.

Face detection, eye detection and facial recognition

The present invention provides a kind of face described in based on image to generate the novelty of face signature at its another aspect Method.By use, face signature can include that the technology of face detection, eye detection and face identification step generates.

Face detection

On the one hand the present invention provides a kind of for using detection algorithm based on texture as detecting for face at it The method of basic skills.One example of face detection algorithm based on texture is known as OPENCV^TMThe open-source of routine Storehouse.

Face detection algorithm based on texture can have for specific facial pose (frontal faces such as risen and fallen) Low True Positive Rate.This is likely due to the face, front in the basic position being different from standard digital photo the change found Textured pattern is trained on portion's image.It practice, for training the textured pattern of face detection algorithm and applying this calculation thereon Mismatch between the posture of the type in the target picture of method will cause higher percentage of errors.Currently face in face detects Two the well-known challenges faced reduce False negative error in the case of being included in the frontal faces of fluctuating and are not increasing False positive mistake is reduced while False negative error.

By application, the present invention includes that three are come in order to the innovative techniques of the step of the accuracy improving face-detection Strengthen face detection based on texture.At first step, it is provided that known skin tone is tested at face detector based on texture On novel application to improve the accuracy of detector.At second step, by the size in the face object region of detection is subtracted Little provide the novel method for reducing false positive face object for predefined size.At third step, it is provided that novel face Orientation compensation method.

The type of face detection method based on texture of the present invention that Figure 11 is shown in an aspect of of the present present invention.

In a particular implementation of type of face detection method based on texture, in first step, face based on texture Portion's detector (131) originally can be arranged for high target True Positive Rate, and it has corresponding high false positive rate.Based on stricture of vagina The face detector of reason can use whole photograph image to run as input.Result from this operation can provide photograph The list of the potential face object in sheet.Can be by any candidate face region record of detecting in data base.For coloured silk Color photo, can perform Face Detection test (133) to reduce false positive rate to potential face object.The test of this colour of skin is permissible Relatively comprise the ratio in the face object region of the colour of skin and the whole region of object.If ratio is less than predetermined threshold, then may be used To skip potential face object (135).

At second step, the face object region detected from first step can be reduced to predefined size (such as 44 × 44 pixel).On region after these reduce, can again run face detector (137) based on texture. The step for purpose be reduce false positive face object.By transporting on the input area of small size (such as 44 × 44 pixel) OK, the false positive mistake of the incorrect textured pattern from non-face can be reduced, true positives texture maps can be retained simultaneously Case.This can cause the reduction of the facial false positive rate detected to retain True Positive Rate simultaneously.(143) can be accepted at second step In be considered face face object as true face portion.Face object not over this second step can be passed to Three steps.

At third step, it is provided that face orientation compensation method.The second step being described above is (wherein at image more Face detection is performed on zonule), True Positive Rate can be reduced in the case of the face risen and fallen.Owing to applying at second step Size reduce, the textured pattern in the face of fluctuating can be considered as non-face by face detector.In order at the face risen and fallen In the case of retain True Positive Rate, can perform face orientation compensate (139) together with face detect.In this process, by office Portion's image-region rotates with incremental angle, and each angle example can pass through face detector.One example is implemented Mode can start image rotation 2 degree at-20 degree with vertical axes and repeat to be rotated up spending away from vertical axes+20.If Postrotational facial zone is identified and there is continuous incremental angle in instances, then may determine that (141) topography Region is real life facial.

These amendments to face detection algorithm based on texture can be substantially reduced false positive error rate.

Eye detection

The eye detection method that Figure 12 is shown in an aspect of of the present present invention.Eye detection method application innovative techniques is to change It is apt to detect the accuracy of pupil position in the facial zone of detection.First step can be to be reduced to very by whole facial zone Likely comprise the smaller area (eye pattern) (145) of actual pupil/eye position.Formula can be used to cut eye from facial zone Graph region.Such as, the formula being used for cutting eye pattern region can be the .23w on the left of removing, the .23w on right side, the .55h of bottom .30h with top；Wherein w is the width of facial zone, and h is the height of facial zone.Figure 14 A diagram cuts from facial zone The specific embodiment of the method in eye pattern region.

If it addition, the third step in type of face detection method detects face on special angle example, then can be to Eye pattern area applications (147) anglec of rotation is to realize the selection more accurately of eye position.

Figure 14 B diagram can change for the formula cutting eye pattern region to guarantee that eye pattern region is sufficiently large satisfied to guarantee Pupil detection result.

Once cut eye pattern region, then eye pattern region can be sized to fixed size (such as 80x80 pixel) again (149) allow to apply fixed form color changing method to determine candidate's eye position.This method can be based on relatively The dark intensity of pupil in the peripheral region in eye pattern region.

The central authorities of color intensity image can be set in intensity map zero/black (151) be typically due to instead to remove Penetrate the potential pupil candidate leaving the light of lens and eyeglass frames and produce.Figure 14 C illustrates this process.

Color modified-image can be obtained from color intensity image, and then can be passed through thresholding filtering Device so that the only small percentage of eye areas is white (such as can apply the thresholding of 2%) (153).Then, can apply " best fit " method is to select the eye position (left side and right side) with the highest color change intensity.Can be to the present invention's Face recognition method transmission (155) optimal candidate pupil position coordinate.

Facial recognition

Figure 13 diagram is according to the face recognition method of various aspects of the invention.Facial recognition generally can pass through pretreatment (157), projection (159), distance calculate (163) and polymerization (167) performs.Face recognition method can use principal component analysis (PCA).PCA is known method, and it is used in the analysis of form of ownership energetically, because it is from chaotic data set Extract the simple nonparametric technique of relevant information.

The present invention can use PCA by novel manner, which overcomes about the high sensitivity to theme illumination and posture Traditional problem, it is assumed that individual's photo can have little postural change, because most theme may tend at camera straight Connect and check.The present invention can exploit the fact that the photograph that can obtain describing unique individual under various postures and illumination condition Sheet set.The invention provides aggregate target the unknown face and and each associated plurality of face of known individual between comparison Method.

Can be first to image application pretreatment (157).The input being provided as the facial recognition stage can be used Facial zone and eyes coordinates facial zone is normalized.Normalization process can include by facial zone translate, Rotate and be scaled to predetermined template size.Normalization process can use eyes coordinates as a reference point so that regulation facial area Area image is to be placed on eyes coordinates in specific image pixel.Then can be to the fixed blind process of image application standard (facial zone is limited to the region of elliptical internal potentially) is to filter non-face part.Patterning process can be used (relate in the intensity of the pixel in facial zone run two-dimensional linear matching) guarantee image pixel intensities on image in space Upper uniform.Finally, histogram image equilibrium (image processing method well known by persons skilled in the art can be performed in gray level territory Method, so that regulating the contrast of image by the rectangular histogram of image).

Then can be to image application projection (159).Pixel obtained by facial zone after normalization can be led to Cross the formula of Based PC A to produce the PCA vector being used as face signature by the present invention.PCA vector can include due to from typically The projection picture that the main constituent that the big collection of image is extracted obtains.

The face signature created according to this method can be that the actual numbers array with given dimension (is vowed from PCA Amount).Although the accurate dimension of vector space can be adaptively determined, wherein its maximum is set to capture such as input picture The value of the 95% of energy, but the default value used can be the dimension in the range of 50 to 100.

Finally, circulation (161) can be applied so that unknown facial and known individual is mated.Linear or non-thread can be used Property sorted logic by each face signature (being expressed as digital array) in number with any other face signature compared with really Spacing distance values (163).For example, it is possible to approach two signatures by calculating normalized inner product distance.

In order to relatively more facial with and all faces of being associated of known individual, can carry out all of single one to one than Relatively, and then can use all results in the next step, or can use by compare determine with certain thresholding (165) set of optimal coupling.Thresholding selects to may be selected so that when keeping the comparison of facial and non-matching individual The average half obtaining distance.

Some form of polymerization can be used to combine the set of the distance value obtained in previously step to determine face With known person-to-person distance.This polymerization can be the geometric average of computed range value (169).Geometric average can be class It is similar to the technology that is averaging of arithmetic average, but it can want average N number of numeral and then by the N of product by being multiplied by Individual root calculates as expectation meansigma methods.Facial with each known person-to-person closest match can by calculate face with This polymerization each known person-to-person in data base apart from (167) and selects minimum range and is found.

Finally, the matching distance that can will be closest to compared with thresholding that is static or that be dynamically determined to reduce false positive Matching rate.If using the thresholding dynamically selected, then this thresholding can be determined in the following manner: assumes first that and is comparing Face and the polymerization distance value normal distribution of acquisition during the non-matching individual with N number of face being associated (every in N number of Individual probable value), and then use inverse accumulation standard distribution function to calculate thresholding, which ensure that the fixing maximum number of unknown face Mesh or fixed ratio the most phonily mate known individual.This thresholding can become to another person from a people Change, because the number N change of the face being associated with each individual.The advantage that this dynamic threshold calculates includes: fixing maximum Number (or ratio) can keep the least and maintain what the true positives of user mate can accept limiting false positive coupling simultaneously Level.

Figure 13 a also illustrates the face recognition method according to various aspects of the invention.Shown face recognition method can be with base Normalization face is carried out in eyes coordinates.Each face can be defined by the bounding box determined by face recognition method.Permissible Extract the pixel in bounding box to produce the image of face.Face-image can be recorded data base and make its with from which The respective sources image extracting face-image is associated.Then after can calculating normalization face PCA vector and deposited Storage is in data base.Can be by PCA vector compared with the known face in data base.If it finds a match, then can be by face Portion's pointer stores in memory, in such as temporary memory.If all known face in passing sequentially through data base it The most do not find coupling, then face recognition method determines whether to find in the search face.Without finding face, then terminate Facial recognition.If it find that facial, then face recognition method may determine that in the case of finding more than one facial match Optimal coupling in memorizer.Face recognition method determines whether optimal coupling exceedes nearness thresholding, as described above. If it exceeds nearness thresholding, then will most preferably mate write into Databasce.Without exceeding nearness thresholding, then terminate face knowledge Not.

The present invention facilitates optimal instruction at its other aspect by the more than one face signature being associated with individual Practice, thus improve accuracy.The present invention is so that user can select the representative subset of known face for given individual And thus cover the subset selected by the present invention.This subset then can be by identifying that engine is for determining proposed coupling also And ignore other known faces of this individual.This may useful example can be the feelings that wherein there are individual multiple postures Condition.It also solves the individual face produced due to aging, eyes or the change (such as being caused) of face by beard or beard The change in portion.By select more nearest photos as representative subset, can increase for following photo successfully from The dynamic probability selected.Increase owing to linking to the number of the face signature of the known individual in local facial database, the present invention Accuracy when automatically detecting the known individual in processed following photo can increase.

The present invention can be learnt by the immediate coupling determining the face being previously identified from the group of known individual Face.It is an advantage of the current invention that the number of the face signature as the known individual linked in local facial database increases Time, the present invention accuracy when automatically detecting the known individual in processed following photo can increase.

Video scanning

Figure 20 illustrates the frame extracting video and the video scanning method that these frames perform face detection.Number can be set Mesh N (wherein N is adjustable) so that can be at every N number of frame (181) scan video of video as single photograph image (183), previously mentioned technology (185) wherein can be applied to detect and identify facial and known individual.Then can basis Video is spread by technology provided herein.

The optional non-limiting example that the present invention provides makes individuality to find, and to have similar face with itself special Other people levied.Such application such as may be used for individual and finds its twins.In the present embodiment, user can submit to Including the photo of its face-image, the present invention can generate face signature according to this image.Then can be by face signature with first Before uploaded other people face signature compare.Based on predetermined similarity thresholding, can to facial match One of body or all one or more similar faces of report.System according to this embodiment can provide a user with to be ratified and it The means of other people contact.The individuality of coupling can select to another people defining in the case of the method for such contact Send message.A similar embodiment part as appointment service based on the long individual that matches can be disposed.

Face and portrait extract and create

Thumbnail that Figure 23 is shown in an aspect of of the present present invention or the establishment of portrait.If face is positioned in image 23a, The method that then the XY coordinate of the position that an aspect of of the present present invention captures the face shown in Figure 23 a captures face with offer, and In size with rotate the images of two aspects regulation face to create portrait, it can be actual size or thumbnail.Such as figure In 23 illustrated, coordinate can be defined in original photo the face with upper left, upper right, lower-left and bottom right pixel position Profile, it is referred to as bounding box.XY coordinate record is in the data base that such as Figure 11 illustrates.

As illustrated in Figure 23, face is positioned at as shown in image 23a and sets up each determined by XY coordinate The corresponding bounding box in portion and being recorded in as in the data base of Figure 11 diagram.As shown in image 23b and 23c of Figure 23, do The XY coordinate going out bounding box can be with scalable to make bounding box, wound subsequently in the greater or lesser region of facial surroundings Build.By using the eye position that represents such as Figure 18 reference 107 and 109 and identification, face can be rotated so that face Portion becomes vertical as shown in the image 23d of Figure 23.

The record record of portrait/thumbnail image 23d can be used for making of future in the data base that such as Figure 13 illustrates With.

According at least one illustrative embodiments of the present invention, at least one terminal, server or other meters Calculation equipment can detect at least one face at least one digital picture.Computer may determine that and store at least one number The area coordinate of the position of at least one the detected face in word image.Computer can be at least one detected face Portion applies at least one conversion to create at least one portrait of at least one detected face.Computer can rotate at least One portrait be illustrated in vertical orientation at least up at least one portrait and be illustrated at least one portrait to A few facial pair of eyes is positioned in the horizontal plane.Then computer can store at least one portrait rotated. Each detected face can correspond to individual.Can prior at least one convert should be for making and tested from including The digital picture of the face surveyed extracts copy.

So that the identification of individual with at one or more computers storage or link to one or more calculating Each stored facial pictures in the data base of machine is associated.

If digital picture illustrates that more than one individual, the most one or more computers can be at least partially based on and be included Corresponding identified individual in digital picture makes the person-to-person relation being identified be associated.

Advertisement and personalization of product

Figure 24 diagram be incorporated in size and the regulation of rotation aspect after portrait or thumbnail and embed it in advertisement and disappear In breath or product or the marketing material of any other type or image.Can regulate and be also illustrated as image 23d's in fig 23 The size of thumbnail or portrait 24a is to mate the size requirements of advertisement 24b.Such as, computer can store at least one Portrait applies at least one additional transformation for the portrait size requirements of coupling selling images.

Figure 32 illustrates the workflow of advertisement creation.As shown in figure 32, when wanting personalized advertisement, an aspect of of the present present invention will From the advertisement of ad server data storehouse 32b retrieval and the use retrieved from user Demographic data and portrait data base 32a Family is mated.Can the 32c portrait sizing again to retrieving from user Demographic data and portrait data base 32a with Join the size requirements of the advertisement retrieved from ad server data storehouse 32b.Then user's portrait can be merged with advertisement or Cover advertisement information personalized with display on advertisement.Except portrait, it is also possible to have by address name or with advertisement information Other information closed merge with advertisement or cover on advertisement to show that the personalized advertisement of such as advertisement 24b demonstration disappears Breath.

Figure 33 diagram is for the workflow of personalization of product.As shown in figure 33, when personalized product, a side of the present invention Face is by the product retrieved from product database 33b and the user retrieved from user Demographic data and portrait data base 33a Coupling.In the 33c portrait sizing again to retrieving from user Demographic data and portrait data base 33a with coupling from product The size requirements of the product of product server database 33b retrieval.Then user's portrait can be merged with product with display such as figure Personalized product 24c shown in 24.Except portrait, it is also possible to address name or other information relevant with product are closed with product And with display such as the personalized message of obtained personalized product 24c demonstration.

Face displacement

Figure 25 illustrates the image selected by use and covers the digital picture photo face with covering object, and also referred to as face is put Change.Application can be to hide negative memory.

Figure 34 diagram is for the workflow of face displacement.When user wants to hide and negatively remembers, an aspect of of the present present invention Can by the face in facial database 34a to be hidden in the photo from picture data storehouse 34d be chosen or by user The images match being stored in negative mental picture data base 34b provided.Data base 34a, 34b and 34d can be at meters In independent data base in individual data storehouse on calculation machine server, in above computer or storage on multiple computers Data base in be found.In the 34c image sizing again to retrieving from negative mental picture data base 34b with coupling from photograph The size requirements of face to be hidden in the photo 25a of sheet data base 34d retrieval.Image is merged with photo 25a so that aobvious When showing photo, the face that image concealing is negatively remembered, as the obtained mug shot 25b hidden is demonstrated.In the present invention Non-limiting aspect, one or more computers can receive the selection of at least one face to be suppressed, optionally as Any type of user inputs.One or more computers then can cover with at least one digital picture to be suppressed extremely The area coordinate that a few facial position is corresponding.

Alternatively, cover and can include covering in the position phase with at least one face to be suppressed selected image Above corresponding area coordinate.Alternatively, one or more computers can regulate selected image with corresponding to cover The size of area coordinate.Alternatively, one or more computers can regulate covering image edge with at least one The picture characteristics mixing of digital picture.For example, it is possible to the edge of the image of fuzzy covering or the opaque of edge can be revised. Alternatively, it is one or more that picture characteristics to be mixed can include in color, intensity, brightness and texture.Alternatively, press down The selection of at least one face of system can include suppressing particular individual to be displayed on the order in any digital picture, method bag Include and identify any digital picture and the labelling including the face corresponding with particular individual to be suppressed (individual identified) The digital picture identified is to cover the area coordinate corresponding with the position of at least one face to be suppressed.This can pass through Metadata that directly amendment respective digital image or renewal are associated or other information or data are to cause non-destructiveness side The display of the respective digital image of formula makes original source digital picture forever not revised in data base.Therefore, Owing to can for good and all apply to digital picture to cover, or cover the metadata that can relate to revising digital picture to cause Digital picture is covered during display.

Figure 26 illustrates to use and is associated to there being corresponding photo in the metadata of photo or elsewhere alternatively in storage Date according to history order from photo capture face display.The corresponding date can be photo create date, from phase Machine replicate photo date, transmit to the computer system of the present invention date of photo or be associated with photo any its He on date.By such sequence, selected face can be merged to create such as the single composograph by 26b diagram.Also Can be automatically fed to such face in deformation application enable to the ageing of individual is carried out animation.

Alternatively, date data can be associated with digital picture.Can come according to the date data being associated accordingly Show the portrait stored corresponding to ordinary individual.It is alternatively possible to generate the portrait stored corresponding to ordinary individual The animation of differentiation in time represent.

Figure 27 diagram from the display of the face/portrait of photo capture and face or portrait are embedded into game, practise or In other interactive application.

Figure 35 diagram according to various aspects of the invention be used for use user's portrait and photo to create to customize or individual character Change game and the workflow of exercise.User can select game to be played or exercise from game database 35c.The present invention is from portrait Data base 35a or picture data storehouse 35b selects portrait for display.At 35d to portrait or photo sizing again with coupling The size requirements of game to be shown.Again the portrait after sizing or photo are merged with the user interface of game for User plays.Data base 35a, 35b and 35c can be in the individual data storehouses on a computer server, in above-mentioned calculating It is found in independent data base on machine or in storage data base on multiple computers.

Alternatively, one or more computers can use stored portrait or relation to fill interactive computer trip Play.Interactive computer game includes that photo is missed old times or old friends and treats game.

Use the directory system of facial recognition automatization

Figure 28 illustrates the establishment of family tree and for indexing the purposes of photo, and Figure 36 illustrates the workflow of correspondence.Can be from The individual portrait data base identified shows portrait.User can use relationship templates 28b as shown in figure 28 by each at 36b Portrait is dragged and dropped on family tree.In a non-limiting example, each user of the system of the present invention can have 6 basic passes System: mother, father, siblings, spouse, daughter and son, as shown in template 28b.When portrait is dragged on family tree by user Time, family tree grows to illustrate new node.When adding more individual, family tree extends, because user is added and relation is determined Justice.When user has pulled all portraits, complete family tree by deleting external relation or individual (main friend and colleague).With Family likely have relate to its people's socialgram but use the friend identified in similar drag and drop method network after the modification and Peer Relationships.

The family tree 28a created can be used as the index to user picture at 36d.When user selects or clicks on photo Time, perform data base querying at 36e and occur in photo therein with the individual selected by display.

Replace family tree 28a, can present with chain form represent between individual the other kinds of organizational structure of relation with In by pulling portrait to it or selecting portrait to fill.For example, it is possible to by providing blank or the most common tissue Chain creates common tissue chain.Then the message from data base can be presented for being placed on chain to user.

According to the non-limiting aspect of the present invention, at least one computer can include storing multiple message image at least One data base or at least one data base interface with the multiple message image of storage are connected.Each portrait images can be with phase The individual of identification shown in portrait is answered to be associated.Can by individual identification be stored in data base, another data base or with phase Answer in the metadata that portrait images is associated.At least one computer can show that at least one being associated with user identifies The corresponding portrait of individual.At least one computer can show the visual representation of at least one personal relationship with user.At least One computer can be according in the portrait of the user's input received at least one distribution display in the personal relationship of display At least one portrait.Personal relationship's distribution can be stored in data base by least one computer.Visual representation can wrap Including and have such as the expression of tree organizational structure of multiple tree nodes of family tree, the most each tree node corresponds at least one One of personal relationship.At least one computer can input according to user multiplies corresponding at least one additional personal relationship's The tree node of visual representation, to add at least one portrait in the portrait shown to visual representation.Such as, family tree structure to Tree can grow when adding additional portrait.Then user can specify that the relation of newly added portrait, or can distribute acquiescence Relation, or at least one computer can to attempt being based, at least in part, on user be to find on the social networking system of its member Data determine the suitable relation of new relation.The data base of portrait or at least one computer another data addressable Storehouse can include multiple digital picture, and at least one computer can index multiple digital picture according to visual representation At least subset.Therefore, family tree structure may be used for linking to other digital pictures of the member of characterization family tree.Especially, point Hit one or more digital pictures that a member of family tree can link at least illustrate this member of family tree.

Figure 29 diagram group photo is as the use of the index of the photo in picture data storehouse, and Figure 37 illustrates the work of correspondence Stream.User can select photo from picture data storehouse 37a.In one aspect of the invention, selected photo is as being used for indexing The photo of photo is recorded and becomes index.When facial during user selects or clicks on photo, one or more computers The identification included corresponding to the face in the photo selected by user can be searched in data base 37a or another data base The more multidigital image of people.

According to the non-limiting aspect of the present invention, at least one computer can include the data base of portrait or digital picture Or being connected with the data base interface of portrait or digital picture, each portrait is relevant to the individual of identification shown in corresponding portrait Connection.At least one computer can be with at least one digital picture in display digit image and use the data base of message to join Draw at least one digital picture of display thus create at least one shown at least one digital picture and identified individual's List.At least one computer can identify according at least one that the list of individual carrys out at least subset of index number image. Select to have identified user's input of at least one individual in individual according to from least one digital picture of display, at least one Computer can show at least one digital picture in the digital picture correspondingly indexed.Alternatively, index can include showing Show by the visual representation of the digital picture identifying the individual index organized.

Request for missing individual is searched for and search automatically

In still another aspect of the invention, the present invention is so that computer program can be from third-party server (such as Centre personal search server) receive digital facial image or signature.Figure 30 illustrates display user and can select for user's The missing child of the potential coupling of picture data library searching and the delivery of the face signature of wanted criminal.Figure 38 shows the work of correspondence Flow.Such as, find the tissue of individual (such as missing child or wanted criminal) can put up shown in image 30a and 30b The face data of body.Face data can be derived from personal search server 38a.These of its facial database 38b are shared in selection User can download data, enables to automatically compare face data and its facial database 38b.Can be at 38c to user Display result of the comparison is for checking.If finding to mate between target individual and the known individual of specific user, then use Family can be at 38d notice tissue.This is so that tissue can determine the recently or currently position of individuality.It is it is also possible that group Knit can determine individuality name because the name of individuality can be listed in known the list of user one or more In.

The further embodiment of the present invention described with reference to the corresponding workflow in Figure 31 and Figure 39 is so that individual physical ability Enough find other people with itself with similar facial characteristics.Such application may be used for individual and finds its twins. In the present embodiment, user can select to submit the photo including its face-image to, and the present invention can be stored according to its generation Face signature in face signature database 39a.Then can by facial characteristics to previously uploaded to data base 39a or similar length The face signature of other individualities of phase data storehouse 39b compares.Based on predetermined similarity thresholding, can be at 39c to its face One of individuality of coupling or all one or more similar faces of report.System according to the present embodiment can provide at 39d and use The means contacted with other people are ratified in user.The individuality of coupling is it has been specified that in the case of its approval to such contact Can select to send message to another people.Another modification of the present invention can be to find to have the phase individually obtained from whole face Individual like facial characteristics (such as face, nose and eyes).

According to the non-limiting aspect of the present invention, at least one computer can include data base or and the portrait of portrait Data base interface connect, each portrait is associated with the individual of identification shown in corresponding portrait.At least one computer can To receive at least one unidentified message and the face signature of at least one unidentified portrait identified known to user The face signature of the portrait of individual compares.According to the positive result compared, at least one computer can point out user certainly The confirmation of result.According to confirmation, at least one computer is so that at least one unidentified portrait confirms at least with user One has identified that individual is associated and is stored in data base by least one unidentified portrait.

Alternatively, unidentified portrait illustrates missing individual.Alternatively, reception can include some steps.It is possible, firstly, to connect Receive digital picture.Then at least one computer can detect at least one face in the digital picture received.At least one Computer may determine that and store the area coordinate of at least one detected face position in the digital picture received.Extremely A few computer can apply at least one conversion to create detected at least one at least one detected face At least one unidentified portrait of face.At least one computer can rotate at least one unidentified portrait at least up at least At least one face that one unidentified portrait is illustrated in vertical orientation and is illustrated at least one unidentified portrait The pair of eyes in portion is positioned in the horizontal plane.

Relationship visualization is made according to photograph meta data

When the set of digital photos comprises the name label identifying the individual represented in these photos, it is possible to " open Adopt " this information and generate potential interesting, pleasant and useful technology for showing in these photos Relation between labeled personnel.How to make name label can be used for computer system to be not limited to use these data to show Interesting figure and this concept of form.The method quickly generating name label is to use face detection and identification technology. This technology accelerates labeling process by automatization's majority manual step.Name label data can simply consist in photo In metadata, all XMP forms of Adobe in this way, or may reside in SQL or other data bases.

Figure 46 describes the figure generated according to an aspect of the present invention.The person-to-person relation individual's occurred in photo Between interconnection line represent.From the beginning of central or selected individual's (being Ray in this example), the present invention illustrates that Ray occurs Several destination nodes of photo wherein.Can be according to the number of photo to node relatively sizing.It is attached directly to central authorities Any node of " Ray " node can identify the individual having now in the photo of Ray.It is therefoie, for example, April occurs in Having in 100 photos of Ray, " friend A " occurs in 135 photos with Ray.Therefore, next layer of figure illustrates not Have together with Ray and occur but the individual occurred together with being connected directly to the individual of Ray node.Therefore, Sandy and Lisa with April occurs in photo together.This method is for drawing all relations being displaced outwardly from central authorities individual.Each node can To be shown in the frequency occurred together of the individual of name at node and to trip out the one of the tight individual named at front nodal point Existing frequency.

This figure in Figure 46 can serve as the index of photo.By clicking on any node in figure, lantern slide can be used Or thumbnail list form shows the corresponding photo that this node represents.At least one computer can receive on any node Inquire about one or more picture data storehouse before click to get out present the photo of correspondence, or at least one computer can To perform any such inquiry after receiving click user's input.

According to the non-limiting aspect of the present invention, at least one computer can include data base and the knowledge of digital picture At least one shown in other respective digital image identified the name of individual respective meta-data or with the number of digital picture Identify that according at least one shown in storehouse and identification respective digital image the respective meta-data interface of the name of individual connects. At least one computer can select to identify one of individual from data base and determine that to identify that individual occurs in therein The counting of the sum of digital picture.For at least one digitized map in there is the selected digital picture identifying individual Each shown in Xiang has identified individual, at least one computer may determine that have selected identify individual corresponding the most Identify that individual occurs in the counting of the sum of digital picture therein, and show visual representation, all figures as shown in figure 46 Or form.Visual representation may include that primary nodal point, primary nodal point represent that selected identification is individual (in such as Figure 46 Ray) and identify that individual occurs in the corresponding counts of the sum of digital picture therein.For having selected identification Each identify individual shown at least one digital picture in the digital picture of individual, secondary nodal point can have been presented, the Two nodes represent shown at least one digital picture having in the selected digital picture identifying individual corresponding the most Identify individual.Each respective nodes can also include having selected identifying, and the corresponding of individual has identified that individual occurs in it In the visual representation of corresponding counts of sum of digital picture.Each secondary nodal point can link to primary nodal point, alternatively Be visual line form or by means of display vision link.

Alternatively, for using what corresponding secondary nodal point showed each to identify individual (second people), for having phase That answers shown at least one digital picture in the digital picture of second people each has identified individual, at least one computer May determine that the corresponding counting having identified that individual occurs in the sum of digital picture therein with corresponding second people.At this In the case of Zhong, visual representation may include that at least one digital picture in the digital picture for having corresponding second people Shown in each the 3rd node having identified individual, the 3rd node represents in the digital picture with corresponding second people extremely Corresponding shown in a few digital picture has identified that individual, each respective nodes also include having the corresponding of corresponding second people Identify that individual occurs in the visual representation of the corresponding counts of the sum of digital picture therein and corresponding secondary nodal point with every Link between individual corresponding 3rd node.

Another non-restrictive illustrative embodiment can illustrate only listings format based on the level leaving central authorities individual Relation.Figure 37 illustrates how to arrange these photos by level.

For representing that another non-restrictive illustrative embodiment of level can be to illustrate that comprise in this level pieces together Photo.Figure 48 shows and demonstrates may showing of this method.

Real for illustrating another non-restrictive illustrative of the present invention of the obtainable data of the name label from photo The mode of executing is to extract the face of individual from photo and show these faces on the time line.Figure 49 provides an example.From institute Select individual's (by least one computer or inputting selection according to user), can from they can be labeled every Individual photo extracts all faces of this individual and shows according to the date or time axle of certain form.If selected is individual The photo more than one of people can be used, then this can illustrate individual's differentiation in time.

According to the non-limiting aspect of the present invention, at least one computer can include data base or and the portrait of portrait Data base interface connect, each portrait is associated with the individual of identification shown in corresponding portrait.At least one computer can To select to identify one of individual from data base；It is at least partially based on the day metadata pair being associated to each corresponding portrait In the selected individual of identification associated plurality of portrait sequence；And show virtual representation.When virtual representation may include that Top-stitching；And multiple portrait according to corresponding sequence along the layout of timeline.

Will now be described and use face detection and the more relevant details identified for labelling purpose.Face detection algorithm is pressed Operate according to different manners, but scanning digital image is can with the one or more final result in these algorithms of application Presenting of the list of the facial zone of energy.These regions generally with by four coordinates (such as left/upper, right/upper, left/under, right/ Under) bounding box that defines identifies.

If provide the set of new photo to the face detection system of the present invention, then system can return face to user List and ask user's labelling face (in other words, title being attached to face).This title can be via keyboard or warp Inputted by the drag and drop from existing contacts list or address book.User generally also provides for input to delete wherein system mistake Ground is by facial for any false positive that the region recognition of photo is face.

The facial-recognition security systems of the present invention can obtain (the list of such as bounding box coordinates and to former of face testing result The pointer of beginning digital picture) and generate digital signature for each face.These signatures are to constitute by bounding box or to limit The subset of boundary's frame or bounding box apply other cover the mathematical notation of pixel of face of region definition.For example, it may be possible to can The eyes coordinates using face concentrates oval mask to attempt eliminating non-specific feature, such as hair, medicated cap on facial zone With other non-face objects.Then, this pixel covering in region is only used to generate face signature.

These digital signatures depend on that algorithm can be sequence or the individual digit string or many of numeral (real number or plural number) Dimension array.Other functions performed by the facial-recognition security systems of the present invention may is that

Compare two face signatures the most together to determine " similarity " score.If this score is by dynamic or predefined Thresholding, then it becomes " suggestion " coupling between both.

2. " similar " face signature is combined into together the cluster of similar face.This also uses dynamic or predefined thresholding Determine that face signature belongs to identical cluster the most together.

Optional function is the representative that the single individual of the list with face signature is found in labelling activity based on user Property signature.It practice, this artificial system " known ", because face is labeled by identical title.

According to various aspects of the invention, the present invention is so that can visualize the data extracted from photograph meta data and close System, is attached to name label and the date of digital photos specifically.These data can be via common metadata form (example Such as EXIF, Adobe XMP etc.) be attached directly to photo, or can with link to name of pictures database table comprises Metadata data base in photo individualism.

Give one to trip out via label (or in data base) and the individual of frequency of occurrences identification and with them In present photo is whom, determines the data relationship that can differently draw.These figures provide and carry out relation visually The mode changed.

Figure can be automatically created by the present invention, or the user of the present invention can ask from specific selected by user People starts to create figure.This unique individual can become the central authorities of the figure in network representation, or " from left to right " represents In left side starting point.

Required data are represented, it may be necessary to comparison comprises the data of metadata in order to gather the display such as figure of Figure 46 Multiple queries is run in storehouse, the name of the individual occurred in photo specifically.If being attached directly to the metadata of digital photos In only exist data, then must gather this metadata via image scanning process.Obtained set can be placed on In computer storage (array of such as the text field), or off permanently or temporarily SQL database can be placed into In for inquiry purpose subsequently.The purpose described for this part, it is assumed that data reside in SQL database.

Initial query can find to comprise and occur in the individual in one or more photo with selected " beginning " individual The counting of all images.These people are referred to as " acquaintance ".Can deposit according to descending based on the occurrence number of each acquaintance List obtained by storage.The node that initial individual can be shown as in figure by figure.Size of node can or can not Number based on photo and change.The number of photo can be that " initial individual " occurs in photo therein in this case Sum.

Shown next node can be first in list or maximum acquaintance.Size of node can or can not Change along with the number of times of they appearance in the photo with " initial individual ".This node can be attached to via adapter Original start node.This process can be continued until all acquaintances using up in list.One non-limiting embodiment can The size of list to be limited to the minimal amount occurred together.Figure can represent and initiate individual and occur at least 3 photographs Individual in sheet.

Once list is exhausted, then for each acquaintance, can run another inquiry and occur together with this acquaintance to determine But all individuals not occurred together with initial individual.Based on this list, need by the adapter returning to acquaintance to Figure adds node.

This process can be continued until all acquaintance's nodes using up in figure.In order to limit the size of figure (especially It is in the case of the individual with thousands of photos), user can specify that the pole away from initial arbitrary X level Limit.

Can illustrate that this " hierarchical relationship " identified is other kinds of according to generating with mode like above Figure, but display packing is different.For Figure 47, " successively level " photo can be shown as thumbnail or rotatable image. This can be to provide the labelling interface of " label of each level ".For Figure 48, for each layer away from the initial individual of central authorities Level shows the photo of colleague's form.Each colleague can be single figure or the page, or can be combined into one big Figure.

In order to generate Figure 49, it is possible to use the date of figure is along the sequence of timeline definition face.Face-image is originally Body can use the coordinate of the predefined bounding box at facial surroundings to extract from original photo.Bounding box can be based on automatically Identify that the type of face detection method of the coordinate comprising face determines, or user can be at mouse, touch pads or touch screen user Manual definition bounding box is carried out by physically drawing frame at facial surroundings with the help of interface.

According to the non-limiting aspect of the present invention, at least one computer can include data base and the knowledge of digital picture At least one shown in other respective digital image identified the name of individual respective meta-data or with the number of digital picture Identify that according at least one shown in storehouse and identification respective digital image the respective meta-data interface of the name of individual connects. Face detection described herein and identification technology can be used formerly to identify each individual.At least one computer can be from data Storehouse selects identified one of individual；Determine at least one shown at least one digital picture in digital picture first Identify that individual is together with the selected individual of identification；And display visual representation.Visual representation may include that for each first Having identified first hierarchy node of individual, the first level node table shows the selected individual of identification and together in digital picture In at least one digital picture shown in corresponding first identified individual.Visual representation can also include for each first Having identified second hierarchy node of individual, the second level node table shows not to be had in the selected digital picture identifying individual Shown at least one digital picture corresponding first has identified individual.

Alternatively, for each first, visual representation has identified that individual can include the second hierarchy node, the second level joint Point expression does not has selected identifying, and individual still has at least one numeral in the second digital picture having identified individual Shown in image corresponding first has identified individual, wherein second identified individual be confirmed as not illustrate selected Identify shown in any digital picture of individual.

Alternatively, for each second, visual representation has identified that individual can include third layer level node, third layer level saves Point has represented corresponding second shown at least one digital picture not having in the corresponding first digital picture having identified individual Identify individual.

Alternatively, each first hierarchy node can include illustrating that selected identification is individual and corresponding first identifies The two the set of digital picture of individual.

Alternatively, each first hierarchy node can include illustrating that selected identification is individual and corresponding first identifies The two the set of digital picture of individual；Each second hierarchy node can include illustrating that corresponding first has identified that individual does not has The set of the selected digital picture identifying individual；Each third layer level node includes illustrating that second has identified that individual does not has There is the set of the corresponding first digital picture having identified individual.

Cluster describes

The optional function of the present invention can be the list that the list with face signature is found in labelling activity based on user The corresponding signature of individual.It practice, this people can think system " known ", because face is marked by identical title Note.For the purpose of the coupling determined between similarity score and two photos of suggestion, representative signature is not for knowing with each Not face compares to determine whether there is coupling.Representative face signature can also be that the set of face signature is so that gathering In each entry can represent this unique individual face signature single cluster.

The representative face of the group finding the face to individual's distribution is probably useful, in order to reduce the most known The time that portion's (being such as potentially hundreds of or more known face) is relatively required compared with each new unidentified face.Separately Outward, personnel are the most ageing and change its appearance for various reasons, therefore use the similar face representing unique individual The set of face signature of cluster more preferable result can be provided, reduce compared with relatively each specific known face simultaneously Necessary compare number.

The other use of cluster can be in order at the purpose of labelling and present " similar " face to user in batches.This is carrying The ability aspect of a lot of faces (and each single faces of non-marked) that confession labelling has a title can be time-consuming.

One challenge of cluster is: for performance reason, it is undesirable to again must cluster when new face is introduced into set The whole set of unidentified face.Can be more efficient, use the various skills making it possible to create and manage the cluster increased Art.

Will now be described relevant with cluster for the purpose finding individual representative face during identification step Details.Can be grouped to use the form face associated plurality of for known individual of hierarchical clustering, wherein as retouched above Compare facial signature with stating, to create the group of the face with similar appearance.In order to compared with facial for the unknown and known individual, All single faces one to one can be made compare, and may then pass through arithmetic mode combined result to form one or many Individual cluster result, has each group of cluster result of similar face.

To enable effectively compare cluster result, can apply linearity correction, it depends on for calculating cluster The number of the single result of result.This linearity correction can determine in the following manner: first compares and has given number Organizing from the known a large amount of single face of different individual that belongs to determine the distribution of result of the known face of " N ".

It is then possible to determine linearity correction so that this distribution is best aligned with standard normal distribution.Finally, Ke Yijin One step joint correction is with by increasing standard deviation or translation average at face big for Shi Xianggeng compared with the unknown face Group demonstrates advantage.This final step ensure that the individual frequently occurred in the collection of photographs of user is suggested to newly at it The probability aspect of coupling of unknown face there is suitable increase.

Cluster result after the multiple corrections that the comparison with whole known individuals facial by single the unknown can be obtained Compared with the thresholding of fixed threshold or dynamically selection.If it is desired that with the thresholding dynamically selected, then it can be selected as producing The thresholding of raw the maximum number of unknown face (only its single individual has the result exceeding thresholding).The door of such dynamic selection The advantage of limit is, it maximizes the number of real individual suggestions, minimizes false positive suggestion simultaneously.

In one embodiment of the invention, this cluster process can use and (see with at term " hierarchical clustering "http://en.wikipedia.org/wiki/Hierarchical clusteringThe similar approach described under).Initial poly- Class can be that " coagulation type " (bottom-to-top method, the most each observation is from the beginning of the cluster of their own, and at upper shifting layer Multipair cluster is merged during level), and the selection of representative face can be that " Split type " (top-to-bottom method wherein owns Observe and start in a cluster, and recursively perform separation when moving down level).Originally, it is believed that each face be it from Oneself independent cluster.Each face and is recorded and stores score (similarity) value compared with other faces each.Abandon Score less than certain predetermined threshold.Then, from the beginning of top score (two most like faces) and towards minimum score Carry out, if every pair of face is not yet concatenated via certain rim path, then link every pair of face by edge.It is, close And two single clusters, but the face in cluster is linked further not over additional edge.Result is some differences Dendrogram (is set), and each expression clusters.Note, in each dendrogram, there is no the number at loop and edge than face Few one of number.Number for its septum reset is more than each cluster of certain specific integer thresholding (such as 20), selects face The subset in portion represents cluster.

The selection of subset can be such as realization of getting off.First, find and there is the edge of minimum score and remove it with will Tree is divided into two less subtrees.Then, it is determined that how many face will be selected from each subtree so that counting the size with subtree Equal proportion.Such as, if finding 15 faces and subtree A is that the twice of subtree B is big, then want to select 10 from subtree A Face and from subtree B select 5 faces.If the number of the face in subtree is equal to the number wanted, the most defeated Go out these faces to terminate as result and the process to this subtree.If the number of face is more than the number wanted, the most again It is iteratively applied this process.It is, in subtree, find next edge with minimum score, and remove it with Further Division tree.Result is the tree of any size to be reduced to certain fixed size so that residue face is original facial collection The statistical representativeness sample closed.

Will now be described and be used for improving by the cluster of unidentified face with the number of clicks required for reduction user The details that time involved by labelling is relevant.For practical reasons and performance reason, according to an aspect of the present invention, create completely Increment (fully incremental) Time constant clustering algorithm.At least one advantage of this clustering algorithm can be constant Time aspect, it does not changes according to the sum of photo in set.Add involved by new face to existing cluster to describe Step, first, it is assumed that to each existing certain cluster token (such as integer) of face distribution.If two faces have phase Same token, then it is assumed that two faces are in identical cluster.Secondly, it is assumed that each face is associated with certain photo and often Individual photo has date and time associated there.This must be generally but not the date and time obtaining photo.Finally, root Photo is kept according to the order of classification according to this date and time metadata.

The given new photo wherein with at least one face, the date/time first by new photo finds it existing There is the position in the tabulation of photo.Then by being displaced outwardly from this position in the two directions at new face with existing Have between face and perform face one to one and compare.It is, before moving to the date in the most farther (past and future), first First by new face compared with the existing face obtained in similar date/time.

If face compares generation and exceedes the similarity of certain fixed threshold one to one, then distribute and existing to new face The cluster token that portion is identical, and the cluster of new face completes.

Three rules can be applied constant to guarantee the time spent to the cluster each new face of interpolation.First, will be new Face is compared with other the new faces in same photograph.Secondly, if new face is facial with one of specific cluster at it Failure in Bi compare, then by new face not with compared with any other face of this cluster.3rd, have now and could be made that The comparison of maximum fixed number.Do not find coupling if reaching this number, then can distribute new cluster to new face Token, and therefore new face is the first facial in newly clustering.

Finally, when processing a collection of photo, process photo according to random order to improve " at a distance " cluster.

Figure 50 shows the photo with the date and time being associated and face arranged according to the age.According to the present invention Analyze " new photo ".The cluster id/ token of the face accordingly of the letter representation on each face.In Figure 50 below cluster token The curve with arrow illustrated represents and to compare face-to-face to what the face in other photos performed face from new photo. Represent that the numeric representation illustrated on the curve compared face-to-face performs the order that respective face opposite is compared.For example, it is possible to will have First the photo of nearest date/time to new photo compares, and can according to the date being associated of photo or face and The chronological of time compares remaining face.Assuming that previous compare face-to-face in do not find coupling, the most not by respective face Three existing faces of the face that portion links to the curve of the face from new photo represent the ratio skipping face corresponding to these Relatively, because already having accounted for belonging to the corresponding face of cluster.

In order to illustrate coupling, compare suspend when finding coupling and new face using identical cluster id letter as coupling face Portion (such as E), as shown in figure 50.

In order to illustrate without coupling, then new cluster id (such as F) can be distributed to new face, not shown in Figure 50.

According to the non-limiting aspect of the present invention, at least one computer can include data base and the knowledge of digital picture The respective meta-data on the date of other corresponding data image or with the data base of digital picture and identify corresponding data image The respective meta-data interface on date connects, and multiple digital pictures illustrate at least one corresponding unidentified individual (Unidentified numeral Image).Unidentified digital picture can be classified by least one computer by corresponding day metadata；And to often The distribution of individual Unidentified digital picture clusters token accordingly.Distribution may include that the subset according to unidentified digital picture is each This determines to illustrate public unidentified individual, public the clustering accordingly of each Unidentified digital picture distribution in subset Token.Unidentified digital picture can be grouped by least one computer by corresponding cluster token.At certain point, at least One computer from another computer, data base, user or can receive new digital picture from any other place.New number Word image and respective meta-data can identify the date of the newest digital picture, and new digital picture can include new Unidentified individual.At least one computer can then attempt to by using or utilizing any clustering technique described herein to taste Examination identifies the new unidentified individual in the image received.Especially, at least one computer can perform newly in order At least one of at least one corresponding unidentified individual of unidentified individual and multiple digital picture compare, wherein for often group Unidentified digital picture, at least one computer can only perform unidentified individual Yu the most public new unidentified individual Single comparison.At least one computer can produce each group according to comparing the digital picture distribution cluster token performed to new The determination of the public new unidentified individual of the corresponding set of Unidentified digital picture in Unidentified digital picture.Optional Ground, compares execution sequence and includes day metadata order of the day metadata of closest new digital picture on the date.Can Selection of land, wherein in the predetermined maximum of at least one the corresponding unidentified individual performing new unidentified individual and multiple digital picture After the comparison of number, at least one computer can stop comparing execution and making to the cluster that new digital picture distribution is new Board.

The various aspects of the face cluster of Figure 51-53 diagram present invention.Figure 51 is shown and can be provided by the present invention The labeling method face packet to corresponding known individual.In Figure 51, respectively known individual facial group of A, B and C is carried out point Group.In Figure 52, the component of the face of known individual can be formed according to the recognizer of the present invention face of known individual One or more clusters.The division of cluster to similar face can be at least partially based in the face packet of known individual Face signature distance between each face.In Figure 53, when unknown face is submitted to the face recognition method of the present invention, Each cluster of facial for the unknown and each known individual can individually be compared by the recognizer of the present invention.

Image metadata is used to improve facial recognition result

Challenge in facial recognition is it may be that owing to using various camera apparatus (to include babyphone, mobile phone and disposable Camera) the free attribute of client's photo that obtains, identifying that the ability of the people in photo (or video) has very much exactly can The accuracy of 100% can not reached.Only have the challenge that too many variable class is similar to be associated with weather forecast accurately.

The accuracy that can strengthen recognition result makes it exceed the cardinar number student movement of the pixel analyzed and comprise in movement images Calculate.The image associated metadata of data token, camera type, position coordinates and event information is included, it is possible to reduce by utilization The false positive data generated by face recognition algorithm.In the other extension of this concept, additionally it is possible to identify and may relate to Other objects (non-human) in the image of locality, event or time.

Except rest image, these concepts are also applied for video image.They can be when user shoots photo or video Apply in real time, or can apply after an image is captured.

The non-limiting embodiment of this concept according to an aspect of the present invention is illustrated in Figure 40.Facial recognition Algorithm acts on the list of unidentified face and by the digital signature of these faces compared with the digital signature of the unknown face. Any face carrying out the unidentified set with known face in comfortable predefined (or algorithm) thresholding will become linking to Know " the suggestion coupling " of the individual of face.If single photo generates 5 facial zones according to face detection scanning, then may not Be enough to provide the individual identical with the suggestion of the more than one face in same photograph.Note, special situation can be there is (photo after such as relating to the photo of mirror and being revised by editing machine), wherein someone can go out in single photo The most more than once, however design meets the system of these rare cases is unpractiaca.Therefore, if face recognition algorithm runs into Having two (or multiple) face advising identical image from unidentified individual, the most other logic will consider two (or multiple) face in identical image the fact that and ask " closest match " as use suggestion.If to Family suggestion closest match refused by user, then in order to improve efficiency, it is generally desirable to, about be not this image closest to That join and be not the most any face of suggestion, also abandon these faces in any other digital picture of suggestion.Then may be used So that the face of any refusal is advised compared with the set of the known individual getting rid of unidentified individual to find next optimal Join (if present).

According to the non-limiting aspect of the present invention, at least one computer can include portrait and the digital picture being associated Data base or be connected with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait The individual of identification be associated.It is individual that at least one computer may determine that at least two shown in one of digital picture has identified The corresponding portrait of people meets comparison threshold with the portrait of the unidentified individual determined according to the digital picture received.At least one Then computer can advise being identified as having by unidentified individual and be confirmed as at least two and identified in individual most closely What at least two of the portrait being associated accordingly mating the portrait of unidentified individual had identified in individual corresponding has identified individual. Then at least one computer can be got rid of and advise subsequently being identified as from any other in digital picture unidentified individual At least two of digital picture has identified that remaining in individual has identified individual.

When suggestion face it is also contemplated that date or time, as shown in figure 41.It shall be assumed that, with the figure for the present invention As relevant metadata (such as date, coordinate, camera type etc.) is accurately, the most undesirably basic side based on metadata Portion's suggestion can increase accuracy.Considering to share the individual of labelling in the photo on date with another photo, it is referred to as P.At other In photo, it is assumed that there is unidentified face, two of which (or multiple) personal satisfaction is for being referred to as these people's by this face The identification thresholding of suggestion coupling.Therefore, if one of these people meeting thresholding are confirmed by the user as individual P, the then present invention Can be based on individual P labeled and confirm as starting at photograph from phase same date after this date exists in another photo The concept again presented in sheet increases individual's P grading in the list of proposed identification.

According to the non-limiting aspect of the present invention, at least one computer can include portrait and the digital picture being associated Data base or be connected with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait The individual of identification be associated.At least one computer may determine that shown at least one digital picture in digital picture At least one has identified that the corresponding portrait of individual meets with the portrait of the unidentified individual determined according to the digital picture received Comparison threshold.The metadata phase on the date according to the day metadata being associated with the digital picture included corresponding to and received At least one digital picture determined by the digital picture of association, at least one computer can be advised unidentified individual It is identified as at least one and identifies individual.

It is also contemplated that with individual relation, as shown in figure 42.Assuming that be referred to as the individual of P in photo labeled and Other unidentified faces are found in this photo.It is also assumed that individual P still belongs to be referred to as the packet of C, and this is grouped into and is Unite known.Accordingly, as identify proposed algorithm a part, other faces in photo also be packet C member probability more Can be taken into account in the following manner in method the fact that of high: reduce each individual as the member being grouped C Thresholding, or in the case of more than one personal satisfaction gives the suggestion thresholding of unidentified face, mobile in potential coupling List in the individual of higher group of C.

According to the non-limiting aspect of the present invention, at least one computer can include portrait and the digital picture being associated Data base or be connected with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait The individual of identification be associated.At least one computer can according to the picture received include with definition group be associated to Few one has identified individual this has determined, the group being at least partially based on and defining be associated at least one identified the phase of individual The portrait answering portrait and the unidentified individual determined according to the digital picture received meets comparison threshold, and this determines and advises The identification of unidentified individual.

It is also contemplated that position or position coordinates, as shown in figure 43.Assuming that individual P is labeled and is confirmed to be in the date It is in the photo of X for A, position, and each recognizer, but individual P can be generally also at date A to shoot at position Y Other photos in unidentified face suggestion coupling, wherein Y away from X more than N hour.In such a case, it is possible to based on The suggestion coupling abandoning individual P considered below: individual P or can not have possibility by the metadata estimation of corresponding photo Two position X and Y in date-time frame.Individual P can not be taken into account at two local probability by this simultaneously.

According to the non-limiting aspect of the present invention, at least one computer can include portrait and the digital picture being associated Data base or be connected with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait The individual of identification be associated.At least one computer may determine that shown at least one digital picture in digital picture At least one has identified that the corresponding portrait of individual meets with the portrait of the unidentified individual determined according to the digital picture received Comparison threshold.According to include (i) corresponding to and date of day metadata of being associated of the digital picture that received and (ii) metadata of the position and in the preset distance thresholding of location metadata that is associated of the digital picture that received is associated Digital picture in determined by least one digital picture, at least one computer can advise by unidentified individual identify Individual has been identified at least one.

It is also contemplated that on social networks obtainable event information or other information, as shown in figure 44.Assuming that system Give the social network information that captures for this user or other data sources know individual P presence event X, then individual P occurs in At event X, the probability of the photo of shooting is higher.Also it is reasonable to assume that individual P is in event Y (occurring at phase same date) place's shooting The concept occurred in photo is lower.For the face found in the photo of shooting at event X, recognition methods can use this Other information distributes higher grading (via one or more particular adjustments) to individual P, and wherein P otherwise can meet and built View is the thresholding of any one the unidentified face in unidentified face.For the face found in the photo of shooting at event Y Portion, recognition methods can also distribute lower grading (via one or more particular adjustments) to individual P.

According to the non-limiting aspect of the present invention, at least one computer can include portrait and the digital picture being associated Data base or be connected with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait The individual of identification be associated.At least one computer may determine that shown at least one digital picture in digital picture At least one has identified that the corresponding portrait of individual meets with the portrait of the unidentified individual determined according to the digital picture received Comparison threshold.At least one computer can also be at least partially based on the metadata being associated with the digital picture received, really The fixed digital picture received is associated with event.According to determined by the digital picture being associated with event at least one Digital picture, at least one computer can advise that unidentified individual is identified as at least one has identified individual.

It is also contemplated that non-human subject or environmental information, as shown in figure 45.For example it is assumed that individual P in photo by really Recognize and be labeled, and this photo is considered to be similar to a part of the set X of photo based on date, event or positional information. Therefore, if individual P uses object identifying method to be also found to wear green sweater in the photo confirmed, then can find This information is used to apply more to individual P in the case of being also attached to green sweater from the face in another photo of set X High priority or grading.Except clothes, can analyze and consider other objects shown in photo, such as furniture, trees, vapour Any other environment of car, animal, cloud, scenery or photographic detail is arranged.Can analyze and consider the face of any such details Color, texture, orientation or layout.Especially, if individual P is based on labeled in photo in set of date, event or position, Then the non-face details of individual P can be compared (non-for the unidentified individual in other photos in set with suggestion individual P Face detail can include garment shape or color, hair color, glasses, the colour of skin etc.).

According to the non-limiting aspect of the present invention, at least one computer can include portrait and the digital picture being associated Data base or be connected with portrait and the data base interface of digital picture being associated, each portrait with shown in corresponding portrait The individual of identification be associated.At least one computer may determine that shown at least one digital picture in digital picture At least one has identified that the corresponding portrait of individual meets with the portrait of the unidentified individual determined according to the digital picture received Comparison threshold.Then at least one computer can extract at least one digital picture determined by from digital picture non- Portrait visual information.According to the non-portrait visual information extracted and the non-portrait visual information from the digital picture received This determines to meet comparison threshold, and at least one computer can advise that unidentified individual is identified as at least one have been identified individual People.

The facial markers of sound auxiliary

It is alternatively possible to use sound or speech recognition help or support facial markers, such as various aspects of the invention Thered is provided.At least one computer can point out user labelling photo.User can be by using any available means (such as By clicking on mouse, touch screen, input through keyboard) provide user to input and respond, or by using phonetic entry to respond.With Family can be connected with local computing device interface, and facial markers method can operate on local computing device, or user can With with and other computers one or more or computer server (it provides the labelling of photo or face individually or together) lead to The computer interface of letter connects.Under any circumstance, user interfaces the local computing device of connection (it can be mobile electricity Words, personal computer or the calculating equipment of any other type) (by wired or wireless connection) can be included or be connected to Mike or other audio frequency input peripherals.One of local computing device or other computers of communicating with can process The audio frequency of user inputs to determine one or more voice command.

Once labeling process starts, and voice recognition is supported and enables, then can present one or more to user Want the photo of labelling.It is alternatively possible to ask the user whether to recognize particular photos.One of computer can be attempted determining photo Date or time.User can be pointed out to confirm or revise this date or time.Can from user provide voice command by by terms of One of calculation machine processes.By the face recognition method described according to various aspects of the invention, at least one of which individual's quilt It is defined as being found in photo, user can be pointed out to identify at least one individual.Voice command can be provided with logical from user Cross name to identify individual.If voice command does not provide the full name of individual, then face recognition method can be advised based on institute The voice command received uses full name to carry out labelling individual.Such as, if individual is identified as " Bob " by user, then method can be with base By other people of named " Bob " of user's labelling or based on by social networks or contacts list in other photos The individual of other named " Bob " that are connected to user is proposed to be used at least one full name of " Bob ".This process can be with pin Whole individuals in photo and the whole individuals in any other photo the most unlabelled for user are repeated.Alternatively, The most labeled existing photo can also again present for labelling to user or re-flag, preferably all unmarked Photo the method according to the invention is processed and after labelling.Method can also point out the labeled any photo of user record Title.The title of any record can be played back to confirm before being applied to corresponding photo to user.Language can also be passed through Sound identification technical finesse title and be saved as the text header being associated with corresponding photo.Mark can also be presented to user Topic text is for looking back before preserving or confirming.

Summary

Should be appreciated that module or the parts of any execution instruction explained herein can include or be able to access that calculating Machine computer-readable recording medium, such as storage medium, computer-readable storage medium or data storage device (removable and/or non-removable), all Computer-readable medium such as such as disk, CD, tape and other forms.Computer-readable storage medium can include using any letter Ceasing storage method or the volatibility of technology realization and non-volatile, removable and non-removable media, such as computer-readable refers to Make, data structure, program module or other data.The example of computer-readable storage medium includes that RAM, ROM, EEPROM, flash memory are deposited Reservoir or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray Disc or other optical storages, magnetic Box, tape, disk storage device or other magnetic storage apparatus or may be used for store expectation information and can by should With, module or both any other medium of accessing.Any such computer-readable storage medium can be mobile device, tracking A part for module, Object tracking application etc., or may have access to or be connectable to mobile device, tracking module, Object tracking should With etc..Any application described herein or module can use the meter that can be stored by such computer-readable medium or keep Calculation machine is readable/and executable instruction realizes.

Therefore, specific embodiment can be made a change by those skilled in the art, modifications and variations, without departing from only by this Disclosed the scope of the present disclosure defined in the appended claims.

At other aspect, present disclose provides computer, equipment, method and computer programming module, including non-transient state Computer-readable memory or non-transient state set of machine-readable instruction, for realizing such method and realizing the merit previously provided Use during energy.

Although in the case of speciality to a certain degree, describe and illustrate the disclosure by exemplary approach, but should Work as attention, be only used as example and make description and explanation.Can be in parts or the structure of step and combination and the details side of layout A large amount of change is made in face.Therefore, such change is intended to be included in the present invention, and the scope of the present invention is limited by claim Fixed.

Except, in addition to statement in described process (including its any optional step or composition) or intrinsic scope, not having Intentional or imply any required order, sequence or combination.Various equivalent modifications should be appreciated that about described herein Process and any system, equipment etc., various modification are possible and the most favourable in all cases, without departing from The scope of the present invention being limited only by the claims.

Claims (24)

1. the method performed by least one computer, described method includes:

At least one face is detected at least one digital picture；

Determine and store the region seat of detected at least one face described position at least one digital picture described Mark；

At least one face described in detected apply at least one conversion with create detected described at least one face At least one portrait；

Rotate at least one portrait described to be illustrated in vertical orientation and described at least up at least one portrait described The pair of eyes of at least one face described being illustrated at least one portrait is positioned in the horizontal plane；And

Store at least one portrait described in being rotated.

Method the most according to claim 1, the wherein detected each face at least one face described corresponds to Individual.

Method the most according to claim 1, including extracting tested prior to described applying from least one digital picture described At least one face described surveyed.

Method the most according to claim 1, is included in selling images and inserts stored at least one portrait described.

Method the most according to claim 4, wherein said selling images is advertisement.

Method the most according to claim 4, adds including applying at least one to stored at least one portrait described Convert for the portrait size requirements mating described selling images.

Method the most according to claim 1, including:

Receive the selection wanting at least one face repressed；

Cover the district corresponding with wanting repressed described at least one face position at least one digital picture described Territory coordinate.

Method the most according to claim 7, wherein said covering includes being stacked in repressed the image of selection Above the area coordinate that the position of at least one face described is corresponding.

Method the most according to claim 8, is corresponding to region to be blanked including the described image adjustment that will select The size of coordinate.

Method the most according to claim 8, including the edge of stacked described image is adjusted to described at least one The picture characteristics mixing of digital picture.

11. methods according to claim 10, wherein said picture characteristics includes in color, intensity, brightness and texture One or more.

12. methods according to claim 7, wherein want the described selection of repressed at least one face described to include using Being displayed on the order in any digital picture in suppression particular individual, described method includes that identification includes and wants repressed institute State any digital picture of the corresponding face of particular individual and the identified described digital picture of labelling with cover with will quilt The area coordinate that the position of at least one face described of suppression is corresponding.

13. methods according to claim 7, wherein said covering includes applying permanent mask to described digital picture.

14. methods according to claim 7, wherein said cover the metadata including revising described digital picture so that Described digital picture is blanked when shown.

15. methods according to claim 1, including: date data is associated with described digital picture；And according to phase The portrait corresponding to ordinary individual of the described date data display storage that should be associated.

16. methods according to claim 15, including generate storage corresponding to described ordinary individual described portrait with The animation of time-evolution represents.

17. methods according to claim 1, including by relevant at least one portrait described in storage for the identity of individual Connection.

18. methods according to claim 17, including being at least partially based on be included in described digital picture corresponding Identified individual makes identified person-to-person relation be associated.

19. methods according to claim 18, fill interactive mode calculate including by described portrait and the described relations of storage Machine game.

20. methods according to claim 19, the game of wherein said interactive computer includes that photo is missed old times or old friends and treats game.

21. 1 kinds include that at least one system for computer, at least one computer described include at least one processor and non- Transient calculation machine readable memory, described non-Transient calculation machine readable memory include process instruction, described process instruction by At least one processor described makes described computer when performing:

At least one face is detected at least one digital picture；

Determine and store the region seat of detected at least one face described position at least one digital picture described Mark；

At least one face described in detected apply at least one conversion with create detected described at least one face At least one portrait；

Rotate at least one portrait described to be illustrated in vertical orientation and described at least up at least one portrait described The pair of eyes of at least one face described being illustrated at least one portrait is positioned in the horizontal plane；And

Store at least one portrait described in being rotated.

22. systems according to claim 1, wherein said process instruction makes when being performed by least one processor described Described computer prior to described applying from least one digital picture described extract detected described at least one face.

23. systems according to claim 1, wherein said process instruction makes when being performed by least one processor described Obtain described computer in selling images, insert stored at least one portrait described.

24. systems according to claim 1, wherein said process instruction makes when being performed by least one processor described Described computer:

Receive the selection wanting at least one face repressed；

Cover the district corresponding with wanting repressed described at least one face position at least one digital picture described Territory coordinate.