CN102200988B - There is the social networking system of recommendation - Google Patents

Abstract

The invention discloses a kind of social networking system with recommendation.Social networks application can identify the image between the image collection and the image collection of the second user of first user with common link.This is jointly linked and can be identified by the metadata of image or other similar portions.Use first user image collection, can identify element interested and it is compared to the image collection of the second user searching mate.Upon finding the match, result can be selected from result group to show different set of matches.Can present to user and select the image of coupling and be added to the set of user and browse the option of the more images mating one or more groups.

Description

There is the social networking system of recommendation

Technical field

The present invention relates to social networking system, particularly relate to the images match in social networking system and mark.

Background technology

Social networks is to allow user to share individual and the application of specialized information with friend.In many cases, Social networks can allow user's post photos, video and other guide in case with its friend or the network of colleague Share.

Summary of the invention

Social networks application can identify between the image collection and the image collection of the second user of first user There is the image of common link.This is jointly linked and can be marked by the metadata of image or other similar portions Know.Use the image collection of first user, element interested can be identified and by itself and the figure of the second user Image set closes and is compared to find coupling.Upon finding the match, result can be selected from result group to show not Same set of matches.Can present to user and select the image of coupling and be added to the set of user and clear Look at the option of more images of one or more groups of coupling.

There is provided this general introduction in order to introduce in simplified form will be described in detail below in further describe Some concepts.This general introduction is not intended as identifying key feature or the essential feature of theme required for protection, It is intended to be used to limit the scope of theme required for protection.

Accompanying drawing explanation

In the accompanying drawings,

Fig. 1 is the diagram of the embodiment illustrating the system with social networks and image matching system.

Fig. 2 is the diagram of the example embodiment illustrating example image.

Fig. 3 is the flow process diagram of the embodiment of the method illustrating the ranking for determining people from image.

Fig. 4 is the flow chart illustrating the embodiment for analyzing the method finding coupling image based on face Show.

Fig. 5 is the flow process diagram of the embodiment of the method illustrating the pretreatment for face's analysis.

Fig. 6 is the flow process diagram of the embodiment illustrating the method for arranging threshold value by training set.

Fig. 7 is the flow process diagram of the embodiment illustrating the method for event matches.

Fig. 8 is to illustrate for using event matches to find the flow process of the embodiment of the method for the image of friend Diagram.

Fig. 9 is the method for the image illustrating the event attended for using event matches to find about user Embodiment flow process diagram.

Figure 10 is the diagram of the example embodiment of the user interface illustrating the output with event matches.

Figure 11 is the flow process diagram of the embodiment illustrating the method for creating cluster.

Figure 12 is to illustrate the flow process diagram for using the embodiment clustering the method mating image.

Detailed description of the invention

Social networks applies image that may be relevant to first user in the image collection that can find other users. Image can be found by the cluster creating the image in the image collection representing first user.Implement at some In example, on some orthogonal axles, image can be clustered, and on each axle, analyze each figure independently Picture.Comparison between one image and cluster can be by calculating this image and the barycenter of cluster and with permissible " distance " that be closest between the image of neighbours performs.

The size of cluster is used as the general tolerance of the relative importance of each cluster.When presenting one to user During group image result, can would indicate that the image mated with each cluster is included in user circle according to cluster size In face.

This specification in the whole text in, in the description of all accompanying drawings, identical reference represents identical element.

When being referred to as being " connected " or " coupled " by element, these elements can be directly connected to or be coupling in Together, or one or more neutral element can also be there is.On the contrary, it is being referred to as " directly being connected by element Connect " or time " direct-coupling ", there is not neutral element.

The present invention can be embodied in equipment, system, method and/or computer program.Therefore, The present invention partly or entirely can with hardware and/or software (include firmware, resident software, microcode, state machine, Gate array etc.) embody.Include for instruction execution system additionally, the present invention can use on it Or the computer being used in combination with can use or the computer of computer readable program code can use or calculate The form of the computer program on machine readable storage medium storing program for executing.In the context of this article, computer can make With or computer-readable medium can be can to comprise, store, communicate, propagate or transmission procedure is for instruction Any medium that execution system, device or equipment use or is used in combination with.

Computer can use or computer-readable medium it may be that such as, but not limited to, electricity, magnetic, light, Electromagnetism, infrared or semiconductor system, device, equipment or propagation medium.Unrestricted as example, meter Calculation machine computer-readable recording medium can include computer-readable storage medium and communication media.

Computer-readable storage medium includes for storing such as computer-readable instruction, data structure, program mould Volatibility that block or any means of the such information of other data or technology realize and non-volatile, can move Move and irremovable medium.Computer-readable storage medium include, but not limited to RAM, ROM, EEPROM, Flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, Cartridge, tape, disk storage or other magnetic storage apparatus, maybe can be used for storing information needed and permissible Any other medium accessed by instruction execution system.Noting, computer can use or computer-readable medium Can be paper or other the suitable media being printed with program on it, because program can be via such as to paper Or the optical scanning of other media and capture electronically, be compiled the most if necessary, explain, or with it He processes suitable mode, and is subsequently stored in computer storage.

Communication media generally embodies calculating with modulated message signal such as such as carrier wave or other transmission mechanisms Machine instructions, data structure, program module or other data, and include any information-delivery media.Art Language " modulated message signal " can be defined as one or more feature to encode information in the signal The signal that mode is set or changes.Unrestricted as example, communication media includes wire medium, if any Gauze network or directly line connect, and such as acoustics, radio frequency (RF), infrared ray and other wireless mediums it The wireless medium of class.Above-mentioned combination in any also should be included in the range of computer-readable medium.

When the present invention embodies in the general context of computer executable instructions, this embodiment can be wrapped Include the program module performed by one or more systems, computer or other equipment.It is said that in general, program Module include perform particular task or realize the routine of particular abstract data type, program, object, assembly, Data structure etc..Generally, the function of program module the most on-demand can be combined or divide Cloth.

Fig. 1 is the diagram of an embodiment 100, and it illustrates the client-server assembly for social networks. Embodiment 100 is to include client devices and the network environment of social networking service accessed by network Simplify example.

The diagram of Fig. 1 illustrates each functional unit of system.In some cases, assembly can be hardware group The combination of part, component software or hardware and software.Some assembly can be application layer software, and other groups Part can be operating system layer assembly.In some cases, an assembly is permissible to the connection of another assembly Being compact siro spinning technology, two of which or more assembly operate on single hardware platform.In other cases, Connection can be carried out by the network connection of span length's distance.Each embodiment can use different hardware, soft Part and interconnection architecture realize described function.

Embodiment 100 is shown in which that user can have an example of the social networks of image collection.This society Handing over network can be web application, and wherein each user can set up account and can be in society in social networks Hand over management image collection in network.In social networks base structure, the service of operation can be analyzed and comparison diagram Image set closes.

The social networks of embodiment 100 can be the explicitly or implicitly relation that can exist the most between users Any kind of social networks.In certain social networks, relation can by user formally with another User's opening relationships is expressed.Some social networks can set up unidirectional relationship by the statement of this relation, and Other social networkies can the opening relationships when two users agree on relation.

Some social networks can have informal relationship between users.Such as, informal relationship can be two Individual user exchanges email message, or sets up when user uses another mechanism to communicate.Such as, society Handing over network can be that the user of communication sets up in chatroom, instant message transrecieving service or other mechanism.? In some cases, people contacts list in e-mail system or mobile phone be used as Set up the implication relation of social network relationships purpose.

In certain social networks, how the image in user may determine that its image collection can be shared.? In some cases, user can select can be shared to the image of the friend that it exists relation.In other feelings Under condition, user can permit any user of shared image.

Social networks can be that each of which user creatable account is to access the formal social network of social networks Network.In these type of embodiments many, user can access social networks, and social network by web browser Network can be web application.In these type of embodiments many, user can upload image in social network environment Create image collection.

In the more informal version of social networks, user can be on a personal computer or by user People ground controls or stores in the storage vault of management and manage image collection.In this social networks, Yong Huke Mark therefrom can share each storage position of image with other people.In some this type of social networks, society Hand over cyberrelationship base structure can be used to safeguard, this base structure can be only address exchange, forum, Or other mechanism that member can be used for being connected to each other.

Client devices 102 can have one group of nextport hardware component NextPort 104 and component software 106.Client devices 102 Any kind of equipment that can communicate can be represented with social networking service 136.

Nextport hardware component NextPort 104 can represent the typical architecture of calculating equipment, such as desk-top or server computer.At certain In a little embodiments, client devices 102 can be personal computer, game console, the network equipment, friendship Formula self-service terminal (kiosk) or other equipment mutually.Client devices 102 can also be portable setting Standby, as laptop computer, netbook computer, personal digital assistant, mobile phone or other move and set Standby.

Nextport hardware component NextPort 104 can include processor 108, random access memory 110 and non-volatile Storage 112.Nextport hardware component NextPort 104 may also include one or more network interface 114 and user interface facilities 116. In many cases, client devices 102 can include photographing unit 118 or the scanner 120 that can catch image, This image can become a part for the image collection of user.

The various application such as component software 106 can include operating system 112, such as web browser 124 can be Perform in operating system.In many social networks application, web browser 124 can be used for and social networks Service 136 communication accesses social networks application.In other embodiments, the client application of specialization can User interface is provided with social networking service communication.In some this type of embodiment, this client application Can perform can be in the many functions described in social networking service 136.

Client devices 102 can have local image library 126, and this this locality image library can include from such as taking a picture Machine 118, scanner 120 maybe can have the figure that many not homologies such as other equipment of image capture capabilities are collected Picture.Local image library 126 can include the image being stored on other equipment, as in being stored in LAN or cloud On server in storage service.

Client devices 102 can have that user can be allowed to check and manage the some of local image library 126 should With.The example of this type of application can be image editor 130 and image viewer 132.In some cases, Client devices can have this type of application some.

Local image library 126 can include rest image and video image.In certain embodiments, rest image Can be stored in different storehouses with video image, and can access by different application, edit and handle.

In certain embodiments, client devices 102 can have image pre-processor 128.Image semantic classification Device can analyze before image is associated with social networks picture material and be associated with image each Plant metadata.Pretreatment can to client computer can image perform face image analysis, context analyzer, color Color rectangular histogram or other analyses.In other embodiments, part performed by image pre-processor 128 or Repertoire can be performed by social networking service 136.When image pre-processor 128 is positioned at client devices Time on 102, server apparatus can be from performing unloading this generic operation.

Client devices 102 can be connected to social networking service 136 by network 135.In some embodiment In, network 134 can be the wide area networks such as such as the Internet.In certain embodiments, network 134 can include The LAN of wide area network can be connected to by gateway or other equipment.

In certain embodiments, client devices 102 can be such as by hardwireds such as such as Ethernet connections It is connected to network 134.The highest, client devices 102 can pass through such as honeybee The wireless connections such as cellular telephone connection or other wireless connections are connected to network 134.

Each user of social networks can use various client devices 138 to connect.

Social networking service 136 can operate on hardware platform 140.Hardware platform 140 can be to have It is similar to the individual server equipment of the hardware platform of the nextport hardware component NextPort 104 of client devices 102.At some In embodiment, hardware platform 140 can be the virtualized or base of operation on two or more hardware devices Hardware platform in cloud.In certain embodiments, hardware platform can be wherein to use thousands of meter Calculation machine hardware platform large data center.

In certain embodiments, social networking service 136 can operate in operating system 142.Have In the embodiment performing environment based on cloud, the concept of single operating system 142 may not exist.

Social networks 144 can include multiple user account 146.Each user account 146 can include and this account The metadata 148 that family is relevant, and the relation 150 can set up between two or more users.

User account metadata 148 can include the information about user, as the name of user, home address, Position and the hobby of user and detest, education and other relevant informations.It is right that some social networks can have Emphasizing of work related information, this can include as duty history, professional association or other work related informations etc. Project.Other social networkies can emphasize friend and family relation, wherein can emphasize individual event.In some society Hand in network, it may include larger numbers of individual's metadata 148, and other social networkies can have considerably less The individual metadata 148 of amount.

One user account can be associated with another by relation 150.In certain embodiments, relation is permissible Being unidirectional relationship, wherein first user can share information with the second user but the second user possibly cannot reply And may not share information with first user or share limited amount information.In other embodiments, relation can To be two way relation, each of which user agrees to share information each other.

In further embodiments, user can allow its part or all of information to be shared to anyone, wraps Including is not the people of social network members.Some this type of embodiment can allow ID can be shared to anyone Information subset, and the subset can shared with other members of social networks.Some embodiment can allow to use Family defines different from the social network members groups of subsets shared.

Each user account 146 can include one or more image collection 152.Image collection 152 can include Image 154.Each image 154 can include metadata 156, and metadata can be such as timestamp, position The general metadata such as information, image size, title and various labels.Label can include wanting about image The identifier of relative different social network members.

In certain embodiments, image metadata 156 can comprise the metadata derived from picture material. Such as, face can be performed analyze any face in identification image and create face and represent or face's vector. Face represents and can be used for such as comparing with other images.Can be used for deriving other picture materials of metadata Can include background area or individual's texture analysis of dress ornament, whole image or the color histograms of image each several part Figure or other analyses.

Image metadata 156 can be used for creating cluster 158.Cluster 158 can be image or from image The packet of element.Such as, face can be analyzed represent to identify and can comprise the cluster that similar face represents.Similar Ground, can create cluster by the image analysis result of the background area from image is carried out packet.

In certain embodiments, cluster 158 can create by being grouped image based on metadata. Such as, in certain time period, some images of shooting can be grouped together being used as a cluster, or A cluster can be formed with the identical tagged image of tag parameter.The example using cluster can be in this theory The embodiment 1100 and 1200 that bright book proposes after a while finds.

In certain embodiments, social networking service 136 can include analyzing the images to deduced image metadata Image pre-processor 160.Image pre-processor 160 can be used for wherein client devices 102 and is likely not to have Image pre-processor 128 or when Image semantic classification be not situation about performing before analysis.Pre-treatment step Shown in the embodiment 500 that example can propose after a while in this specification.

Relatively engine 162 can use image analysis technology or metadata analysis to compare two or more figures As to determine cluster 158.The relatively example of the operation of engine 162 can propose after a while in this specification The each several part of embodiment 400 finds.

Rank engine 164 can compare each cluster 158 to extract information, as to image or be attached to image The ranking of information or importance.The example of the operation of rank engine 164 can propose after a while in this specification Embodiment 300 in find.

Analysis engine 166 can analyze the coupling between also movement images collection incompatible identification image set.Analyze Engine 166 can use metadata analysis and analysis of image content to identify coupling.

In many examples, social networking service 136 can operate together with web services 168, web Service 168 can be with the browser operated on a client device or other application communications.Web services 168 Can receive the request of HTML (Hypertext Markup Language) (HTTP) form, and with webpage or other defer to HTTP Response respond.In certain embodiments, web services 168 can have application programming interface (API), By this API, the application on client devices can be mutual with social networking service.

Fig. 2 is the diagram of an example embodiment 200, and it illustrates two figures can analyzed by graphical analysis Picture.Embodiment 200 illustrates two images 202,204, and the two image respectively illustrates birthday party and sail Ship is travelled.These images can represent the example image that can find in the image collection of user.

Image 202 can represent the birthday party with two people.From image 202, Liang Ge face can be identified 206 and 208.Some different face recognition mechanism or algorithm can be used to identify face 206 and 208.

Once identifying, face 206 and 208 just can be processed to create the expression of face.This expression can be Can allow to carry out different faces each other numeric ratio compared with face's vector or other represent.

In certain embodiments, other graphical analysis can be performed.Such as, can be by respectively from face 206 With determine in 208 geometrical relationship and catch may be relevant with the dress ornament that corresponding people wears in image part mark Know dress ornament region 210 and 212.

The graphical analysis of dress ornament can be used for comparing two images to determine whether these images are in same event Shooting.When two images comprise similar face and these images additionally comprise similar dress ornament texture or color During rectangular histogram, it can be deduced that this conclusion.This analysis can assume that the same event of graphical representation, because figure People in Xiang wears identical clothing.

It addition, texture analysis, color histogram or other analyses can be carried out analysis background region 214.This Similarity that a little results can be compared to other images to determine between image and mating.

In image 204, can identify and catch face 216 and 218.Because face 216 and 218 Size may be relatively small, and therefore the dress ornament region of the people of image 204 may be not carried out, but can identify And analysis background region 220.

Fig. 3 is the flow chart of the embodiment 300 of the method illustrating the ranking for determining people from image collection Show.Embodiment 300 is can be by comparing engine and rank engine, such as comparison engine 162 He of embodiment 100 The example of the method that rank engine 164 performs.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

Embodiment 300 can be that the face of individual occurrence number in the image collection of user is used as using Family is to the interest of this individual or this individual approximation to the importance of user.

Face in image can analyzed, compare and be grouped into cluster together.Size based on cluster, can be right Ranking is carried out with the individual of cluster correlation connection.

At frame 302, image collection can be received.Can be with this image set of pretreatment incompatible mark face and face's table Show.Shown in the embodiment 500 that the example of this preprocess method can propose after a while in this specification.

In block 304, each image can be processed.For each image in frame 304, if at frame There is not face in 306, then this process can return to frame 304 to process next image.If at frame 306 In one or more faces occur in the picture, then can in frame 308 each face of individual processing.For frame Each face in 308, can add face's object and associated image benchmark to list in a block 310. Graphic based could be for therefrom obtaining the pointer of the image of this face or other indicators.

After all images in having processed frame 304, can be to gained list ordering in frame 312.

At frame 314, this list can be analyzed and come in block 314 based on threshold value identified cluster.Cluster can define The one group face relevant to single people represents.

Determine that a mechanism of cluster can be to represent face to be considered vector.Between any two vector Similarity can be considered as the distance in vector space.The many of same person is reflected when multiple faces represent During different images, then face represents that vector can create vector clusters.

In many examples, can use threshold value be used as determining given face indicate whether " close " another Face represents to become a part for the mechanism of coupling.Threshold value can determine with some different modes, and One such mode can be shown in embodiment 600.

In frame 316, each cluster can be analyzed.For each cluster in frame 316, if at frame In 318, any member of this cluster does not has label or other metadata being associated, then this process can return to Frame 316 processes another cluster.

If one or more members of the cluster in frame 318 comprise label or other metadata, then can be at frame In 320, these labels are applied to other cluster members.In some cases, can frame 322 to user in Existing user interface facilities, wherein user can ratify or disapprove label.If the user while approval mark in frame 324 Sign, then in frame 326, label can be applied to all members of this cluster.If the user while in frame 324 not Approval label, then be not applied to each member in frame 328 by label.

In many social networks application, image can be tagged by user with the identifier of such as particular person. The process of frame 316 to 328 can represent the method that this type of label can be automatically applied to other images.At some In embodiment, the label being applied to cluster member can be the label that people denotable to this cluster is relevant.One Individual simple example can be the label of the name defining this people.

Cluster can be analyzed in frame 330 come according to size cluster ranking.Ranking can reflect people for The relative importance at family.Cluster ranking can be used in frame 332 in various applications people to be distinguished preferentially Level.

Such as, news sources can include that message, state update or relevant to the people in the social networks of user Other information.Those projects relevant to important people can highlighted or with catch user's attention side Formula presents.Sundry item about the people in the image collection infrequently occurring in user can be with secondary Or the non-mode emphasized presents.

Fig. 4 is the flow process illustrating the embodiment 400 for analyzing the method finding coupling image based on face Diagram.Embodiment 400 is can be compared, by the analysis engine 166 etc. of such as embodiment 100, the side that engine performs One example of method.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

Embodiment 400 shows and can be compared by image and the first image collection from the second image collection Example compared with the method identifying the image comprising the people identical with the first image collection in the second image collection.

At frame 402, the second image collection can be received.At frame 404, can pretreatment the second image collection.With In shown in the embodiment 500 that an example of the method for pretreatment can propose after a while in this specification.

At frame 406, each image in the second image collection can be processed.For each figure in frame 406 Picture, if the most not finding face, then this process can return to frame 406 to process next image.

If finding face at frame 408, then can process each face object at frame 410.For frame 410 In each face object, can cluster with the first image collection be compared to find the most in block 412 Close coupling.If being unsatisfactory for threshold value in this coupling of frame 414, then this process can return to frame 410 place Manage next face's object.If mating in threshold value at frame 414, then at frame 416, this image is associated with this Cluster.

After all images in having processed frame 406, result can be from the second image collection, Join the list of the image of cluster in the first image collection.In frame 418, can come this list according to ranking Sorting and be presented to user, this ranking can determine during embodiment 300.

Fig. 5 is the flow process diagram of the embodiment 500 of the method illustrating the pretreatment for face's analysis.Implement Example 500 is can be taken by the image pre-processor 128 of the client computer 102 of such as embodiment 100 or social networks The example of the method that preprocessor 160 image pre-processor such as grade of business 136 performs.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

The pretreatment of embodiment 500 can be to all image identification faces in image collection and create face Vector or certain other numeric representations of face image.

Image file can be received at frame 502, and this image file can be scanned to identify at frame 504 There is face.

If finding face at frame 506, then can be with each face of individual processing at frame 508.For frame 508 In each face, can be by image cropping to this face in frame 510, and can be from sanction at frame 512 The image creation face object cut.Can create face's vector at frame 514, this face's vector can be face The numeric representation of image.At frame 516, vector sum face of face object can be deposited as the metadata of image Storage.

After having processed all faces in frame 508, if having another image to use at frame 518, then this mistake Journey can loop back to frame 502, and otherwise this process stops in frame 520.

Fig. 6 is the flow chart of the embodiment 600 illustrating the method for arranging threshold value with training image collection Show.Embodiment 600 be can from the friend of user collect example image and use these example image to arrange can Minimize the example of the method for the false threshold value certainly compared.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

Embodiment 600 can determine that a threshold value is arranged, and can minimize when this threshold value is arranged on movement images set Vacation is compared certainly.In many social networks application, relatively high confidence threshold can have for the most just minimizing Really identify the probability of coupling.When the image collection from the second user selects photo or next of video image When joining the image collection of first user, incorrect coupling may give the user the low confidence of matching process Degree.But, the coupling of omission, i.e. coupling exist but threshold value does not allow this coupling to be detected, may not The confidence level of user can there be is the biggest infringement.

The process of embodiment 600 is used from the image collection collection presentation graphics of the friend of user and is acted on Training set relatively.Face compares can ethnic groups based on those people being associated with user, the colour of skin and other things Manage characteristic and have any different.Selected image can come from the friends of friends of user, and can reflect the figure of user The possible physical characteristic of the people in image set conjunction.

The process of embodiment 600 can attempt to remove from training set may appointing in the image collection of user Who.This can be by checking that any label being associated with the image of friend is to guarantee that this label does not mate use The friend at family performs.

At frame 602, the friend of user can be identified.The friend of user can from the relation in social networks and Any other source determines.In some cases, user can belong to some social networkies, and each social networks has There is a different set of relation.In such cases, those relations are considered as much as possible.

At frame 604, each friend of user can be processed.For each friend in frame 604, at frame 606 Process each image in the image collection of this friend.For each image in frame 606, can at frame 608 The label that mark is associated with this image.If be associated with the friend of user, then at frame at frame 610 label 610 do not consider this image.By getting rid of the friend of user at frame 610, this training set may not include possibility The image of the coupling to user, but can include having to may people in the image collection of user similar The image of people of characteristic.

If may not be relevant to user at frame 610 label instruction image, then this image be selected at frame 612 For training set.In many cases, the image selected for training set can be in the image collection of friend The subset of all images.Such as, during a process can select every 100 or 1000 candidate images It is used as a part for training set.In certain embodiments, training set can be made and randomly choosing.

Have selected in frame 604 to 612 will be after the image in training set, can be to this instruction at frame 614 Practice collection and perform face's pretreatment.This pretreatment can be similar to the pretreatment of embodiment 500.

Matching threshold can be used as default at frame 616.

At frame 618, each image of image collection of user can be processed to arrange threshold value so that user's In image collection, neither one image mates with training set.For each image in frame 618, if at frame 620 these images do not comprise face, then this process returns to frame 618.

When image comprises face in frame 620, frame 622 can process each face.For frame 622 In each face, can be compared to look for the face's object in training set by this face's object at frame 624 To most like face's object.If similarity is less than threshold value in frame 626, then this process can return to frame 622.If similarity is more than threshold value in frame 626, then in frame 628, adjust threshold value so that this threshold value Less than the similarity in frame 628.

After having processed all images in the image collection of user in frame 618, can store in frame 630 Present threshold value is also used for follow-up comparison.

Fig. 7 is the flow process diagram of the embodiment 700 illustrating the method for event matches.Embodiment 700 It is can be shown by a simplification of the method for the execution such as analysis engine 166 analysis engine such as grade of such as embodiment 100 Example.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

Embodiment 700 is the example that can be used for detecting the method for event from metadata.Metadata can be from Image, such as the metadata analyzed from face or derive in other graphical analyses.Metadata can also be not from The metadata that image is derived, such as title, timestamp or positional information.

Embodiment 700 can infer event from the common factor of the image collection of two users.This common factor is permissible Occur to occur when two users attend same event and have taken the image of this event.Such as, two User can attend birthday party or family party, and have taken the photo of the family having a dinner party.In another example, Two users can attend a meeting, competitive sports or other public accidents, and can shoot the image of this rally. In some cases, user may learn that and attends event each other, and in other cases, user may Do not know that another person attends.

At frame 702, image collection can be received from first user.At frame 704, figure can be received from the second user Image set closes.In certain embodiments, the information received can be only the unit relevant to the image in set Data, and be not real image itself.

Coupling is found in the comparable metadata from each image collection of frame 706.Coupling can be based on image Analyze, as found the face of coupling in the image from two different sets.Coupling can be divided based on metadata Analysis, as found the image with the timestamp of coupling, label, positional information or other metadata.

In many cases, coupling can with a certain tolerance or bias levels come in decision block 706 mark Join and can have a large amount of deviation or tolerance, thus each coupling can be assessed in later step further.Frame 706 In coupling can be rough or preliminary coupling, this is rough or preliminary matches can be further refined and mark Know and there is the biggest deterministic coupling.

The result of frame 706 can be from a pair image of each set.In some cases, result is permissible From each set, one group of image of shared similar metadata.

At frame 708, the image of each group of coupling can be compared.Figure for each group of coupling in frame 708 Picture, may compare metadata in block 710 to determine whether deducibility event.

Event can be based on several factors which infer.Some factor can highly be weighted, and other factors are permissible There is secondary characteristic.Whether indicate the judgement of event that various exploration or formula can be used to determine coupling, And this type of is soundd out or formula can be depending on embodiment.Such as, some embodiment can have a large amount of metadata can With, and other embodiments can have less metadata parameters.Some embodiment can have the figure of complexity As analyzing, and other way of example can have more uncomplicated or even without graphical analysis.

The factor highly weighted can be in one of image of this second user of the second ID wherein In the case of first user.This type of metadata has specifically identified the link between two image collections, and refers to Show that two users may be in the same time in same place.

In certain embodiments, user can be that the image during it is gathered with the people from its social networks adds Label.In this type of embodiment, user can manually select an image and create the friend's identified in this image Label.Some this type of embodiment can allow user to point to face the position appending tags on image.This Class label can be considered as reliable indicator, and is given weight more higher than other metadata.

Other height weighting factors can be on room and time closely.Timestamp closely May indicate that with physical location information two users are once at identical when and where.In certain embodiments, Point that image can include shooting this image and when shooting this image photographing unit towards direction.When this type of When metadata can use, the overlap in the region of two image coverings can be the evidence of event.

Some image can tag with the various descriptors manually added by user.Such as, image is permissible Tag with " birthday party of Anna " or " technical conference ".When the figure from two image collections During as being coupled with similar label, label can be the good indicator of event.

Graphical analysis can be used to analyze mate to identify common event.Such as, the image in two set Between face image coupling can be the good indicator of event that two users attend and catch.Face schemes As coupling can be by similar background image region and by dividing the dress ornament of the people being associated with the face of coupling Analysis further confirms that.

When identifying common event, different situations and different embodiment can use different groups of each factor Close.Such as, in some cases, event can determine separately through graphical analysis, even at unit's number According to time incoherent.Such as, user has been likely to purchase camera apparatus and may the most correctly Time and date in photographing unit is set, or may be the time zone different from another user by set of time. In this case, timestamp metadata is probably incorrect, but graphical analysis can identify common event.

In another example, though graphical analysis possibly cannot identify any common face, background or other Similarity, metadata also can identify common event.

Different embodiments can have the different threshold values for identified event.In the typical society to embodiment 700 Hand in Web vector graphic, can perform to analyze based on event from trend image application label.In this embodiment, The definitiveness of higher degree is probably desirable so that incorrect label will not introduce as noise In image collection.In another kind of purposes, coupling can be used for identifying Possible event, and user can manual examination (check) Possible event determines that event the most once occurred.In this purposes, determine the threshold of event Value can have than in other service conditions much lower qualitative extent really.

If not determining event in frame 712, then this process can return to frame 708 to process another coupling.

If mark is got over part in frame 712, then can identify be associated with this event all in frame 714 Image.To this event definition metadata tag, and this label can be answered in frame 718 in block 716 For image.

The image being associated to event can by mark or shared common metadata relevant with the image mated or The image of other features determines.Such as, can mate two images, each image is from an image collection. Once have matched these images, the image of coupling any phase in its respective set can be identified at frame 714 Close image.

By scanning associated picture, metadata tag in frame 716 can determine that whether event tag is with arbitrary Individual associated picture is associated and generates.Such as, the available such as " Anna of one of image collected in frame 714 Birthday " etc. event tag tag.At frame 718, then this label can be applied to all relevant figures Picture.

In certain embodiments, the event tag of frame 716 can be that can to identify coupling be how to determine automatic The event tag generated.Such as, the coupling determined by having the common metadata of time and positional information can There is the label including " Jerusalem, on February 22nd, 2010 ".Each embodiment can have for really The different mechanisms that calibration is signed.

In certain embodiments, in frame 718, the label of application may be invisible to user.This label can be by Social networks is used for different images set together to provide the search strengthened or browse ability, and not Show that label is for checking or revising to user.

Fig. 8 is to illustrate the event matches between the image collection and the image collection of user friend of user Method embodiment 800 flow process diagram.Embodiment 800 is the event matches described in embodiment 700 One use scene of method.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

The image collection of the image collection of user with user friend is compared by embodiment 800.This compares can The event shared by two users of mark, and first user in the image collection of friend can be identified may wish to Add the image of his or her image collection to.

It is powerful that embodiment 800 could be for being linked together by two image collections in social networks Instrument.In some purposes, two users may know that they have attended same event and may want to each other Share their image.In other purposes, user may not remember attend same event or may not recognize Know to two people the most there.The method of embodiment 800 can by identify its life in common factor and allow They carry out shared events by its image and strengthen the mutual of user.

At frame 802, the image collection of user can be received.At frame 804, the friend of user can be identified, and And each friend can be processed at frame 806.For each friend in frame 806, can be at this at frame 808 Perform event matches between user and the friend of user and identify common event.Event matches can be by embodiment Similar mode described in 700 performs.

At frame 810, can be with each new events found in analysis block 808.For in frame 810 each newly Event, can select to mate the image of this event in frame 812 from the image collection of friend.At frame 814, Any metadata since the image selected by the image collection of friend can be identified, and answered at frame 816 Image for the user relevant to event.

Label and other metadata can be traveled to use by the operation of frame 814 and 816 from the image collection of friend The image collection at family.In certain embodiments, user's approval can be given or disapprove tagged option.Mark Sign and other metadata can enrich the image set of user by automatically or semi-automatically applying useful label Close.

At frame 818, the image of friend can be presented to user, and can image is grouped by event. Shown in the embodiment 1000 that the example of user interface can propose after a while in this specification.

After having processed each event in frame 810, at frame 820, user may browse through the image of friend also Select one or more images of friend.At frame 822 to, selected image can be added the image collection of user.

Fig. 9 be illustrate for user friend between the flow process of embodiment 900 of method of event matches Diagram.Embodiment 900 is a use scene of the event matches method described in embodiment 700.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

In the image collection of the friend that embodiment 900 compares user two identify can be from the two of user The event that friend infers.Image from the event inferred can be presented to user and user can be by these Image adds the image collection of user to.

Embodiment 900 is probably useful in social networks scene, and wherein user may attend or do not attend Event and may want to checks the image of this event can some add user's to by certain in these images Image collection.Such as, it is impossible to the grand parents of the party attending grandson generation may want to see the image of this party. This party can be by analyzing the incompatible deduction of image set from two or more people attending this party.Pass through From the analysis to image collection, infer event, all associated pictures of this event can be collected and be presented to Grand parents appreciates for them.

Embodiment 900 operates in the way of similar to embodiment 800, but for the image set of event matches Close the set of the friend that can be from user to rather than user gather and the set of his or her friend Compare.

At frame 902, the friend of user can be identified and be placed in list.Friend can pass through social network Network identifies.At frame 904, each friend can be processed.For each friend in frame 904, at frame 906 The each remaining friend in list of friends can be analyzed.Remaining friend is that it not yet processes image collection Those friends.For each remaining friend in frame 906, can be two friends' in frame 908 Perform event matches process between image collection and identify common event.The process of frame 904 and 906 can be pacified Line up so that every a pair friend can be processed to identify common event.

At frame 910, each common event can be processed.For each common event in frame 910, some Embodiment can include that the checking in frame 912 is to determine whether this user may be on the scene.

The checking of frame 912 can be used for preventing from illustrating the event not inviting user.Such as, two friends of user Can get together an evening of seeking pleasure, but user may not invited.For preventing user from being offended, certain A little embodiments can include that the checking of such as frame 912 prevents from the user discover that event occurs.In other embodiments In, such as the example for above-mentioned grand parents, the checking that maybe can ignore frame 912 can not be included.

In certain social networks, user may be able to select whether will with other user's shared events, and Its common event and which user cannot which user may can be selected to check.

At frame 914, the image of the image collection from friend can be selected from common event and at frame 916 Middle it is presented to user according to event packets.After having processed all common event in frame 910, Frame 918, user is browsable and selects image, and selected image can add to the collection of user at frame 920 Close.

Figure 10 is the example embodiment 1000 illustrating the user interface with the result from event matches analysis Diagram.Embodiment 1000 is the event matches that can be used for presenting such as embodiment 800 or 900 to user One simplification example of the user interface of the result that the event matches such as analysis are analyzed.

User interface 1002 can show the result of event matches process.In user interface 1002, it is shown that Result from three events.Event 1004 can have label " birthday party ", and event 1006 can have Having label " sandy beach holiday ", event 1008 can have label " ski vacation ".Can be from definition from friend Image collection label in identify various label.In some cases, label can be detected from coupling Event user image in determine.

Each event can present together with the source of image.Such as, event 1004 can have " from mother Set with Joe " image source 1010.Event 1006 can have the image of " from the set of Joe " Source 1012, and event 1008 can have the image source 1014 of " from the set of Lora ".Image source can The user's labelling about the friend of user is used to create.

User interface 1002 may also include the various metadata about event.Such as, event 1004 is permissible Which present together with the metadata 1016 being determined to be in this event with the friend of instruction user.Similarly, Event 1006 and 1008 can be respectively provided with metadata 1018 and 1020.

Each event can have the selected works of presented image.Event 1004 and image 1022,1024 and 1026 illustrate together.Event 1006 illustrates together with image 1028 and 1030, event 1008 and image 1032 illustrate together.Each image side can be the image collection that user can be used for selecting user to be added to The button of one or more images or other mechanism.

The user interface of embodiment 1000 only can serve as the result that the images match such as such as event matches are analyzed Present to an example of some assembly of user.User interface can be that user can be used for browsing the matching analysis Result and to result perform operation mechanism.

Figure 11 is to illustrate for creating the flow process that can be used for mating the embodiment 1100 of the method for the cluster of image Diagram.Event 1100 is the one that image packet can create cluster by analyzing the merging of single image collection The simplification example of method.Cluster can use in image comparative analysis and metadata comparative analysis.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

Embodiment 1100 can illustrate the method for simplifying for creating image clustering.Cluster can be to share With one group of image of feature, and can be to have when to face's packet and image being grouped as entirety ?.

Cluster by the vector of mark representative image and can create by being grouped together by vector.Cluster can There is barycenter and radius, and numeric ratio can be made between image and cluster and relatively determine image and cluster Between " distance " to determine coupling.

At frame 1102, can receive image collection, and at frame 1104, that can analyze in image collection is each Image.Use face recognition embodiment in, image can from bigger image cropping, can only wrap Face's object of the face feature containing people.In this type of embodiment, this analysis can create and represent face's object Vector.In other embodiments, whole creation of image image vector can be analyzed.

At frame 1106, can analyze the images to create image vector.Image vector can comprise each yuan of image The numeric representation of element, including face image analysis, dress ornament analysis, background image analysis and texture analysis.

In certain embodiments, the analysis of frame 1106 can create some image vectors.Such as, there are two The image of face can with represent face two image vectors, represent two people dress ornament two images to The one or more vectors measuring and representing the various textures in background image or image represent.

After analyzing each image in frame 1104, image can be grouped together in frame 1108.Point Group can use metadata groupings and graphical analysis packet.A kind of mechanism for packet can be for every unitary Data category or graphical analysis type, be grouped together image on independent or orthogonal packet axle.Such as, Can be that a packet axle is set up in face image analysis.On this axle, all face images can be represented or to Amount packet.Individually, each image can be grouped according to different metadata such as such as timestamp or position etc..

In each axle, can be with identified cluster at frame 1110.The definition of cluster can use can be by cluster limit The threshold value of the strict packet making image controls.Cluster can be used for determining that degree represents the reality of image with height Border is mated so that such as image compares and can have height with other operations such as ranking and determine degree.

On it, each axle to image packet can have the different threshold values for identified cluster.Such as, face's figure As coupling can have relatively stringent threshold value so that the coupling only with the highest similarity degree could be recognized For being cluster.On the contrary, analyzed the image mated by background image and can have the threshold value being relatively not intended to, make Obtain and the image of wider range can be grouped.

Each cluster can have the barycenter and radius calculated in frame 1112.Barycenter and radius can be used for inciting somebody to action Other images determine when comparing with image collection and mate.At frame 1114, can store cluster and barycenter and Radius.

Figure 12 is to illustrate for using the barycenter of cluster and radius analysis to mate the embodiment of the method for image The flow process diagram of 1200.Embodiment 1200 can illustrate that the image that embodiment 1100 can be used to be analyzed is marked Know the coupling between image collection and the image collection of friend of user, then select the most suitably or most preferably to mate It is shown to a kind of method of user.

Other embodiments can use different order, additional or less step and different titles Or term realizes the function that is similar to.In some embodiments, various operations or one group of operation can be by same Step or asynchronous mode operate executed in parallel with other.These steps of selecting at this selected come to simplify Form illustrates some principles of operation.

At frame 1202, the image collection of user can be received, and at frame 1204, the image of friend can be received Set.At frame 1205, can the image collection of friend of pretreatment user.One example of pretreatment image can To be embodiment 500.The pretreatment of embodiment 500 can apply to face image analysis, and can be expanded Exhibition is pre-to background image analysis, texture analysis, color histograms map analysis, dress ornament analysis and other graphical analyses Process.

What the pretreatment of frame 1205 can correspond to perform before clustering the image collection of user appoints What is analyzed.

At frame 1206, each image in the image collection of friend can be analyzed.Each in frame 1206 Image, at frame 1208, can analyze each cluster that the image collection with user is associated.

As described in embodiment 1100, each image collection can comprise the multiple clusters in multiple normal axis. Each cluster can represent importance or the element of the image collection of user, and these aspects can be used for from The image of the image collection of friend compares.

For each cluster in frame 1208, at frame 1210, it may be determined that from the image analyzed to the most poly- The distance of class.At frame 1212, if this distance is in barycenter matching threshold, then at frame 1218, by this figure As joining with this cluster correlation.

If in this distance of frame 1212 not in barycenter matching threshold, then can determine that arest neighbors at frame 1214 The distance occupied.If in the distance of frame 1216 to nearest-neighbors not in neighbours' threshold value, it is determined that do not have Join.

Nearest-neighbors can be the image in cluster.Nearest-neighbors assessment can identify cluster outside but It it is the image of one of the image that is closely grouped together with this cluster.In an exemplary embodiments, when with matter Heart threshold ratio relatively time, neighbours' threshold value may be less.

After analyzing all images in the image collection of friend in frame 1206, the figure of optional friend As presenting to user.

At frame 1220, can be according to the size cluster ranking to user.Ranking can be used as the importance to user Representative.In frame 1222, it can be estimated that each cluster.For each cluster in frame 1222, frame 1224 In can be compared to find image immediate with neighbours by the image of coupling and cluster, and at frame 1226 In find and cluster the immediate image of barycenter.Optimal coupling is can determine that and in frame 1230 in frame 1228 It is added to user interface show.

It can be the most relevant to user and most likely matched well that the process of frame 1220 to 1230 can identify Those coupling.Dependency can be determined by the ranking of the cluster of derivation from the image collection of user. Optimal coupling can be and the barycenter of cluster is recently or very close to those images of another image, and this can be by Nearest-neighbors represents.

Images match may be susceptible to noise, and many image matching algorithms may result in false positive result, its Middle image is mated improperly.In the social networks with images match is applied, user is to matching mechanisms Satisfaction can be higher when presenting the coupling of quality to user.

The process of frame 1220 to 1230 can be from presenting to user with selecting optimal coupling coupling.This One process can be the representative coupling of each Clustering and selection one and present each coupling to user so that Yong Huneng Enough check various coupling.

After have selected image, the image according to cluster tissue can be presented to user at frame 1232.? Frame 1234, user is browsable and selects image, and at frame 1236, can add the image to user's Set.

In certain embodiments, user the degree of depth may excavate the coupling of a certain cluster to check additional Join.In this case, the process of frame 1220 to 1230 can be used for tissue and from the figure mating specific cluster As subset selects optimal image.

Description to this theme is in order at the purpose of illustration and description and proposes above.It is not intended to exhaustive Theme or this theme is limited to disclosed precise forms, and in view of other amendments of teachings above and modification are all Possible.Select and describe embodiment to explain principle and the practical application thereof of the present invention best, thus Others skilled in the art are enable at various embodiments and various to be suitable to conceived special-purpose Amendment in utilize the technology of the present invention best.Appended claims is intended to include except by prior art institute Other outside the scope of limit replace embodiment.

Claims (9)

1. a method, including:

Receive the first set (1202) including image, described first set and the first user of social networks It is associated；

Receive the second set (1204) including image, described second set and the second user of social networks It is associated；

Analyze described first set (1205) and create image clustering, described in the most each cluster representative first The face of a people in the social networks of user represents；

Based on cluster size to described cluster ranking；

Apply described cluster ranking that the people joined with described cluster correlation is distinguished priority, wherein said ranking Reflect everyone relative importance for described first user；

The similarity being determined by between each described image and each described cluster in described second set To identify cluster match (1228) from described second set；And

Described cluster match (1230) is shown to described first user.

2. the method for claim 1, it is characterised in that also include:

Present to described first user and at least one in described cluster match is added to described first set Option.

3. method as claimed in claim 2, it is characterised in that also include:

Determining the relation between described first user and described second user, described relation includes permitting described One user receives image from described second user.

4. method as claimed in claim 3, it is characterised in that described allowance include described first user and The most shared agreement between described second user.

5. a method, including:

Receiving the first set including image, described first set is associated with the first user of social networks；

Be determined by the expression vector of each described image to analyze described first set, the most each expression to Amount includes that the face of a people in the social networks of described first user represents；

Mark has the cluster of the similar image representing vector；

Based on cluster size to described cluster ranking；And

Apply described cluster ranking people's prioritized to joining, wherein said ranking with described cluster correlation Reflect everyone relative importance for described first user.

6. method as claimed in claim 5, it is characterised in that apply described cluster ranking to come with described People's prioritized of cluster correlation connection includes using to described first via news sources based on described cluster ranking Family presents the first user information being associated with the one or more people in the social networks of described first user.

7. method as claimed in claim 6, it is characterised in that described news sources includes that state updates.

8. method as claimed in claim 5, it is characterised in that also include receiving being associated with the second user Image second set, and by described second set in image compared with the cluster of described image with Identified cluster mates.

9. method as claimed in claim 8, it is characterised in that also include showing to described first user and gather Class is mated.