KR20130108125A - System and method for recommending buddies in social network - Google Patents

Abstract

PURPOSE: A system for recommending a friend of a social network and a method thereof are provided to form the social network with various people. CONSTITUTION: A data management module manages information about users composing a social network, user friend information, and photo album data uploaded by the users. A face detecting and feature extracting module extracts face feature data and detects a face by analyzing a photo provided from the outside and generates a face feature table including the extracted data. A friend recommendation computing module generates friend recommendation data based on friend information and user information in a data management module and the face feature table. A face matching module determines the matching of the face feature data received from the computing module and feeds the result back to the computing module. A friend recommendation control module provides photos uploaded by the user to the face detecting and feature extracting module, controls the face feature data to store it through the data management module, generates the friend recommendation data by controlling the friend recommendation computing module, and provides the friend recommendation information to one or more users according to the friend recommendation data. [Reference numerals] (AA) Start; (BB) End; (S210) Identify a face's features and make a face feature table of a photo album by analyzing all the photos of the photo album; (S220) Determinate and save a set of user avatar features for each user; (S230) Generate friend recommendation data by matching user avatars with each other; (S240) Generate the friend recommendation data by matching several users' face feature tables with each other

Description

System and method of recommending friends in social networks {SYSTEM AND METHOD FOR RECOMMENDING BUDDIES IN SOCIAL NETWORK}

The present invention relates to a system and method for recommending friends in a social network, and more particularly, to detect and recognize a face appearing in a picture uploaded by a user on a social network, and to match it with the avatars of all users so as not to match the current friend status but matched. A system and method for recommending users.

Network technology integrates the resources provided on the Internet into organic combinations so that resources can be fully shared and achieve organic collaboration, enabling people to transparently use the full power of resources and obtain information as needed. The emergence of social networks further expands the social relationships of people and allows more information and resources to be shared and acquired. Users can have their own subnetwork on the social network, that is, a subnetwork, which is a diagram of their relationship with their friends, so that the friend recommendation function has appeared over time.

Traditional friend recommendation features are generally based on basic personal referral information on social networks, including schools, interests, and hobbies. Friends of other users can be recommended, and users who are close to the user's current online location can be recommended.

This friend recommendation feature has some limitations. Users are not easy to find other related users in other forms. For example, if user A has a group picture and does not have information related to others in this group picture, the existing friend recommendation function cannot recommend that these people be friends of user A.

Accordingly, an object of the present invention is to detect and match a face in a picture uploaded by a user (such as a picture in an album) and recommend a system for and recommend a user to other users who are not yet friends of the user but appear in the picture uploaded by the user. To provide.

Another object of the present invention is to detect and match faces in user-uploaded photos (such as photos in albums) so that a third user may appear in the photos of two users, but if they are not friends of these two users, It is to provide a system and method for recommending friends with each other.

The object of the present invention is a friend recommendation system of a social network including a data management module, a face detection and feature extraction module, a face matching module, a user avatar determination module, a friend recommendation computing module and a friend recommendation control module, wherein the data The management module manages user information including a user ID and a user avatar feature, friend information including an ID of at least one user who is a friend, and photo album data for each user; The face detection and feature extraction module analyzes a picture provided by the friend recommendation control module, detects a face area in the picture, and extracts the face feature data from the detected face area of each user; The face matching module matches the first face feature data and the second face feature data provided by the friend recommendation computing module and feeds back a positive or negative matching result to the friend recommendation computing module; The friend recommendation control module extracts photo album data of all users from the data management module according to the received friend recommendation command, and includes the extracted facial feature data extracted from the user's photo album data for all users. Controlling the face detection and feature extraction module to extract face detection and facial feature data with respect to the extracted photo album data of each user to generate a photo album face feature table, the photo album face generated for each user Control the user avatar determination module to determine the user avatar feature of each user based on a feature table, store the determined user avatar feature in the user avatar feature item of the user information of each user, and generate the Photo albums Feature table and controls the Friends computing module to generate each user in the user information and contact suggestion data based on the friend information in the data management module; The user avatar determining module determines, for each user, the facial feature data extracted from the avatar picture selected by the user on the photo album face feature table generated for the user as the avatar feature of the user, and the determined avatar Store a feature in the user avatar feature item of the user; The friend recommendation computing module performs a first process on the photo album facial feature table for each user, the first process using the user information and friend information in the data management module through the face matching module. Matching each facial feature data of the user's photo album facial feature table with the user avatar feature of each user in turn, the face matching module feeds back a positive matching result and the matched user avatar feature is the user or user's avatar feature. Each time the friend recommendation computing module does not correspond to a friend, the friend recommendation computing module includes generating a friend recommendation data for recommending a user to the user having the matched user avatar feature and vice versa. Social network friend recommendation system Can be achieved by

The friend recommendation control module may store a photo album facial feature table through the data management module.

The friend recommendation control module counts each facial feature data in the photo album facial feature table of each user and generates the duplicate facial feature data in the photo album facial feature table of each user when generating the photo album facial feature table. If the friend recommendation computing module performs a first process on the photo album face feature table of each user, the face feature data matching the user avatar feature of a user or a user friend is displayed on the photo album face. Can be deleted from the feature table.

The friend recommendation computing module performs the first process on the photo album feature table of each user and then performs a second process on the photo album feature table of each user, wherein the second process is the face matching module. Each face feature data of the user's photo album face feature table is sequentially matched with each face feature data of the user's photo album face feature table through the feedback function, and each time the face matching module feeds back a positive matching result. The friend recommendation data may be generated for recommending a user and other matched users and vice versa.

The friend recommendation data generated by the first process and the second process includes different contents, and the friend recommendation data generated by the second process includes photo information extracted from the matched facial avatar feature data. It may include.

If the user avatar determination module determines that an avatar picture is not set on the user's photo album data, the user avatar determination module is configured to retrieve the largest number of facial feature data extracted from the user's photo album feature table. Can be used as the user avatar feature of the user.

When the friend recommendation control module receives a command for adding a photo to a user's album, the friend recommendation control module controls the data management module to add the photo to the user's photo album data, The friend recommendation computing module is configured to control the face detection and feature extraction module to extract facial feature data, add the extracted facial feature data to the user's photo album facial feature table, and generate friend recommendation data. The controller may be configured to perform the first process and the second process on the face feature data newly added to the photo album face feature table of the user.

When the friend recommendation control module receives a command for adding a friend for a designated user, the friend recommendation control module adds the user ID of the friend to the friend information of the designated user through the data management module, and the facial matching module. A user avatar feature of a friend may be sequentially matched with the face feature data on the photo album face feature table of the designated user, and the matched face feature data may be deleted from the photo album face feature table of the designated user.

The system further includes a security control module for selecting and displaying a picture on the user's photo album data for which the user approves analysis and detection, wherein the friend recommendation control module is the user's data management module in the data management module. When extracting photo album data, only the selected photo approved by the user for analysis, detection, and feature extraction may be provided to the face detection and feature extraction module.

The face detection and feature extraction module includes a template match model, a skin color model, an active appearance model (AAM), a support vector machine (SVM) model, and an Adaboost model to perform face detection. To extract the facial feature data using at least one of: template-based feature extraction, algebraic feature extraction, feature matching based on flexibility matching, feature extraction based on neural network And feature extraction based on wavelet multi-resolution, and the face matching module may match the facial feature data using a principal component analysis method.

The object of the present invention is to provide a method of recommending a friend in a social network for storing user information including a user ID and a user avatar feature, friend information including at least one user ID as a friend, and photo album data for each user. Analyzing pictures in the photo album of each user of a social network, detecting a facial area in the photo, extracting facial feature data from the facial area of a person on the photo, and extracting from the photo in the photo album Generating a photo album facial feature table comprising the facial feature data; The facial feature data extracted from the avatar picture designated by the user in the photo album feature table generated for each user is determined as the avatar feature of the user, and the determined avatar feature is the user avatar feature among the user information of the user. Storing in the item; A first process is performed on the photo album face feature table of each user, and the first process uses the stored user information and friend information of each user to retrieve each face feature data of the user's photo album face feature table. Match the user avatar feature of each user in turn, and recommend a user to the user with the matched user avatar feature whenever a positive matching result is fed back and the matched user avatar does not correspond to the user or a friend of the user. And vice versa, may be achieved by a friend recommendation method of a social network comprising the step of generating friend recommendation data for recommendation.

The photo album facial feature table may be stored in a database for each user.

When generating a photo album feature table for each user, each facial feature data in the user's photo album facial feature table is counted, and the duplicate facial feature data can be deleted from the user's photo album facial feature table. have.

When performing the first processing on the photo album face feature table of each user, the face feature data matching the user avatar feature of a user or a friend of the user may be deleted from the photo album face feature table.

After the first process is performed on the photo album feature table of each user, a second process is performed on the photo album feature table of each user, and the second process is performed in the photo album face feature table of the user. Each facial feature data is sequentially matched with each facial feature data in the photo album facial feature table of another user, and whenever the face matching module feeds back a positive matching result, the user is recommended to the other matched user. In the opposite case, it may also include generating friend recommendation data for recommendation.

The friend recommendation data generated by the first process and the second process includes different contents, and the friend recommendation data generated by the second process includes photo information extracted from the matched facial avatar feature data. It may include.

If it is determined that there is no avatar picture specified on the user's photo album data, the largest number of facial feature data extracted from the user's photo album feature table may be determined as the user avatar feature of the user. .

When a command to add a photo to an album for a user is received, the photo is added to the user's photo album data, facial feature data is extracted from the photo, and the extracted facial feature data is the user's photo album. The first and second processing may be performed on the facial feature data added to the facial feature table and newly added to the user's photo album facial feature table.

When a command to add a friend is received for the designated user, the user ID of the friend is added to the friend information of the designated user, and the user avatar feature of the friend is the photo album facial feature table of the designated user. Matched with the facial feature data on the image in turn, and the matched facial feature data may be deleted from the photo album facial feature table of the designated user.

The method selects and displays the photos in the photo album data of the user that the user has approved for analysis and detection, and if the photo album data of each user is extracted, analyzes only for the selected pictures that have been approved for analysis and detection. The method may further include performing detection and feature extraction.

Effects according to the present invention are as follows.

First, social networking with various people is possible.

Second, it is possible to form a friendship relationship between other users who are not yet friends of the user but appear in the picture uploaded by the user.

1 is a block diagram illustrating a structure of a friend recommendation system according to an exemplary embodiment.
2 is a flowchart illustrating a friend recommendation method according to an exemplary embodiment.
3 is a flowchart illustrating a process of generating a photo album face feature table according to an embodiment of the present invention.
4 is an exemplary view of a picture uploaded by a user according to an embodiment of the present invention.
5 is an exemplary view of a picture uploaded by a user according to another embodiment of the present invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings to describe the present invention.

Modules defined on the friend recommendation system according to the present invention are not limited to the corresponding modules, but may be recombined into fewer modules or divided into more modules to implement the functions. In addition, at the server side of the social network, one or more computers / processors may be equipped with and run this module. Similarly, the steps specified in the friend recommendation method according to the present invention are not limited to the corresponding steps and may be recombined into fewer steps or divided into more steps in order to implement the function.

The friend recommendation system and method according to the present invention can be installed and operated on a server side of a social network. A general inventive step of the present invention is to analyze a photo in each user's photo album, detect facial areas, extract facial feature data, and construct a photo album facial feature table containing the extracted facial feature data of each user; Determine a user avatar feature of each user based on each user's photo album facial feature table; When facial feature data of each user's photo album facial feature table is matched with user avatar features of other users on a social network, and it is determined that the matched user avatar feature does not correspond to the user himself or his friends, By generating friend recommendation data in order to recommend the user having the matched user avatar feature to the user from which the matched facial feature data has been extracted and vice versa, it appears in another user's picture, Rather, a user can be recommended to these other users. In addition, in order to generate friend recommendation data for recommending users who have matched facial feature data but are not currently friends of each other, the facial feature data of each user's photo album facial feature table is used in the photo album facial feature table of other users. Match facial feature data.

According to the present invention, various existing algorithms and models may be applied for analysis, detection, and extraction of facial feature data on a photo for matching between facial feature data and matching between facial feature data and user avatar features. For example, conventional facial detection can be performed using a template match model, skin color model, active appearance model (AAM), support vector machine (SVM), and Adaboost model, which have different detection accuracy and speed. Template-based feature extraction, algebra-based feature extraction, feature matching based on flexible matching, feature extraction based on neural networks, and feature extraction based on wavelet multi-resolution Etc.

There are various methods for describing facial feature data, including principal component analysis. Facial matching is to calculate facial features to obtain similarity values and to match facial features through machine learning. The above-described face detection, face feature data extraction, and face matching techniques are not limited to the above described techniques, and those skilled in the art may perform the above process using other algorithms and models. The present invention does not repeatedly describe other algorithms and models.

1 is a block diagram illustrating a structure of a friend recommendation system according to an exemplary embodiment. Referring to FIG. 1, the friend recommendation system according to the present invention includes a face matching module 110, a face detection and feature extraction module 120, a user avatar determination module 160, a friend recommendation computing module 130, and a friend recommendation control. Module 140 and data management module 150.

The data management module 150 manages user information, friend information, and photo album data of all users on a social network. Unidentified user IDs are used to distinguish each user on a social network. The user information of each user according to the present invention includes at least a user ID and a user avatar feature, and includes the user's network name, gender, age and region of residence. And other social information such as symbols. The first avatar feature of the user is in a blank state, and the user information may further include other information about the user avatar feature. The friend information of each user includes the user ID of one or more users who are friends of the user. In general, users on social networks can classify and organize their friends according to taxonomy ("Family", "School Friends", "Workmates", etc.). Unassigned friends, etc.). Each user's photo album data includes at least one photo uploaded by the user.

The data management module 150 may separately create a table including user information, friend information, and photo album data for each user in a database to be managed, a user information table including user information of all users, and all users. A friend information table including friend information of and a photo album information table including photo album data of all users can also be generated. In the following storage structure, the record of each table must include the ID of the user who is the target of the record, and the user's ID is one of the main indexes of the table. Of course, the user information, friend information, and photo album data storage are not limited to the two types of storage structures, and the user information as long as the user information, friend information, and photo album information of the user can be retrieved with a specific user ID. , Friend information and photo album data may be stored utilizing other data structures desired by the user.

According to an exemplary embodiment of the present disclosure, the data management module 150 further stores and manages the photo album face feature table of each user under the control of the friend recommendation control module 140. The photo album face feature table includes face feature data extracted from photo album data of a user who is an object of the photo album face feature table.

The face detection and feature extraction module 120 analyzes a picture provided by the friend recommendation control module 140, detects a face area of the picture, and extracts face feature data from the detected face area.

The face matching module 110 matches the first face feature data and the second face feature data provided by the friend recommendation computing module 130, and then matches the friend recommendation computing module 130 with a positive or negative matching result. Feedback).

In the system of the present invention, the friend recommendation control module 140 controls another module to perform a corresponding task according to the received command. The corresponding operation controls the data management module 150 for managing the user information, friend information and photo album data according to the received user management command, friend management command and photo album management command; Extracting photo album data of each user from the data management module 150 according to the received friend recommendation command; All facial feature data extracted from the photo album data of the user is controlled through the control of the face detection and feature extraction module 120 for detecting a facial area from the extracted photo album data of each user and extracting facial feature data. Generate a photo album facial feature table that includes; The user avatar determining module for determining the user avatar feature of each user based on the photo album face feature table generated for each user and storing the determined user avatar feature in a user avatar feature item of each user's user information Control 160; And a control of the friend recommendation computing module 130 for generating friend recommendation data based on the generated photo album face feature table of all the users, user information and friend information of the data management module 150.

According to an embodiment of the present invention, the friend recommendation control module 140 counts each face feature data on the photo album face feature table of each user when generating a photo album face feature table for the user. That is, the friend recommendation control module 140 counts the number of times each facial feature data appears in the photo album facial feature table of each user, and deletes duplicate facial feature data from the photo album face feature table of each user. The friend recommendation control module 140 may store the photo album face feature table of each user through the data management module 150. When the user uploads a new photo, the system may change the user's previously created user's profile. Based on the photo album facial feature table, friend recommendation data may be accumulated and generated.

The user avatar determining module 160 determines the facial feature data extracted from the avatar picture specified by the user from the photo album face feature table generated for each user as the avatar feature of the user, and determines the determined avatar feature. Stored in the user avatar feature item of the user information of the user. If it is determined that the user does not specify the avatar picture, the user avatar determination module 160 determines the largest number of facial feature data extracted from the photo album feature table of the user as the user avatar feature of the user. do.

The friend recommendation computing module 130 performs a first process on the photo album face feature table of each user using the user information and the friend information managed by the data management module 150, and the face matching module ( 110, each face feature data on the photo album face feature table of the user is sequentially matched with the user avatar feature of each user, so that the face matching module 110 feeds back a positive matching result and the matched If the user avatar feature does not correspond to the user or a friend of the user, the friend recommendation computing module 130 recommends the user to the user having the matched user avatar feature, and vice versa. Generate recommendation data. That is, the user having the matched user avatar feature is recommended to the user, and the user is recommended to the user having the matched user avatar feature. The photo having the user avatar feature extracted from the photo album data of the recommended user may be included in the generated recommendation data for more intuitively recommending a friend.

When the friend recommendation computing module 130 performs a first process on the photo album face feature table of each user, the friend recommendation computing module 130 may be a user or a user on the photo album face feature table of the user. Recognize the facial feature data that matches the user avatar feature of a friend of a; Accordingly, according to an exemplary embodiment of the present disclosure, the friend recommendation computing module 130 deletes the face feature data matching the user avatar feature of the user or a friend of the user from the photo album face feature table.

According to a preferred embodiment of the present invention, after performing the first processing on the photo album face feature table of each user, the friend recommendation computing module 130 may be configured on the photo album face feature table of each user. 2 Perform the process. The second process may sequentially match each facial feature data on the photo album facial feature table of the user with each facial feature data on the photo album face feature table of other users through the face matching module 110, and Whenever the face matching module 110 feeds back a positive matching result, the face matching module 110 generates friend recommendation data regarding mutual recommendation for the user and other matched users. That is, the other matched user is recommended to the user, and the user is recommended to the matched other users. One of the photographs of the extracted matching facial feature data is included in the generated recommendation data for more intuitive friend recommendation.

The friend recommendation computing module 130 uses the data management module 150 to display the friend recommendation data generated for a related user when the user refreshes a specific page or logs in next time. Store the friend recommendation data generated according to.

When the friend recommendation control module 140 receives a command for adding a picture to a photo album for a user, the friend recommendation control module 140 may add the picture to the user's photo album data. Control the management module 150, and control the face detection and feature extraction module 120 to extract the face feature data from the picture, and the extracted face feature data to the photo album face feature table of the user Add and control the friend recommendation computing module 130 to generate friend recommendation data. The friend recommendation computing module 130 performs the first process on the face feature data newly added to the photo album face feature table of the user, and preferably performs the second process.

When receiving a command for adding a friend to the designated user, the friend recommendation control module 140 adds the user ID of the friend to the friend information of the designated user through the data management module 150. And matching the user avatar feature of the friend with the face feature data on the photo album face feature table of the designated user through the face matching module 110 and matching the matched face feature data with the designated user. Delete from the photo album facial features table.

According to a preferred embodiment of the present invention, the system further includes a security control module (not shown) for selecting and displaying a photo which allows the user to perform analysis and detection on his photo album data. Accordingly, when the friend recommendation control module 140 extracts the photo album data of each user from the data management module 150, the security control module (not shown) performs analysis, detection, and feature extraction. And only the photo is detected is provided to the face detection and feature extraction module 120.

This can be seen with reference to the above-described description shown in FIG. 1, wherein the friend recommendation system of the social network according to the present invention detects and extracts facial features from a photo uploaded by a user in a social network (such as a photo in a photo album). A user who appears on another user's picture but is not yet a friend may match a facial feature extracted from the user uploaded picture with another user's user avatar feature to be recommended as a friend. In addition, the friend recommendation system according to the present invention sequentially matches facial feature data extracted from photos in all user's photo albums and recommends two users who are not yet friends but appear to the same album in their respective photo albums.

The friend recommendation method of the social network according to the present invention is described in more detail with reference to FIGS. The method is implemented on the system at the server side of the social network as shown in FIG. As described above, user information, friend information, and photo album data of a user are stored for each user in a social network on the system according to the present invention, wherein the user information includes a user ID and a user avatar feature, and the friend information. Contains at least one user ID of friends.

2 is a flowchart illustrating a friend recommending method according to an exemplary embodiment.

Referring to FIG. 2, in step 210, the system performs face detection and analysis on pictures on the photo album data of each user of a social network, detects a face area in a picture, and detects the face area in the detected face area. Facial feature data is extracted, and a photo album feature table is generated that includes all facial feature data extracted from the photos in each user's photo album. 3 illustrates a process of performing face detection and analysis on a specific picture of a user and extracting facial feature data in step 210. In operation 212, an analysis and facial detection may be performed on a specific photo to determine whether a facial region is detected in the specific photo. If a facial area is detected, in step 215, the system extracts the facial feature data from the detected facial area according to a predetermined algorithm. The system then inserts the extracted facial feature data into the user's photo album facial feature table in step 218. On the contrary, when the facial region is not detected in the specific picture in step 210, the system terminates the processing of the specific picture. The process shown in FIG. 3 is performed for all the photos on the photo album data of each user of the social network.

According to a preferred embodiment of the present invention, the system counts each facial feature data of each user's photo album facial feature table while generating the photo album facial feature table of each user (ie, each facial feature data is counted. The number of times appearing in the photo album face feature table of each user) and delete the duplicated face feature data from the photo album face feature table. The system also stores each user's photo album facial feature table on an inactive storage medium.

In step 220, the system determines facial feature data extracted from the avatar picture specified by the user on the photo album face feature table generated for each user as the avatar feature of the corresponding user, and determines the determined avatar feature. Stored in the user avatar feature item of the user information of the user. If it is determined that the user does not have a designated avatar picture, the most facial feature data extracted from the user's photo album feature data is determined as the user avatar feature of the user, and the determined avatar feature is determined by the user. Is stored in the user avatar feature item of the user information.

Thereafter, in step 230, the system performs a first process on the photo album face feature table of each user. The first process uses each user's stored user information and friend information to sequentially match each facial feature data of the user's photo album facial feature table with the user avatar feature of each user, with a positive matching result. When the matched user avatar is fed back and does not correspond to the user or a friend of the user, for the mutual recommendation of the user and the user having the matched user avatar feature, that is, the user having the matched user avatar feature is referred to as the user. And recommend the user to the user having the matched user avatar feature to generate the friend recommendation data. The photo having the user avatar feature extracted from the photo album data of the recommended user is included in the generated recommendation data for more intuitive friend recommendation. Accordingly, other users who are not friends of the user but appear in the photo album of the social network are recommended to the user and vice versa.

The system preferably deletes the facial feature data that matches the user avatar feature of a user or user friend from the user's photo album facial feature table.

Based on this, according to another preferred embodiment of the present invention, the system performs a first process on the photo album feature table of each user and then performs a second process on the photo album feature table of each user. Perform step 240. The second process sequentially matches each facial feature data of the user's photo album facial feature table with each facial feature data of another user's photo album facial feature table, and the face matching module returns a positive matching result. Each time the feedback is generated, the friend recommendation data is generated for mutual recommendation of the user and the matched other users, ie, recommending the matched other users to the user and recommending the user to the matched other users. do. One of the pictures having the extracted matching facial feature data is included in the generated recommendation data for more intuitive friend recommendation. Accordingly, users who have photo albums in which the same person appears may be connected to each other and recommended to each other.

According to another embodiment of the present invention, when receiving a command to add a photo to an album for a user, the system adds the photo to the user's photo album data and through the face detection and feature extraction module. Extract facial feature data from the photo, add the extracted facial feature data to the user's photo album facial feature table, perform the first processing, and add the newly added facial feature data to the user's photo album facial feature table Preferably, the second process is performed on the facial feature data.

According to another embodiment of the present invention, when receiving a command for adding a friend for a specified user, the system adds the user ID of the friend to the friend information of the designated user, and the user avatar of the friend. A feature is sequentially matched with the face feature data of the photo album face feature table of the designated user, and the matched face feature data in the feedback matching result is deleted from the photo album face feature table of the designated user.

According to a preferred embodiment of the present invention, the system displays the selected photo that the user is authorized to analyze and detect in his photo album data, and when extracting the photo album data of each user, the selected photo Analyze, detect, and extract features only.

As can be seen from the foregoing description according to Figs. 1 to 3, according to the present invention, the friend recommendation system and method of the social network are face detection for a photo uploaded by a user in a social network (such as a photo album). And performing a feature extraction and appearing in another user's picture but not a friend of another user to be recommended by matching the facial feature extracted from the picture uploaded by the user to the user avatar feature of all users. Can be. In addition, the friend recommendation system according to the present invention sequentially matches the facial feature data extracted from the photos on the photo albums of all users, and if the same person appears on the photo albums of two users, Recommend each other to further improve the interaction between users.

4 is an exemplary view of a picture uploaded by a user according to an embodiment of the present invention. As illustrated in FIG. 4, a friend recommendation process when the user (A) 200 is to take a picture with the friends (B, C) 210 and 220 and upload the picture will be described. In this case, the information of the friends B and C 210 and 220 is stored in the friend list 152 of the user A and the photo album face feature table 154. That is, the user A's friend list 152 stores information of friends B 210 and friends C 220, and the photo album facial feature table 154 is also a friend B 210. ) And avatar information of each friend together with facial feature information of the friend (C) 220, but the friend (B) 210 or the friend (C) 220 are friends of each other in the data management module 150. If it is not set.

The friend recommendation control module 140 uses the face detection and feature extraction module 120 to extract face detection and facial features of the people included in the picture. Subsequently, the friends 210 and 220 (B and C) are connected through the user information 151, the friend list 152, the photo album data 153, and the photo album facial feature table 154 in the data management module 150. Make sure they are included in each other's friends list. If it is determined that the friend (B) 210 and the friend (C) 220 are not friends with each other, the user avatar determination module 160 may be used to match each friend (B, C) 210, 220. Create an avatar and include friend recommendation data. That is, the data recommending the friend (C) 220 to the friend (B) 210 and the data recommending the friend (B) 210 to the friend (C) 220 are provided.

5 is an exemplary view of a picture uploaded by a user according to another embodiment of the present invention. Unlike the embodiment of FIG. 4, an example of performing a friend recommendation using two photos uploaded by the same user (B) 210 is illustrated. That is, the same user (B) 210 uploads a picture taken with a friend (A) 200 and a picture taken by the same user (B) 210 with a friend (D) 230. One case is shown.

Through a process similar to the above, the friend recommendation control module 140 generates an avatar suitable for each of the friends A and D 200 and 230 using the user avatar determination module 160 and generates friend recommendation data including the same. do. That is, the data for recommending the friend (D) 230 is provided to the friend (A) 200, and the data for recommending the friend (A) 200 is provided to the friend (D) 230.

Although the present invention has been described with reference to the preferred embodiments, it will be understood by those skilled in the art that the technical spirit or essential features of the present invention may be embodied in other specific forms without modification. will be. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

110: face matching module 120: face detection and feature extraction module
130: friend recommendation computing module 140: friend recommendation control module
150: data management module 160: user avatar determination module

Claims (24)

A data management module managing information of a plurality of users constituting a social network, friend information of the user, and photo album data uploaded by the plurality of users;
A face detection and feature extraction module configured to analyze a photograph provided from the outside to extract face detection and face feature data, and to generate a face feature table including the extracted data;
A friend recommendation computing module configured to generate friend recommendation data based on a facial feature table generated by the face detection and feature extraction module and user information and friend information of each user in the data management module;
A face matching module configured to determine whether to match a plurality of facial feature data received from the friend recommendation computing module and to feed back the result to the friend recommendation computing module;
The user uploads the uploaded picture to the face detection and feature extraction module, controls to store the face feature data extracted by the face detection and feature extraction module through a data management module, and sets the friend recommendation computing module. And a friend recommendation control module to control and generate friend recommendation data, and to provide friend recommendation information to at least one user according to the friend recommendation data. The method of claim 1,
A user avatar determination module for determining a user avatar feature of each user based on the facial feature table,
And the control unit controls to store the user avatar feature extracted by the user avatar determination module in user information of the data management module. 3. The method of claim 2,
The friend recommendation control module deletes duplicate facial feature data in a facial feature table. The method of claim 1,
The friend recommendation computing module sequentially matches each facial feature data in the facial feature table with the user avatar feature of each user, and the face matching module feeds back a positive matching result and the matched user avatar feature is the user or the user. Whenever it does not correspond to a friend of the friend recommendation system of the social network, characterized in that recommended to the user having the matched user avatar feature. The method of claim 2, wherein the user avatar determination module,
If it is determined that an avatar picture is not set on the user's photo album data, the friend of the social network using the largest number of facial feature data extracted from the photo album feature table as the user's avatar feature. Referral system. The method of claim 1,
And a security control module for providing the face detection and feature extraction module with only the pictures approved by the user. 7. The method according to any one of claims 1 to 6,
The face detection and feature extraction module performs face detection using at least one of a template match model, a skin color model, an active appearance model (AAM), a support vector machine (SVM) model, and an adaboost model. Friend Referral System. 7. The method according to any one of claims 1 to 6,
The face detection and feature extraction module includes template-based feature extraction, algebra-based feature extraction, feature matching based on flexibility matching, feature extraction based on neural networks, and wavelet multi-resolution. Extracting facial features using at least one of feature extraction based on the; 7. The method according to any one of claims 1 to 6,
The facial matching module is a friend recommendation system of a social network, characterized in that for matching facial feature data using a principal component analysis method (principal component analysis method). According to claim 1, The friend recommendation control module,
The facial detection and feature extraction module is controlled to detect facial features of at least two other users other than the user included in one picture of the user, and to confirm each friend information through the data management module, and to remove other features except the user. A friend recommendation system of a social network, characterized by performing a friend recommendation between users. According to claim 1, The friend recommendation control module,
The facial detection and feature extraction module is controlled to detect facial features of at least one other user except for users included in different pictures of the same user, and to check each friend information through the data management module, thereby identifying the user. A friend recommendation system of a social network, characterized in that for performing friend recommendations among other users. Generating data of each user of the social network;
Extracting facial features of a person included in the picture by analyzing a picture of each user;
And generating friend recommendation data using the extracted facial features. The method of claim 12,
And analyzing each user's picture to determine a user avatar characteristic of each user. Analyze the photos in the photo album of each user of the social network, detect a facial area in the photo, extract facial feature data from the facial area of the person on the photo, and extract the facial in the photo in the photo album Generating a photo album facial feature table comprising feature data;
The facial feature data extracted from the avatar picture designated by the user in the photo album feature table generated for each user is determined as the avatar feature of the user, and the determined avatar feature is the user avatar feature among the user information of the user. Storing in the item;
Matching each facial feature data of the user's photo album facial feature table with the user avatar feature of each user using the stored user information and friend information of each user;
And generating friend recommendation data whenever the matched user avatar does not correspond to a user or a friend of the user. 15. The method of claim 14,
And storing the photo album facial feature table in a database for each user. 15. The method of claim 14,
Counting each facial feature data in the photo album facial feature table of each user and deleting the duplicate facial feature data from the user's photo album facial feature table. 15. The method of claim 14,
And generating friend recommendation data by sequentially matching each facial feature data in the user's photo album facial feature table with each facial feature data in the user's photo album facial feature table. 15. The method of claim 14,
And the friend recommendation data includes different content from each other. 15. The method of claim 14,
And the friend recommendation data includes photo information extracted from the matched facial avatar feature data. 15. The method of claim 14,
If it is determined that the designated avatar picture does not exist on the user's photo album data, the facial feature data extracted from the user's photo album feature table is determined as the user avatar feature of the user. How to recommend friends on social networks. 15. The method of claim 14,
And receiving from the user information on whether to approve the analysis and detection of the photos in the photo album data of the user. 22. The method according to any one of claims 14 to 21,
A method of recommending friends in a social network, comprising detecting a face using at least one of a template match model, a skin color model, an active appearance model (AAM), a support vector machine (SVM) model, and an Adaboost model. 22. The method according to any one of claims 14 to 21,
Feature extraction based on template, feature extraction based on algebra, feature extraction based on flexible matching, feature extraction based on neural networks, and feature extraction based on wavelet multi-resolution A method of recommending friends in a social network, comprising extracting facial features using one. 22. The method according to any one of claims 14 to 21,
A method of recommending friends in a social network, characterized by matching facial feature data using a principal component analysis method.

Images