KR20200014723A - Model training method and data similarity determination method, apparatus and device thereof - Google Patents

Abstract

Embodiments of the present application disclose model training methods, apparatus and devices, and data similarity determination methods, apparatuses and devices. The model training method includes obtaining a plurality of user data pairs, wherein the data fields of two sets of user data within each user data pair have the same portion; Obtaining user similarity corresponding to each user data pair, wherein the user similarity is similarity between users corresponding to two sets of user data in each user data pair; Determining sample data for training a preset classification model according to a plurality of user data pairs and user similarities corresponding to each pair of user data; And training the classification model based on the sample data to obtain a similarity classification model. By the embodiment of the present application, fast training of the model can be implemented, model training efficiency can be improved, and resource consumption can be reduced.

Description

Model training method and data similarity determination method, apparatus and device thereof

TECHNICAL FIELD The present invention relates to the field of computer technology, and more particularly, to a model training method, apparatus and device, and a method, apparatus and device for determining data similarity.

As a new identity verification method, face recognition created new risks while providing convenience to the user. In the case of multiple users with very similar appearances (such as twins), it is difficult to effectively distinguish between different users through face recognition, and face recognition is a mistake in account registration and account fraud due to incorrect identification of users. It is very likely to cause a risk of usefulness. In the most known case of very similar appearances, twins, especially identical twins, are closely related to each other and are very likely to be associated with such risk-related behavior. Determining twin user data from large amounts of data has become an important problem to solve.

In general, based on a supervised machine learning method, a recognition model is constructed using preselected sample data. Specifically, investigators conduct social investigations through questionnaires, prize-winning questions, or manual observations, collect user data, ask for investigations through manual observations, or ask people to be investigated, or twin relationships. Acquire and label it. Based on manually sorted associations or twin relationships, an identification model is built using the corresponding user data as sample data.

However, the above-mentioned identification model built using supervised machine learning methods requires manual labeling of the sample data, and the manual classification process consumes a large amount of human resources and also requires a lot of time for classification. This makes the model training inefficient and leads to high resource consumption.

It is an object of embodiments of the present application to provide a model training method, apparatus and device, and data similarity determination method, apparatus and device for implementing fast training of a model, improving model training efficiency, and reducing resource consumption.

In order to solve the above-mentioned technical problem, an embodiment of the present application is implemented as follows:

 An embodiment of the present application to provide a model training method,

Obtaining a plurality of user data pairs, wherein the data fields of two sets of user data in each user data pair have the same portion;

Obtaining user similarity corresponding to each user data pair, wherein the user similarity is similarity between users corresponding to two sets of user data in each user data pair;

Determining sample data for training a preset classification model according to a plurality of user data pairs and user similarities corresponding to each user data pair; And

Training the classification model based on the sample data to obtain a similarity classification model.

Optionally, acquiring user similarity corresponding to each user data pair may include:

Acquiring a biological characteristic of a user corresponding to the first user data pair, wherein the first user data pair is any user data pair of the plurality of user data pairs; And

Determining user similarity corresponding to the first user data pair according to a biological characteristic of the user corresponding to the first user data pair.

Optionally, the biological feature comprises a face image feature;

Acquiring a biological characteristic of a user corresponding to the first user data pair may include:

Obtaining a face image of a user corresponding to the first user data pair; And

Performing feature extraction on the face image to obtain a face image feature,

Correspondingly, determining user similarity corresponding to the first user data pair according to the biological characteristics of the user corresponding to the first user data pair,

Determining user similarity corresponding to the first user data pair according to a facial image feature of the user corresponding to the first user data pair.

Optionally, the biological features include negative features;

Acquiring a biological characteristic of a user corresponding to the first user data pair may include:

Acquiring voice data of a user corresponding to the first user data pair; And

Performing feature extraction on speech data to obtain a speech characteristic;

Correspondingly, determining user similarity corresponding to the first user data pair according to the biological characteristics of the user corresponding to the first user data pair,

Determining user similarity corresponding to the first user data pair according to a voice characteristic of the user corresponding to the first user data pair.

Optionally, determining sample data for training the classification model according to the user similarities and the plurality of user data pairs corresponding to each user data pair,

Performing feature extraction on each user data pair in the plurality of user data pairs to obtain an associated user feature between two sets of user data in each user data pair; And

Determining sample data for training the classification model, in accordance with associated user characteristics between the user data in each user data pair and user similarity corresponding to each user data pair.

Optionally, according to the associated user characteristics between the two sets of user data in each user data pair and the user similarity corresponding to each user data pair, determining the sample data for training the classification model may include:

Selecting a positive sample feature and a negative sample feature from user features corresponding to the plurality of user data pairs according to a user similarity corresponding to each user data pair and a predetermined similarity threshold; And

Using the positive sample feature and the negative sample feature as sample data for training the classification model.

Optionally, the user feature includes a household registration dimension feature, a name dimension feature, a social feature, and a feature of interest, where the household registration dimension feature comprises a feature of user identity information, wherein the name dimension feature is a user The characteristics of the name information and the degree of scarcity of the user's surname, and the social characteristics include the characteristics of the user's social relation information.

Optionally, the positive sample feature includes the same quantity of features as the negative sample feature.

Optionally, the similarity classification model is a binary classifier model.

An embodiment of the present application further provides a method for determining data similarity,

Acquiring a to-be-detected user data pair;

Performing feature extraction on each set of detection target user data in the detection target user data pair; And

Determining similarity between the users corresponding to the two sets of detection user data in the detection user data pair according to the detection user feature and the pre-trained similarity classification model.

Optionally, the method

If the similarity between the users corresponding to the two sets of detection user data pairs in the detection user data pair is greater than a predetermined similarity classification threshold, determining the detection users corresponding to the detection user data pairs as twins.

An embodiment of the present application to provide a model training apparatus,

A data acquisition module configured to acquire a plurality of user data pairs, the data fields of two sets of user data in each user data pair having the same portion;

A similarity obtaining module, configured to obtain user similarity corresponding to each user data pair, wherein the user similarity is similarity between users corresponding to two sets of user data in each user data pair;

A sample data determination module configured to determine sample data for training a preset classification model according to a plurality of user data pairs and user similarities corresponding to each user data pair; And

And a model training module configured to train the classification model based on the sample data to obtain a similarity classification model.

Optionally, the similarity acquisition module may include:

A biological feature acquisition unit configured to acquire a biological feature of a user corresponding to the first user data pair, wherein the first user data pair is any user data pair of the plurality of user data pairs; And

And a similarity obtaining unit, configured to determine user similarity corresponding to the first user data pair according to a biological characteristic of the user corresponding to the first user data pair.

Optionally, the biological feature comprises a face image feature;

The biological feature acquisition unit acquires a face image of the user corresponding to the first user data pair; Perform feature extraction on the face image to obtain a face image feature;

Correspondingly, the similarity obtaining unit is configured to determine the user similarity corresponding to the first user data pair according to the facial image feature of the user corresponding to the first user data pair.

Optionally, the biological features include negative features;

The biological feature acquisition unit obtains voice data of a user corresponding to the first user data pair; Perform feature extraction on speech data to obtain a speech characteristic;

Correspondingly, the similarity obtaining unit is configured to determine the user similarity corresponding to the first user data pair according to the voice characteristic of the user corresponding to the first user data pair.

Optionally, the sample data determination module,

A feature extraction unit configured to perform feature extraction on each user data pair in the plurality of user data pairs to obtain an associated user feature between two sets of user data in each user data pair; And

And a sample data determination unit configured to determine sample data for training the classification model, according to associated user characteristics between two sets of user data in each user data pair and user similarity corresponding to each user data pair.

Optionally, the sample data determination unit selects a positive sample feature and a negative sample feature from user features corresponding to the plurality of user data pairs according to user similarities and predetermined similarity thresholds corresponding to each user data pair; The positive sample feature and the negative sample feature are configured to use as sample data for training the classification model.

Optionally, the user feature includes a household registration dimension feature, a name dimension feature, a social feature, and a feature of interest, wherein the household registration dimension feature includes a feature of user identity information, wherein the name dimension feature is a feature of the user name information and the user. Includes features of the degree of scarcity of sex, and social features include features of the user's social relationship information.

Optionally, the positive sample feature includes the same quantity of features as the negative sample feature.

Optionally, the similarity classification model is a binary classifier model.

An embodiment of the present application further provides an apparatus for determining data similarity,

A detection target data acquisition module configured to acquire a detection target user data pair;

A feature extraction module configured to perform feature extraction on each set of detection target user data in the detection target user data pair to obtain a detection target user feature; And

And a similarity determination module configured to determine similarity between the users corresponding to the two sets of detected user data in the detected user data pair according to the detected user feature and the pre-trained similarity classification model.

Optionally, the device is

A similarity classification module configured to determine, as twins, a detected user corresponding to the detected user data pair as twin if the similarity between the users corresponding to the two sets of detected user data in the detected user data pair is greater than a predetermined similarity classification threshold. It includes more.

An embodiment of the present application to provide a model training device,

A processor; And

And a memory configured to store computer executable instructions, wherein when executed the computer executable instructions cause the processor to:

Obtaining a plurality of user data pairs, wherein the data fields of two sets of user data in each user data pair have the same portion;

Obtaining user similarity corresponding to each user data pair, wherein the user similarity is similarity between users corresponding to two sets of user data in each user data pair;

Determining sample data for training a preset classification model, according to a plurality of user data pairs and user similarities corresponding to each pair of user data; And

Train the classification model based on the sample data to obtain a similarity classification model.

An embodiment of the present application provides a data similarity determination device,

A processor; And

And a memory configured to store computer executable instructions, wherein when executed the computer executable instructions cause the processor to:

Acquiring a detection target user data pair;

Performing feature extraction on each detected user data in the detected user data pair to obtain a detected user feature; And

Determining similarity between the users corresponding to the two sets of detected user data in the detected user data pair according to the detected user feature and the pre-trained similarity classification model.

As can be seen from the technical solution provided by the embodiments of the present application, in the embodiments of the present application, a plurality of user data pairs are obtained-the data fields of two sets of user data in each user data pair are the same part. Has-; User similarity corresponding to each user data pair is obtained; Sample data for training a preset classification model is determined; Then, since the classification model is trained based on the sample data to obtain a similarity classification model, the similarity between the users corresponding to the two sets of detection user data in the detection user data pair may be determined according to the similarity classification model. In this manner, a plurality of user data pairs are obtained through the same data field, and the association between the users corresponding to two sets of user data in each user data pair to obtain sample data for training a preset classification model. As determined by the similarity, that is, sample data can be obtained without manual classification, so that fast training of the model can be implemented, model training efficiency can be improved, and resource consumption can be reduced.

BRIEF DESCRIPTION OF DRAWINGS To describe the technical solutions in the embodiments of the present application or in the prior art more clearly, the drawings that need to be used in the description of the embodiments or the prior art are briefly introduced. In the following description the drawings are only some embodiments of the present application, and it will be apparent to one skilled in the art that additional drawings may be obtained according to these drawings without creative efforts.
1 illustrates an embodiment of a model training method according to the present application.
2 illustrates an embodiment of a data similarity determination method according to the present application.
3 is a schematic diagram of an interface of a detection application according to the present application.
4 illustrates an embodiment of a data similarity determination method according to the present application.
5 is a schematic diagram of processing logic of a data similarity determination process according to the present application.
6 shows an embodiment of a model training apparatus according to the present invention.
7 shows an embodiment of a data similarity determining apparatus according to the present invention.
8 shows an embodiment of a model training device according to the present application.
9 illustrates an embodiment of a data similarity determining device according to the present application.

Embodiments of the present application provide a model training method, apparatus and device, and a data similarity determination method, apparatus and device.

BRIEF DESCRIPTION OF DRAWINGS To enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some but not all of the embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application are within the protection scope of the present application.

Example 1

As shown in FIG. 1, an embodiment of the present application provides a model training method. The method may be performed by a terminal device or a server. The terminal device may be a personal computer or the like. The server may be a single independent server or may be a server cluster formed by a plurality of servers. Embodiments of the present application are described in detail using examples in which the method is executed by the server to improve model training efficiency. The method may specifically include the following steps:

In step S102, a plurality of user data pairs are obtained, wherein the data fields of two sets of user data in each user data pair have the same portion.

Each user data pair may include user data of a plurality of different users. For example, the plurality of user data pairs includes user data pair A and user data pair B. User data pair A includes user data 1 and user data 2, and user data pair B includes user data 3 and user data 4. User data may be data related to the user, for example, identity information such as the user's name, age, height, address, identity card number, social security card number, and information such as the user's interest, purchased goods, travel, etc. It may also include. A data field is a field or character that can represent a user's identity corresponding to two sets of different user data within a user data pair, as well as a predetermined quantity of associations between users, for example, surname, identity card number. May be a number (e.g., the first 14 digits of an identity card number), a social security card number, or another identity number from which a user identity or information may be determined.

In implementations, user data may be obtained in a variety of ways. For example, user data can be purchased from different users; Or information entered by the user when registering with a website or application, such as information entered when the user registers with Alipay® or user data actively uploaded by the user. The specific manner in which user data is obtained is not limited to the embodiments of the present application. After the user data is acquired, the data fields included in the obtained user data can be compared to find user data having data fields that share the same portion. User data having data fields that share the same portion can be grouped together to form a pair of user data. By the above method, a plurality of user data pairs can be obtained, and the data fields of the user data in each user data pair have the same part.

For example, in practical applications, in order to reduce the amount of computation and improve the processing efficiency as much as possible, the data field may be set as an identity card number and a surname, and one or more numbers in the identity card number, for example, of the identity card number Given that the first 14 digits may represent a relationship between two users, information such as identity card numbers and user names may be retrieved from the user data. In an embodiment of the present application, as an example, the first 14 digits of the identity card number are used as the basis for determining whether the data field has the same part. Specifically, the first 14 digits of the identity card number and each user's last name can be obtained, and the first 14 digits of the identity card number can be compared with the last user's last name. Two sets of user data with the same last name and the same first 14 digits of the identity card number can be grouped into one user data pair. Specifically, the user data pair is in the form of a user pair, for example, {identity card number of user 1, identity card number of user 2, name of user 1, name of user 2, other data of user 1, user 2 And other data}.

A data field of two sets of user data having the same part may be interpreted as some content in the data field, for example, the first 14 digits of an 18-digit identity card number, or all the content of the data field is the same. It should be understood that it can be interpreted.

In step S104, user similarity corresponding to each user data pair is obtained, where user similarity is similarity between users corresponding to two sets of user data in each user data pair.

User similarity can be used to indicate the degree of similarity between a plurality of users, for example 99% or 50%. In practical applications, user similarity can also be expressed in other ways. For example, user similarity can also be expressed as twins and not twins, or identical twins and fraternal twins.

In an embodiment, the main purpose of this embodiment is to train a classification model that requires sample data for training the classification model, as well as user similarity corresponding to the sample data. User similarity may be stored in advance on the server or terminal device. User similarity can be determined in various ways. One optional method of processing is provided below, and for details, reference may be made to the following: An image of the user may be acquired in advance, where the image is displayed to the user when registering with the application or website. Uploaded by the user, and the user may be a user corresponding to two sets of user data included in each user data pair. Images of each user data pair may be compared, and similarity between the users corresponding to two sets of user data included in the user data pair may be calculated through the comparison of the images. During the image comparison process, processing methods such as image preprocessing, image feature extraction and image feature comparison can be used, which is not limited to the embodiments of the present application.

In step S106, according to the user similarity and the plurality of user data pairs corresponding to each user data pair, sample data for training a preset classification model is determined.

Classification models include naive Bayesian classification models, logistic regression classification models, decision tree classification models, or support vector machine classification models. The same may be any classification model. In an embodiment of the present application, considering that the classification model is used only to determine whether two different users are similar, the classification model may be a binary classification model. Sample data may be data used to train a classification model. The sample data may be two sets of user data in the user data pair, or may be data obtained after the user data has been processed in a particular manner. For example, feature extraction may be performed on the above user data to obtain a corresponding user feature, and the data of the user feature may be used as sample data.

In an embodiment, for example, a similarity threshold of 80% or 70% may be set in advance. User similarity corresponding to each user data pair may be compared with a similarity threshold, respectively. User data pairs corresponding to user similarities that are greater than the similarity threshold may be grouped into one set, and user data pairs corresponding to user similarities that are less than the similarity threshold may be grouped into one set, with a predetermined quantity (e.g., For example, 40000 or 50000 user data pairs may be selected from each of the two sets above, and the selected user data pairs are used as sample data for training a preset classification model.

It should be noted that the sample data for training the preset classification model can be selected in various ways different from the above. For example, features of two sets of user data included in each user data pair are extracted to obtain corresponding user features, and then the user features are located according to the user similarity and similarity thresholds corresponding to each user data pair. Can be grouped into two sets. Two sets of user characteristic data can be used as sample data for training a preset classification model.

In step S108, the classification model is trained based on the sample data to obtain a similarity classification model.

The similarity classification model may be a model used to determine the degree of similarity between different users.

In an embodiment, feature extraction is performed on two sets of user data in each selected user data pair based on the case where the user data pair selected above is used as sample data for training a preset classification model, thereby corresponding user features. Then, the user characteristics of each pair of user data of the sample data can be input to the classification model for calculation. After the calculation, the calculation result can be output. The calculation result can be compared with the user similarity corresponding to the corresponding user data pair to determine whether the two are the same. If the two are not the same, the relevant parameters of the classification model can be changed, and then the user characteristics of the user data pair are entered into the modified classification model and calculated, and it is determined whether the calculation result is equal to the user similarity. The procedure is repeated until both are identical. If both are the same, the above processing procedure may be performed for the next selected user data pair. Finally, if the calculation result obtained after the user feature of each user data pair is input to the classification model is the same as the user similarity corresponding to the corresponding user data pair, the obtained classification model is a similarity classification model.

By the above method, a similarity classification model can be obtained. To use the similarity classification model, reference may be made to the following related contents:

As shown in FIG. 2, an embodiment of the present application provides a method for determining similarity. The method may be performed by a terminal device or a server. The terminal device may be a personal computer or the like. The server may be a single independent server or may be a server cluster formed by a plurality of servers. The method may specifically include the following steps:

In step S202, a detection target user data pair is obtained.

The detection target user data pair may be a user data pair formed by user data of two users of the detection target.

In an implementation, a corresponding detection application can be set up to detect similarities between two different users. As shown in FIG. 3, the detection application may include a button for uploading data. When similarity between two different users is to be detected, a button for uploading data may be pressed. The detection application may pop up a prompt box for uploading the data. The data uploader may input data of the detection target user data pair into the prompt box, and then tap the confirmation button in the prompt box when the input is completed. The detection application may acquire the detection target user data pair input by the data uploader. The detection application may be installed on the terminal device or may be installed on the server. When the similarity determination method provided by the embodiment of the present application is executed by the server and the detection application is installed in the terminal device, the detection application can transmit the detection target user data pair to the server after acquiring the detection target user data pair, The server can acquire the detection target user data pair. When the detection application is installed in the server, the server can directly acquire the detection target user data pair from the detection application.

In step S204, feature extraction is performed on each set of detection target user data in the detection target user data pair to obtain the detection target user feature.

The detection target user feature may be a feature of user data of the detection target user.

In an implementation, each set of detected user data in the detected user data pair may be acquired. For any set of detection user data, the corresponding feature can be extracted from the detection user data by using a preset feature extraction algorithm, and the extracted feature can be used as detection user data corresponding to the detection user. have. By the above method, the detection target user feature corresponding to each set of detection user data in the detection target user data pair can be obtained.

It should be appreciated that the feature extraction algorithm can be any algorithm that can extract a predetermined feature from the user data, and specifically can be set according to the actual situation.

In step S206, the similarity between the users corresponding to the two sets of the detected user data in the detected user data pair is determined according to the detected user feature and the pre-trained similarity classification model.

In an embodiment, the detected target user feature obtained through step S204 may be input and calculated in the similarity classification model obtained through steps S102 through S108. The result output from the similarity classification model may be the similarity between the users corresponding to the two sets of detection user data in the detection user data pair.

In practical applications, it should be appreciated that the direct output of the similarity classification model can be presented in percentage, for example 90% or 40%. In order to make the output result more intuitive to the user, the direct output result of the similarity classification model is the same as when the identical twins and non-identical twins need to be distinguished, or when the identical twins and fraternal twins need to be distinguished. It may be further set according to the situation. In consideration of the above case, the classification threshold may be set. If the direct output result is greater than the classification threshold, it is determined that the users corresponding to the two sets of detected user data in the detected user data pair are identical twins; If not, the user is determined to be identical or not fraternal twins. In this way, the similarity between the users corresponding to the two sets of the detected user data pairs in the detected user data pair can be quickly determined according to the pre-trained similarity classification model, thereby improving the efficiency of determining the similarity between users. Can be.

In the above description, both the user data pair and the detected user data pair both comprise two sets of user data, and in practical applications, the model training method and the similarity determination method provided by the present application may also have more than two sets of user data. It should be appreciated that the present invention may also be applied to a combination of user data and a detected target user data combination. For specific embodiments, reference may be made to the relevant content of embodiments of the present application, and details are not described herein again.

An embodiment of the present application provides a model training method and a similarity determining method in which a plurality of user data pairs are obtained, wherein two sets of user data data fields in each user data pair have the same portion; User similarity corresponding to each user data pair is obtained; Sample data for training a preset classification model is determined; Then, since the classification model is trained based on the sample data to obtain the similarity classification model, the similarity between the users corresponding to the two sets of detection user data in the detection user data pair may be determined according to the similarity classification model. In this way, a plurality of user data pairs are obtained only through the same data field, and the association between users corresponding to two sets of user data in each user data pair to obtain sample data for training a preset classification model. As determined by the similarity, that is, sample data can be obtained without manual classification, fast training of the model can be implemented, efficiency of model training can be improved, and resource consumption can be reduced.

Example 2

As shown in FIG. 4, an embodiment of the present application provides a method of determining data similarity. The method may be executed by the server or jointly by the terminal device and the server. The terminal device may be a personal computer or the like. The server may be a single independent server or may be a server cluster formed by a plurality of servers. In an embodiment of the present application, to improve model training efficiency, the detailed description is made using an example in which the method is executed by the server. If the method is jointly implemented by a terminal device and a server, reference may be made to the following related contents, and details are not described herein again. The method specifically includes:

Currently, as a novel user identity verification method, face recognition has created new risks while providing convenience to the user. With current face recognition technology, the user's image is captured in the field and compared with the user's image pre-stored in the database of the face recognition system, and when the value obtained through the comparison reaches a predetermined threshold, the user It is determined that the user corresponds to the pre-stored user image, thereby verifying the identity of the user. However, it is difficult to effectively verify the identity of a user with a very similar face by using the above method, which is likely to cause incorrect registration of the account and fraudulent usefulness of the account funds due to the inability to verify the identity.

As the most typical known case involving very similar appearances, twins, especially identical twins, are closely related to each other and are very likely to result in denial public opinion. If there is a list containing as many twin users as possible, a special facial recognition coping strategy can be designed for these users to avoid such risks. Therefore, a model for effectively identifying twins can be constructed to output a list of twins for monitoring the facial recognition behavior of these users while ensuring high accuracy, thereby achieving risk control. For an embodiment of building a model for effectively identifying twins, reference may be made to the model training method provided by steps S402 to S412 below, the details of which are as follows:

In step S402, a plurality of user data pairs are obtained, wherein the data fields of two sets of user data in each user data pair have the same portion.

In an embodiment, considering that twins generally have the same first 14 digits of the same last name and identity card number, the first 14 digits of the last name and identity card number can be used as a data field to select a user data pair. For a particular implementation of step S402, reference may be made to the relevant content of step S102 of Embodiment 1, and details are not described herein again.

Note that the process of selecting a user data pair is implemented based on the first 14 digits of the last name and identity card number. In another embodiment of the present application, the process of selecting user data pairs is also implemented based on other information, for example, based on gender and social security card numbers, or identity card numbers and first 14 numbers of social security cards. It may be, but is not limited to the embodiment of the present application.

Considering that the degree of similarity between users corresponding to two sets of user data in the user data pair should be determined during model training, the following provides an associated processing scheme, specifically the following steps S404 and S406. May be referenced.

In step S404, biological characteristics of a user corresponding to the first user data pair are obtained, where the first user data pair is any user data pair of the plurality of user data pairs.

Biological features include physiological and behavioral features of the human body such as fingerprint features, iris features, facial features, DNA or other physiological features, or voiceprint features, handwriting features, keystroke habits or other behavioral features. Can be.

In an implementation, after the plurality of user data pairs are acquired through step S402 above, the user data pairs (ie, the first user data pair) may be arbitrarily selected from the plurality of user data pairs. When the user logs in to the server for registration using the terminal device, the user may upload one or more of the above biological characteristics of the user to the server. The server may store the biological feature and the user's identifier in an associated manner. The identifier of the user may be the user's last name or first name of the user entered by the user during registration. The above information stored in the server in an associated manner may be as shown in Table 1.

User's identifier Biological characteristics User 1 Biological feature A User 2 Biological feature B User 3 Biological Characteristics C

After selecting the first user data pair, the server extracts the identifier of the user included in the first user data pair, and then acquires a corresponding biological characteristic according to the user's identifier, thereby corresponding to the first user data pair. Obtain biological characteristics of the user. For example, the identifiers of the users included in the first user data pair are User 2 and User 3, and by querying the corresponding relationship in the table above, User 2 corresponds to Biological Feature B, and User 3 corresponds to Biological Feature C. The biological characteristics of the user corresponding to the first user data pair may be determined to be biological characteristic B and biological characteristic C. In step S406, the biological characteristics of the user corresponding to the first user data pair may be determined. User similarity is determined according to the biological characteristics of the user corresponding to the first user data pair.

In an embodiment, after a biological characteristic of a user corresponding to the first user data pair is obtained through step S404 above, to determine the degree of similarity (ie, user similarity) between the two corresponding users, Similarity calculations are performed for each of the identified biological features. The similarity calculation can be implemented in various ways, for example, depending on the Euclidean distance between feature vectors, and is not limited to the embodiments of the present application.

It should be appreciated that the threshold can be set to determine if the user is similar. For example, the threshold is set at 70. When the user similarity corresponding to the two biological features is 70 or more, the user corresponding to the two sets of user data in the first user data pair is determined to be similar; When the user similarity corresponding to the two biological features is less than 70, it is determined that the users corresponding to the two sets of user data in the first user data pair are not similar.

By the above method, the above processing procedure is performed on user data pairs other than the first user data pair in the plurality of user data pairs to obtain user similarity corresponding to each user data pair in the plurality of user data pairs. can do.

In step S404 and step S406 above, the user similarity is determined according to the biological characteristics of the user. In practical applications, user similarity can be specifically determined in various ways. Steps S404 and S406 are described in detail below using examples in which the biological features are facial features, and the following steps 1 and 2 may be referred to for details.

In step 1, a face image of a user corresponding to the first user data pair is obtained, where the first user data pair is any user data pair of the plurality of user data pairs.

In an implementation, after a plurality of user data pairs are acquired through step S402 above, one user data pair (ie, a first user data pair) may be arbitrarily selected from the plurality of user data pairs. When the user logs in to the server for registration using the terminal device, the user can upload an image containing the user's face to the server. The server may store the image and the user's identifier in an associated manner. The identifier of the user may be the user's last name or first name of the user entered by the user during registration. The above information stored in the server in an associated manner may be as shown in Table 2.

User's identifier Image containing the user's face User 1 Image A User 2 Image B User 3 Image C

After acquiring the first user data pair, the server may extract the identifier of the user included in the first user data pair, and then obtain a corresponding image according to the identifier of the user, thereby obtaining the first user A face image of the user corresponding to the data pair may be obtained. For example, the identifiers of the users included in the first user data pair are User 2 and User 3, and by querying the corresponding relationship in the table above, the image containing the user's face and corresponding to User 2 is image B It may be determined that the image including the face of the user and corresponding to the user 3 is the image C, that is, the face image of the user corresponding to the first user data pair is determined to be the image B and the image C. Step 2 In FIG. 4, feature extraction is performed on a face image to obtain a face image feature, and a user similarity corresponding to the first user data pair is determined according to a face image feature of the user corresponding to the first user data pair.

In an embodiment, after the face image of the user corresponding to the first user data pair is obtained through step 1, feature extraction may be performed on the obtained face image, respectively, to obtain a corresponding face image feature, A corresponding feature vector is obtained based on the extracted feature of each face image; The Euclidean distance between the feature vectors of any two face images can then be calculated, and the degree of similarity (ie, user similarity) between the corresponding two users can be determined according to the value of the Euclidean distance between the feature vectors. The larger the value of the Euclidean distance between feature vectors, the lower the similarity of the user; The smaller the value of the Euclidean distance between feature vectors, the higher the user similarity.

It should be noted that for face images, the two face images may or may not be similar. Thus, a threshold for determining whether the images are similar can be set. For example, the threshold is set at 70. When the user similarity corresponding to the two face images is 70 or more, it is determined that the users corresponding to the two sets of user data in the first user data pair are similar; When the user similarity corresponding to the two face images is less than 70, it is determined that the users corresponding to the two sets of user data in the first user data pair are not similar.

For example, based on the example of step 1, feature extraction is performed on image B and image C, respectively, and corresponding feature vectors are constructed according to the extracted features, respectively, so that feature vector of image B and feature vector of image C. Acquire. The Euclidean distance between the feature vector of Image B and the feature vector of Image C is calculated, and the user similarity between User 2 and User 3 is determined according to the obtained Euclidean distance value.

By the above method, the above processing procedure is performed on user data pairs other than the first user data pair in the plurality of user data pairs, so that user similarity corresponding to each user data pair in the plurality of user data pairs is obtained. Can be obtained.

In addition, for the processing of the above step (S404) and step (S406), the following further provides an optional processing scheme, the following steps 1 and 2 can be referred to for details.

In step 1, voice data of a user corresponding to the first user data pair is obtained, wherein the first user data pair is any user data pair of the plurality of user data pairs.

In an implementation, after a plurality of user data pairs are acquired through step S402 above, one user data pair (ie, a first user data pair) may be arbitrarily selected from the plurality of user data pairs. When the user logs in to the server for registration using the terminal device, the user has a predetermined duration (eg 3 seconds or 5 seconds) and / or predetermined voice content (eg one or more). Voice data including a word or a sentence) can be uploaded to the server. The server may store the voice data and the user's identifier in an associated manner. After selecting the first user data pair, the server extracts each user's identifier included in the first user data pair, and then acquires corresponding voice data according to the user's identifier, thereby corresponding to the second user data pair. It is possible to obtain the voice data of the user.

In step 2, feature extraction is performed on the voice data to obtain a voice feature, and the user similarity corresponding to the first user data pair is determined according to the voice feature of the user corresponding to the first user data pair.

In an embodiment, after the voice data of the user corresponding to the first user data pair is obtained through step 1 above, feature extraction is performed on the obtained voice data, respectively, and the extracted feature of the voice data of each piece is extracted. Based on this, the degree of similarity (ie, user similarity) between the two corresponding users may be determined. For specific implementations, reference may be made to the related contents in step S406 above. Alternatively, user similarity can be determined through a one-to-one comparison of features; Or speech spectral analysis may be performed on any two pieces of speech data to determine user similarity. By the above method, the above processing procedure is performed on user data pairs other than the first user data pair in the plurality of user data pairs, so that user similarity corresponding to each user data pair in the plurality of user data pairs is obtained. Can be obtained.

In step S408, feature extraction is performed for each user data pair in the plurality of user data pairs to obtain an associated user feature between two sets of user data in each user data pair.

In an implementation where a user data pair (which may be referred to as a third user data pair) may be arbitrarily selected from a plurality of user data pairs, feature extraction may be performed on two different sets of user data, respectively, within the third user data pair. Can be. For example, the third user data pair may include user data 1 and user data 2, and feature extraction may be performed on user data 1 and user data 2, respectively. Then, an associated user feature between two sets of user data in the third user data pair can be obtained by comparing features extracted from different user data. By the above method, the above processing procedure is performed on other user data pairs in addition to the third user data pairs in the plurality of user data pairs, so as to obtain an associated user characteristic between two sets of user data in each user data pair. can do.

In practical applications, user features may include, but are not limited to, household registration dimension features, name dimension features, social features, features of interest, and the like. The household registration dimension feature may include a feature of user identity information. The generation registration dimension is mainly based on China's household registration management system. The identity card information included in the household registration includes the date of birth and the place of household registration, and the household registration includes the name of the parent and the address of the residents. However, for historical and other reasons, some citizens' registration information is not the same as the actual situation. For example, a registered date of birth may be earlier than the actual date, two children may each follow the parents' last name, or a divorce may result in separation of household registration. Thus, the generation registration dimension feature can serve as a criterion for determining whether two users are twins. In this way, the association between different users is determined according to characteristics such as whether different users corresponding to the user data pair have the same date of birth, the same household registration place, the same parent, or the same current address.

Name dimension features include features of the user name information and the degree of scarcity of the user's last name. In the case of name dimension features, based on Natural Language Processing (NLP) theory and social experience, the names of two people generally look similar to Zhang Jinlong and Zhang Jinhu, or Zhang Zhang Meimei and Zhang Lili. If there is a specific semantic relationship, there is a relation between the two. In an embodiment of the present application, the relationship between two user's names can be evaluated using a dictionary, and the user's registered personal information and demographic data are used to calculate the degree of scarcity of the gender as a feature. In this way, the association between different users is such that whether different users corresponding to the user data pair have the same last name or the same length name, the degree of synonym of the name, the combination of the names is the phrase, the degree of scarcity of the last name, etc. It depends on the characteristics.

Social features include features of the user's social relationship information. Social features may be obtained by extracting social relationships of user data pairs based on big data. In general, twins will often interact with each other and have large overlapping social relationships, such as the same relatives or even classmates. In an embodiment of the present application, the user data pairs are associated and obtain corresponding features based on the personal information of the user stored in the server and the relational network formed by existing data, address book, and the like. In this way, the association between different users corresponding to a pair of user data means that in a social networking application, different users follow each other, different users transfer funds between each other, and different users can read their contact information. Is stored in the network, different users display specific names in the address book in the address book, and the number of common contacts among these address books.

Further, given that twins can travel together with many hobbies and shopping preferences in common, the user features may further include e-commerce, tourism, entertainment and other dimensions. In embodiments of the present application, data relating to e-commerce, tourism, entertainment, and other dimensional features may be obtained from a predetermined database or website. In this way, the association between different users corresponding to a pair of user data is the amount of common shopping records between different users, whether they traveled together, whether they checked in to the hotel at the same time, the similarity between their shopping preferences, and the same delivery. It depends on features such as having an address.

It should be noted that the process of determining user similarity (including step S404 and step S406) and the feature extraction process above (i.e., step S408) are executed in chronological order. In practical applications, the processing of determining user similarity and the processing of feature extraction may also be executed simultaneously or in reverse order, which is not limited to the embodiments of the present application.

In step S410, sample data for training the classification model is determined according to the associated user characteristics between the two sets of user data in each user data pair and the user similarity corresponding to each user data pair.

In an implementation, the threshold may be set in advance. Depending on the threshold, user data pairs corresponding to user similarities greater than the threshold may be selected from the plurality of user data pairs. The associated user feature between two sets of user data in each selected user data pair can be used as a user feature to train the classification model. The user similarity corresponding to the selected user feature and the selected user data pair may be determined as sample data for training the classification model.

The processing of step S410 may be implemented in various other ways besides the manners described above. The following further provides an optional treatment scheme specifically comprising the following Step 1 and Step 2:

In step 1, a positive sample feature and a negative sample feature are selected from user features corresponding to the plurality of user data pairs according to a user similarity corresponding to each user data pair and a predetermined similarity threshold.

In an embodiment, based on the common sense that identical twins have very similar appearance, same date of birth and place of birth, and generally the same last name, to calculate whether two users are identical twins, the user similarity between two user's face images is calculated do. Specifically, for example, a similarity threshold of 80% or 70% may be set in advance. The user data pair corresponding to the user similarity larger than the similarity threshold may be determined as the user data pair of identical twins, and the user data pair corresponding to the user similarity smaller than the similarity threshold may be determined as the user data pair of the twin not identical. On the other hand, since identical twins and fraternal twins have basically the same characteristics except for their appearance, the user features corresponding to the user data pairs of identical twins can be used as positive sample features of the similarity classification model, User features corresponding to user data pairs of twins that are not identical may be used as negative sample features of the similarity classification model.

It should be noted that a negative sample feature does not mean that the features contained therein are all user features of the fraternal twins. In practical applications, user features of fraternal twins can occupy a very small portion of negative sample features, or negative sample features can include a small number of positive sample features, which do not affect the training of the classification model, but have similarities. It will improve the robustness of the classification model.

In addition, the positive sample feature may include the same quantity of features as the negative sample feature. For example, 10000 user data pairs corresponding to less than 10% user similarity are selected from a plurality of user data pairs, and 10000 user data corresponding to greater than 10% and less than 20% user similarity from a plurality of user data pairs. A pair is selected and 10000 user data pairs corresponding to more than 20% and less than 30% user similarity from the plurality of user data pairs are selected and corresponding to more than 30% and less than 40% user similarity from the plurality of user data pairs. 10000 user data pairs are selected, and 10000 user data pairs corresponding to more than 40% and less than 50% user similarity are selected from the plurality of user data pairs. The user feature of the 50000 user data pair above is used as the negative sample feature. 40000 user data pairs corresponding to more than 80% and less than 90% user similarity are selected from the plurality of user data pairs, and 10000 user data pairs corresponding to more than 90% and less than 100% user similarity are multiple user data pairs Is selected from. The user feature of the 50000 user data pair above is used as a positive sample feature.

In step 2, the positive sample feature and the negative sample feature are used as sample data for training the classification model.

In an implementation, data of the user characteristic and corresponding user similarity may be combined, and the combined data may be used as sample data for training the classification model.

In step S412, the classification model is trained based on the sample data to obtain a similarity classification model.

Since the main purpose of the classification model is to identify twins, the similarity classification model is a binary classifier model, specifically the Gradient Boosting Decision Tree (GBDT) to simplify and realize the embodiments of the present application. It may be a binary classifier model.

In implementations, the positive sample features may each be entered into a classification model for calculation. The obtained calculation result can be compared with user similarity corresponding to the positive sample feature. If the two match each other, the next positive sample feature or negative sample feature may be selected and entered into the classification model for calculation. The obtained calculation result is continuously compared with the user similarity corresponding to the positive sample feature. If the two do not match, the relevant parameters of the classification model may be adjusted, and then the positive sample feature is input into the adjusted classification model for calculation, and the obtained calculation result is again compared with the user similarity corresponding to the positive sample feature. The procedure is repeated until the two match each other. By the above method, all positive sample features and all negative sample features can be entered into the classification model for calculation, thereby training the classification model. The final classification model obtained through training can be used as a similarity classification model.

Thus, the similarity classification model is obtained through the above processing procedure. The similarity classification model can be applied to face recognition scenarios. For twin users who may be involved in risk-related behavior, the similarity classification model can be used for separate risk control.

As shown in FIG. 5, after the similarity classification model is obtained, it may be determined whether the detection target users corresponding to the detection target user data pairs are twins by using the similarity classification model. For specific embodiments, reference may be made to the related contents in the above steps (S414) to (S420).

In step S414, a detection target user data pair is obtained.

The content of step S414 is the same as the content of step S202 of the first embodiment, and for specific embodiments of step S414, reference may be made to the related content of step S202, and details are described herein again. It doesn't work.

In step S416, feature extraction is performed on each set of detection target user data in the detection target user data pair to obtain the detection target user feature.

For the process of performing feature extraction on each set of detection target user data in the detection target user data pair to obtain the detection target user feature in step S416, the related contents of step S408 may be referred to. . That is, the features extracted from the detection target user data may include, but are not limited to, household registration dimension features, name dimension features, social features, and features of interest. Reference may be made to the relevant contents of step S408, and details are not described herein again.

In step S418, the similarity between the users corresponding to the two sets of the detected user data in the detected user data pair is determined according to the detected user feature and the pretrained similarity classification model.

The content of step S418 is the same as the content of step S206 of Embodiment 1, and for the specific implementation of step S418, reference may be made to the related content of step S206, and details are described herein again. It doesn't work.

In step S420, if the similarity between the users corresponding to the two sets of the detected user data pairs in the detected user data pair is greater than the predetermined similarity classification threshold, it is determined that the detected users corresponding to the detected user data pair are twins. .

In an embodiment, the outputted twin list affects the use of the facial recognition of the target user, and therefore is preferably a high accuracy similarity classification model to use. In practical applications, the similarity classification threshold may be set to a large value, for example 95% or 97%. The user features to be detected are predicted and scored using a trained similarity classification model. The scoring process is to calculate the probability that the users corresponding to the corresponding user data pair are twins. For example, if the probability is 80%, the score is 80; If the probability is 90%, the score is 90. The higher the score, the more likely the users corresponding to the user data pair are twins.

Embodiments of the present application provide a method of determining data similarity in which a plurality of user data pairs are obtained, wherein two sets of user data data fields in each user data pair have the same portion; User similarity corresponding to each user data pair is obtained; Sample data for training a preset classification model is determined; Then, since the classification model is trained based on the sample data to obtain the similarity classification model, the similarity between the users corresponding to the two sets of detection user data in the detection user data pair may be later determined according to the similarity classification model. . In this way, a plurality of user data pairs are obtained only through the same data field, and the association between users corresponding to two sets of user data in each user data pair to obtain sample data for training a preset classification model. As determined by the similarity, that is, sample data can be obtained without manual classification, fast training of the model can be implemented, efficiency of model training can be improved, and resource consumption can be reduced.

Example 3

The method of determining data similarity provided by the embodiments of the present application is described above. Based on the same concept, an embodiment of the present application further provides a model training apparatus as shown in FIG. 6.

The model training device may be deployed on a server. The apparatus includes a data acquisition module 601, a similarity acquisition module 602, a sample data determination module 603, and a model training module 604, where:

The data acquisition module 601 is configured to acquire a plurality of user data pairs, wherein the data fields of two sets of user data in each user data pair have the same portion;

Similarity obtaining module 602 is configured to obtain user similarity corresponding to each user data pair, where the user similarity is similarity between users corresponding to two sets of user data in each user data pair;

The sample data determination module 603 is configured to determine, according to a user similarity corresponding to each user data pair and a plurality of user data pairs, sample data for training a preset classification model;

Model training module 604 is configured to train the classification model based on the sample data to obtain a similarity classification model.

In an embodiment of the present application, the similarity obtaining module 602 is,

A biological feature acquisition unit configured to acquire a biological feature of a user corresponding to the first user data pair, wherein the first user data pair is any user data pair of the plurality of user data pairs; And

And a similarity obtaining unit, configured to determine user similarity corresponding to the first user data pair according to a biological characteristic of the user corresponding to the first user data pair.

In an embodiment of the present application, the biological feature comprises a face image feature;

The biological feature acquisition unit acquires a face image of the user corresponding to the first user data pair; Perform feature extraction on the face image to obtain a face image feature;

Correspondingly, the similarity obtaining unit is configured to determine the user similarity corresponding to the first user data pair according to the facial image feature of the user corresponding to the first user data pair.

In an embodiment of the present application, the biological feature comprises a negative feature;

The biological feature acquisition unit obtains voice data of a user corresponding to the first user data pair; Perform feature extraction on speech data to obtain a speech characteristic;

Correspondingly, the similarity obtaining unit is configured to determine the user similarity corresponding to the first user data pair according to the voice characteristic of the user corresponding to the first user data pair.

In an embodiment of the present application, the sample data determination module 603,

A feature extraction unit configured to perform feature extraction on each user data pair in the plurality of user data pairs to obtain an associated user feature between two sets of user data in each user data pair; And

And a sample data determination unit configured to determine sample data for training the classification model, according to associated user characteristics between two sets of user data in each user data pair and user similarity corresponding to each user data pair.

In an embodiment of the present application, the sample data determination unit selects a positive sample feature and a negative sample feature from user features corresponding to the plurality of user data pairs according to user similarities and predetermined similarity thresholds corresponding to each user data pair and ; The positive sample feature and the negative sample feature are configured to use as sample data for training the classification model.

In an embodiment of the present application, the user feature includes a household registration dimension feature, a name dimension feature, a social feature, and a feature of interest, wherein the generation registration dimension feature includes a feature of user identity information, and the name dimension feature includes user name information. The characteristics of the characteristics and the degree of scarcity of the user characteristics, and the social characteristics include the characteristics of the user's social relationship information.

In an embodiment of the present application, the positive sample feature includes the same quantity of features as the negative sample feature.

In an embodiment of the present application, the similarity classification model is a binary classifier model.

Embodiments of the present application provide a model training apparatus from which a plurality of user data pairs are obtained, wherein two sets of user data data fields in each user data pair have the same portion; User similarity corresponding to each user data pair is obtained; Sample data for training a preset classification model is determined; Then, since the classification model is trained based on the sample data to obtain the similarity classification model, the similarity between the users corresponding to the two sets of detection user data in the detection user data pair may be determined according to the similarity classification model. In this way, a plurality of user data pairs are obtained only through the same data field, and the association between users corresponding to two sets of user data in each user data pair to obtain sample data for training a preset classification model. As determined by the similarity, that is, sample data can be obtained without manual classification, fast training of the model can be implemented, efficiency of model training can be improved, and resource consumption can be reduced.

Example 4

The model training apparatus provided by the embodiment of the present application is described above. Based on the same concept, an embodiment of the present application further provides an apparatus for determining data similarity as shown in FIG.

The data similarity determination apparatus includes a detection target data acquisition module 701, a feature extraction module 702, and a similarity determination module 703, where:

The detection target data acquisition module 701 is configured to acquire the detection target user data pair;

The feature extraction module 702 is configured to obtain a detection target user feature for each set of detection target user data in the detection target user data pair;

The similarity determination module 703 is configured to determine the similarity between the users corresponding to the two sets of the detected user data in the detected user data pair according to the detected user feature and the pretrained similarity classification model.

In an embodiment of the present application, the apparatus,

A similarity classification module configured to determine, as twins, a detected user corresponding to the detected user data pair as twin if the similarity between the users corresponding to the two sets of detected user data in the detected user data pair is greater than a predetermined similarity classification threshold. It includes more.

Embodiments of the present application provide a data similarity determining apparatus in which a plurality of user data pairs are obtained, wherein two sets of user data data fields in each user data pair have the same portion; User similarity corresponding to each user data pair is obtained; Sample data for training a preset classification model is determined; Then, since the classification model is trained based on the sample data to obtain the similarity classification model, the similarity between the users corresponding to the two sets of detection user data in the detection user data pair may be determined according to the similarity classification model. In this way, a plurality of user data pairs are obtained only through the same data field, and the association between users corresponding to two sets of user data in each user data pair to obtain sample data for training a preset classification model. As determined by the similarity, that is, sample data can be obtained without manual classification, fast training of the model can be implemented, efficiency of model training can be improved, and resource consumption can be reduced.

Example 5

Based on the same concept, an embodiment of the present application further provides a model training device as shown in FIG. 8.

The model training device may be a server or the like provided in the above-described embodiment.

The model training device may vary greatly depending on different configurations or capabilities, and may include one or more processors 801 and memory 802. Memory 802 may store one or more storage applications or data. Memory 802 may be temporary or permanent storage. The application stored in memory 802 may include one or more modules (not shown), where each module may include a series of computer executable instructions on a model training device. In addition, the processor 801 may be configured to communicate with the memory 802 and execute a series of computer executable instructions in the memory 802 on the model learning device. The model training device may further include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input / output interfaces 805, and one or more keyboards 806.

Specifically, in this embodiment, the model training device includes a memory and one or more programs. One or more programs are stored in memory. One or more programs may include one or more modules, each module comprising a series of computer executable instructions on a model training device. One or more processors execute one or more programs,

Acquire a plurality of user data pairs, the data fields of two sets of user data in each user data pair having the same portion;

Obtain user similarities corresponding to each user data pair, wherein the user similarities are similarities between users corresponding to two sets of user data in each user data pair;

Determine, according to the user similarity and the plurality of user data pairs corresponding to each user data pair, sample data for training a preset classification model;

And execute computer executable instructions that train the classification model based on the sample data to obtain a similarity classification model.

Optionally, when executed, the executable instruction further causes the processor to:

Acquire biological characteristics of a user corresponding to the first user data pair, wherein the first user data pair is any user data pair of the plurality of user data pairs; And

Determine user similarity corresponding to the first user data pair according to a biological characteristic of the user corresponding to the first user data pair.

Optionally, when executed, the executable instruction may cause the processor to additionally operate in the following manner, that is,

Biological features include facial image features;

Acquiring a biological characteristic of a user corresponding to the first user data pair is:

Acquiring a face image of a user corresponding to the first user data pair; And

Performing feature extraction on the face image to obtain a face image feature,

Correspondingly, determining the user similarity corresponding to the first user data pair according to the biological characteristics of the user corresponding to the first user data pair:

Determining user similarity corresponding to the first user data pair according to a facial image feature of the user corresponding to the first user data pair.

Optionally, when executed, the executable instruction may cause the processor to additionally operate in the following manner, that is,

Biological features include negative features;

Acquiring a biological characteristic of a user corresponding to the first user data pair is:

Acquiring voice data of a user corresponding to the first user data pair; And

Performing feature extraction on speech data to obtain speech features;

Correspondingly, determining user similarity corresponding to the first user data pair according to the biological characteristics of the user corresponding to the first user data pair,

Determining user similarity corresponding to the first user data pair according to a voice characteristic of the user corresponding to the first user data pair.

Optionally, when executed, the executable instruction further causes the processor to:

Perform feature extraction on each user data pair of the plurality of user data pairs to obtain an associated user feature between the two sets of user data of each user data pair;

Depending on the associated user characteristics between the two sets of user data in each user data pair and the user similarity corresponding to each user data pair, it may be possible to determine sample data for training the classification model.

Optionally, when executed, the executable instruction further causes the processor to:

Select a positive sample feature and a negative sample feature from user features corresponding to the plurality of user data pairs according to user similarities and predetermined similarity thresholds corresponding to each user data pair;

Positive sample features and negative sample features can be used as sample data for training classification models.

Optionally, the user feature includes a household registration dimension feature, a name dimension feature, a social feature, and a feature of interest, wherein the household registration dimension feature includes a feature of user identity information, wherein the name dimension feature is a feature of the user name information and the user Includes features of the degree of scarcity of gender, and social features include features of the user's social relationship information.

Optionally, the positive sample feature includes the same quantity of features as the negative sample feature.

Optionally, the similarity classification model is a binary classifier model.

Embodiments of the present application provide a model training device from which a plurality of user data pairs are obtained, wherein the data fields of two sets of user data in each user data pair have the same portion; User similarity corresponding to each user data pair is obtained; Sample data for training a preset classification model is determined; Then, since the classification model is trained based on the sample data to obtain the similarity classification model, the similarity between the users corresponding to the two sets of detection user data in the detection user data pair may be determined according to the similarity classification model. In this way, a plurality of user data pairs are obtained only through the same data field, and the association between users corresponding to two sets of user data in each user data pair to obtain sample data for training a preset classification model. As determined by the similarity, that is, sample data can be obtained without manual classification, fast training of the model can be implemented, efficiency of model training can be improved, and resource consumption can be reduced.

Example 6

Based on the same concept, an embodiment of the present application further provides a data similarity determination device as shown in FIG. 9.

The data similarity determining device may be a server, a terminal device, or the like provided in the above-described embodiment.

The data similarity determining device can vary greatly depending on different configurations or capabilities, and can include one or more processors 901 and memory 902. Memory 902 may store one or more storage applications or data. Memory 902 may be temporary or permanent storage. The application stored in memory 902 may include one or more modules (not shown), where each module may include a series of computer executable instructions at a data similarity determining device. In addition, the processor 901 may be configured to communicate with the memory 902 and to execute a series of computer executable instructions within the memory 902 on the data similarity determination device. The data similarity determining device may further include one or more power supplies 903, one or more wired or wireless network interfaces 904, one or more input / output interfaces 905, and one or more keyboards 906.

Specifically, in this embodiment, the data similarity determining device includes a memory and one or more programs. One or more programs are stored in memory. One or more programs may include one or more modules, each module comprising a series of computer executable instructions on a data similarity determining device. One or more processors are configured to execute one or more programs to execute computer executable instructions, such as:

Acquiring a detection target user data pair;

Performing feature extraction on each set of detection target user data in the detection target user data pair;

Determining similarity between the users corresponding to the two sets of detected user data in the detected user data pair according to the detected user feature and the pre-trained similarity classification model.

Optionally, when executed, the executable instruction further causes the processor to:

If the similarity between two users corresponding to the two sets of detection user data in the detection user data pair is larger than a predetermined similarity classification threshold, the detection user corresponding to the detection user data pair may be determined to be twins.

Embodiments of the present application provide a data similarity determining device from which a plurality of user data pairs are obtained, wherein the data fields of two sets of user data in each user data pair have the same portion; User similarity corresponding to each user data pair is obtained; Sample data for training a preset classification model is determined; Then, since the classification model is trained based on the sample data to obtain the similarity classification model, the similarity between the users corresponding to the two sets of detection user data in the detection user data pair may be determined according to the similarity classification model. In this way, a plurality of user data pairs are obtained only through the same data field, and the association between users corresponding to two sets of user data in each user data pair to obtain sample data for training a preset classification model. As determined by the similarity, that is, sample data can be obtained without manual classification, fast training of the model can be implemented, efficiency of model training can be improved, and resource consumption can be reduced.

Certain embodiments of the present specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps set forth in the claims may be performed in a different order than that described in the embodiments and may still achieve the desired results. In addition, the processes depicted in the figures do not necessarily have to be in the specific order or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing may also be possible or advantageous.

In the 1990s, improvements in technology could be clearly distinguished as improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, and switches) or improvements in software (improvements in method procedures). However, with advances in technology, improvements in many current method procedures can now be regarded as direct improvements in hardware circuit structure. Most all designers program the improved method procedures into hardware circuits to obtain corresponding hardware circuit structures. Thus, it is inappropriate to assume that improvements in method procedures cannot be implemented using material hardware modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by a device programmed by a user. to be. Designers program themselves to "integrate" digital systems into a single piece of PLD without having to ask the chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, at present, programming is mostly implemented using logic compiler software, instead of manually manufacturing integrated circuit chips. Logic compiler software is similar to the software compiler used to develop and write programs, and the original code must be written using a specific programming language called Hardware Description Language (HDL) before compiling. Many types of HDLs, such as Advanced Boolean Expression Language (ABEL), Altera Hardware Description Language (AHDL), Confluence, Cornell University Programming Language (CUPL), HDCal, Java Hardware Description Language (JHDL), Raver (Lava), Lola, MyHDL, PALASM, and Ruby Hardware Description Language (RHDL), among which the most commonly used is High-Speed Integrated Circuit Hardware Description Language (VHDL) and Verilog. do. One of ordinary skill in the art also knows that a hardware circuit for implementing a logic method procedure can be easily obtained by programming the method procedure slightly logically and programming it in an integrated circuit using some of the hardware description languages above. Should be

The controller can be implemented in any suitable manner. For example, the controller may be a computer executable by, for example, a microprocessor or processor, and (micro) processors, logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Computer-readable media that stores readable program code (eg, software or firmware). Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicon Labs C8051F320. The memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will appreciate that the controller may be implemented using only pure computer readable program code, and that the method steps allow the controller to perform the same functions as logic gates, switches, application specific integrated circuits, programmable logic controllers, and embedded microcontrollers. It also knows that it can be logically programmed to implement in the form of a controller. Thus, this type of controller can be considered as a hardware component, and the apparatus included herein to implement various functions can also be considered as a structure inside the hardware component. Or, even an apparatus configured to implement various functions may be regarded as both a software module for implementing a method and a structure inside a hardware component.

The systems, devices, modules or units shown in the above embodiments may be specifically implemented using computer chips or material objects or products with specific functions. Typical implementation device is a computer. Specifically, the computer is, for example, a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email device, game console, tablet computer or wearable device or any such It can be a combination of devices.

For ease of explanation, when the above apparatus is described, the apparatus is divided into various units in terms of functions for each description. Clearly, when the present application is implemented, the functionality of the unit may be implemented in the same or multiple pieces of software and / or hardware.

As will be appreciated by those skilled in the art, embodiments of the present application may be implemented as a method, system, or computer program product. Thus, the present application may employ the form of a hardware embodiment as a whole, a software embodiment as a whole, or an embodiment combining software and hardware aspects. In addition, embodiments of the present application may include a storage medium usable by one or more computers, including but not limited to, program code usable by a computer, including but not limited to magnetic disk memory, CD-ROM, optical memory, and the like. It may be in the form of a computer program product implemented on.

This application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products in embodiments of the present application. It should be understood that computer program instructions may implement each process and / or block in the flowcharts and / or block diagrams and a combination of processes and / or blocks in the flowcharts and / or block diagrams. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of another programmable data processing device to generate a machine to implement the specified functions in one or more processes in the flowcharts and / or one or more blocks in the block diagram. The configured apparatus may be generated using instructions executed by a processor of a computer or other programmable data processing device.

These computer program instructions may also be stored in computer readable memory, which may direct a computer or other programmable data processing device to operate in a specified manner, such that the instructions stored in the computer readable memory may produce a product that includes the instruction device. Wherein the instruction device implements the functionality specified in one or more processes in the flowcharts and / or one or more blocks in the block diagrams.

These computer program instructions may also be loaded into a computer or other programmable data processing device such that a series of operating steps may be performed on the computer or other programmable data processing device to produce processing implemented by the computer, and the computer or other programmable data processing Instructions executed on the device provide steps for implementing the specified functionality in one or more processes in the flowcharts and / or one or more blocks in the block diagram.

In a typical configuration, the computing device includes one or more central processing units (CPUs), input / output interfaces, network interfaces and memory.

The memory may be in the form of computer readable media: volatile memory, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or It may include flash RAM. Memory is an example of a computer readable medium.

Computer-readable media includes volatile and nonvolatile, mobile and non-mobile media, and can store information using any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media in a computer are, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), which can be used to store computer accessible information. ), Dynamic random access memory (DRAM), other types of RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, Compact disc read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, cassette tape, tape disk storage or other magnetic storage device, or any Other non-transmission media. According to the definition herein, computer readable media does not include transitory computer readable media (transitory media), eg, modulated data signals and carriers.

“Include”, “include” or other variations thereof are intended to encompass non-exclusive inclusions, so that a process, method, article or device comprising a series of elements not only includes the element, but also explicitly enumerates it. It should be further appreciated that it includes other elements that are not, or further includes inherent elements of a process, method, product or device. In the absence of further limitations, an element defined by “comprising one / one…” does not exclude that a process, method, article or device comprising that element further has another identical element.

As will be appreciated by those skilled in the art, embodiments of the present application may be implemented as a method, system, or computer program product. Thus, the present application may employ the form of a hardware embodiment as a whole, a software embodiment as a whole, or an embodiment combining software and hardware aspects. In addition, embodiments of the present application may include a storage medium usable by one or more computers, including but not limited to, program code usable by a computer, including but not limited to magnetic disk memory, CD-ROM, optical memory, and the like. It may be in the form of a computer program product implemented on.

The present application may be described in the general context of computer-executable instructions, eg, program modules, executed by a computer. Generally, program modules include routines, programs, objects, assemblies, data structures, etc. that are used to perform particular tasks or to implement particular abstract data types. The present application can also be implemented in a distributed computing environment, where tasks are performed using remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

Embodiments of the present specification are described gradually, and the same or similar parts of the embodiments may be referred to each other, and each embodiment emphasizes different aspects from the other embodiments. In particular, the system implementation is basically similar to the method embodiment and therefore briefly described. For related parts, reference may be made to the description of the parts of the method embodiments.

The above description is only an embodiment of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application are within the scope of the claims of the present application.

Claims (24)

As a model training method,
Obtaining a plurality of user data pairs, the data fields of two sets of user data in each user data pair having the same portion;
Obtaining user similarity corresponding to each user data pair, wherein the user similarity is similarity between users corresponding to the two sets of user data in each user data pair;
Determining, according to the user similarity and the plurality of user data pairs corresponding to each user data pair, sample data for training a preset classification model; And
Training the classification model based on the sample data to obtain a similarity classification model. The method of claim 1,
Acquiring user similarity corresponding to each pair of user data,
Acquiring a biological characteristic of a user corresponding to a first user data pair, wherein the first user data pair is any user data pair of the plurality of user data pairs; And
Determining user similarity corresponding to the first user data pair according to the biological characteristic of the user corresponding to the first user data pair. The method of claim 2,
The biological feature comprises a face image feature;
Acquiring a biological characteristic of a user corresponding to the first user data pair includes:
Acquiring a face image of the user corresponding to the first user data pair; And
Performing feature extraction on the face image to obtain a face image feature;
Correspondingly, determining a user similarity corresponding to the first user data pair according to the biological characteristic of the user corresponding to the first user data pair:
Determining the user similarity corresponding to the first user data pair according to a facial image feature of the user corresponding to the first user data pair. The method of claim 2,
The biological characteristic comprises a negative characteristic;
Acquiring a biological characteristic of a user corresponding to the first user data pair may include:
Acquiring voice data of the user corresponding to the first user data pair; And
Performing feature extraction on the speech data to obtain a speech feature;
Correspondingly, determining the user similarity corresponding to the first user data pair according to the biological characteristic of the user corresponding to the first user data pair,
Determining the user similarity corresponding to the first user data pair in accordance with a voice characteristic of the user corresponding to the first user data pair. The method of claim 1,
Determining sample data for training a classification model according to the user similarity and the plurality of user data pairs corresponding to each user data pair,
Performing feature extraction on each user data pair in the plurality of user data pairs to obtain an associated user feature between the two sets of user data in each user data pair; And
Determining the sample data for training the classification model according to the associated user characteristics between the user data in each user data pair and the user similarity corresponding to each user data pair. Way. The method of claim 5,
Determining, according to the associated user characteristics between the two sets of user data in each user data pair and the user similarity corresponding to each user data pair, the sample data for training the classification model,
Selecting a positive sample feature and a negative sample feature from user features corresponding to the plurality of user data pairs according to the user similarity and a predetermined similarity threshold corresponding to each user data pair; And
Using the positive sample feature and the negative sample feature as the sample data for training the classification model. The method of claim 6,
The user feature includes a household registration dimension feature, a name dimension feature, a social feature, and a feature of interest, the household registration dimension feature includes a feature of user identity information, and the name dimension feature is a user name. A characteristic of the information and a degree of scarcity of the user's surname, wherein the social characteristic comprises a characteristic of the user's social relationship information. The method of claim 6,
The positive sample feature comprises a quantity of features equal to the negative sample feature. The method according to any one of claims 1 to 8,
The similarity classification model is a binary classifier model. As a data similarity determination method,
Acquiring a to-be-detected user data pair;
Performing feature extraction on each set of detection user data in the detection user data pair to obtain a detection user feature; And
Determining similarity between the users corresponding to the two sets of detected user data in the detected user data pair according to the detected user feature and a pre-trained similarity classification model. The method of claim 10,
The method,
If the similarity between the users corresponding to the two sets of detection user data in the detection user data pair is greater than a predetermined similarity classification threshold, determining the detection user corresponding to the detection user data pair as twin. Further comprising a data similarity determination method. As a model training device,
A data acquisition module configured to acquire a plurality of user data pairs, the data fields of two sets of user data in each user data pair having the same portion;
A similarity obtaining module configured to obtain user similarity corresponding to each user data pair, wherein the user similarity is a similarity between users corresponding to the two sets of user data in each user data pair;
A sample data determination module, configured to determine sample data for training a preset classification model according to the user similarity and the plurality of user data pairs corresponding to each user data pair; And
And a model training module configured to train the classification model based on the sample data to obtain a similarity classification model. The method of claim 12,
The similarity acquisition module,
A biological feature acquisition unit configured to acquire a biological feature of a user corresponding to a first user data pair, wherein the first user data pair is any user data pair of the plurality of user data pairs; And
And a similarity obtaining unit configured to determine a user similarity corresponding to the first user data pair according to the biological characteristic of the user corresponding to the first user data pair. The method of claim 13,
The biological feature comprises a face image feature;
The biological feature acquisition unit acquires a face image of the user corresponding to the first user data pair; Perform feature extraction on the face image to obtain a face image feature;
Correspondingly, the similarity obtaining unit is configured to determine the user similarity corresponding to the first user data pair according to the face image feature of the user corresponding to the first user data pair. The method of claim 13,
The biological characteristic comprises a negative characteristic;
The biological feature acquisition unit acquires voice data of the user corresponding to the first user data pair; Perform feature extraction on the speech data to obtain a speech feature;
Correspondingly, the similarity obtaining unit is configured to determine the user similarity corresponding to the first user data pair according to the voice characteristic of the user corresponding to the first user data pair. The method of claim 12,
The sample data determination module,
A feature extraction unit configured to perform feature extraction on each user data pair in the plurality of user data pairs to obtain an associated user feature between the two sets of user data in each user data pair; And
A sample data determination unit, configured to determine the sample data for training the classification model according to the associated user characteristics between the two sets of user data in each user data pair and the user similarity corresponding to each user data pair. Including, model training device. The method of claim 16,
The sample data determination unit selects a positive sample feature and a negative sample feature from user features corresponding to the plurality of user data pairs according to the user similarity and predetermined similarity threshold corresponding to each user data pair; And use the positive sample feature and the negative sample feature as the sample data for training the classification model. The method of claim 17,
The user feature includes a household registration dimension feature, a name dimension feature, a social feature, and a feature of interest, the household registration dimension feature includes a feature of user identity information, and the name dimension feature includes a feature of user name information and A feature of a degree of scarcity, wherein the social feature comprises a feature of the user's social relationship information. The method of claim 17,
And the positive sample feature comprises a quantity of features equal to the negative sample feature. The method according to any one of claims 12 to 19,
The similarity classification model is a binary classifier model. A device for determining data similarity,
A detection target data acquisition module configured to acquire a detection target user data pair;
A feature extraction module configured to perform feature extraction on each set of detection target user data in the detection target user data pair to obtain a detection target user feature; And
And a similarity determination module configured to determine similarity between users corresponding to the two sets of detected user data in the detected user data pair according to the detected user feature and a pre-trained similarity classification model. . The method of claim 21,
The device,
If the similarity between the users corresponding to the two sets of detection target user data in the detection target user data pair is greater than a predetermined similarity classification threshold, determine that the detection target user corresponding to the detection target user data pair is twin. And a similarity classification module configured. As a model training device,
A processor; And
A memory configured to store computer executable instructions,
When executed, the computer executable instructions cause the processor to perform the following actions:
Obtaining a plurality of user data pairs, wherein the data fields of two sets of user data in each user data pair have the same portion;
Obtaining user similarity corresponding to each user data pair, wherein the user similarity is similarity between users corresponding to the two sets of user data in each user data pair;
Determining, according to the user similarity and the plurality of user data pairs corresponding to each user data pair, sample data for training a preset classification model; And
Training the classification model based on the sample data to obtain a similarity classification model
A model training device, which makes it run. A data similarity determining device,
A processor; And
A memory configured to store computer executable instructions,
When executed, the computer executable instructions cause the processor to perform the following actions:
Acquiring a detection target user data pair;
Performing feature extraction on each set of detection user data in the detection user data pair to obtain a detection user feature; And
Determining similarity between users corresponding to the two sets of detected user data in the detected user data pair according to the detected user feature and a pre-trained similarity classification model.
Device for determining similarity of data.

Images