CN111340053A - Order classification method, classification system, computer device and readable storage medium - Google Patents
Order classification method, classification system, computer device and readable storage medium Download PDFInfo
- Publication number
- CN111340053A CN111340053A CN201811467925.1A CN201811467925A CN111340053A CN 111340053 A CN111340053 A CN 111340053A CN 201811467925 A CN201811467925 A CN 201811467925A CN 111340053 A CN111340053 A CN 111340053A
- Authority
- CN
- China
- Prior art keywords
- order
- abnormal
- information
- sample
- orders
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/01—Customer relationship services
- G06Q30/012—Providing warranty services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/01—Customer relationship services
- G06Q30/015—Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
- G06Q30/016—After-sales
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0282—Rating or review of business operators or products
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0633—Lists, e.g. purchase orders, compilation or processing
- G06Q30/0635—Processing of requisition or of purchase orders
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- General Physics & Mathematics (AREA)
- Marketing (AREA)
- Economics (AREA)
- General Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Entrepreneurship & Innovation (AREA)
- Game Theory and Decision Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the disclosure provides an order classification method, a classification system, computer equipment and a readable storage medium. The order classification method comprises the following steps: acquiring an abnormal order and extracting the characteristic information of the abnormal order; and inputting the characteristic information of the abnormal order into a preset classification model, and performing responsibility party classification on the abnormal order. By adopting the embodiment of the disclosure, the responsible party of the abnormal order can be accurately judged, so that intervention measures can be performed, the abnormal situation of the order is reduced, and the success rate of the order is improved.
Description
Technical Field
The disclosed embodiment relates to the technical field of computers, in particular to an order classification method, a classification system, computer equipment and a readable storage medium.
Background
In a project managed by a network appointment platform for drivers and users, for an abnormal order such as cancellation, complaint or bad comment, a responsible party of the abnormality needs to be determined, wherein the responsible party mainly comprises the drivers, the users or the network appointment platform, the abnormal order is easy to obtain, but the labeling of the responsible party is difficult, namely the labels of training samples are very difficult to obtain, and the quantity of the obtained training samples is very limited.
In the related art, a part of samples are marked through manual marking and according to feedback of a driver or a user, and comprise positive samples and negative samples, but the samples marked in the marking mode are very limited and cannot be guaranteed to be unbiased relative to an integral data set, so that the problem that training samples are inconsistent with actual estimated samples can occur when the part of samples are directly used for training, the effect possibly appearing on the training set is good, but the effect during actual estimation is poor, namely, the overfitting is one.
Disclosure of Invention
The disclosed embodiments are directed to solving at least one of the technical problems of the related art or related art
To this end, an aspect of the embodiments of the present disclosure is to provide an order classification method.
Another aspect of an embodiment of the present disclosure is to provide an order classification system.
It is yet another aspect of an embodiment of the present disclosure to provide a computer apparatus.
It is yet another aspect of an embodiment of the present disclosure to provide a computer-readable storage medium.
In view of this, according to an aspect of the embodiments of the present disclosure, an order classification method is provided, including: acquiring an abnormal order and extracting the characteristic information of the abnormal order; and inputting the characteristic information of the abnormal order into a preset classification model, and performing responsibility party classification on the abnormal order.
According to the order classification method provided by the embodiment of the disclosure, an abnormal order with abnormal conditions such as cancellation, complaint or bad comment is obtained, and one or more pieces of feature information of the abnormal order, which are related to the abnormal conditions, are extracted. Further, the responsibility party of the abnormal order is classified by using a preset classification model according to one or more characteristic information, namely whether the responsibility party of the abnormal order is a driver, a user or a network taxi appointment platform is distinguished. By adopting the embodiment of the disclosure, the responsible party of the abnormal order can be accurately judged, so that intervention measures can be performed, the abnormal situation of the order is reduced, and the success rate of the order is improved.
The order classification method according to the embodiment of the present disclosure may further have the following technical features:
in the above technical solution, preferably, the method further includes: collecting a plurality of sample abnormal orders, and dividing the plurality of sample abnormal orders into a first type of sample abnormal orders and a second type of sample abnormal orders; acquiring the responsibility party marking information of each first-class sample abnormal order according to the characteristic information of each first-class sample abnormal order; training a responsible party to label information, and establishing a preset classification model; inputting the characteristic information of the second-class abnormal sample orders into a preset classification model, performing responsibility party classification on the second-class abnormal sample orders and obtaining the responsibility party confidence coefficient of each second-class abnormal sample order; and performing iterative training on the first type of abnormal sample orders and the second type of abnormal sample orders with the responsibility side confidence coefficient larger than the first threshold value until the preset classification model reaches the preset convergence condition.
In the technical scheme, PU-Learning (Learning from Positive and Unlablebedsample, formal and unmarked exemplar) is a semi-supervised binary classification model, a binary classifier is trained by using a marked Positive sample and a large number of unmarked samples, and different from the common binary classification problem, the scale of P in the PU problem is usually quite small, and the enlargement of a Positive sample set is difficult; the scale of U is usually very large, for example, the webpage resources which are not identified in the webpage classification can be obtained from the network very cheaply and conveniently, and the purpose of introducing U is to reduce the preparation workload of manual classification, improve the precision and achieve the effect of automatic classification as far as possible. In the embodiment of the disclosure, a PU-learning idea is introduced, a large number of acquired sample abnormal orders are divided into a first type of sample abnormal order (a sample to be labeled) and a second type of abnormal sample order (a sample not to be labeled), feature information of the first type of sample abnormal order is obtained, responsibility party labeling is performed according to the feature information, and a preset classification model is initially established by training labeling information. Further, the unlabelled second-class abnormal sample orders are estimated by using a preset classification model, that is, the characteristic information of the second-class abnormal sample orders is input into the preset classification model to obtain the confidence coefficient (probability of responsible party), for example, the probability that the responsible party of a certain second-class abnormal sample order is a driver, the probability that the responsible party is a user, or the probability that the responsible party is a network taxi appointment platform (only one responsible party of one abnormal sample order). And adding the second type of abnormal sample orders with the confidence coefficient of the responsible party larger than the first threshold value into model training, continuing to train the model together with the first type of abnormal sample orders, and continuously iterating according to the method until the preset classification model reaches the preset convergence condition. The method for continuously and iteratively labeling the samples based on PU-learning solves the classification problem in label-free machine learning by using very limited labeled samples, and obtains the classification model of the accurate abnormal order.
In any of the foregoing technical solutions, preferably, the preset convergence condition includes that the sum of the numbers of the second-class abnormal sample orders and the first-class abnormal sample orders whose confidence of the responsible party is greater than the first threshold is greater than a second threshold, the accuracy of the first-class abnormal sample orders is greater than a third threshold, and the recall rate of the first-class abnormal sample orders is greater than a fourth threshold.
In the technical scheme, the preset convergence condition includes that the scale of the labeled sample is large enough (the sum of the numbers of the second-class abnormal sample orders with the responsibility party confidence degree larger than the first threshold and the first-class abnormal sample orders is larger than a second threshold), the accuracy of the part of the sample (the first-class abnormal sample order) obviously having the label is larger than a third threshold, the recall rate of the part of the sample (the first-class abnormal sample order) is larger than a fourth threshold, the accuracy is the ratio of the sample of the responsibility party determined in the first-class abnormal sample order to the total sample, the recall rate is the ratio of the sample of a certain responsibility party in the first-class abnormal sample order to all samples of the responsibility party (namely the ratio of the sample of the responsibility party as a user to all samples of the responsibility party as a user, a driver or a network reservation platform), and when the preset convergence condition is reached, a classification model of the accurate abnormal order is established.
In any of the above technical solutions, preferably, the acquiring of the liability party labeling information of each abnormal order of the first type sample specifically includes: receiving first responsibility party marking information of a first type sample abnormal order acquired according to first preset order information; and/or marking second responsibility party marking information for the first type sample abnormal order according to second preset order information; the responsibility party marking information comprises first responsibility party marking information and second responsibility party marking information.
In the technical scheme, positive samples are provided for establishing a preset classification model by marking responsible parties of each abnormal order of the first samples. When the responsible party of each abnormal order of the first type is marked, two methods can be included. A manual marking is carried out on a first type sample abnormal order by a marking team according to first preset order information, wherein the first preset order information is non-obvious information related to abnormal conditions, such as abnormal conditions which can be known only by asking a driver or a user by a telephone; and the other method is that the system automatically marks the first type sample abnormal order with second preset order information, wherein the second preset order information comprises information obviously fed back by a driver, a user or a network booking platform customer service, such as complaint or poor comment information on the order.
In any of the above technical solutions, preferably, the first preset order information includes one or a combination of the following: order track information, user historical order cancellation information, driver historical order cancellation information, user historical poor evaluation or complaint information, driver historical poor evaluation or complaint information, user and driver communication information, user and network car booking platform communication information, driver and network car booking platform communication information, network car booking platform call return information to users, and network car booking platform call return information to drivers; the second preset order information comprises one or a combination of the following: user bad comment or complaint information, driver bad comment or complaint information.
In the technical scheme, the preset order information is the characteristic information of each order, the first preset order information and the second preset order information include but are not limited to the above information, and the responsibility parties of the sample abnormal orders can be accurately marked through the first preset order information and the second preset order information.
In any of the above technical solutions, preferably, the responsible party of the abnormal order includes any one of: a user, a driver, or a network appointment platform.
In the technical scheme, the abnormal orders are classified into the abnormal orders with the responsibility party as the user, the abnormal orders with the responsibility party as the driver or the abnormal orders with the responsibility party as the network taxi appointment platform, so that intervention measures can be carried out to reduce abnormal conditions of the orders, the service quality of the user is guaranteed, and benefits of the driver and the network taxi appointment platform are guaranteed.
According to another aspect of the disclosed embodiments, there is provided an order classification system, including: the characteristic extraction unit is used for acquiring the abnormal order and extracting the characteristic information of the abnormal order; and the classification unit is used for inputting the characteristic information of the abnormal order into a preset classification model and classifying the responsibility parties of the abnormal order.
The order classification system provided by the embodiment of the disclosure acquires an abnormal order with abnormal conditions such as cancellation, complaint or bad comment, and extracts one or more pieces of characteristic information of the abnormal order, which is related to the abnormal conditions. Further, the responsibility party of the abnormal order is classified by using a preset classification model according to one or more characteristic information, namely whether the responsibility party of the abnormal order is a driver, a user or a network taxi appointment platform is distinguished. By adopting the embodiment of the disclosure, the responsible party of the abnormal order can be accurately judged, so that intervention measures can be performed, the abnormal situation of the order is reduced, and the success rate of the order is improved.
The order classification system according to the embodiment of the present disclosure may further have the following technical features:
in the above technical solution, preferably, the method further includes: the model establishing unit is used for acquiring a plurality of sample abnormal orders and dividing the plurality of sample abnormal orders into a first type of sample abnormal orders and a second type of sample abnormal orders; acquiring the responsibility party marking information of each first-class sample abnormal order according to the characteristic information of each first-class sample abnormal order; training a responsible party to label information, and establishing a preset classification model; inputting the characteristic information of the second-class abnormal sample orders into a preset classification model, performing responsibility party classification on the second-class abnormal sample orders and obtaining the responsibility party confidence coefficient of each second-class abnormal sample order; and performing iterative training on the first type of abnormal sample orders and the second type of abnormal sample orders with the responsibility side confidence coefficient larger than the first threshold value until the preset classification model reaches the preset convergence condition.
In the technical scheme, PU-Learning (Learning from Positive and Unlablebedsample, formal and unmarked exemplar) is a semi-supervised binary classification model, a binary classifier is trained by using a marked Positive sample and a large number of unmarked samples, and different from the common binary classification problem, the scale of P in the PU problem is usually quite small, and the enlargement of a Positive sample set is difficult; the scale of U is usually very large, for example, the webpage resources which are not identified in the webpage classification can be obtained from the network very cheaply and conveniently, and the purpose of introducing U is to reduce the preparation workload of manual classification, improve the precision and achieve the effect of automatic classification as far as possible. In the embodiment of the disclosure, a PU-learning idea is introduced, a large number of acquired sample abnormal orders are divided into a first type of sample abnormal order (a sample to be labeled) and a second type of abnormal sample order (a sample not to be labeled), feature information of the first type of sample abnormal order is obtained, responsibility party labeling is performed according to the feature information, and a preset classification model is initially established by training labeling information. Further, the unlabelled second-class abnormal sample orders are estimated by using a preset classification model, that is, the characteristic information of the second-class abnormal sample orders is input into the preset classification model to obtain the confidence coefficient (probability of responsible party), for example, the probability that the responsible party of a certain second-class abnormal sample order is a driver, the probability that the responsible party is a user, or the probability that the responsible party is a network taxi appointment platform (only one responsible party of one abnormal sample order). And adding the second type of abnormal sample orders with the confidence coefficient of the responsible party larger than the first threshold value into model training, continuing to train the model together with the first type of abnormal sample orders, and continuously iterating according to the method until the preset classification model reaches the preset convergence condition. The method for continuously and iteratively labeling the samples based on PU-learning solves the classification problem in label-free machine learning by using very limited labeled samples, and obtains the classification model of the accurate abnormal order.
In any of the foregoing technical solutions, preferably, the preset convergence condition includes that the sum of the numbers of the second-class abnormal sample orders and the first-class abnormal sample orders whose confidence of the responsible party is greater than the first threshold is greater than a second threshold, the accuracy of the first-class abnormal sample orders is greater than a third threshold, and the recall rate of the first-class abnormal sample orders is greater than a fourth threshold.
In the technical scheme, the preset convergence condition includes that the scale of the labeled sample is large enough (the sum of the numbers of the second-class abnormal sample orders with the responsibility party confidence degree larger than the first threshold and the first-class abnormal sample orders is larger than a second threshold), the accuracy of the part of the sample (the first-class abnormal sample order) obviously having the label is larger than a third threshold, the recall rate of the part of the sample (the first-class abnormal sample order) is larger than a fourth threshold, the accuracy is the ratio of the sample of the responsibility party determined in the first-class abnormal sample order to the total sample, the recall rate is the ratio of the sample of a certain responsibility party in the first-class abnormal sample order to all samples of the responsibility party (namely the ratio of the sample of the responsibility party as a user to all samples of the responsibility party as a user, a driver or a network reservation platform), and when the preset convergence condition is reached, a classification model of the accurate abnormal order is established.
In any of the above technical solutions, preferably, the model establishing unit is specifically configured to receive first responsibility party marking information of the first type sample abnormal order acquired according to the first preset order information; and/or marking second responsibility party marking information for the first type sample abnormal order according to second preset order information; the responsibility party marking information comprises first responsibility party marking information and second responsibility party marking information.
In the technical scheme, positive samples are provided for establishing a preset classification model by marking responsible parties of each abnormal order of the first samples. When the responsible party of each abnormal order of the first type is marked, two methods can be included. A manual marking is carried out on a first type sample abnormal order by a marking team according to first preset order information, wherein the first preset order information is non-obvious information related to abnormal conditions, such as abnormal conditions which can be known only by asking a driver or a user by a telephone; and the other method is that the system automatically marks the first type sample abnormal order with second preset order information, wherein the second preset order information comprises information obviously fed back by a driver, a user or a network booking platform customer service, such as complaint or poor comment information on the order.
In any of the above technical solutions, preferably, the first preset order information includes one or a combination of the following: order track information, user historical order cancellation information, driver historical order cancellation information, user historical poor evaluation or complaint information, driver historical poor evaluation or complaint information, user and driver communication information, user and network car booking platform communication information, driver and network car booking platform communication information, network car booking platform call return information to users, and network car booking platform call return information to drivers; the second preset order information comprises one or a combination of the following: user bad comment or complaint information, driver bad comment or complaint information.
In the technical scheme, the preset order information is the characteristic information of each order, the first preset order information and the second preset order information include but are not limited to the above information, and the responsibility parties of the sample abnormal orders can be accurately marked through the first preset order information and the second preset order information.
In any of the above technical solutions, preferably, the responsible party of the abnormal order includes any one of: a user, a driver, or a network appointment platform.
In the technical scheme, the abnormal orders are classified into the abnormal orders with the responsibility party as the user, the abnormal orders with the responsibility party as the driver or the abnormal orders with the responsibility party as the network taxi appointment platform, so that intervention measures can be carried out to reduce abnormal conditions of the orders, the service quality of the user is guaranteed, and benefits of the driver and the network taxi appointment platform are guaranteed.
According to a further aspect of the embodiments of the present disclosure, a computer device is provided, which includes a memory, a processor and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the steps of the order classification method according to any of the above technical solutions are implemented.
According to the computer device provided by the embodiment of the disclosure, when the processor executes the computer program, the steps of the order classification method according to any one of the above technical schemes are implemented, so that the computer device has all the beneficial effects of the order classification method according to any one of the above technical schemes.
According to yet another aspect of the embodiments of the present disclosure, a computer-readable storage medium is proposed, on which a computer program is stored, which when executed by a processor implements the steps of the order classification method according to any of the above-mentioned technical solutions.
The computer-readable storage medium provided in the embodiments of the present disclosure, when being executed by a processor, implements the steps of the order classification method according to any one of the above technical solutions, and therefore the computer-readable storage medium includes all the benefits of the order classification method according to any one of the above technical solutions.
Additional aspects and advantages of the disclosed embodiments will be set forth in part in the description which follows or may be learned by practice of the disclosed embodiments.
Drawings
The above and/or additional aspects and advantages of the embodiments of the present disclosure will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 illustrates a flow diagram of an order classification method of one embodiment of the disclosure;
FIG. 2 illustrates a flow diagram of an order classification method of another embodiment of the disclosed embodiments;
FIG. 3 is a diagram illustrating an order classification method according to one embodiment of the present disclosure;
FIG. 4 illustrates a schematic diagram of an order classification system of one embodiment of the present disclosure;
FIG. 5 shows a schematic diagram of an order classification system of another embodiment of the present disclosure;
FIG. 6 shows a schematic diagram of a computer device of one embodiment of the disclosed embodiments.
Detailed Description
In order that the above objects, features and advantages of the embodiments of the present disclosure can be more clearly understood, embodiments of the present disclosure will be described in further detail below with reference to the accompanying drawings and detailed description. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure, however, the embodiments of the disclosure may be practiced in other ways than those described herein, and therefore the scope of the embodiments of the disclosure is not limited to the specific embodiments disclosed below.
An embodiment of a first aspect of the embodiments of the present disclosure provides an order classification method, and fig. 1 illustrates a flowchart of the order classification method according to an embodiment of the present disclosure. Wherein, the method comprises the following steps:
102, acquiring an abnormal order and extracting the characteristic information of the abnormal order;
and 104, inputting the characteristic information of the abnormal order into a preset classification model, and classifying the abnormal order by responsibilities.
According to the order classification method provided by the embodiment of the disclosure, an abnormal order with abnormal conditions such as cancellation, complaint or bad comment is obtained, and one or more pieces of feature information of the abnormal order, which are related to the abnormal conditions, are extracted. Further, the responsibility party of the abnormal order is classified by using a preset classification model according to one or more characteristic information, namely whether the responsibility party of the abnormal order is a driver, a user or a network taxi appointment platform is distinguished. By adopting the embodiment of the disclosure, the responsible party of the abnormal order can be accurately judged, so that intervention measures can be performed, the abnormal situation of the order is reduced, and the success rate of the order is improved.
The characteristic information comprises order track information, user historical order cancellation information, driver historical order cancellation information, user historical poor evaluation or complaint information, driver historical poor evaluation or complaint information, user and driver communication information, user and network car booking platform communication information, driver and network car booking platform communication information, network car booking platform call return visit information for users, network car booking platform call return visit information for drivers, order user poor evaluation or complaint information, order driver poor evaluation or complaint information and the like.
For example, an abnormal order with an abnormal cancellation condition is obtained, order track information (characteristic information) of the abnormal order is extracted, the order track information is analyzed through a preset classification model, when a driver goes to a user position to pick up the user, the driver cancels the order at a position one kilometer away from the user position, and the preset drive receiving time is not reached, namely the driver does not time out, then the responsible party for canceling the order condition is judged as the user, and therefore the abnormal order is divided into the order with the responsible party as the user. For another example, an abnormal order with a customer complaint condition is obtained, customer complaint information (characteristic information) of the abnormal order is extracted, and if the complaint information is that the attitude of the driver is bad, the responsible party of the abnormal order is judged to be the driver, so that the abnormal order is divided into orders with the responsible party being the driver. For another example, an abnormal order with an abnormal cancellation condition is obtained, cancellation information (characteristic information) of the abnormal order is extracted, the cancellation information is that the network car-booking platform does not assign a vehicle to the user even after the user places an order for a long time, so that the user cancels the order, the responsibility party of the abnormal order is judged to be the network car-booking platform, and therefore the abnormal order is divided into orders with the responsibility party being the network car-booking platform.
Fig. 2 shows a flow diagram of an order classification method according to another embodiment of the disclosure. Wherein, the method comprises the following steps:
In this embodiment, PU-Learning (Learning from Positive and un-labeled sample), which is a semi-supervised binary classification model, trains a binary classifier by using labeled Positive samples and a large number of unlabelled samples, and unlike the general binary classification problem, the size of P in the PU problem is usually quite small, and it is difficult to expand the Positive sample set; the scale of U is usually very large, for example, the webpage resources which are not identified in the webpage classification can be obtained from the network very cheaply and conveniently, and the purpose of introducing U is to reduce the preparation workload of manual classification, improve the precision and achieve the effect of automatic classification as far as possible. In the embodiment of the disclosure, a PU-learning idea is introduced, a large number of acquired sample abnormal orders are divided into a first type of sample abnormal order (a sample to be labeled) and a second type of abnormal sample order (a sample not to be labeled), feature information of the first type of sample abnormal order is obtained, responsibility party labeling is performed according to the feature information, and a preset classification model is initially established by training labeling information. Further, the unlabelled second-class abnormal sample orders are estimated by using a preset classification model, that is, the characteristic information of the second-class abnormal sample orders is input into the preset classification model to obtain the confidence coefficient (probability of responsible party), for example, the probability that the responsible party of a certain second-class abnormal sample order is a driver, the probability that the responsible party is a user, or the probability that the responsible party is a network taxi appointment platform (only one responsible party of one abnormal sample order). And adding the second type of abnormal sample orders with the confidence coefficient of the responsible party larger than the first threshold value into model training, continuing to train the model together with the first type of abnormal sample orders, and continuously iterating according to the method until the preset classification model reaches the preset convergence condition. The method for continuously and iteratively labeling the samples based on PU-learning solves the classification problem in label-free machine learning by using very limited labeled samples, and obtains the classification model of the accurate abnormal order.
Preferably, the preset convergence condition includes that the sum of the numbers of the second-class abnormal sample orders and the first-class abnormal sample orders with the responsibility party confidence degree larger than the first threshold is larger than the second threshold, the accuracy of the first-class abnormal sample orders is larger than the third threshold, and the recall rate of the first-class abnormal sample orders is larger than the fourth threshold.
In this embodiment, the preset convergence condition includes that the scale of the labeled sample is large enough (the sum of the number of the second-class abnormal sample orders with the responsibility side confidence degree larger than the first threshold and the first-class abnormal sample orders is larger than the second threshold), the accuracy of the part of the sample (the first-class abnormal sample order) obviously having the label is larger than the third threshold, and the recall rate of the part of the sample (the first-class abnormal sample order) is larger than the fourth threshold, wherein the accuracy is the ratio of the sample of the responsibility side determined in the first-class abnormal sample order to the total sample, and the recall rate is the ratio of the sample of a certain responsibility side in the first-class abnormal sample order to all samples of the responsibility side (i.e. the ratio of the sample of the responsibility side being the user to all samples of the responsibility side being the user, the driver or the network reservation platform), and when the preset convergence condition is reached, the classification model of the.
Preferably, in step 204, the obtaining of the responsibility party labeling information of each abnormal order of the first type sample specifically includes: receiving first responsibility party marking information of a first type sample abnormal order acquired according to first preset order information; and/or marking second responsibility party marking information for the first type sample abnormal order according to second preset order information; the responsibility party marking information comprises first responsibility party marking information and second responsibility party marking information.
In this embodiment, positive samples are provided by labeling responsible parties for each abnormal order of the first type of sample as a pre-set classification model. When the responsible party of each abnormal order of the first type is marked, two methods can be included. A manual marking is carried out on a first type sample abnormal order by a marking team according to first preset order information, wherein the first preset order information is non-obvious information related to abnormal conditions, such as abnormal conditions which can be known only by asking a driver or a user by a telephone; and the other method is that the system automatically marks the first type sample abnormal order with second preset order information, wherein the second preset order information comprises information obviously fed back by a driver, a user or a network booking platform customer service, such as complaint or poor comment information on the order.
Preferably, the first preset order information includes one or a combination of the following: order track information, user historical order cancellation information, driver historical order cancellation information, user historical poor evaluation or complaint information, driver historical poor evaluation or complaint information, user and driver communication information, user and network car booking platform communication information, driver and network car booking platform communication information, network car booking platform call return information to users, and network car booking platform call return information to drivers; the second preset order information comprises one or a combination of the following: user bad comment or complaint information, driver bad comment or complaint information.
In this embodiment, the preset order information is feature information of each order, and the first preset order information and the second preset order information include, but are not limited to, the above information, and the first preset order information and the second preset order information can be used to accurately mark the responsible party of the sample abnormal order. For example, when it is determined that the driver goes to the user position to pick up the user through the order track information, the user cancels the order at a distance of one kilometer from the user position, and the preset pick-up time is not reached at this time, that is, the driver does not time out, then the responsible party for canceling the order is determined as the user. The bad comment or complaint information included in the first preset order information is history information, and the bad comment or complaint information included in the second preset order information is information in the current abnormal order.
Preferably, the responsible party for the exception order includes any one of: a user, a driver, or a network appointment platform.
In the embodiment, the abnormal orders are classified, and are divided into the abnormal orders with the responsibility party as the user, the abnormal orders with the responsibility party as the driver or the abnormal orders with the responsibility party as the network taxi appointment platform, so that intervention measures can be performed to reduce abnormal conditions of the orders, the service quality of the user is guaranteed, and benefits of the driver and the network taxi appointment platform are guaranteed.
FIG. 3 is a diagram illustrating an order classification method according to one embodiment of the present disclosure. Wherein, the method comprises the following steps:
and step 304, manually marking and feeding back information, randomly extracting some abnormal orders (thousand levels), and manually marking the abnormal orders by a marking team according to the order track information, the history information (history canceling orders, complaints or poor comments) of the driver and the user, the call information, the call return information of the driver and the user when necessary and the like. In addition, some abnormal orders have obvious direct feedback information of drivers, users or network appointment customer service, such as complaint or poor comment information and the like, the feedback information is used for marking, labels of a part of abnormal orders (namely, labels of responsibility parties of the abnormal orders) are marked according to the two ways, and the part of abnormal orders are the abnormal orders with the labels;
And 308, estimating unlabeled abnormal order samples by using the model to obtain the abnormal order samples with higher confidence coefficient, performing iterative training until the scale of the samples of the training model is large enough and the effect of the model is optimal, estimating the unlabeled abnormal order samples by using the model, selecting a part of the unlabeled abnormal order samples with higher confidence coefficient from the unlabeled abnormal order samples, adding the part of the unlabeled abnormal order samples into the training of the model, and continuously performing iterative training according to the method until convergence. The condition of model convergence is that the scale of the abnormal order sample of the training model is large enough, and the accuracy and recall rate of the model to the obviously labeled abnormal order are high enough (namely the effect of the model is best);
In a second aspect of the embodiments of the present disclosure, an order classification system is provided, and fig. 4 shows a schematic diagram of an order classification system 40 according to an embodiment of the embodiments of the present disclosure. Wherein the system 40 comprises:
a feature extraction unit 402, configured to obtain an abnormal order and extract feature information of the abnormal order;
the classifying unit 404 is configured to input the feature information of the abnormal order into a preset classification model, and perform responsibility classification on the abnormal order.
The order classification system 40 provided in the embodiment of the present disclosure obtains an abnormal order having abnormal situations such as cancellation, complaint, or bad comment, and extracts one or more pieces of feature information of the abnormal order, which are related to the abnormal situations. Further, the responsibility party of the abnormal order is classified by using a preset classification model according to one or more characteristic information, namely whether the responsibility party of the abnormal order is a driver, a user or a network taxi appointment platform is distinguished. By adopting the embodiment of the disclosure, the responsible party of the abnormal order can be accurately judged, so that intervention measures can be performed, the abnormal situation of the order is reduced, and the success rate of the order is improved.
The characteristic information comprises order track information, user historical order cancellation information, driver historical order cancellation information, user historical poor evaluation or complaint information, driver historical poor evaluation or complaint information, user and driver communication information, user and network car booking platform communication information, driver and network car booking platform communication information, network car booking platform call return visit information for users, network car booking platform call return visit information for drivers, order user poor evaluation or complaint information, order driver poor evaluation or complaint information and the like.
For example, an abnormal order with an abnormal cancellation condition is obtained, order track information (characteristic information) of the abnormal order is extracted, the order track information is analyzed through a preset classification model, when a driver goes to a user position to pick up the user, the driver cancels the order at a position one kilometer away from the user position, and the preset drive receiving time is not reached, namely the driver does not time out, then the responsible party for canceling the order condition is judged as the user, and therefore the abnormal order is divided into the order with the responsible party as the user. For another example, an abnormal order with a customer complaint condition is obtained, customer complaint information (characteristic information) of the abnormal order is extracted, and if the complaint information is that the attitude of the driver is bad, the responsible party of the abnormal order is judged to be the driver, so that the abnormal order is divided into orders with the responsible party being the driver. For another example, an abnormal order with an abnormal cancellation condition is obtained, cancellation information (characteristic information) of the abnormal order is extracted, the cancellation information is that the network car-booking platform does not assign a vehicle to the user even after the user places an order for a long time, so that the user cancels the order, the responsibility party of the abnormal order is judged to be the network car-booking platform, and therefore the abnormal order is divided into orders with the responsibility party being the network car-booking platform.
FIG. 5 shows a schematic diagram of an order classification system 50 of another embodiment of the disclosed embodiment. Wherein the system 50 comprises:
the model establishing unit 502 is configured to collect a plurality of sample exception orders, and divide the plurality of sample exception orders into a first type of sample exception order and a second type of sample exception order; acquiring the responsibility party marking information of each first-class sample abnormal order according to the characteristic information of each first-class sample abnormal order; training a responsible party to label information, and establishing a preset classification model; inputting the characteristic information of the second-class abnormal sample orders into a preset classification model, performing responsibility party classification on the second-class abnormal sample orders and obtaining the responsibility party confidence coefficient of each second-class abnormal sample order; performing iterative training on the first type sample abnormal orders and the second type abnormal sample orders with the responsibility side confidence coefficient larger than a first threshold value until a preset classification model reaches a preset convergence condition;
a feature extraction unit 504, configured to obtain an abnormal order and extract feature information of the abnormal order;
the classifying unit 506 is configured to input the feature information of the abnormal order into a preset classification model, and perform responsibility classification on the abnormal order.
In this embodiment, PU-Learning (Learning from Positive and un-labeled sample), which is a semi-supervised binary classification model, trains a binary classifier by using labeled Positive samples and a large number of unlabelled samples, and unlike the general binary classification problem, the size of P in the PU problem is usually quite small, and it is difficult to expand the Positive sample set; the scale of U is usually very large, for example, the webpage resources which are not identified in the webpage classification can be obtained from the network very cheaply and conveniently, and the purpose of introducing U is to reduce the preparation workload of manual classification, improve the precision and achieve the effect of automatic classification as far as possible. In the embodiment of the disclosure, a PU-learning idea is introduced, a large number of acquired sample abnormal orders are divided into a first type of sample abnormal order (a sample to be labeled) and a second type of abnormal sample order (a sample not to be labeled), feature information of the first type of sample abnormal order is obtained, responsibility party labeling is performed according to the feature information, and a preset classification model is initially established by training labeling information. Further, the unlabelled second-class abnormal sample orders are estimated by using a preset classification model, that is, the characteristic information of the second-class abnormal sample orders is input into the preset classification model to obtain the confidence coefficient (probability of responsible party), for example, the probability that the responsible party of a certain second-class abnormal sample order is a driver, the probability that the responsible party is a user, or the probability that the responsible party is a network taxi appointment platform (only one responsible party of one abnormal sample order). And adding the second type of abnormal sample orders with the confidence coefficient of the responsible party larger than the first threshold value into model training, continuing to train the model together with the first type of abnormal sample orders, and continuously iterating according to the method until the preset classification model reaches the preset convergence condition. The method for continuously and iteratively labeling the samples based on PU-learning solves the classification problem in label-free machine learning by using very limited labeled samples, and obtains the classification model of the accurate abnormal order.
Preferably, the preset convergence condition includes that the sum of the numbers of the second-class abnormal sample orders and the first-class abnormal sample orders with the responsibility party confidence degree larger than the first threshold is larger than the second threshold, the accuracy of the first-class abnormal sample orders is larger than the third threshold, and the recall rate of the first-class abnormal sample orders is larger than the fourth threshold.
In this embodiment, the preset convergence condition includes that the scale of the labeled sample is large enough (the sum of the number of the second-class abnormal sample orders with the responsibility side confidence degree larger than the first threshold and the first-class abnormal sample orders is larger than the second threshold), the accuracy of the part of the sample (the first-class abnormal sample order) obviously having the label is larger than the third threshold, and the recall rate of the part of the sample (the first-class abnormal sample order) is larger than the fourth threshold, wherein the accuracy is the ratio of the sample of the responsibility side determined in the first-class abnormal sample order to the total sample, and the recall rate is the ratio of the sample of a certain responsibility side in the first-class abnormal sample order to all samples of the responsibility side (i.e. the ratio of the sample of the responsibility side being the user to all samples of the responsibility side being the user, the driver or the network reservation platform), and when the preset convergence condition is reached, the classification model of the.
Preferably, the model establishing unit 502 is specifically configured to receive first responsibility party marking information of the first type sample abnormal order obtained according to the first preset order information; and/or marking second responsibility party marking information for the first type sample abnormal order according to second preset order information; the responsibility party marking information comprises first responsibility party marking information and second responsibility party marking information.
In this embodiment, positive samples are provided by labeling responsible parties for each abnormal order of the first type of sample as a pre-set classification model. When the responsible party of each abnormal order of the first type is marked, two methods can be included. A manual marking is carried out on a first type sample abnormal order by a marking team according to first preset order information, wherein the first preset order information is non-obvious information related to abnormal conditions, such as abnormal conditions which can be known only by asking a driver or a user by a telephone; and the other method is that the system automatically marks the first type sample abnormal order with second preset order information, wherein the second preset order information comprises information obviously fed back by a driver, a user or a network booking platform customer service, such as complaint or poor comment information on the order.
Preferably, the first preset order information includes one or a combination of the following: order track information, user historical order cancellation information, driver historical order cancellation information, user historical poor evaluation or complaint information, driver historical poor evaluation or complaint information, user and driver communication information, user and network car booking platform communication information, driver and network car booking platform communication information, network car booking platform call return information to users, and network car booking platform call return information to drivers; the second preset order information comprises one or a combination of the following: user bad comment or complaint information, driver bad comment or complaint information.
In this embodiment, the preset order information is feature information of each order, and the first preset order information and the second preset order information include, but are not limited to, the above information, and the first preset order information and the second preset order information can be used to accurately mark the responsible party of the sample abnormal order. For example, when it is determined that the driver goes to the user position to pick up the user through the order track information, the user cancels the order at a distance of one kilometer from the user position, and the preset pick-up time is not reached at this time, that is, the driver does not time out, then the responsible party for canceling the order is determined as the user. The bad comment or complaint information included in the first preset order information is history information, and the bad comment or complaint information included in the second preset order information is information in the current abnormal order.
Preferably, the responsible party for the exception order includes any one of: a user, a driver, or a network appointment platform.
In the embodiment, the abnormal orders are classified, and are divided into the abnormal orders with the responsibility party as the user, the abnormal orders with the responsibility party as the driver or the abnormal orders with the responsibility party as the network taxi appointment platform, so that intervention measures can be performed to reduce abnormal conditions of the orders, the service quality of the user is guaranteed, and benefits of the driver and the network taxi appointment platform are guaranteed.
In an embodiment of the third aspect of the embodiments of the present disclosure, a computer device is provided, and fig. 6 shows a schematic diagram of a computer device 60 according to an embodiment of the present disclosure. Wherein the computer device 60 comprises:
a memory 602, a processor 604 and a computer program stored on the memory 602 and executable on the processor 604, the steps of the order classification method according to any of the embodiments described above being implemented when the computer program is executed by the processor 604.
In the computer device 60 provided by the embodiment of the present disclosure, when the processor 604 executes the computer program, the steps of the order classification method according to any of the above embodiments are implemented, so that the computer device includes all the beneficial effects of the order classification method according to any of the above embodiments.
An embodiment of the fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the order classification method according to any of the embodiments described above.
The computer-readable storage medium provided by the embodiments of the present disclosure, when being executed by a processor, implements the steps of the order classification method according to any of the above embodiments, and therefore, the computer-readable storage medium includes all the benefits of the order classification method according to any of the above embodiments.
In the description herein, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance unless explicitly stated or limited otherwise; the terms "connected," "mounted," "secured," and the like are to be construed broadly and include, for example, fixed connections, removable connections, or integral connections; may be directly connected or indirectly connected through an intermediate. Specific meanings of the above terms in the embodiments of the present disclosure can be understood by those of ordinary skill in the art according to specific situations.
In the description herein, reference to the term "one embodiment," "some embodiments," "a specific embodiment," or the like, means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the embodiments of the disclosure. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the disclosed embodiments should be included in the scope of protection of the disclosed embodiments.
Claims (14)
1. An order classification method, comprising:
acquiring an abnormal order and extracting the characteristic information of the abnormal order;
and inputting the characteristic information of the abnormal order into a preset classification model, and performing responsibility party classification on the abnormal order.
2. The order classification method according to claim 1, further comprising:
collecting a plurality of sample abnormal orders, and dividing the sample abnormal orders into a first type of sample abnormal orders and a second type of abnormal sample orders;
acquiring the responsibility party marking information of each first-class sample abnormal order according to the characteristic information of each first-class sample abnormal order;
training the responsibility party labeling information, and establishing the preset classification model;
inputting the characteristic information of the second-class abnormal sample orders into the preset classification model, performing responsibility party classification on the second-class abnormal sample orders and obtaining responsibility party confidence of each second-class abnormal sample order;
and performing iterative training on the first class of abnormal sample orders and the second class of abnormal sample orders with the responsibility side confidence coefficient larger than a first threshold value until the preset classification model reaches a preset convergence condition.
3. The order classification method according to claim 2,
the preset convergence conditions comprise that the sum of the numbers of the second type abnormal sample orders with the responsibility party confidence degree larger than the first threshold value and the first type abnormal sample orders is larger than a second threshold value, the accuracy rate of the first type abnormal sample orders is larger than a third threshold value, and the recall rate of the first type abnormal sample orders is larger than a fourth threshold value.
4. The order classification method according to claim 2, wherein the obtaining of the liability party labeling information of each abnormal order of the first type includes:
receiving first responsibility party marking information of the first type sample abnormal order acquired according to first preset order information; and/or
According to second preset order information, second responsibility party marking information is marked on the first type sample abnormal order;
the responsibility party marking information comprises the first responsibility party marking information and the second responsibility party marking information.
5. The order classification method according to claim 4,
the first preset order information comprises one or a combination of the following information: order track information, user historical order cancellation information, driver historical order cancellation information, user historical poor evaluation or complaint information, driver historical poor evaluation or complaint information, user and driver communication information, user and network car booking platform communication information, driver and network car booking platform communication information, network car booking platform call return information to users, and network car booking platform call return information to drivers;
the second preset order information comprises one or a combination of the following information: user bad comment or complaint information, driver bad comment or complaint information.
6. The order classification method according to any one of claims 1 to 5,
the responsible party of the abnormal order comprises any one of the following: a user, a driver, or a network appointment platform.
7. An order classification system, comprising:
the characteristic extraction unit is used for acquiring an abnormal order and extracting the characteristic information of the abnormal order;
and the classification unit is used for inputting the characteristic information of the abnormal order into a preset classification model and classifying the abnormal order by responsible parties.
8. The order classification system of claim 7, further comprising:
the model establishing unit is used for acquiring a plurality of sample abnormal orders and dividing the sample abnormal orders into a first type of sample abnormal orders and a second type of sample abnormal orders; acquiring the responsibility party marking information of each first-class sample abnormal order according to the characteristic information of each first-class sample abnormal order; training the responsibility party labeling information, and establishing the preset classification model; inputting the characteristic information of the second-class abnormal sample orders into the preset classification model, performing responsibility party classification on the second-class abnormal sample orders and obtaining responsibility party confidence of each second-class abnormal sample order; and performing iterative training on the first class of abnormal sample orders and the second class of abnormal sample orders with the responsibility side confidence coefficient larger than a first threshold value until the preset classification model reaches a preset convergence condition.
9. The order classification system of claim 8,
the preset convergence conditions comprise that the sum of the numbers of the second type abnormal sample orders with the responsibility party confidence degree larger than the first threshold value and the first type abnormal sample orders is larger than a second threshold value, the accuracy rate of the first type abnormal sample orders is larger than a third threshold value, and the recall rate of the first type abnormal sample orders is larger than a fourth threshold value.
10. The order classification system of claim 8,
the model establishing unit is specifically used for receiving first responsibility party marking information of the first type sample abnormal order acquired according to first preset order information; and/or marking second responsibility party marking information for the first type sample abnormal order according to second preset order information;
the responsibility party marking information comprises the first responsibility party marking information and the second responsibility party marking information.
11. The order classification system of claim 10,
the first preset order information comprises one or a combination of the following information: order track information, user historical order cancellation information, driver historical order cancellation information, user historical poor evaluation or complaint information, driver historical poor evaluation or complaint information, user and driver communication information, user and network car booking platform communication information, driver and network car booking platform communication information, network car booking platform call return information to users, and network car booking platform call return information to drivers;
the second preset order information comprises one or a combination of the following information: user bad comment or complaint information, driver bad comment or complaint information.
12. The order classification system according to any one of claims 7 to 11,
the responsible party of the abnormal order comprises any one of the following: a user, a driver, or a network appointment platform.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor when executing the computer program realizes the steps of the order classification method according to any of claims 1 to 7.
14. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the order classification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811467925.1A CN111340053A (en) | 2018-12-03 | 2018-12-03 | Order classification method, classification system, computer device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811467925.1A CN111340053A (en) | 2018-12-03 | 2018-12-03 | Order classification method, classification system, computer device and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111340053A true CN111340053A (en) | 2020-06-26 |
Family
ID=71183129
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811467925.1A Pending CN111340053A (en) | 2018-12-03 | 2018-12-03 | Order classification method, classification system, computer device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111340053A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348537A (en) * | 2020-11-09 | 2021-02-09 | 南京领行科技股份有限公司 | Information processing method, device, electronic equipment and storage medium |
| CN112837013A (en) * | 2021-02-02 | 2021-05-25 | 拉扎斯网络科技（上海）有限公司 | Service processing method, device and equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150278751A1 (en) * | 2014-03-31 | 2015-10-01 | Level 3 Communications, Llc | Systems and methods for quality milestone management |
| CN107273454A (en) * | 2017-05-31 | 2017-10-20 | 北京京东尚科信息技术有限公司 | User data sorting technique, device, server and computer-readable recording medium |
| CN107464169A (en) * | 2017-08-10 | 2017-12-12 | 北京小度信息科技有限公司 | Information output method and device |
| CN108805660A (en) * | 2018-05-24 | 2018-11-13 | 北京三快在线科技有限公司 | Order processing method, apparatus and server |
-
2018
- 2018-12-03 CN CN201811467925.1A patent/CN111340053A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150278751A1 (en) * | 2014-03-31 | 2015-10-01 | Level 3 Communications, Llc | Systems and methods for quality milestone management |
| CN107273454A (en) * | 2017-05-31 | 2017-10-20 | 北京京东尚科信息技术有限公司 | User data sorting technique, device, server and computer-readable recording medium |
| CN107464169A (en) * | 2017-08-10 | 2017-12-12 | 北京小度信息科技有限公司 | Information output method and device |
| CN108805660A (en) * | 2018-05-24 | 2018-11-13 | 北京三快在线科技有限公司 | Order processing method, apparatus and server |
Non-Patent Citations (2)
| Title |
|---|
| 赵婕: "《图像特征提取与语义分析》", 重庆：重庆大学出版社, pages: 134 - 136 * |
| 马廷淮等: "动作识别训练数据的扩展研究", 《计算机与数字工程》, no. 11, pages 22 - 25 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112348537A (en) * | 2020-11-09 | 2021-02-09 | 南京领行科技股份有限公司 | Information processing method, device, electronic equipment and storage medium |
| CN112348537B (en) * | 2020-11-09 | 2022-06-07 | 南京领行科技股份有限公司 | Information processing method, device, electronic equipment and storage medium |
| CN112837013A (en) * | 2021-02-02 | 2021-05-25 | 拉扎斯网络科技（上海）有限公司 | Service processing method, device and equipment |
| CN112837013B (en) * | 2021-02-02 | 2023-08-11 | 拉扎斯网络科技（上海）有限公司 | Service processing method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3432197B1 (en) | Method and device for identifying characters of claim settlement bill, server and storage medium | |
| US11514271B2 (en) | System and method for automatically adjusting strategies | |
| US11138524B2 (en) | Cascaded boosted predictive models | |
| US8706729B2 (en) | Systems and methods for distributed data annotation | |
| JP5506722B2 (en) | Method for training a multi-class classifier | |
| CN110472524B (en) | Invoice information management method and system based on deep learning and readable medium | |
| CN109598566B (en) | Ordering prediction method, ordering prediction device, computer equipment and computer readable storage medium | |
| CN109067643B (en) | Reply method and device based on keywords, computer equipment and storage medium | |
| CA2908952A1 (en) | Evaluation control | |
| CN105787133B (en) | Advertisement information filtering method and device | |
| CN111340053A (en) | Order classification method, classification system, computer device and readable storage medium | |
| CN103226563B (en) | To the method and system that the client activities in automatic client back-up system are classified | |
| CN107733967A (en) | Processing method, device, computer equipment and the storage medium of pushed information | |
| CN108596632A (en) | A kind of anti-cheating recognition methods and system based on order attributes and user behavior | |
| CN110363228B (en) | Noise label correction method | |
| CN106297489A (en) | A kind of control system for driving the person's of taking an entrance examination self-clocking reservation training | |
| AU2017208340A1 (en) | Information processing apparatus, information processing program, recording medium, and information processing method | |
| Prelipcean et al. | Workshop Synthesis: New developments in travel diary collection systems based on smartphones and GPS receivers | |
| CN105765612A (en) | System and method of determining missing interval change information in radiology reports | |
| CN111310961A (en) | Data prediction method, data prediction device, electronic equipment and computer readable storage medium | |
| CN107368565A (en) | Data processing method, data processing equipment and computer-readable recording medium | |
| CN111159241B (en) | Click conversion estimation method and device | |
| CN112488222A (en) | Crowdsourcing data labeling method, system, server and storage medium | |
| CN110310028B (en) | Method and apparatus for crowdsourcing | |
| DE112019005823T5 (en) | EVENT SYSTEM THAT USES USER MOBILITY BEHAVIOR |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |