CN113112030B - Method and system for training model and method and system for predicting sequence data - Google Patents

Method and system for training model and method and system for predicting sequence data Download PDF

Info

Publication number
CN113112030B
CN113112030B CN202110497221.4A CN202110497221A CN113112030B CN 113112030 B CN113112030 B CN 113112030B CN 202110497221 A CN202110497221 A CN 202110497221A CN 113112030 B CN113112030 B CN 113112030B
Authority
CN
China
Prior art keywords
data
sequence
shared
hidden state
personalized
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110497221.4A
Other languages
Chinese (zh)
Other versions
CN113112030A (en
Inventor
姚权铭
时鸿志
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
4Paradigm Beijing Technology Co Ltd
Original Assignee
4Paradigm Beijing Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 4Paradigm Beijing Technology Co Ltd filed Critical 4Paradigm Beijing Technology Co Ltd
Priority to CN202110497221.4A priority Critical patent/CN113112030B/en
Publication of CN113112030A publication Critical patent/CN113112030A/en
Application granted granted Critical
Publication of CN113112030B publication Critical patent/CN113112030B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Abstract

A method and system for training a model and a method and system for predicting sequence data are provided. The method and system for training a model can acquire a sequence training sample set and train a machine learning model based on the sequence training sample set, wherein the machine learning model is a hidden Markov model comprising two hidden state layers, wherein a first hidden state layer comprises a personalized hidden state of each object in a plurality of objects, and a second hidden state layer comprises a plurality of shared hidden states shared by the plurality of objects. A method and system of predicting sequence data may obtain a sequence prediction sample of an object and perform prediction on the sequence prediction sample using the machine learning model to provide a prediction result regarding next sequence data after the plurality of sequence data, wherein the machine learning model is trained in advance to predict the next sequence data after the series of sequence data for a chronological series of sequence data.

Description

Method and system for training model and method and system for predicting sequence data
The present application is a divisional application of patent application with application date 2019, 4, 28, 201910349922.6, entitled "method and System for training model and method and System for predicting sequence data".
Technical Field
The present application relates generally to the field of artificial intelligence, and more particularly, to a method and system for training a machine learning model for predicting sequence data, and a method and system for predicting sequence data using the machine learning model.
Background
With the advent of mass data, artificial intelligence technology has evolved rapidly, and machine learning is an inevitable product of the evolution of artificial intelligence to a certain stage, which aims at mining valuable potential information from a large amount of data by means of computation.
Mining the law behind sequence data by modeling continuously occurring sequence data (e.g., moving position data, music listening sequence, etc.) through machine learning is very important for various application scenarios. For example, personalized sequential behavior is ubiquitous in our daily lives, and simulating such behavior is very important for many application scenarios. For example, modeling trajectory data (one of the sequence data) helps to understand the mobility laws of the user, which may facilitate improved ride sharing services and traffic; modeling a music listening sequence helps to reveal a continuous law behind human behavior, thereby facilitating enhancement of accuracy of content recommendation; modeling the order in which the user purchased the merchandise is beneficial to analyzing the user's preferences, thereby facilitating targeted advertising; the scenes such as this are also numerous and not limited thereto. In all these application scenarios, an important feature is that the sequence pattern reflected by the sequence data is highly personalized, and different objects may have completely different sequence laws, so a model for effectively learning the personalized sequence data is needed.
Hidden Markov Models (HMMs) are one of the models for modeling sequence data, which not only can characterize sequence patterns, but also can discover the states behind the hidden sequence patterns, and are therefore often used for sequence modeling. However, sequence modeling with HMMs often has the problem that, for example, if we train one HMM for each object, then a reliable HMM model cannot be trained with very limited data since there is too little data for that object; if we train one HMM for all objects using their data, this will result in the trained model losing personalization. At present, although researchers propose grouping objects according to the similarity of the sequence data of the objects and training an HMM for each group, this method still forces different objects (objects in the same group) to share an HMM, so that the model is still not enough for the individualization of the objects, and further, the prediction accuracy is difficult to meet the requirement when the trained model is used for predicting the sequence data of the objects.
Disclosure of Invention
The present invention aims to solve the problem that the existing HMM model cannot simultaneously process the scarcity of training data and the sequence pattern diversity of different objects, for example, to improve the prediction accuracy of sequence data in a scene involving the prediction of object sequence data (e.g., sequence behavior).
According to an exemplary embodiment of the present application, a method of training a machine learning model for predicting sequence data is provided, the method may include: obtaining a sequence training sample set, wherein the sequence training sample set comprises a plurality of sequence training samples for each of a plurality of objects, and each sequence training sample comprises a plurality of sequence data arranged in time sequence; training the machine learning model based on the sequence training sample set, wherein the machine learning model is a hidden markov model comprising two hidden state layers, wherein a first hidden state layer comprises personalized hidden states of each of the plurality of objects and a second hidden state layer comprises a plurality of shared hidden states shared by the plurality of objects.
According to another exemplary embodiment of the present application, a computer-readable storage medium storing instructions is provided, wherein the instructions, when executed by at least one computing device, cause the at least one computing device to perform a method of training a machine learning model for predicting sequence data as described above.
According to another exemplary embodiment of the present application, a system is provided comprising at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, cause the at least one computing device to perform a method of training a machine learning model for predicting sequence data as described above.
According to another exemplary embodiment of the present application, a system for training a machine learning model for predicting sequence data is provided, the system may include: a training sample acquisition device configured to acquire a sequence training sample set, wherein the sequence training sample set includes a plurality of sequence training samples for each of a plurality of objects, and each sequence training sample includes a plurality of sequence data arranged in chronological order; training means configured to train the machine learning model based on the set of sequence training samples, wherein the machine learning model is a hidden markov model comprising two hidden state layers, wherein a first hidden state layer comprises personalized hidden states for each of the plurality of objects and a second hidden state layer comprises a plurality of shared hidden states shared by the plurality of objects.
According to another exemplary embodiment of the present application, a method of predicting sequence data using a machine learning model is provided, the method may include: obtaining a sequence prediction sample of an object, wherein the sequence prediction sample comprises a plurality of sequence data of the object arranged in time sequence; performing prediction on the sequence prediction samples to provide a prediction result on next sequence data after the plurality of sequence data using the machine learning model, wherein the machine learning model is trained in advance to predict the next sequence data after the series of sequence data for a series of sequence data arranged in time order, and the machine learning model is a hidden markov model including two hidden state layers, wherein a first hidden state layer includes a personalized hidden state of each of the plurality of objects and a second hidden state layer includes a plurality of shared hidden states shared by the plurality of objects.
According to another exemplary embodiment of the present application, a computer-readable storage medium storing instructions is provided, wherein the instructions, when executed by at least one computing device, cause the at least one computing device to perform a method of predicting sequence data using a machine learning model as described above.
According to another exemplary embodiment of the present application, a system is provided comprising at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, cause the at least one computing device to perform a method of predicting sequence data using a machine learning model as described above.
According to another exemplary embodiment of the present application, a system for predicting sequence data using a machine learning model is provided, the system may include: a prediction sample acquisition device configured to acquire a sequence prediction sample of an object, wherein the sequence prediction sample includes a plurality of sequence data of the object arranged in chronological order; a prediction means configured to perform prediction for the sequence prediction samples using the machine learning model to provide a prediction result regarding next sequence data after the plurality of sequence data, wherein the machine learning model is trained in advance to predict the next sequence data after the series of sequence data for a series of sequence data arranged in time order, and the machine learning model is a hidden markov model including two hidden state layers, wherein a first hidden state layer includes a personalized hidden state of each of a plurality of objects, and a second hidden state layer includes a plurality of shared hidden states shared by the plurality of objects.
According to the method and the system for training the machine learning model, the hidden Markov model comprising two hidden state layers can be trained, and the first hidden state layer of the hidden Markov model comprises the personalized hidden state of each object in a plurality of objects, and the second hidden state layer comprises a plurality of shared hidden states shared by the plurality of objects, so that the scarcity of training data can be overcome, the sequence mode diversity of different objects can be ensured, and the trained hidden Markov model can provide more accurate sequence data prediction results for different objects.
Since the method for predicting sequence data using the machine learning model according to the exemplary embodiment of the present application predicts sequence data using the hidden markov model including two hidden state layers described above, personalized sequence data prediction is provided for different objects, and thus it is possible to improve the prediction accuracy of sequence data.
Drawings
These and/or other aspects and advantages of the present application will become more apparent and more readily appreciated from the following detailed description of the embodiments of the present application, taken in conjunction with the accompanying drawings, wherein:
FIG. 1 is a block diagram illustrating a system for training a machine learning model for predicting sequence data according to an exemplary embodiment of the present application;
FIG. 2 is a schematic diagram of a hidden Markov model sharing hidden states according to an exemplary embodiment of the present application;
FIG. 3 is a flowchart illustrating a method of training a machine learning model for predicting sequence data according to an exemplary embodiment of the present application;
FIG. 4 is a block diagram illustrating a system for predicting sequence data using a machine learning model according to an exemplary embodiment of the present application;
fig. 5 is a flowchart illustrating a method of predicting sequence data using a machine learning model according to an exemplary embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, exemplary embodiments of the present application will be described in further detail below with reference to the accompanying drawings and detailed description.
Fig. 1 is a block diagram showing a system (hereinafter, simply referred to as "model training system" for convenience of description) 100 for training a machine learning model for predicting sequence data according to an exemplary embodiment of the present application. As shown in fig. 1, model training system 100 may include a training sample acquisition device 110 and a training device 120.
Specifically, training sample acquisition device 110 may acquire a set of sequential training samples. Here, the sequence training sample set may include a plurality of sequence training samples for each of a plurality of objects, and each sequence training sample may include a plurality of sequence data arranged in time order. As an example, the plurality of sequence data may relate to behavior data of the object at different points in time. Here, the behavior data may include continuous feature data reflecting the behavior of the object, for example, position data of the object (e.g., position data on which the user rides), and the like, but is not limited thereto. Alternatively, the behavior data may include discrete feature data reflecting the behavior of the object, such as, but not limited to, a content ID of content accepted by the object (for example, content may be various types of content such as music, video, advertisement, image, etc.). As another example, the plurality of sequence data may relate to status data of the object at different points in time, e.g., physiological status data of the user (e.g., blood pressure, blood glucose, etc.), price of goods, price of stocks, etc.
For example, in a scenario in which a movement trajectory of an object (e.g., a user or a vehicle) is predicted, the training sample acquisition means 110 may acquire a series of historical position data of each of a plurality of objects arranged in time series to construct the above-described sequence training sample; in the content recommendation scenario, the training sample acquiring means 110 may acquire a series of content IDs of history accepted contents arranged in time order for each of a plurality of users to construct the sequence training sample; in a scenario involving physiological state prediction of a subject (human or animal), the training sample acquisition means 110 may acquire a series of historical physiological state data of each of a plurality of subjects arranged in time series to construct the above-described sequence training sample; in a scenario involving price prediction of commodities or stocks, the training sample acquiring means 110 may acquire a series of historical price data of each commodity or stock of the same class of commodities or stocks arranged in time series to construct the above-described series training sample.
In this application, an object may be a living person or an inanimate object (e.g., a machine, a commodity, a stock, or the like). Furthermore, the sequence data may be the behavior or properties of the object at different points in time in a particular aspect, and is not limited to behavior data or state data.
Specifically, as an example, the training sample acquisition device 110 may acquire a set of historical data records for a plurality of objects and construct the sequence training sample set based on the set of historical data records for the plurality of objects. Alternatively, the training sample acquisition device 110 may acquire the set of sequential training samples generated by other devices directly from the outside. Here, the training sample acquisition device 110 itself performs the operation to construct the sequence training sample set will be described as an example. For example, the training sample acquisition device 110 may acquire the history data record manually, semi-automatically, or fully automatically, or process the acquired history data record such that the processed history data record is in a suitable format or form. Here, the training sample acquiring device 110 may receive the history data record manually input by the user through an input device (e.g., a workstation), or the training sample acquiring device 110 may acquire the history data record set from the data source in a fully automatic manner, for example, by systematically requesting the data source to send the history data record set to the training sample acquiring device through a timer mechanism implemented in software, firmware, hardware, or a combination thereof, or may automatically perform the acquisition of the history data record set in the presence of a manual intervention, for example, request the acquisition of the history data record set in the presence of a specific user input. Each time a history data record is acquired, the data record acquisition device 110 may preferably store the captured data in a non-volatile memory. As an example, a data warehouse may be utilized to store acquired historical data records as well as processed historical data records.
Here, when constructing the sequence training sample set, for a plurality of history data records of each object arranged in time sequence, if a time interval between two adjacent history data records satisfies a preset condition, the training sample acquiring apparatus 110 may segment the plurality of history data records, thereby obtaining a plurality of sequence training samples of the object. For example, the preset condition may be that the time interval between any two adjacent pieces of history data is greater than a predetermined time threshold, but is not limited thereto, and for example, the preset condition may also be that the time interval between any two adjacent pieces of history data is within a predetermined time range. Here, each history data record may include a plurality of data attribute fields, such as an object identification field, an object behavior data field, a behavior occurrence time field, and the like, as examples. The training sample acquiring means 110 may first acquire a plurality of history data records of each object according to the object identification field in the acquired history data record set of the plurality of objects, and then may arrange the plurality of history data records of each object in time sequence, and if a time interval between two adjacent history data records in the arranged plurality of history data records is greater than a preset threshold, may perform slicing such that a time interval between any two adjacent history data records in each of the sliced subset of history data records is less than or equal to the preset threshold.
To more intuitively represent the segmentation process, it is assumed that a history of objects is defined as a tuple r n ＝<u n ，t n ，e n >, where u n Is the user ID, e n Is historical sequence data, t n Is a time stamp (i.e. and e n Corresponding time stamps). Here, e n Either continuous or discrete data. As an example, when the history data record relates to object behavior, e in a scenario involving mobile location prediction n For example, may be position data of an object and may be represented as a two-dimensional continuous vector e n ＝(l o ，l a ) Where lo represents longitude and la represents latitude. As another example, in a scenario involving content recommendation, e n For example, it may be the singer ID of the music the user listens to. As another example, in a scenario involving physiological state prediction of a subject (human or animal), e n May be data of the physiological state of the subject, for example, blood pressure value, blood glucose value, etc. As another example, in a scenario involving price prediction of a commodity or stock, e n May be the price of the commodity or stock. However, the kind or the expression form of the history data record is not limited to the above example. In this case, assume thatIs a set of history data records of a plurality of objects, if +. >Is->A subsequence and satisfies->(wherein Deltat > 0) and +.>Then->May be a sequence training sample of the object being constructed.
After a set of sequence training samples (including a plurality of sequence training samples for each of a plurality of objects) is obtained in the manner described above, the training device 120 may train a machine learning model based on the set of sequence training samples. In the present application, the machine learning model herein may be a hidden markov model including two hidden state layers, wherein a first hidden state layer includes a personalized hidden state of each of the plurality of objects, and a second hidden state layer includes a plurality of shared hidden states shared by the plurality of objects.
To facilitate an understanding of the hidden markov models (hereinafter, may be referred to as shared hidden markov models) comprising two hidden state layers proposed herein, a brief description of a classical Hidden Markov Model (HMM) will be presented herein. The HMM assumes that the sequence data of an object is controlled by a plurality of hidden states and that transitions between these hidden states follow a markov assumption, i.e. the probability that the object is in the next hidden state depends only on the current hidden state. Assuming that M is the number of hidden states, the classical HMM model comprises three parameter sets, which are:
(1) M-dimensional vector pi epsilon R M Wherein pi m =p (z=m) defines an initial probability that the object initially accesses the mth hidden state, where z represents the initial hidden stateA state;
(2) MxM transition probability matrixWhich defines transition probabilities between M hidden states following a markov assumption, where a ij ＝p(z n ＝j|z n-1 =i) and represents the probability of an object from the i-th hidden state to the j-th hidden state;
(3) Parameter set d= { D m M=1.2, … M, which defines a set of probability distributions of M hidden states in the observation space, where d m Defining probability distribution of mth hidden state in observation space.
Next, a description is given of a hidden markov model sharing hidden states according to an exemplary embodiment of the present application with reference to fig. 2.
As shown in fig. 2, compared to a classical HMM, the hidden markov model of the shared hidden state in the present application may include two hidden state layers, a first hidden state layer may include a personalized hidden state of each of the plurality of objects, and a second hidden state layer may include a plurality of shared hidden states shared by the plurality of objects. The first hidden state layer comprises the personalized hidden state of each object in the plurality of objects, so that the personalized sequence mode of each object is ensured, and the plurality of objects in the second hidden state layer share the plurality of shared hidden states, so that the problem of scarcity of training data is effectively solved, and in conclusion, the hidden Markov model sharing the hidden states not only overcomes the scarcity of the training data, but also can ensure the sequence mode diversity of different objects.
The hidden markov model sharing hidden states in the present application fully follows and can reflect objective rules in practical application scenarios, for example, many people come together to form hot spots or groups with similar interests often listen to a type of music that is commonly shared by users and unlikely to be affected by a single user. On the other hand, for example, user sequence behavior patterns are extremely diverse. For example, two users work together at site a, and they often go home after work. Their home is likely not in the same area, so it is not appropriate to use a single transfer model to predict their way to behind site a. In addition, for example, user 1 likes rock music and ballad, while user 2 likes rock music and rap. Without personalized information, we have little to predict what music they will listen to after rock music. In this application, it is through the first hidden state layer that the personalized sequence mode of each object is ensured, and through the second hidden state layer, multiple objects can share the hidden state that is unlikely to be affected by a single object.
The hidden Markov model of the shared hidden state of the present application is described in further detail below with reference to FIG. 2. For convenience of description, three objects are included in the first hidden state layer and each object has three hidden states (in the first hidden state layer, the first three circles represent three personalized hidden states of the first object, the middle three circles represent three personalized hidden states of the second object, and the last three circles represent three personalized hidden states of the third object), but it should be clear that: the present application is not limited in any way with respect to the number of objects and the number of hidden states.
Referring to fig. 2, the number of personalized hidden states of each object in the first hidden state layer is smaller than the number of the plurality of shared hidden states in the second hidden state layer. As an example, in a scenario where the mobile location of an object (user or vehicle) is predicted, the personalized hidden state may include, for example, that the location of the object is in a work area, living area, rest area, etc., and the shared hidden state may include some hot spots in the viewing space that are shared by the object, such as shopping malls, restaurants, leisure centers, etc. As another example, in a content recommendation scenario, the personalized hidden state may include the type of content that is commonly accepted by a particular user, e.g., ballad music, rock music, rap music, etc., while the shared hidden state may include the type of content accepted by most users, e.g., soothing music, rhythmic music, etc. As another example, in a scenario involving physiological state prediction of a subject (human or animal), the personalized hidden state may be a physiological state index interval (e.g., a blood pressure change interval) typical of a particular subject, and the shared hidden state may be a physiological state index interval typical of a homogeneous subject; as another example, in a scenario involving price prediction of a commodity or stock, the personalized hidden state may be a common price interval for a certain commodity or stock, and the shared hidden state may be a price interval in which the like commodity or stock is generally located. Although the number of shared hidden states shared by three objects is shown as 8 in fig. 2, it should be clear that the present application is not limited to the number of shared hidden states, as long as the number is greater than the number of personalized hidden states for each object.
As shown in fig. 2, each shared hidden state in the second hidden state layer corresponds to a probability distribution (denoted by d in fig. 2, e.g., d 1 To d 8 ). As an example, when the above-described behavior data is continuous feature data (e.g., position data) reflecting the behavior of the object, the probability distribution corresponding to each shared hidden state may include a gaussian distribution, but is not limited thereto. As another example, when the above-described behavior data includes discrete feature data (e.g., content ID) reflecting the behavior of the object, the probability distribution corresponding to each shared hidden state may include a polynomial distribution, but is not limited thereto.
According to an exemplary embodiment of the present application, the model parameters of the markov model sharing the hidden states may include a personalized parameter set for each object and a shared parameter set shared by the plurality of objects. Specifically, the personalized parameter set includes a probability of a personalized hidden state of each object in the first hidden state layer, a transition probability between personalized hidden states of each object, and a transmission probability of each object from the personalized hidden state to a shared hidden state, the shared parameter set including a set of probability distributions corresponding to each shared hidden state.
Next, a detailed description will be given of the training process of the model. Heretofore, the above-mentioned personalized parameter set and shared parameter set are visually represented for convenience of description.
In particular, assuming that the number of shared hidden states in the second hidden state layer is M, a shared parameter set shared by a plurality of objects may beExpressed as d= { D m M is equal to or less than M and equal to or greater than 1), each d m Is the probability distribution corresponding to the mth shared hidden state, which defines the probability distribution of the mth shared hidden state over the observation space.
Furthermore, the personalized parameter set for each object u may be represented as Φ u ＝{π u ，A u ，B u }. Assuming the personalized hidden state of object u uses z n Representing and sharing hidden states with c n The expression here, pi u Is the probability of the personalized hidden state of the object u in the first hidden state layer, if the object has K personalized hidden statesIs the probability that object u is initially in the ith personalized hidden state, where z 1 Is the initial personalized hidden state of the object, i is less than or equal to K and greater than or equal to 1.Is a transition probability matrix between K personalized hidden states of the object u, wherein +.>Representing the transition probability of the object u from the ith personalized hidden state to the jth personalized hidden state. / >Is a transmission probability matrix, wherein ∈>Representing the probability of transmission from the ith personalized hidden state in the first hidden state layer to the mth shared hidden state in the second hidden state layer.
In general, in a real scene, objects have only a few states of distribution in the viewing space, for example, users shift between only a few areas (e.g., home and office), or each user tends to listen to only a few types of music in a collection. Thus, if, during the training process, the personalized hidden states located in the first hidden state layer in the hidden markov model of the present application are caused to be transmitted to the few shared hidden states in the second hidden state layer with a highly concentrated probability distribution, the trained model will be easier to interpret (in other words, more in line with the objective rules in the actual scenario).
To this end, according to an exemplary implementation of the present application, the training apparatus 120 may construct an objective function for training the machine learning model to include a loss function and a regularization term, where the regularization term is used to constrain a concentration of emission probability distributions for each object from the personalized hidden state to the shared hidden state. Since entropy may measure the degree of uncertainty or measure diversity, the canonical term herein may include, as an example, a constraint term related to the entropy of each object's emission probability from personalized hidden states to shared hidden states. For example, the constraint term may be constructed as Wherein (1)>Wherein λ is a real number greater than 0, < >>Indicating a transmission probability of the ith object from the ith personalized hidden state to the mth shared hidden state, wherein u, i and m are each positive integers greater than 0.
Although the constraint term related to entropy is taken as an example of the above-described regular term here, it should be noted that the regular term here is not limited to the constraint term related to entropy, but may be any function term capable of constraining the concentration degree of the emission probability distribution from the personalized hidden state to the shared hidden state. Alternatively, the objective function of the hidden markov model for training the shared hidden state may not include a regularization term for constraining the concentration degree of the emission probability distribution from the personalized hidden state to the shared hidden state for each object (at this time, λ=0 described above). Or,the objective function may include other regularities in addition to the regularization term described above that constrain the complexity of the model. Furthermore, the above-mentioned constraint terms related to entropy are not limited to being constructed asBut may be constructed as a combination of any function term with respect to entropy.
As an example, an objective function according to an exemplary embodiment of the present application may be constructed as follows:
Wherein,is a loss function, +.>Is a sequence training sample of the object (i.e. each sequence of sequence data observed in the observation space), and +.>(where N is the length of the sequence), J u Is the set of sequence training samples for all objects u, and λ > 0 is the constraint coefficients of the constraint term.
As shown in fig. 2, for each sequence composed of sequence data observed in the observation spaceTwo hidden state sequences corresponding to the two hidden state sequences are respectively personalized hidden state sequences +.>And shared hidden state sequence->Thus at the above objective functionIn equation (1) of>Can be expressed as follows:
wherein,meaning +.>
By continuously optimizing the above objective function by using the sequence training samples, the personalized parameter set phi can be finally determined u ＝{π u ，A u ，B u Sum shared parameter set d= { D m }. Finally, if the plurality of sequence data in the sequence training samples relates to behavior data of the object at different points in time, the hidden markov model of the shared hidden state of the present application may be trained to predict the next row of data of the object after a series of historical behavior data of the object in chronological order. Alternatively, if multiple sequence data in a sequence training sample relate to state data of an object at different points in time, a hidden Markov model sharing hidden states may be trained to predict next state data of the object after a series of historical attribute data for the object's chronological series of historical state data.
For example, if the behavior data is position data of an object, the machine learning model is trained to predict the position data of the object at a next point in time for a series of historical position data of the object that are chronologically arranged. If the behavior data is a content ID of the content accepted by the user, the machine learning model is trained to predict the content ID of the content that the user will accept at a next point in time for a chronological sequence of historically accepted content IDs of the user. If the state data is physiological state data of the subject, the machine learning model is trained to predict physiological state data of the subject at a next point in time for a chronological series of historical physiological state data of the subject. If the status data is price data for a commodity or stock, the machine learning model is trained to predict price data for the commodity or stock at a next point in time for a chronological series of historical price data for the commodity or stock.
Next, a process of training the hidden markov model sharing the hidden states using the above objective function will be described in detail.
Specifically, the training device 120 may determine the lower bound of the objective function based on the jensen inequality using the personalized hidden state sequence and the shared hidden state sequence corresponding to each sequence training sample, and determine the model parameters of the model by maximizing the lower bound of the objective function.
First, the training device 120 may utilize a personalized hidden state sequenceAnd sharing hidden state sequencesTo find a lower bound for the target based on the jensen inequality, and then optimize the lower bound to update the model parameters and find a new lower bound until convergence. Here, the lower bound of the objective function L (Φ, D) can be determined as follows:
in a specific training model, the personalized parameter set { pi } may be initialized first u }、{A u Sum { B } u And a shared parameter set D, and then, for each sequence training sample of the input object u, updating the posterior probabilities of the personalized hidden state sequences and the shared hidden state sequences corresponding to the sequence training sample(update +.for convenience of description below)>Is referred to as the E-step) and can be achieved by maximizing L' 1 (phi, D) to update model parameters { pi ] u }、{A u Sum { B } u And D (hereinafter, for convenience of description, a step of updating model parameters is referred to as an M-step). The training device 120 may repeat the E-step and the M step until the objective function L (Φ, D) is maximum, and the model parameter corresponding to the maximum objective function is the model parameter of the model finally trained.
By maximizing L 'in the M-step as described above' 1 (phi, D) to update model parameters { pi ] u }、{A u Sum { B } u And D, the M-step is described in detail below.
First, in the M-step, the training device 120 may set the lower bound L 'of the objective function' 1 (Φ, D) is transformed to include functional terms affected only by the probability of the personalized hidden state, functional terms affected only by the transition probability, functional terms affected only by the emission probability, and functional terms affected only by the shared parameter set, and corresponding model parameters are determined by maximizing the respective functional terms separately. In particular, L 'can be, for example' 1 (Φ, D) becomes:
here, three auxiliary variables ζ can be defined n (i，j)、γ n (i) And ρ n (i, m) to estimateWherein,andWhere n=1, 2. In the M-step, ζ may be used n (i，j)、γ n (i) And ρ n (i, m) substitution->To make L' 1 (Φ, D) becomes a form including the above items (4) to (7), wherein the function item (4) is a function item affected only by the probability of the personalized hidden state, the function item (5) is a function item affected only by the transition probability, the function item (6) is a function item affected only by the emission probability, and the function item (7) is a function item affected only by the shared parameter set. The training device 120 may then determine the corresponding model parameters Φ and D by maximizing the respective function terms.
Since the above function terms (4), (5) and (7) are concave without any other additional terms, the training device 120 may determine its maximum value based on the conventional Baum Welch algorithm, and thus determine the corresponding model parameters, and it is clear to those skilled in the art how to determine the maximum value of the function terms (4), (5) and (7) based on the conventional Baum Welch algorithm, and thus, a detailed description thereof will be omitted herein. However, the function term (6) is not always concave and is affected by the constraint term mentioned above, and thus its maximum cannot be determined based on the conventional Baum Welch algorithm. Here, for a function term affected by the emission probability, the present application proposes a way in which the emission probability can be determined by converting the problem of maximizing the function term into a one-dimensional nonlinear equation problem under the framework of a difference of projection planning (DCP, difference of Convex Programming). Next, this will be described.
For simplicity, can letAnd b= { b m }. Next, the problem of finding b for each i and u (i.e., the maximization problem of function term (6)) can be translated into a minimization problem:
wherein,it has been estimated in the E-step that the presence of lambda > 0 makes (8) a non-convex optimization problem.
To optimize such non-convex functions with convergence guaranteed b, (8) can be decomposed into convex termsAnd concave item->To meet the formal requirements of the DCP framework.
DCP is a general and powerful framework for solving non-convex problems, according to which convex upper bound f can be minimized by locally linearizing concave terms (t+1) (b) Wherein f is (t+1) (b) Is represented as follows:
how to solve (9) effectively is the key to achieving a fast solution to the non-convex problem. To achieve this objective, the present application converts (9) into a one-dimensional nonlinear equation problem, i.e., there is η such that:
the optimal solution of equation (9) can be determined byη in (2) is obtained. Equation (10) is a simple one-dimensional nonlinear equation problem that can be solved, for example, using newton's method. Specifically, the process of solving (8) under the DCP framework is as follows:
first, initialize b (1) Subsequently, for t=1, …, T, with current b (t) Converting equation (9) to equation (10) and obtaining b by solving (10) using Newton's method (t+1) . Repeating the above operation to obtain b (T) At this time, the emission probability at which the function term (6) is maximum is determined.
A system for training a machine learning model for predicting sequence data, a structure of the machine learning model, and the like according to an exemplary embodiment of the present application have been described in detail above with reference to fig. 1 and 2. In one aspect, since the machine learning model of the present application includes two hidden state layers (where a first hidden state layer includes a personalized hidden state for each object and a second hidden state layer includes a plurality of shared hidden states shared by a plurality of objects), it can not only overcome the scarcity of training data but also guarantee sequence pattern diversity for different objects. On the other hand, since the objective function for training the machine learning model is constructed in the present application to include the regularities for constraining the concentration degree of the emission probability distribution from the personalized hidden state to the shared hidden state for each object, the trained machine learning model is easier to explain and more conforms to the objective law. In addition, in the model training process, the problem of maximizing the function term is converted into the one-dimensional nonlinear equation problem under the DCP framework to determine the emission probability, so that the emission probability can be rapidly solved, and the model training speed can be improved.
It should be noted that, although the model training system 100 is described above as being divided into the devices for performing the respective processes (for example, the training sample acquiring device 110 and the training device 120), it is clear to those skilled in the art that the processes performed by the respective devices described above may be performed without any specific device division or without explicit demarcation between the respective devices by the model training system 100. Furthermore, the model training system 100 described above with reference to fig. 1 is not limited to include the above-described devices, but some other devices (e.g., storage devices, data processing devices, etc.) may be added as needed, or the above devices may be combined.
Fig. 3 is a flowchart illustrating a method of training a machine learning model for predicting sequence data (hereinafter, simply referred to as "model training method" for convenience of description) according to an exemplary embodiment of the present application.
Here, by way of example, the model training method illustrated in fig. 3 may be performed by the model training system 100 illustrated in fig. 1, may be implemented entirely in software by a computer program or instructions, and may also be performed by a specifically configured computing system or computing device, e.g., by a system comprising at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform the model training method described above. For convenience of description, it is assumed that the model training method shown in fig. 3 is performed by the model training system 100 shown in fig. 1, and that the model training system 100 may have the configuration shown in fig. 1.
Referring to fig. 3, in step S310, the training sample acquiring apparatus 110 may acquire a sequence training sample set. Here, the sequence training sample set may include a plurality of sequence training samples for each of a plurality of objects, and each sequence training sample includes a plurality of sequence data arranged in time order. As an example, the plurality of sequence data herein may relate to behavior data of an object at different points in time, the machine learning model being trained to predict a next row of data of the object after a chronological series of historical behavior data of the object. As another example, the plurality of sequence data herein relates to state data of an object at different points in time, the machine learning model being trained to predict next state data of the object after a series of historical attribute data for a chronological series of historical state data of the object. The sequence data, the behavior data, the status data, etc. have been described above with reference to fig. 1, and the description thereof will not be repeated here, but the related contents described with reference to fig. 1 are equally applicable thereto.
Specifically, in step S310, the training sample acquiring device 110 may acquire a set of history data records of the plurality of objects, and construct the sequence training sample set based on the set of history data records of the plurality of objects. Here, for a plurality of history data records of each object arranged in time sequence, if a time interval between two adjacent history data records satisfies a preset condition, slicing is performed, so as to obtain a plurality of sequence training samples of the object. For example, if the time interval between any two adjacent histories is greater than a predetermined time threshold, then a cut is made. Since the manner in which the sequence training samples of each object are obtained by means of segmentation has been described in the description of fig. 1, a detailed description thereof will be omitted.
Next, at step S320, the training device 120 may train the machine learning model based on the set of sequence training samples acquired at step S310. Here, the machine learning model is a hidden markov model including two hidden state layers. Specifically, the first hidden state layer may include a personalized hidden state of each of the plurality of objects, and the second hidden state layer may include a plurality of shared hidden states shared by the plurality of objects.
According to an exemplary embodiment, each shared hidden state may correspond to a probability distribution. As described above, the sequence data may include behavior data of the object. As an example, the behavior data may include continuous feature data reflecting the behavior of the object, in which case the probability distribution corresponding to each shared hidden state may include a gaussian distribution, but is not limited thereto. As another example, the behavior data may include discrete feature data reflecting the behavior of the object, in which case the probability distribution corresponding to each shared hidden state may include a polynomial distribution, but is not limited thereto. Here, the continuous feature data may include location data of the object, in which case the machine learning model may be trained to predict next location data of the object for a series of historical location data of the object (i.e., the machine learning model is trained to predict a moving location of the object). As an example, the discrete feature data may include a content ID of the content accepted by the object, in which case the machine learning model may be trained to predict the content ID of the next content that the object will accept for a series of historically accepted content IDs of the object. It should be noted that, for different application scenarios, the continuous feature data and the discrete feature data may include different types of data of the object.
In the hidden markov model comprising two hidden state layers of the present application, the number of personalized hidden states of each object in the first hidden state layer may be smaller than the number of the plurality of shared hidden states in the second hidden state layer. Furthermore, the model parameters of the hidden markov model may include a personalized parameter set for each object and a shared parameter set shared by the plurality of objects. In particular, the personalized parameter set may include a probability of a personalized hidden state of each object in the first hidden state layer, a transition probability between personalized hidden states of each object, and a transmission probability of each object from the personalized hidden state to the shared hidden state, and the shared parameter set may include a set of probability distributions corresponding to each shared hidden state.
Furthermore, an objective function for training the machine learning model may be constructed to include a loss function and a regularization term. Here, the regularization term is used to constrain the concentration of the emission probability distribution from the personalized hidden state to the shared hidden state for each object. By constructing the objective function to include a constraint term for the concentration of the emission probability distribution from the personalized hidden state to the shared hidden state for each object, the trained model can be made easier to interpret, i.e. more in line with the objective rules in the actual scenario. As an example, the canonical term herein may include constraint terms related to the entropy of the emission probability of each object from the personalized hidden state to the shared hidden state. For example, the constraint term may be constructed as Wherein,wherein λ is a real number greater than 0, < >>And indicating a transmission probability of a ith personalized hidden state to an mth shared hidden state of a ith object in the plurality of objects, wherein u, i and m are positive integers greater than 0.
The descriptions of the machine learning model of the present application mentioned above in the descriptions of fig. 1 and 2 are all adapted to fig. 3, and thus, are not repeated here.
In step S320, the training device 120 may determine a lower bound of the objective function based on the jensen inequality using the personalized hidden state sequence and the shared hidden state sequence corresponding to each sequence training sample, and determine the model parameters by maximizing the lower bound of the objective function. Specifically, in step S320, the training device 120 may transform the lower bound of the objective function to include a function term affected only by the probability of the personalized hidden state, a function term affected only by the transition probability, a function term affected only by the emission probability, and a function term affected only by the shared parameter set, and determine the corresponding model parameters by maximizing the respective function terms. In particular, for a function term that is affected by the emission probability, the training device 120 can determine the emission probability under the DCP framework by converting the problem of maximizing the function term into a one-dimensional nonlinear equation problem. In the description of fig. 1 above, the process of determining the model parameters has been described, and will not be repeated here.
In addition, the descriptions of the respective devices included in the model training system with reference to fig. 1 are applicable here, so for the relevant details related to the above steps, reference may be made to the corresponding descriptions of fig. 1, and no further description is given here.
The model training method according to the exemplary embodiments of the present application described above can not only overcome the scarcity of training data but also ensure the sequence pattern diversity of different objects due to the inclusion of two hidden state layers, thereby enabling the trained model to provide more accurate predictions of the sequence data, and furthermore, by including the regular terms for constraining the emission probability in the objective function used for training the model, the trained model can be more easily interpreted.
Hereinafter, a process of predicting sequence data using the machine learning model trained as described above will be described with reference to fig. 4 and 5.
Fig. 4 is a block diagram showing a system (hereinafter, simply referred to as "prediction system" for convenience of description) 400 for predicting sequence data using a machine learning model according to an exemplary embodiment of the present application.
Referring to fig. 4, a prediction system 400 may include a prediction sample acquisition device 410 and a prediction device 420. In particular, the prediction sample acquisition device 410 may be configured to acquire a sequence prediction sample of the object. Here, the sequence prediction sample includes a plurality of sequence data of the object arranged in time order. The prediction device 420 may perform prediction on the sequence prediction samples acquired by the prediction sample acquisition device 410 using a machine learning model to provide a prediction result regarding the next sequence data after the plurality of sequence data.
Here, the machine learning model may be trained in advance to predict the next sequence data after the series of sequence data for the series of sequence data arranged in time series, and the machine learning model may be a hidden markov model including two hidden state layers. Specifically, the first hidden state layer may include a personalized hidden state of each of the plurality of objects, and the second hidden state layer may include a plurality of shared hidden states shared by the plurality of objects. The machine learning model is herein referred to as a hidden markov model with shared hidden states in the descriptions of fig. 1 to 3, and the training process thereof may be the training process described with reference to fig. 3, which will not be described herein.
As an example, the plurality of sequence data may relate to behavior data of the object at different points in time (e.g., movement position data of the object, behavior of clicking content of the object, etc.), or may relate to status data of the object at different points in time (e.g., physiological status data of an organism, price of goods, trading price of stocks, etc.). In particular, the behavior data may include both continuous feature data reflecting the behavior of the object and discrete feature data reflecting the behavior of the object. For example, the continuous feature data may include location data of the object, and the discrete feature data may include a content ID of the content accepted by the object.
For example, in a scenario where a moving trajectory of an object (e.g., a user or a vehicle) is predicted, the predicted sample acquiring device 110 may acquire a series of historical position data of the object arranged in time order to construct the above-described sequence predicted sample; in the content recommendation scenario, the prediction sample acquisition means 110 may acquire a series of content IDs of history-accepted contents of the user arranged in time series to construct the above-described sequence prediction sample; in a scenario involving the prediction of the physiological state of a subject (human or animal), the prediction sample acquisition device 110 may acquire a series of historical physiological state data of the subject arranged in time series to construct the above-described sequence prediction sample; in a scenario involving price prediction of commodities or stocks, the prediction sample acquisition means 110 may acquire a series of historical price data of commodities or stocks arranged in time series to constitute the above-described series prediction sample.
In the hidden Markov model of the shared hidden state of the present application, each shared hidden state may correspond to a probability distribution. If the behavior data is continuous feature data reflecting the behavior of the object, the probability distribution corresponding to each shared hidden state may include a gaussian distribution, but is not limited thereto. If the behavior data is discrete feature data reflecting the behavior of the object, the probability distribution corresponding to each shared hidden state may include a polynomial distribution, but is not limited thereto.
As described above with reference to fig. 1-3, the number of personalized hidden states for each object in the first hidden state layer may be less than the number of the plurality of shared hidden states in the second hidden state layer. Further, the model parameters of the machine learning model described above may include a personalized parameter set for each object and a shared parameter set shared by the plurality of objects. In particular, the personalized parameter set may include a probability of a personalized hidden state of each object in the first hidden state layer, a transition probability between personalized hidden states of each object, and a transmission probability of each object from the personalized hidden state to the shared hidden state, and the shared parameter set may include a set of probability distributions corresponding to each shared hidden state.
As described above, the prediction sample acquisition device 410 may acquire a sequence prediction sample of an object. Specifically, the prediction sample acquisition device 410 may acquire a plurality of history data records of the object, arrange the plurality of history data records in chronological order, and construct the sequence prediction sample based on the plurality of arranged history data records. Here, if the time interval between two adjacent historical data records in the plurality of arranged historical data records meets a preset condition, slicing is performed, and then a sequence prediction sample of the object is obtained.
As an example, each of the plurality of historical data records of the object may include a plurality of data attribute fields, e.g., an object identification field, an object behavior data field, a behavior occurrence time field, etc. For example, the object behavior data field may include location data of the object at a point in time indicated by the behavior occurrence time field (e.g., the location data may be represented by a vector including longitude and latitude). Alternatively, the object behavior data field may include a content ID of the content that the object accepts at a point in time corresponding to the time indicated by the behavior occurrence time field (e.g., a news ID of news clicked by the user, or a singer ID of music listened to by the user, etc.). It should be noted that the present application is not limited in the type of the sequence data as long as it is a series of data appearing continuously in time series. In addition, the present application also has no limitation on the type of behavior data as long as it is data reflecting a series of behaviors of the subject in chronological order.
The sequence data record of the object may be an online data record, a data record that is generated and stored in advance, or a data record that is received from an external data source (e.g., a server, a database, etc.) via an input device or a transmission medium. The data records may be stored, for example, in the form of data tables, in a local storage medium or in a cloud computing platform having data storage capabilities (including, but not limited to, public and private clouds, etc.). In addition, regarding the manner in which the data records are obtained, the above-described history data records may be input to the prediction sample obtaining device 410 through an input device, or may be automatically generated by the prediction sample obtaining device 410 from the obtained data, or may be obtained by the prediction sample obtaining device 410 from a network (e.g., a storage medium (e.g., a data warehouse) on the network), and further, an intermediate data exchange device such as a server may assist the prediction sample obtaining device 410 in obtaining the corresponding data from an external data source. Here, the acquired history data record may be further converted into a format that is easy to handle, for example, form data. According to an exemplary embodiment of the present application, the above-mentioned plurality of history data records of the object may refer to a series of sequence data having a certain continuity (e.g., a continuity of behavior over time), for example, a content ID of a content continuously clicked by a user after opening an App of a certain news information class until exiting the App.
As described above, the prediction device 410 may perform prediction on the sequence prediction samples using the machine learning model to provide a prediction result regarding the next sequence data after the plurality of sequence data. Specifically, the prediction apparatus 420 may first determine a personalized parameter set for the object among model parameters of the trained machine learning model, then determine a probability that each next candidate sequence data occurs after the plurality of sequence data using the determined personalized parameter set for the object to be predicted and a shared parameter set, and determine the next sequence data after the plurality of sequence data based on the determined probabilities. Here, the predictive device 420 may first predict the probability of the personalized hidden state of the object based on the probability transitions shown in the schematic diagram shown in fig. 2 (e.g., pi shown in fig. 2 u ) And the transition probabilities between personalized hidden states (e.g., a shown in fig. 2 u ) Determining a personalized hidden state sequence of the object (e.g., as shown in FIG. 2) Second, the probability of transmission of the object from personalized hidden state to shared hidden state (e.g., B shown in FIG. 2 u ) Determining a shared hidden state sequence corresponding to the personalized hidden state sequence (e.g., in FIG. 2Shown +.>) Finally, a probability of occurrence of each next candidate sequence data after the plurality of sequence data may be determined based on the determined shared hidden state sequence and the shared parameter set (e.g., D shown in fig. 2).
Here, assuming that the plurality of sequence data described above relate to position data of an object at different points of time, for example, the object is located at positions 1 to 5 (for example, the positions may be expressed in terms of longitude and latitude) at the first to fifth points of time, respectively, the prediction device 420 may predict the probability that the object appears at the next candidate position according to the above-described prediction process. For example, assuming that there are three candidate locations (candidate location 1 to candidate location 3, e.g., the three candidate locations correspond to building 1, building 2, and building 3, respectively), the prediction apparatus 420 may calculate probabilities that the object is next located at candidate location 1 to candidate location 3, respectively. Subsequently, the prediction device 420 may determine the next sequence data after the plurality of sequence data based on the determined probability. For example, the prediction device 420 may select a candidate position having the highest calculated probability among the candidate positions 1 to 3 as the next sequence data. Assuming that the probability that the predicted object is next located at candidate position 3 is highest, the prediction apparatus 420 may determine the position data of building 3 as the next sequence data here.
For example, if the behavior data is position data of an object, the prediction device 420 may utilize the machine learning model to predict the position data of the object at a next point in time for a series of historical position data of the object that are chronologically arranged. After predicting a user or a location to which the vehicle will move next using the machine learning model, for example, the prediction system 400 may provide the prediction to a ride service provider, which may then deploy the vehicle (e.g., a sharing bicycle) to the location to better provide ride service to the user.
If the behavior data is a content ID of the content accepted by the user, the prediction means 420 may predict the content ID of the content accepted by the user at the next point in time for a series of historically accepted content IDs of the user in chronological order using the machine learning model. After predicting the content ID of the content that the user is likely to accept next, for example, the prediction system 400 may provide the prediction result to the content service provider, and then the content provider may recommend the content corresponding to the content ID to the user, thereby facilitating accurate content recommendation.
If the state data is physiological state data of the subject, the prediction device 420 may utilize the machine learning model to predict physiological state data of the subject at a next point in time for a chronological series of historical physiological state data of the subject. For example, after predicting the user's next physiological state data, the prediction system 400 may provide the prediction results to the healthcare provider, which may then instruct the user to take countermeasures in advance of the change in physiological state based on the prediction results.
If the status data is price data of a commodity or stock, the predicting means 420 may predict price data of the commodity or stock at a next point in time for a series of historical price data of the commodity or stock arranged in chronological order using the machine learning model. After predicting the price of the good or stock at the next point in time, for example, the prediction system 400 may provide the prediction result to the user to assist the user in making decisions, such as helping the user determine whether to purchase the good or stock.
It should be noted that, although only four application scenarios involving sequence data prediction are listed above, it is clear to a person skilled in the art that the scenarios to which the prediction system 400 may be applied are not limited to the four application scenarios described above, but may be applied to any scenario involving generating sequence data of an object.
The prediction system according to the exemplary embodiment may predict sequence data using a hidden markov model including two hidden state layers, so that personalized sequence data prediction may be effectively provided for different objects, and the accuracy of the prediction may be improved.
In addition, it should be noted that, although the prediction system 400 is described above as being divided into the devices (e.g., the prediction sample acquisition device 410 and the prediction device 420) for performing the respective processes, it is clear to those skilled in the art that the processes performed by the respective devices described above may be performed without any specific device division or explicit demarcation between the respective devices. Furthermore, the predictive system 400 described above with reference to fig. 4 is not limited to include the above-described devices, but may also add some other devices (e.g., storage devices, data processing devices, etc.) as desired, or the above devices may be combined. Also, as an example, the model training system 100 and the prediction system 400 described above with reference to fig. 1 may be combined into one system or be independent systems from each other, which is not limited in this application.
Fig. 5 is a flowchart illustrating a method of predicting sequence data using a machine learning model (hereinafter, simply referred to as a "prediction method" for convenience of description) according to an exemplary embodiment of the present application.
Here, by way of example, the prediction method illustrated in fig. 5 may be performed by the prediction system 400 illustrated in fig. 4, may be implemented entirely in software by a computer program or instruction, and may also be performed by a specifically configured computing system or computing device, e.g., by a system that comprises at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform the above-described prediction method. For convenience of description, it is assumed that the prediction method shown in fig. 5 is performed by the prediction system 400 shown in fig. 4, and that the prediction system 400 may have the configuration shown in fig. 4.
Referring to fig. 5, in step S510, the prediction sample acquisition device 410 may acquire a sequence prediction sample of an object. Here, the sequence prediction sample may include a plurality of sequence data of the object arranged in time order. As an example, the plurality of sequence data may relate to behavior data of the object at different points in time, or the plurality of sequence data may relate to state data of the object at different points in time. Here, the behavior data may include continuous feature data reflecting the behavior of the object, for example, the continuous feature data may include position data of the object, but is not limited thereto. Alternatively, the behavior data may include discrete feature data reflecting the behavior of the object, for example, the discrete feature data including, but not limited to, a content ID of the content accepted by the object.
Specifically, in step S510, the prediction sample acquisition device 410 may acquire a plurality of history data records of the object, arrange the plurality of history data records in chronological order, and construct the sequence prediction sample based on the plurality of arranged history data records. Here, if the time interval between two adjacent historical data records in the plurality of arranged historical data records meets a preset condition, slicing is performed, and then a sequence prediction sample of the object is obtained.
Next, in step S520, the prediction device 420 may perform prediction on the sequence prediction samples using a machine learning model to provide a prediction result regarding the next sequence data after the plurality of sequence data. Here, the machine learning model may be trained in advance to predict next sequence data after a series of sequence data arranged in time order for the series of sequence data, and the machine learning model is a hidden markov model including two hidden state layers, wherein a first hidden state layer may include therein a personalized hidden state of each of a plurality of objects, and a second hidden state layer may include therein a plurality of shared hidden states shared by the plurality of objects. Here, each shared hidden state may correspond to a probability distribution. As described above, the behavior data may include continuous feature data reflecting the behavior of the object, and at this time, the probability distribution corresponding to each shared hidden state may include a gaussian distribution, but is not limited thereto. As described above, the behavior data may also include discrete feature data reflecting the behavior of the object, and at this time, the probability distribution corresponding to each shared hidden state may include a polynomial distribution, but is not limited thereto. Further, in the machine learning model described above, the number of personalized hidden states for each object in the first hidden state layer may be less than the number of the plurality of shared hidden states in the second hidden state layer.
For example, if the above-described behavior data is the position data of the object, the prediction apparatus 420 may predict the position data of the object at the next time point using the machine learning model for a series of historical position data of the object arranged in time series in step S520. For example, if the above-described behavior data is a content ID of a content accepted by the user, the prediction apparatus 420 may predict a content ID of a content that the user will accept at a next time point using the machine learning model for a series of historically accepted content IDs of the user that are chronologically arranged at step S520. For example, if the above-mentioned state data is physiological state data of the subject, the prediction device 420 may predict the physiological state data of the subject at the next time point for a series of historical physiological state data of the subject arranged in time series using the machine learning model at step S520. For example, if the above-mentioned status data is price data of goods or stocks, the prediction means 420 may predict price data of goods or stocks at the next time point using the machine learning model for a series of historical price data of goods or stocks arranged in time series at step S520.
According to an exemplary embodiment, the model parameters of the machine learning model described above may include a personalized parameter set for each object and a shared parameter set shared by the plurality of objects. In particular, the personalized parameter set may comprise a probability of a personalized hidden state of each object in the first hidden state layer, a transition probability between personalized hidden states of each object, and a transmission probability of each object from the personalized hidden state to a shared hidden state, the shared parameter set comprising a set of probability distributions corresponding to each shared hidden state.
Specifically, in step S520, the prediction apparatus 420 may first determine a personalized parameter set for the object among model parameters of the machine learning model, then determine a probability that each next candidate sequence data appears after the plurality of sequence data using the determined personalized parameter set for the object and a shared parameter set, and finally determine the next sequence data after the plurality of sequence data based on the determined probability. For example, in determining the probability of each next candidate sequence data occurring after the plurality of sequence data, the prediction device 420 may first determine a personalized hidden state sequence for the object based on the probability of the personalized hidden state of the object and the transition probability between personalized hidden states. The prediction means 420 may then determine a shared hidden state sequence corresponding to the personalized hidden state sequence based on the determined personalized hidden state sequence and the probability of transmission of the object from the personalized hidden state to the shared hidden state, and finally, the prediction means 420 may determine a probability of occurrence of each next candidate sequence data after the plurality of sequence data based on the determined shared hidden state sequence and the set of shared parameters.
Since the prediction method shown in fig. 5 may be performed by the prediction system 400 shown in fig. 4, for the relevant details related to the above steps, reference may be made to the corresponding description of fig. 4, and a detailed description thereof will be omitted.
The prediction method according to the exemplary embodiment described above predicts sequence data by using a hidden markov model including two hidden state layers, so that personalized sequence data prediction can be effectively provided for different objects, and thus the prediction accuracy of the sequence prediction data can be improved.
Model training apparatuses and model training methods, and prediction systems and prediction methods according to exemplary embodiments of the present application have been described above with reference to fig. 1 to 5.
However, it should be understood that: the systems and their devices illustrated in fig. 1 and 4, respectively, may be configured as software, hardware, firmware, or any combination thereof that performs a particular function. For example, these systems or devices may correspond to application specific integrated circuits, to pure software code, or to modules of software in combination with hardware. Furthermore, one or more functions implemented by these systems or apparatuses may also be performed uniformly by components in a physical entity device (e.g., a processor, a client, a server, or the like).
Furthermore, the above-described methods may be implemented by instructions recorded on a computer-readable storage medium, for example, according to an exemplary embodiment of the present application, a computer-readable storage medium storing instructions may be provided, wherein the instructions, when executed by at least one computing device, cause the at least one computing device to perform the steps of: obtaining a sequence training sample set, wherein the sequence training sample set comprises a plurality of sequence training samples for each of a plurality of objects, and each sequence training sample comprises a plurality of sequence data arranged in time sequence; training the machine learning model based on the sequence training sample set, wherein the machine learning model is a hidden markov model comprising two hidden state layers, wherein a first hidden state layer comprises personalized hidden states of each of the plurality of objects and a second hidden state layer comprises a plurality of shared hidden states shared by the plurality of objects.
Furthermore, according to another exemplary embodiment of the present application, a computer-readable storage medium storing instructions may be provided, wherein the instructions, when executed by at least one computing device, cause the at least one computing device to perform the steps of: obtaining a sequence prediction sample of an object, wherein the sequence prediction sample comprises a plurality of sequence data of the object arranged in time sequence; performing prediction on the sequence prediction samples to provide a prediction result on next sequence data after the plurality of sequence data using the machine learning model, wherein the machine learning model is trained in advance to predict the next sequence data after the series of sequence data for a series of sequence data arranged in time order, and the machine learning model is a hidden markov model including two hidden state layers, wherein a first hidden state layer includes a personalized hidden state of each of the plurality of objects and a second hidden state layer includes a plurality of shared hidden states shared by the plurality of objects.
The instructions stored in the above-described computer-readable storage medium may be executed in an environment deployed in a computer device, such as a client, a host, a proxy device, a server, etc., and it should be noted that the instructions may also perform more specific processes when performing the above-described steps, the contents of which are already mentioned in the processes described with reference to fig. 3 and 5, and thus, a detailed description will not be repeated here.
It should be noted that the model training system and the predictive system according to the exemplary embodiments of the present disclosure may rely entirely on the execution of a computer program or instructions to achieve the respective functions, i.e. the respective means correspond to the respective steps in the functional architecture of the computer program, such that the entire system is invoked by a dedicated software package (e.g. lib library) to achieve the respective functions.
On the other hand, when the systems and apparatuses shown in fig. 1 and 4 are implemented in software, firmware, middleware or microcode, the program code or code segments to perform the corresponding operations may be stored in a computer-readable medium, such as a storage medium, so that at least one processor or at least one computing device can perform the corresponding operations by reading and executing the corresponding program code or code segments.
For example, according to an exemplary embodiment of the present application, a system may be provided that includes at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform the steps of: obtaining a sequence training sample set, wherein the sequence training sample set comprises a plurality of sequence training samples for each of a plurality of objects, and each sequence training sample comprises a plurality of sequence data arranged in time sequence; training the machine learning model based on the sequence training sample set, wherein the machine learning model is a hidden markov model comprising two hidden state layers, wherein a first hidden state layer comprises personalized hidden states of each of the plurality of objects and a second hidden state layer comprises a plurality of shared hidden states shared by the plurality of objects.
For example, according to another exemplary embodiment of the present application, a system may be provided that includes at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform the steps of: obtaining a sequence prediction sample of an object, wherein the sequence prediction sample comprises a plurality of sequence data of the object arranged in time sequence; performing prediction on the sequence prediction samples to provide a prediction result on next sequence data after the plurality of sequence data using the machine learning model, wherein the machine learning model is trained in advance to predict the next sequence data after the series of sequence data for a series of sequence data arranged in time order, and the machine learning model is a hidden markov model including two hidden state layers, wherein a first hidden state layer includes a personalized hidden state of each of the plurality of objects and a second hidden state layer includes a plurality of shared hidden states shared by the plurality of objects.
In particular, the above-described system may be deployed in a server or client, as well as on a node in a distributed network environment. Furthermore, the system may be a PC computer, tablet device, personal digital assistant, smart phone, web application, or other device capable of executing the above set of instructions. In addition, the system may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). Additionally, all components of the system may be connected to each other via a bus and/or a network.
Here, the system is not necessarily a single system, but may be any device or aggregate of circuits capable of executing the above-described instructions (or instruction set) alone or in combination. The system may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces with locally or remotely (e.g., via wireless transmission).
In the system, the at least one computing device may include a Central Processing Unit (CPU), a Graphics Processor (GPU), a programmable logic device, a special purpose processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the at least one computing device may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, and the like. The computing device may execute instructions or code stored in one of the storage devices, wherein the storage devices may also store data. Instructions and data may also be transmitted and received over a network via a network interface device, which may employ any known transmission protocol.
The storage device may be integrated with the computing device, for example, with RAM or flash memory disposed within an integrated circuit microprocessor or the like. Further, the storage devices may include stand-alone devices, such as external disk drives, storage arrays, or other storage devices usable by any database system. The storage device and the computing device may be operatively coupled or may communicate with each other, such as through an I/O port, network connection, or the like, such that the computing device is capable of reading instructions stored in the storage device.
The foregoing description of various exemplary embodiments of the present application has been presented for purposes of illustration and description, and is not intended to be exhaustive or to limit the application to the precise embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The scope of the application should, therefore, be determined with reference to the appended claims.

Claims (52)

1. A method of training a machine learning model for predicting sequence data, comprising:
externally obtaining a sequence training sample set, wherein the sequence training sample set comprises a plurality of sequence training samples for each of a plurality of objects, and each sequence training sample comprises a plurality of sequence data arranged in time sequence;
Training the machine learning model based on the set of sequence training samples,
wherein the machine learning model is a hidden Markov model comprising two hidden state layers, wherein a first hidden state layer comprises personalized hidden states for each of the plurality of objects and a second hidden state layer comprises a plurality of shared hidden states shared by the plurality of objects,
wherein the plurality of sequence data relates to behavior data of the object at different points in time, the machine learning model being trained to predict a next row of data of the object after a chronological series of historical behavior data of the object; or alternatively
The plurality of sequence data relates to state data of the object at different points in time, the machine learning model being trained to predict next state data of the object after a series of historical state data of the object for the series of historical state data of the object arranged in chronological order.
2. The method of claim 1, wherein each shared hidden state corresponds to a probability distribution.
3. The method of claim 2, wherein the behavior data comprises continuous feature data reflecting object behavior, the probability distribution comprising a gaussian distribution; or alternatively
Wherein the behavior data comprises discrete feature data reflecting behavior of the object, and the probability distribution comprises a polynomial distribution.
4. A method as claimed in claim 3, wherein the continuous feature data comprises location data of the object and the discrete feature data comprises a content ID of the content accepted by the object.
5. The method of claim 1, wherein a number of personalized hidden states for each object in the first hidden state layer is less than a number of the plurality of shared hidden states in the second hidden state layer.
6. The method of claim 1, wherein the model parameters of the machine learning model include a personalized parameter set for each object and a shared parameter set shared by the plurality of objects.
7. The method of claim 6, wherein the personalized parameter set comprises a probability of personalized hidden states for each object in the first hidden state layer, a transition probability between personalized hidden states for each object, and a transmission probability from personalized hidden states to shared hidden states for each object, the shared parameter set comprising a set of probability distributions corresponding to each shared hidden state.
8. The method of claim 7, wherein the objective function for training the machine learning model is constructed to include a loss function and a regularization term, wherein the regularization term is used to constrain a concentration of emission probability distributions for each object from personalized hidden states to shared hidden states.
9. The method of claim 8, wherein the regularization term includes a constraint term related to entropy of emission probability of each object from personalized hidden state to shared hidden state.
10. The method of claim 9, wherein the constraint term is structured asWherein,wherein λ is a real number greater than 0, < >>And indicating a transmission probability of a ith personalized hidden state to an mth shared hidden state of a ith object in the plurality of objects, wherein u, i and m are positive integers greater than 0.
11. The method of claim 9, wherein training the machine learning model comprises:
the lower bound of the objective function is determined based on the jensen inequality using the personalized hidden state sequence and the shared hidden state sequence corresponding to each sequence training sample, and the model parameters are determined by maximizing the lower bound of the objective function.
12. The method of claim 11, wherein determining the model parameters by maximizing a lower bound of the objective function comprises:
transforming the lower bound of the objective function to include a function term affected only by the probability of the personalized hidden state, a function term affected only by the transition probability, a function term affected only by the emission probability, and a function term affected only by the shared parameter set, and determining the corresponding model parameters by maximizing the respective function terms,
Wherein the emission probability is determined for a function term affected by the emission probability by converting a problem of maximizing the function term into a one-dimensional nonlinear equation problem under a robust planning framework.
13. The method of claim 1, wherein,
if the behavioral data is location data of the object, the machine learning model is trained to predict the location data of the object at a next point in time for a chronological series of historical location data of the object;
if the behavior data is a content ID of the content accepted by the user, the machine learning model is trained to predict the content ID of the content that the user will accept at a next point in time for a chronological sequence of historically accepted content IDs of the user;
if the state data is physiological state data of the subject, the machine learning model is trained to predict physiological state data of the subject at a next point in time for a chronological series of historical physiological state data of the subject;
if the status data is price data for a commodity or stock, the machine learning model is trained to predict price data for the commodity or stock at a next point in time for a chronological series of historical price data for the commodity or stock.
14. A computer readable storage medium storing instructions which, when executed by at least one computing device, cause the at least one computing device to perform the method of any of claims 1 to 13.
15. A system comprising at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform the method of any of claims 1-13.
16. A system for training a machine learning model for predicting sequence data, comprising:
a training sample acquisition device configured to acquire a sequence training sample set from outside, wherein the sequence training sample set includes a plurality of sequence training samples for each of a plurality of objects, and each sequence training sample includes a plurality of sequence data arranged in chronological order;
training means configured to train the machine learning model based on the set of sequence training samples,
wherein the machine learning model is a hidden Markov model comprising two hidden state layers, wherein a first hidden state layer comprises personalized hidden states for each of the plurality of objects and a second hidden state layer comprises a plurality of shared hidden states shared by the plurality of objects,
Wherein the plurality of sequence data relates to behavior data of the object at different points in time, the machine learning model being trained to predict a next row of data of the object after a chronological series of historical behavior data of the object; or alternatively
The plurality of sequence data relates to state data of the object at different points in time, the machine learning model being trained to predict next state data of the object after a series of historical state data of the object for the series of historical state data of the object arranged in chronological order.
17. The system of claim 16, wherein each shared hidden state corresponds to a probability distribution.
18. The system of claim 17, wherein the behavior data comprises continuous feature data reflecting object behavior, the probability distribution comprising a gaussian distribution; or alternatively
Wherein the behavior data comprises discrete feature data reflecting behavior of the object, and the probability distribution comprises a polynomial distribution.
19. The system of claim 18, wherein the continuous feature data comprises location data of the object and the discrete feature data comprises a content ID of the content accepted by the object.
20. The system of claim 16, wherein a number of personalized hidden states for each object in the first hidden state layer is less than a number of the plurality of shared hidden states in the second hidden state layer.
21. The system of claim 16, wherein the model parameters of the machine learning model include a personalized parameter set for each object and a shared parameter set shared by the plurality of objects.
22. The system of claim 21, wherein the personalized parameter set comprises a probability of personalized hidden states for each object in the first hidden state layer, a probability of transitions between personalized hidden states for each object, and a probability of emissions from personalized hidden states to shared hidden states for each object, the shared parameter set comprising a set of probability distributions corresponding to each shared hidden state.
23. The system of claim 22, wherein the objective function for training the machine learning model is constructed to include a loss function and a regularization term, wherein the regularization term is used to constrain a concentration of emission probability distributions for each object from personalized hidden states to shared hidden states.
24. The system of claim 23, wherein the regularization term includes a constraint term related to entropy of emission probability of each object from personalized hidden state to shared hidden state.
25. The system of claim 24, wherein the constraint is configured toWherein,wherein λ is a real number greater than 0, < >>And indicating a transmission probability of a ith personalized hidden state to an mth shared hidden state of a ith object in the plurality of objects, wherein u, i and m are positive integers greater than 0.
26. The system of claim 24, wherein the training device is configured to:
the lower bound of the objective function is determined based on the jensen inequality using the personalized hidden state sequence and the shared hidden state sequence corresponding to each sequence training sample, and the model parameters are determined by maximizing the lower bound of the objective function.
27. The system of claim 26, wherein the training means is configured to transform the lower bound of the objective function to include a function term affected only by the probability of the personalized hidden state, a function term affected only by the transition probability, a function term affected only by the emission probability, and a function term affected only by the shared parameter set, and to determine the corresponding model parameters by maximizing the respective function terms,
wherein for a function term affected by the emission probability, the training means determines the emission probability by converting a problem of maximizing the function term into a one-dimensional nonlinear equation problem under a robust planning framework.
28. The system of claim 16, wherein,
if the behavioral data is location data of the object, the machine learning model is trained to predict the location data of the object at a next point in time for a chronological series of historical location data of the object;
if the behavior data is a content ID of the content accepted by the user, the machine learning model is trained to predict the content ID of the content that the user will accept at a next point in time for a chronological sequence of historically accepted content IDs of the user;
if the state data is physiological state data of the subject, the machine learning model is trained to predict physiological state data of the subject at a next point in time for a chronological series of historical physiological state data of the subject;
if the status data is price data for a commodity or stock, the machine learning model is trained to predict price data for the commodity or stock at a next point in time for a chronological series of historical price data for the commodity or stock.
29. A method of predicting sequence data using a machine learning model, comprising:
constructing a sequence prediction sample of an object based on a plurality of historical data records of the object, wherein the sequence prediction sample comprises a plurality of sequence data of the object arranged in time sequence;
Performing prediction on the sequence prediction samples using the machine learning model to provide a prediction result regarding next sequence data after the plurality of sequence data,
wherein the machine learning model is trained in advance to predict a next sequence data after a series of sequence data arranged in time order for the series of sequence data, and the machine learning model is a hidden Markov model including two hidden state layers, wherein a first hidden state layer includes a personalized hidden state of each of a plurality of objects, a second hidden state layer includes a plurality of shared hidden states shared by the plurality of objects,
wherein the plurality of sequence data relates to behavior data of the object at different points in time or the plurality of sequence data relates to state data of the object at different points in time.
30. The method of claim 29, wherein,
if the behavior data is location data of the object, the step of performing the prediction comprises: predicting position data of the object at a next point in time for a series of historical position data of the object arranged in time sequence using the machine learning model;
if the behavior data is a content ID of the content accepted by the user, the step of performing the prediction includes: predicting a content ID of the content that the user will accept at a next point in time for a series of historically accepted content IDs of the user arranged in time order using the machine learning model;
If the state data is physiological state data of the subject, the step of performing the prediction comprises: predicting physiological state data of the subject at a next point in time for a series of historical physiological state data of the subject arranged in time sequence using the machine learning model;
if the status data is price data for a commodity or stock, the step of performing the prediction includes: price data of a commodity or stock at a next point in time is predicted for a series of historical price data of the commodity or stock in chronological order using the machine learning model.
31. The method of claim 29, wherein each shared hidden state corresponds to a probability distribution.
32. The method of claim 31, wherein the behavior data comprises continuous feature data reflecting object behavior, the probability distribution comprising a gaussian distribution; or alternatively
Wherein the behavior data comprises discrete feature data reflecting behavior of the object, and the probability distribution comprises a polynomial distribution.
33. The method of claim 32, wherein the continuous feature data comprises location data of the object and the discrete feature data comprises a content ID of the content accepted by the object.
34. The method of claim 29, wherein constructing the sequence prediction samples comprises: and arranging the plurality of historical data records in time sequence, and constructing the sequence prediction sample based on the arranged plurality of historical data records, wherein if the time interval between two adjacent historical data records in the arranged plurality of historical data records meets a preset condition, the sequence prediction sample of the object is obtained by cutting.
35. The method of claim 29, wherein the model parameters of the machine learning model include a personalized parameter set for each object and a shared parameter set shared by the plurality of objects.
36. The method of claim 35, wherein the personalized parameter set comprises a probability of personalized hidden states for each object in the first hidden state layer, a probability of transitions between personalized hidden states for each object, and a probability of emissions from personalized hidden states to shared hidden states for each object, the shared parameter set comprising a set of probability distributions corresponding to each shared hidden state.
37. The method of claim 36, wherein the step of performing prediction comprises:
Determining a personalized parameter set for the object among model parameters of the machine learning model;
determining a probability of occurrence of each next candidate sequence data after the plurality of sequence data using the determined personalized parameter set and shared parameter set for the object;
based on the determined probability, a next sequence data after the plurality of sequence data is determined.
38. The method of claim 37, wherein determining the probability of each next candidate sequence data occurring after the plurality of sequence data comprises:
determining a personalized hidden state sequence of the object according to the probability of the personalized hidden state of the object and the transition probability between the personalized hidden states;
determining a shared hidden state sequence corresponding to the personalized hidden state sequence according to the determined personalized hidden state sequence and the emission probability of the object from the personalized hidden state to the shared hidden state;
and determining the probability of each next candidate sequence data after the plurality of sequence data according to the determined shared hidden state sequence and the shared parameter set.
39. The method of claim 29, wherein a number of personalized hidden states for each object in the first hidden state layer is less than a number of the plurality of shared hidden states in the second hidden state layer.
40. A computer-readable storage medium storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform the method of any of claims 29-39.
41. A system comprising at least one computing device and at least one storage device storing instructions that, when executed by the at least one computing device, cause the at least one computing device to perform the method of any of claims 29 to 39.
42. A system for predicting sequence data using a machine learning model, comprising:
a prediction sample acquisition device configured to construct a sequence prediction sample of an object based on a plurality of historic data records of the object, wherein the sequence prediction sample includes a plurality of sequence data of the object arranged in chronological order;
prediction means configured to perform prediction on the sequence prediction samples using the machine learning model to provide a prediction result regarding next sequence data after the plurality of sequence data,
wherein the machine learning model is trained in advance to predict a next sequence data after a series of sequence data arranged in time order for the series of sequence data, and the machine learning model is a hidden Markov model including two hidden state layers, wherein a first hidden state layer includes a personalized hidden state of each of a plurality of objects, a second hidden state layer includes a plurality of shared hidden states shared by the plurality of objects,
Wherein the plurality of sequence data relates to behavior data of the object at different points in time or the plurality of sequence data relates to state data of the object at different points in time.
43. The system of claim 42, wherein,
if the behavior data is the position data of the object, the predicting means predicts the position data of the object at the next point in time using the machine learning model for a series of historical position data of the object arranged in time sequence;
if the behavior data is a content ID of the content accepted by the user, the predicting means predicts the content ID of the content accepted by the user at the next point in time using the machine learning model for a series of historically accepted content IDs of the user in chronological order;
if the state data is physiological state data of the subject, the predicting means predicts the physiological state data of the subject at a next point in time using the machine learning model for a series of historical physiological state data of the subject arranged in time order;
if the status data is price data of a commodity or stock, the predicting means predicts price data of the commodity or stock at the next point in time using the machine learning model for a series of historical price data of the commodity or stock arranged in chronological order.
44. The system of claim 42, wherein each shared hidden state corresponds to a probability distribution.
45. The system of claim 44, wherein the behavior data comprises continuous feature data reflecting object behavior, the probability distribution comprising a gaussian distribution; or alternatively
Wherein the behavior data comprises discrete feature data reflecting behavior of the object, and the probability distribution comprises a polynomial distribution.
46. The system of claim 45, wherein the continuous feature data comprises location data of the object and the discrete feature data comprises a content ID of content accepted by the object.
47. The system of claim 42, wherein constructing sequence prediction samples of the object comprises: and arranging the plurality of historical data records in time sequence, and constructing the sequence prediction sample based on the arranged plurality of historical data records, wherein if the time interval between two adjacent historical data records in the arranged plurality of historical data records meets a preset condition, the sequence prediction sample of the object is obtained by cutting.
48. The system of claim 42, wherein the model parameters of the machine learning model include a personalized parameter set for each object and a shared parameter set shared by the plurality of objects.
49. The system of claim 48, wherein the personalized parameter set comprises a probability of personalized hidden states for each object in the first hidden state layer, a probability of transitions between personalized hidden states for each object, and a probability of emissions from personalized hidden states to shared hidden states for each object, the shared parameter set comprising a set of probability distributions corresponding to each shared hidden state.
50. The system of claim 49, wherein the predictive device is configured to:
determining a personalized parameter set for the object among model parameters of the machine learning model;
determining a probability of occurrence of each next candidate sequence data after the plurality of sequence data using the determined personalized parameter set and shared parameter set for the object;
based on the determined probability, a next sequence data after the plurality of sequence data is determined.
51. The system of claim 50, wherein the predictive device is configured to:
determining a personalized hidden state sequence of the object according to the probability of the personalized hidden state of the object and the transition probability between the personalized hidden states;
determining a shared hidden state sequence corresponding to the personalized hidden state sequence according to the determined personalized hidden state sequence and the emission probability of the object from the personalized hidden state to the shared hidden state;
And determining the probability of each next candidate sequence data after the plurality of sequence data according to the determined shared hidden state sequence and the shared parameter set.
52. The system of claim 42, wherein the number of personalized hidden states for each object in the first hidden state layer is less than the number of the plurality of shared hidden states in the second hidden state layer.
CN202110497221.4A 2019-04-28 2019-04-28 Method and system for training model and method and system for predicting sequence data Active CN113112030B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110497221.4A CN113112030B (en) 2019-04-28 2019-04-28 Method and system for training model and method and system for predicting sequence data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910349922.6A CN110097193B (en) 2019-04-28 2019-04-28 Method and system for training model and method and system for predicting sequence data
CN202110497221.4A CN113112030B (en) 2019-04-28 2019-04-28 Method and system for training model and method and system for predicting sequence data

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201910349922.6A Division CN110097193B (en) 2019-04-28 2019-04-28 Method and system for training model and method and system for predicting sequence data

Publications (2)

Publication Number Publication Date
CN113112030A CN113112030A (en) 2021-07-13
CN113112030B true CN113112030B (en) 2023-12-26

Family

ID=67446102

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201910349922.6A Active CN110097193B (en) 2019-04-28 2019-04-28 Method and system for training model and method and system for predicting sequence data
CN202110497221.4A Active CN113112030B (en) 2019-04-28 2019-04-28 Method and system for training model and method and system for predicting sequence data

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201910349922.6A Active CN110097193B (en) 2019-04-28 2019-04-28 Method and system for training model and method and system for predicting sequence data

Country Status (1)

Country Link
CN (2) CN110097193B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110457329B (en) * 2019-08-16 2022-05-06 第四范式（北京）技术有限公司 Method and device for realizing personalized recommendation
CN110852442B (en) * 2019-10-29 2022-03-15 支付宝(杭州)信息技术有限公司 Behavior identification and model training method and device
CN111191834A (en) * 2019-12-26 2020-05-22 北京摩拜科技有限公司 User behavior prediction method and device and server
CN111597121B (en) * 2020-07-24 2021-04-27 四川新网银行股份有限公司 Precise test method based on historical test case mining
CN111881355B (en) * 2020-07-28 2023-03-10 北京深演智能科技股份有限公司 Object recommendation method and device, storage medium and processor
CN112199095B (en) * 2020-10-16 2022-04-26 深圳大学 Encryption API (application program interface) use analysis method and system
CN112785371A (en) * 2021-01-11 2021-05-11 上海钧正网络科技有限公司 Shared device position prediction method, device and storage medium
CN113509726B (en) * 2021-04-16 2023-12-05 超参数科技(深圳)有限公司 Interaction model training method, device, computer equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103201754A (en) * 2010-11-18 2013-07-10 索尼公司 Data processing device, data processing method, and program
CN105181898A (en) * 2015-09-07 2015-12-23 李岩 Atmospheric pollution monitoring and management method as well as system based on high-density deployment of sensors
CN107657280A (en) * 2013-03-15 2018-02-02 英特尔公司 Real-time continuous interactive study and detection

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2001269521A1 (en) * 2000-07-13 2002-01-30 Asahi Kasei Kabushiki Kaisha Speech recognition device and speech recognition method
CN101840699B (en) * 2010-04-30 2012-08-15 中国科学院声学研究所 Voice quality evaluation method based on pronunciation model
US9836576B1 (en) * 2012-11-08 2017-12-05 23Andme, Inc. Phasing of unphased genotype data
CN103035236B (en) * 2012-11-27 2014-12-17 河海大学常州校区 High-quality voice conversion method based on modeling of signal timing characteristics
CN104021390B (en) * 2013-03-01 2018-01-02 佳能株式会社 Model generating means, pattern recognition apparatus and its method
JP6679898B2 (en) * 2015-11-24 2020-04-15 富士通株式会社 KEYWORD DETECTION DEVICE, KEYWORD DETECTION METHOD, AND KEYWORD DETECTION COMPUTER PROGRAM
CN106845319A (en) * 2015-12-03 2017-06-13 佳能株式会社 Hand-written register method, hand-written recognition method and its device
CN105931271B (en) * 2016-05-05 2019-01-18 华东师范大学 A kind of action trail recognition methods of the people based on variation BP-HMM
CN106503267A (en) * 2016-12-07 2017-03-15 电子科技大学 A kind of personalized recommendation algorithm suitable for user preference dynamic evolution
CN108615525B (en) * 2016-12-09 2020-10-09 中国移动通信有限公司研究院 Voice recognition method and device
CN108241872A (en) * 2017-12-30 2018-07-03 北京工业大学 The adaptive Prediction of Stock Index method of Hidden Markov Model based on the multiple features factor
CN108648748B (en) * 2018-03-30 2021-07-13 沈阳工业大学 Acoustic event detection method under hospital noise environment
CN109413587A (en) * 2018-09-20 2019-03-01 广州纳斯威尔信息技术有限公司 User trajectory prediction technique based on WiFi log
CN109326277B (en) * 2018-12-05 2022-02-08 四川长虹电器股份有限公司 Semi-supervised phoneme forced alignment model establishing method and system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103201754A (en) * 2010-11-18 2013-07-10 索尼公司 Data processing device, data processing method, and program
CN107657280A (en) * 2013-03-15 2018-02-02 英特尔公司 Real-time continuous interactive study and detection
CN105181898A (en) * 2015-09-07 2015-12-23 李岩 Atmospheric pollution monitoring and management method as well as system based on high-density deployment of sensors

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DiscoveringPeriodi Patterns for Large Scale MobileTraffic Data Method and Applications;Hongzhi Shi等;《IEEE Transactions or Mobile Computing》;第17卷;全文 *
Selection of Shared-State Hidden Markov Model Structure Using Bayesian Criterion;Shinji WATANABE等;《IEICE TRANSACTIONS on information and Systems》;第E88-D卷(第1期);全文 *
雷达高分辨距离像目标识别技术研究;潘勉;《中国博士学位论文全文数据库信息科技辑》;全文 *

Also Published As

Publication number Publication date
CN110097193A (en) 2019-08-06
CN113112030A (en) 2021-07-13
CN110097193B (en) 2021-03-19

Similar Documents

Publication Publication Date Title
CN113112030B (en) Method and system for training model and method and system for predicting sequence data
US11521221B2 (en) Predictive modeling with entity representations computed from neural network models simultaneously trained on multiple tasks
AU2023203298A1 (en) System to collect and identify skin conditions from images and expert knowledge
WO2019015631A1 (en) Method for generating combined features for machine learning samples and system
US20230153857A1 (en) Recommendation model training method, recommendation method, apparatus, and computer-readable medium
Urena et al. GDM-R: A new framework in R to support fuzzy group decision making processes
US11556567B2 (en) Generating and visualizing bias scores representing bias in digital segments within segment-generation-user interfaces
CN102289444A (en) Methods, apparatus, and articles of manufacture to rank users in online social network
Jabbari et al. Discovery of causal models that contain latent variables through Bayesian scoring of independence constraints
Lucherini et al. T-RECS: A simulation tool to study the societal impact of recommender systems
US20200111575A1 (en) Producing a multidimensional space data structure to perform survival analysis
US20220129709A1 (en) Systems and Methods for Preference and Similarity Learning
WO2020174233A1 (en) Machine-learned model selection network planning
CN113032676B (en) Recommendation method and system based on micro-feedback
US20230030341A1 (en) Dynamic user interface and machine learning tools for generating digital content and multivariate testing recommendations
CN116578774A (en) Method, device, computer equipment and storage medium for pre-estimated sorting
CN115631008B (en) Commodity recommendation method, device, equipment and medium
US20210231449A1 (en) Deep User Modeling by Behavior
CN113705698A (en) Information pushing method and device based on click behavior prediction
CN114298118B (en) Data processing method based on deep learning, related equipment and storage medium
CN113095592A (en) Method and system for performing predictions based on GNN and training method and system
US20220375549A1 (en) Method and system of generating, delivering and displaying cross platform personalized digital software applications
JP7462254B1 (en) Program, method, information processing device, and system
CN111382346B (en) Method and system for recommending content
US20230342799A1 (en) Generating segments of users based on unobserved behaviors

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant