CN112612768B - Model training method and device - Google Patents
Model training method and device Download PDFInfo
- Publication number
- CN112612768B CN112612768B CN202011459323.9A CN202011459323A CN112612768B CN 112612768 B CN112612768 B CN 112612768B CN 202011459323 A CN202011459323 A CN 202011459323A CN 112612768 B CN112612768 B CN 112612768B
- Authority
- CN
- China
- Prior art keywords
- data
- offline
- behavior
- characteristic data
- behavior characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/45—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Abstract
The embodiment of the application provides a model training method, which comprises the steps of obtaining an off-line behavior characteristic data set, and orderly storing the off-line behavior characteristic data set to a target storage unit; receiving a stream batch data pulling request of an offline behavior characteristic data set stored in a target storage unit; sequentially pulling off-line behavior characteristic data of a corresponding time section from the off-line behavior characteristic data set based on the stream batch data pulling request, and generating first training sample data according to the pulled off-line behavior characteristic data; inputting all generated first training sample data into a model to be trained for training to obtain an initial estimation model; acquiring online behavior characteristic data, and generating second training sample data according to the online behavior characteristic data; and inputting the second training sample data into the initial prediction model for training to obtain a target prediction model. The method and the device can improve the training efficiency of the click rate pre-estimation model.
Description
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a model training method and device.
Background
With the development of computer technology, Artificial Intelligence (AI) is beginning to be focused on by people, and is applicable to various fields such as language recognition, image recognition, natural language processing, and target tracking.
AI training is an important loop in AI engineering, determining the effectiveness of an AI. In the existing AI training, an online feature is obtained through SingleTask (single task), an offline feature is obtained through Crontab offline calculation task, and feature vectorization calculation is performed on the above feature through SingleTask, so as to obtain a training data stream.
However, the training data stream obtained by the method takes a lot of time to obtain a sufficient number of training data streams, and thus the training of the model is very time-consuming.
Disclosure of Invention
An object of the embodiments of the present application is to provide a model training method, system, computer device and computer-readable storage medium, which can be used to solve the following problems: the training of the model is very time consuming.
One aspect of an embodiment of the present application provides a model training method, including:
acquiring an offline behavior characteristic data set, and orderly storing the offline behavior characteristic data set to a target storage unit, wherein the offline behavior characteristic data set is orderly stored according to a preset time section;
receiving a stream batch data pulling request of an offline behavior characteristic data set stored in the target storage unit;
sequentially pulling off-line behavior characteristic data of a corresponding time section from the off-line behavior characteristic data set based on the stream batch data pulling request, and generating first training sample data according to the pulled off-line behavior characteristic data;
inputting all generated first training sample data into a model to be trained for training to obtain an initial estimation model;
acquiring online behavior characteristic data, and generating second training sample data according to the online behavior characteristic data;
and inputting the second training sample data into the initial prediction model for training to obtain a target prediction model.
Optionally, the target storage unit is a distributed file system HDFS, and the obtaining the offline behavior feature data set and sequentially storing the offline behavior feature data set to the target storage unit includes:
identifying a generation time of each piece of offline behavior feature data in the set of offline behavior feature data;
and storing the offline behavior characteristic data belonging to the same time section into the same data block of the HDFS according to the generation time, and sequentially storing the offline behavior characteristic data of different time sections into different data blocks of the HDFS.
Optionally, the generating first training sample data according to the pulled offline behavior feature data includes:
and performing aggregation calculation on the pulled offline behavior characteristic data through an offline behavior characteristic data processing module written based on the BSQL language to obtain the first training sample data.
Optionally, the obtaining the online behavior feature data and generating second training sample data according to the online behavior feature data includes:
acquiring online behavior characteristic data of a plurality of users in a sliding time window mode;
and counting the online behavior data of each user in the time window, and taking the online behavior data of each user as second training sample data.
Optionally, the online behavior feature data includes online click behavior feature data of the user and display behavior feature data of the user for exposing the media file, and the statistics of the online behavior data of each user in the time window and the taking of the online behavior data of each user as a second training sample data includes:
and splicing the click behavior characteristic data and the display behavior characteristic data of each user in the time window through an online behavior characteristic data processing module written based on the BSQL language to obtain spliced characteristic data, and taking the spliced characteristic data as the second training sample data.
Optionally, the splicing processing is performed on the click behavior feature data and the display behavior feature data of each user in the time window by using the online behavior feature data processing module written based on the BSQL language, so as to obtain spliced feature data, and the step of taking the spliced feature data as the second training sample data includes:
splicing the click behavior characteristic data and the display behavior characteristic data of each user in the time window through an online behavior characteristic data processing module compiled based on the BSQL language to obtain spliced characteristic data;
and acquiring portrait feature data of each user, and performing feature fusion processing on the spliced feature data and the portrait feature data corresponding to each user to obtain the second training sample data.
Optionally, the offline behavior feature data set includes offline click behavior feature data of a user and display behavior feature data of an exposed media file for the user, and the obtaining of the first training sample data by performing aggregation calculation on the pulled offline behavior feature data through an offline behavior feature data processing module written based on BSQL language includes:
performing aggregation calculation on the pulled offline behavior characteristic data through an offline behavior characteristic data processing module written based on a BSQL language to obtain offline real-time characteristic data of each user in a current time section;
and acquiring portrait feature data of each user, and performing feature fusion processing on the offline real-time feature data and the portrait feature data corresponding to each user to obtain the first training sample data.
Yet another aspect of embodiments of the present application provides a model training apparatus, including:
the system comprises an acquisition module, a target storage unit and a storage module, wherein the acquisition module is used for acquiring an offline behavior characteristic data set and sequentially storing the offline behavior characteristic data set to the target storage unit, and the offline behavior characteristic data set is sequentially stored according to a preset time section;
the receiving module is used for receiving a stream batch data pulling request of the offline behavior characteristic data set stored in the target storage unit;
the pull module is used for sequentially pulling the off-line behavior characteristic data of the corresponding time section from the off-line behavior characteristic data set based on the stream batch data pull request and generating first training sample data according to the pulled off-line behavior characteristic data;
the first training module is used for inputting all the generated first training sample data into a model to be trained for training so as to obtain an initial pre-estimated model;
the generating module is used for acquiring online behavior characteristic data and generating second training sample data according to the online behavior characteristic data;
and the second training module is used for inputting the second training sample data into the initial prediction model for training so as to obtain a target prediction model.
Yet another aspect of an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the model training method as described in any one of the above when executing the computer program.
A further aspect of embodiments of the present application provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, is adapted to carry out the steps of the model training method according to any one of the preceding claims.
According to the model training method, the model training device, the computer equipment and the computer readable storage medium, an offline behavior feature data set is obtained and stored to a target storage unit in order, wherein the offline behavior feature data set is stored in order according to a preset time section; receiving a stream batch data pulling request of an offline behavior characteristic data set stored in the target storage unit; sequentially pulling off-line behavior characteristic data of a corresponding time section from the off-line behavior characteristic data set based on the stream batch data pulling request, and generating first training sample data according to the pulled off-line behavior characteristic data; inputting all generated first training sample data into a model to be trained for training to obtain an initial estimation model; acquiring online behavior characteristic data, and generating second training sample data according to the online behavior characteristic data; and inputting the second training sample data into the initial prediction model for training to obtain a target prediction model. According to the embodiment, offline data are sequentially stored in the target storage unit, so that a batch of data can be pulled to be processed based on a batch data pulling request, first training sample data are generated quickly to perform model training, and the training efficiency of the click rate estimation model can be improved. Meanwhile, an initial estimation model is obtained through training, the initial estimation model is trained again through second training sample data generated through the online behavior characteristic data, fine adjustment of the model is achieved, and estimation accuracy of a target estimation model obtained through final training can be improved.
Drawings
Fig. 1 schematically shows a link diagram of a streaming data transmission link;
FIG. 2 schematically illustrates a partition slicing diagram;
FIG. 3 schematically shows a flow chart of a model training method according to a first embodiment of the present application;
FIG. 4 is a flow chart schematically illustrating a detailed step of obtaining an offline behavior feature data set and storing the offline behavior feature data set to a target storage unit in an ordered manner according to the present application;
fig. 5 schematically shows a detailed flowchart of the step of obtaining the first training sample data by performing aggregation calculation on the pulled offline behavior feature data through an offline behavior feature data processing module written based on BSQL language;
FIG. 6 is a flow chart schematically illustrating a detailed step of obtaining online behavior feature data and generating second training sample data according to the online behavior feature data;
fig. 7 is a flowchart schematically illustrating a step refinement of obtaining the first training sample data by performing aggregation calculation on the pulled offline behavior feature data through an offline behavior feature data processing module written based on BSQL language;
FIG. 8 schematically shows a block diagram of a model training apparatus according to a second embodiment of the present application; and
fig. 9 schematically shows a hardware architecture diagram of a computer device suitable for implementing the model training method according to the third embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the descriptions relating to "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present application.
In the description of the present application, it should be understood that the numerical references before the steps do not identify the order of performing the steps, but merely serve to facilitate the description of the present application and to distinguish each step, and therefore should not be construed as limiting the present application.
The following are some explanations of terms that the present application refers to:
crontab, a timed execution script.
HDFS (Hadoop Distributed File System) is a Hadoop Distributed File system.
The Flink Cluster (Flink Cluster) is a distributed system for stateful computation of unbounded and bounded data streams. Flink is designed to run in all common clustered environments, performing calculations at memory speed and any scale.
And the task deployer (JobManager) is used as a Master node (main node) of the Flink cluster and is responsible for task scheduling and resource management of the Flink cluster.
Task performer (TaskManager) as the Worker node (slave node) of the Flink cluster. And the TaskManager receives the tasks to be deployed from the JobManager and is responsible for specific task execution and resource application and management of the corresponding tasks on each node.
And the data input module (Source) is used as a data input interface and used for consuming data from a corresponding theme (Topic) in the data cache layer 3.
And the data output module (Sink) is used as a data output interface and is used for distributing the data to the storage terminals of the data storage layer 5.
Watermark (watermark), a mechanism proposed by Apache Flink to handle EventTime window computations, is essentially a time stamp.
The Airflow is a distributed task scheduling framework, and a workflow with an upper-level dependency relationship and a lower-level dependency relationship is assembled into a directed acyclic graph.
The flow id (LogId) may be defined by a three-segment semantic (e.g., department + project + business) so that the category to which the data belongs can be quickly locked, and may also be defined with other attached information, such as creator information, etc. The data stream may be defined with schema (organization and structure of the database) such as information of fields, types, necessity or not. The schema may be used for analysis and evaluation operations of the data stream. According to the defined schema, the metadata information of the data stream may be sent with corresponding field values, such as Service scenarios, and different Service scenarios may be configured with different SLA (Service-Level agent) quality guarantees. It should be noted that these field values may be sent and modified by a user or administration.
Fig. 1 schematically shows a streaming data transmission link according to an embodiment of the present application, said streaming data transmission link consisting in providing a streaming data transmission service, such as data collection and distribution for both real-time streaming and offline streaming scenarios. The real-time streaming scene is mainly used for writing data into databases such as Kafka and Hbase, and corresponds to the data timeliness of the second level. The offline flow scene corresponds to the timeliness of data at an hour level or a day level and is mainly used for writing the data into databases such as HDFS (Hadoop distributed file system) and Hive. The streaming data transmission system may be composed of: data source 1, network routing layer 2, data buffer layer 3, data distribution layer 4, data storage layer 5, etc.
The data source 1 may be an internal data source, or may be connected to a data interface of an external data source. The data source 1 may have data in multiple formats, for example, the reported data of APP and Web are data in HTTP (HyperText Transfer Protocol), and the internal communication data of the server is data in RPC (Remote Procedure Call) format. As shown in fig. 1, the data of the data source 1 may be log data reported by the mobile terminal and received by one or more edge nodes, or may be data provided by various systems or devices such as a database, a data coordination system, and a log agent.
The network routing layer 2, which may be implemented by one or more gateway nodes, is used to forward data provided by the BFE layer 1 to the data buffer layer 3. Specifically, the network routing layer 2 is configured to be connected to the BFE layer 1, and may be adapted to various service scenarios and data protocols, for example, APP and Web data configured to be compatible with a HyperText Transfer Protocol (HTTP) Protocol, and internal communication data of a GRPC Protocol.
The data buffer layer 3 may be implemented by a message distribution subscription system or the above system cluster. In some embodiments, the data buffer layer 3 may be composed of multiple sets of kafka clusters, which function as data peak clipping and valley filling. Data with different importance, priority and data throughput can be distributed to different kafka clusters to guarantee the value of different types of data and avoid the influence of system faults on the whole data.
The data distribution layer 4 may be implemented by a model training method (composed of a plurality of traffic distribution nodes Collector), and is used for content conversion and distribution storage, that is, to ensure that data is acquired from the data buffer layer 3 and written into a corresponding storage terminal in the data storage layer 5. Specifically, the data distribution layer 4 is used for landing of data distribution, and supported distribution scenarios include HDFS, Kafka, Hbase, ES (elastic search), and the like, and in the distribution process, due to the fact that timeliness requirements of data landing of different storage terminals may be different, for example, data writing of HDFS is calculation and application of tasks performed by day, data writing of Kafka is generally calculation and application of tasks performed by second, and is generally used in scenarios such as real-time recommendation, real-time calculation, and the like. The data distribution layer 4 may perform service grouping management according to the storage terminal according to the distribution requirements of different scenarios of data. For example, the lines may be divided into Kafka Collector groups, HDFS Collector groups, and the like. Different Collector groups will take data of the corresponding topic (topic) from the data buffer layer 3 and distribute it downstream.
By way of example, the data distribution layer 4 is composed of clusters, e.g., a Flume cluster or a Flink cluster. Taking the Flink cluster as an example: the Fink cluster can consume the data stream under the corresponding Topic in the data cache layer 3, process the data stream and then issue the processed data stream to the data storage layer 5. The Flink cluster may include two modules, the JobManager and the TaskManager. The TaskManager comprises Source and Sink. Source and Sink are introduced as follows: (1) source: data may be consumed from an upstream node (e.g., data cache layer 3). (2) Sink: a write of the partial data to a downstream node may be received, e.g., under a directory specified by the HDFS.
And a data storage layer 5 for storing data, which may be composed of various forms of databases, such as HDFS, ES, Hive, Kafka, Hbase, and the like.
Namely, the data flow of the streaming data transmission link is as follows: data source 1 → network routing layer 2 → data buffer layer 3 → data distribution layer 4 → data storage layer 5. Through the streaming data transmission link, data in a data source can be transmitted to a target terminal. The method comprises the following specific steps: the data source can output a data stream with the LogId as a stream identifier, and sequentially passes through the gateway routing layer 2, the data buffer layer 3, the data distribution layer 4, and finally enters the data storage layer 5.
(1) In an offline scenario:
(1.1) the partition slice order characteristic:
when storing the offline stream, the generated data of different time partitions can be orderly saved in the HDFS. The method comprises the following specific steps:
in this embodiment, the data consumption (inflow) speed of each Source is controlled to store data in order according to time, so that the HDFS is obtained. Each Source consumes a respective data stream; reporting the watermark of the corresponding data stream to the JobManager. And the JobManager generates a speed control instruction according to the watermark reported by each Source and issues the speed control instruction to each Source. And each Source controls the consumption speed of each data stream according to the received speed control instruction so as to keep consistent with the consumption speed of other data streams in other sources, so that the HDFS can acquire all data in one time window at a time. In this embodiment, it is ensured that the data streams in each Partition flow into the HDSF through the Flink cluster at substantially the same speed. As shown in fig. 2, data P1_ T1, P2_ T1, P3_ T1, P4_ T1, P5_ T1, P6_ T1, etc. corresponding to the T1 partition are stored in the HDFS cluster at approximately the same time. Since all the partition data of this time slice T1 are stored into the HDFS cluster at almost the same time, the corresponding data processing tasks can be executed for all the partition data of this time slice T1, performing the tasks at minute level.
The transmission and storage of off-line data are introduced above, and the model is trained on the near line by the off-line data and the on-line data.
Example one
Fig. 3 schematically shows a flowchart of a model training method according to a first embodiment of the present application. The following description is made by taking a computer device as an execution subject. It should be noted that the computer device may be a Flink cluster. It is to be understood that the flow charts in the embodiments of the present application are not used to limit the order of executing the steps.
As shown in fig. 3, the model training method may include steps S30 to S35, wherein:
step S30, obtaining an offline behavior feature data set, and sequentially storing the offline behavior feature data set to a target storage unit, where the offline behavior feature data set is sequentially stored according to a preset time segment.
Specifically, the offline behavior feature data set may include click data and presentation data of a plurality of users within a historical preset time period, for example, click data and presentation data of the users in the past 30 days.
As an example, the click data may include the following feature-related data: 1) behavior characteristics of various multimedia files (including video manuscripts, television play videos, movie videos, art videos and the like) clicked by users, such as: clicking, like, collecting, etc.; (2) the user clicks on the label category characteristics of various multimedia files, such as: video tags, video types, video attributes, partitions, video uploader identifications, and the like. Among them, video tags such as "game", "absolutely living", "make-up in summer", "eat chicken", and the like. (3) The basic characteristic data of the user comprises the age and sex, the member grade, the screen type, the province, the city and the model of the used equipment, such as Huacheng mate 40.
It should be noted that each click action of the user generates one piece of click data.
As an example, the presentation data may include the following feature-related data: 1) the label category characteristics of multimedia files (including video manuscripts, television show videos, movie videos, art videos, etc.) exposed to the user, such as: video tags, video types, video attributes, partitions, video uploader identifications, and the like. Among them, video tags such as "game", "absolutely living", "make-up in summer", "eat chicken", and the like. (2) The basic characteristic data of the user comprises the age and sex, the membership grade, the screen type, the province, the city and the model of the used equipment, such as Huazhimate 40.
It should be noted that one piece of presentation data is generated for each exposure by the user.
The target storage unit can be a distributed file system HDFS, a hive database, an HBase database and the like. In this embodiment, the target memory unit is preferably an HDFS.
As an example, the time zone may be a default time zone or a time period set by a user, for example, the time zone is 5 minutes.
As shown in fig. 4, in order to facilitate storing the offline behavior feature data sets in order to the target storage unit, the step of obtaining the offline behavior feature data sets and storing the offline behavior feature data sets in order to the target storage unit may include steps S40-S41, where: step S40, identifying a generation time of each piece of offline behavior feature data in the offline behavior feature data set; step S41, storing the offline behavior feature data belonging to the same time segment into the same data block of the HDFS according to the generation time, and storing the offline behavior feature data of different time segments into different data blocks of the HDFS in sequence. As an example, assuming that the offline behavior feature data set includes click data and presentation data of a user in the past 1 day, and the time zone is 4 hours, when the offline behavior feature data set is stored in the HDFS, the generation time of each piece of offline behavior feature data may be identified first, and after all the generation times of the offline behavior feature data are identified, all the offline behavior feature data from 0 point to 4 points in the past may be stored in the first data block under a certain file in the HDFS, that is, block 1; all offline behavior feature data of past 4 points-8 points are stored in the second data block under the file in the HDFS, namely block62, and the like, and all offline behavior feature data of past 20 points-24 points are stored in the sixth data block under the file in the HDFS, namely block 6.
Step S31, receiving a stream batch data pull request for the offline behavior feature data set stored in the target storage unit.
Specifically, the streaming batch data pulling request is used for a batch of offline behavior characteristic data belonging to the same time section in the target storage unit.
And step S32, sequentially pulling the off-line behavior feature data of the corresponding time section from the off-line behavior feature data set based on the stream batch data pulling request, and generating first training sample data according to the pulled off-line behavior feature data.
Specifically, offline behavior feature data belonging to the same time segment may be stored in different data blocks under different files of the target storage unit.
Taking the target storage unit as the HDFS as an example, based on the batch data pull request, all the offline behavior feature data of past 0 point to 4 points stored in block1 may be pulled from the HDFS first, and then the offline behavior feature data of this time interval is processed to generate the first training sample data. And then, continuously pulling all the offline behavior characteristic data of the past 4 points to 8 points stored in the block2 from the HDFS, then processing the offline behavior characteristic data of the time interval to generate first training sample data, and repeating the operation until the offline behavior characteristic data stored in the HDFS is processed.
It should be noted that, when a plurality of files in the HDFS all store offline behavior feature data, when pulling data, offline behavior feature data belonging to the same time interval may be pulled from data blocks under the plurality of files at the same time.
As an example, in order to reduce the training threshold of the model, the step of generating the first training sample data according to the pulled offline behavior feature data may include: and performing aggregation calculation on the pulled offline behavior characteristic data through an offline behavior characteristic data processing module written based on BSQL language to obtain the first training sample data.
Specifically, the BSQL is an SQL intelligent query analysis tool, and supports databases of Oracle, SQLServer, MySQL, Access, Sybase, and SQLAnywhere, and the BSQL supports rapid export of data of the databases to Excel, HTML, RTF, PDF, XML, and the like.
The main characteristics of BSQL:
1) the SQL grammar is automatically completed quickly.
2) The fast script is automatically generated.
3) A dedicated SQL query editor is provided.
4) And automatically and quickly generating professional database reports. And providing a report generation guide and performing self-definition.
5) The database data is exported to a variety of available file formats including MS Excel, HTML, RTF, PDF, XML, etc.
6) Supporting multiple database platforms.
The offline behavior characteristic data processing module written by the BSQL language can simply realize the aggregation calculation of the pulled offline behavior characteristic data to obtain the first training sample data.
As an example, when performing aggregation calculation on the pulled offline behavior feature data, the aggregation calculation of the line behavior feature data may be implemented by using a GroupBy statement. Specifically, offline behavior feature data of each user is calculated through GroupBy statement aggregation, and the offline behavior feature data of each user is used as a piece of first training sample data.
By way of example, the offline behavior feature dataset includes click behavior feature data for a user offline and presentation behavior feature data for a user exposed media file.
As shown in fig. 5, in order to improve accuracy of the model, the obtaining, by performing aggregation calculation on the pulled offline behavior feature data through an offline behavior feature data processing module written based on BSQL language, the first training sample data includes: step S50, performing aggregation calculation on the pulled offline behavior characteristic data through an offline behavior characteristic data processing module written based on BSQL language to obtain offline real-time characteristic data of each user in the current time section; and step S51, obtaining portrait feature data of each user, and performing feature fusion processing on the offline real-time feature data and the portrait feature data corresponding to each user to obtain the first training sample data.
The fusion process is used to fuse a plurality of pieces of data into one piece of data, for example, fuse data a and data B into data C.
For ease of understanding, a preferred example is provided below to illustrate the process of obtaining a first training sample data. Specifically, first, the offline behavior feature data extracted by the offline behavior feature data processing module written based on BSQL language is aggregated and calculated to obtain the offline real-time feature data of the user a in the current time segment (taking the past 0-4 points as an example), which is composed of the following features: video characteristics 1, video characteristics 2, user basic characteristics 1 and user basic characteristics 2; then, image feature data of the user A are obtained, for example, the obtained image feature data are a feature A and a feature B; then, the features are fused to obtain first training sample data, wherein the obtained first training sample data comprises the following features: video feature 1, video feature 2, user basic feature 1, user basic feature 2, feature A and feature B.
The user interest tags indicated by the portrait feature data of the user may include at least one user interest tag of an educational resource preference tag, a ghost video preference tag, an animation video preference tag, a quadratic video preference tag, a film and television video preference tag, a synthesis video preference tag, and the like of the user.
And step S33, inputting all the generated first training sample data into the model to be trained for training to obtain an initial estimation model.
In particular, the model to be trained may be an LR model, or other model. The initial estimation model can be an initial click estimation model used for estimating the click probability of the user on various multimedia files, and can also be an initial conversion estimation model used for estimating the conversion probability of the user on various multimedia files, and the like.
It should be noted that the initial estimation model obtained by training is not a model finally used for estimating the click or conversion probability of the user on various multimedia files, but is an intermediate model.
And step S34, acquiring online behavior feature data, and generating second training sample data according to the online behavior feature data.
Specifically, the online behavior feature data is behavior feature data generated within a preset time period from the current time to a time after the current time, for example, if the current time is 5 points, the online behavior feature data may be behavior feature data generated by 5 points to 5 points and 5 points.
As shown in fig. 6, in order to obtain online behavior feature data, the step of obtaining online behavior feature data and generating second training sample data according to the online behavior feature data may include: step S60, acquiring online behavior characteristic data of a plurality of users in a sliding time window mode; and step S61, counting the online behavior data of each user in the time window, and taking the online behavior data of each user as a second training sample data.
As an example, if the sliding time window takes 1 hour as an example, the online behavior feature data of multiple users in one hour may be obtained, and then statistics is performed on the online behavior feature data of each user in the one hour to obtain second training sample data.
As an example, the online behavior feature data may include click behavior feature data of the user online and presentation behavior feature data for the user exposed media file.
As an example, to reduce the training threshold of the model, the counting online behavior data of each user in the time window, and taking the online behavior data of each user as a second training sample data includes: and splicing the click behavior characteristic data and the display behavior characteristic data of each user in the time window through an online behavior characteristic data processing module compiled based on the BSQL language to obtain spliced characteristic data, and taking the spliced characteristic data as the second training sample data.
Specifically, the on-line behavior feature data processing module written in the BSQL language can simply implement the splicing processing of the click behavior feature data and the display behavior feature data to obtain the first training sample data.
As an example, it is assumed that the click behavior feature data acquired at the 40 th minute consists of the following feature data: feature 1, feature 2, feature 3, feature 4; the exhibited behavior feature data acquired at the 30 th minute consisted of the following feature data: feature 1, feature 3, feature 4, and feature 5, and after the splicing processing, the obtained feature data is composed of the following feature data: feature 1, feature 2, feature 3, feature 4, feature 5. As shown in fig. 7, in order to improve the accuracy of the model, the obtaining of the first training sample data by performing aggregation calculation on the pulled offline behavior feature data through the offline behavior feature data processing module written based on the BSQL language includes: step S70, splicing the click behavior characteristic data and the display behavior characteristic data of each user in the time window through an online behavior characteristic data processing module written based on the BSQL language to obtain spliced characteristic data; and step S71, obtaining portrait feature data of each user, and performing feature fusion processing on the spliced feature data and the portrait feature data corresponding to each user to obtain the second training sample data.
The fusion process is used to fuse a plurality of pieces of data into one piece of data, for example, fuse data a and data B into data C.
For ease of understanding, a preferred example is provided below to illustrate the process of obtaining a second training sample data. Specifically, firstly, the click behavior feature data and the display behavior feature data of the user A in the time window (0 (the time window takes 1 hour as an example) are spliced through an online behavior feature data processing module written based on the BSQL language to obtain spliced feature data comprising the following features, namely feature 1, feature 2, feature 3, feature 4 and feature 5, then image feature data of the user A is obtained, for example, the obtained image feature data are feature A and feature B, and then the features are fused to obtain second training sample data, namely the obtained second training sample data comprises the following features, namely feature 1, feature 2, feature 3, feature 4, feature 5, feature A and feature B.
The user interest tags indicated by the portrait feature data of the user may include at least one user interest tag of an educational resource preference tag, a ghost video preference tag, an animation video preference tag, a quadratic video preference tag, a film and television video preference tag, a synthesis video preference tag, and the like of the user.
And step S35, inputting the second training sample data into the initial estimation model for training to obtain a target estimation model.
Specifically, second training sample data is input into the initial prediction model for training, so that fine adjustment of the initial prediction model is achieved, and a final target prediction model for predicting the probability of clicking the multimedia file by the user is obtained.
In the embodiment of the application, an offline behavior feature data set is obtained and stored to a target storage unit in order, wherein the offline behavior feature data set is stored in order according to a preset time section; receiving a stream batch data pulling request of an offline behavior characteristic data set stored in the target storage unit; sequentially pulling off-line behavior characteristic data of a corresponding time section from the off-line behavior characteristic data set based on the stream batch data pulling request, and generating first training sample data according to the pulled off-line behavior characteristic data; inputting all generated first training sample data into a model to be trained for training to obtain an initial estimation model; acquiring online behavior characteristic data, and generating second training sample data according to the online behavior characteristic data; and inputting the second training sample data into the initial prediction model for training to obtain a target prediction model. According to the embodiment, offline data are stored in the target storage unit in order, so that a batch of data can be pulled to be processed based on a batch data pulling request, first training sample data is generated quickly to perform model training, and the training efficiency of the click rate pre-estimation model can be improved. Meanwhile, an initial prediction model is obtained through training, the initial prediction model is trained again through second training sample data generated by the online behavior characteristic data, fine tuning of the model is achieved, and the prediction accuracy of the target prediction model obtained through final training can be improved.
Example two
FIG. 8 is a block diagram of a model training apparatus according to the second embodiment of the present application, which may be partitioned into one or more program modules, stored in a storage medium, and executed by one or more processors to implement the second embodiment of the present application. The program modules referred to in the embodiments of the present application refer to a series of computer program instruction segments that can perform specific functions, and the following description will specifically describe the functions of the program modules in the embodiments. As shown in fig. 8, the model training apparatus 800 may include the following components:
an obtaining module 801, configured to obtain an offline behavior feature data set, and store the offline behavior feature data set to a target storage unit in order, where the offline behavior feature data set is stored in order according to a preset time segment.
A receiving module 802, configured to receive a stream batch data pull request for the offline behavior feature data set stored in the target storage unit.
A pulling module 803, configured to sequentially pull the offline behavior feature data of the corresponding time segment from the offline behavior feature data set based on the batch data pulling request, and generate first training sample data according to the pulled offline behavior feature data.
The first training module 804 is configured to input all generated first training sample data into a model to be trained for training, so as to obtain an initial prediction model.
The generating module 805 is configured to obtain online behavior feature data, and generate second training sample data according to the online behavior feature data.
A second training module 806, configured to input the second training sample data into the initial prediction model for training to obtain a target prediction model.
In an exemplary embodiment, the target storage unit is a distributed file system HDFS, and the obtaining module 801 is further configured to identify a generation time of each piece of offline behavior feature data in the offline behavior feature data set; and storing the offline behavior characteristic data belonging to the same time section into the same data block of the HDFS according to the generation time, and sequentially storing the offline behavior characteristic data of different time sections into different data blocks of the HDFS.
In an exemplary embodiment, the pulling module 803 is further configured to perform aggregation calculation on the pulled offline behavior feature data through an offline behavior feature data processing module written based on BSQL language to obtain the first training sample data.
In an exemplary embodiment, the generating module 805 is further configured to obtain online behavior feature data of a plurality of users in a sliding time window manner; and counting the online behavior data of each user in the time window, and taking the online behavior data of each user as second training sample data.
In an exemplary embodiment, the online behavior feature data includes online click behavior feature data of the user and display behavior feature data for the user exposed media file, and the generating module 805 is further configured to perform splicing processing on the click behavior feature data and the display behavior feature data of each user in the time window through an online behavior feature data processing module written based on BSQL language to obtain spliced feature data, and use the spliced feature data as the second training sample data.
In an exemplary embodiment, the generating module 805 is further configured to perform splicing processing on the click behavior feature data and the display behavior feature data of each user in the time window through an online behavior feature data processing module written based on BSQL language, so as to obtain spliced feature data; and acquiring portrait feature data of each user, and performing feature fusion processing on the spliced feature data and the portrait feature data corresponding to each user to obtain the second training sample data.
In an exemplary embodiment, the pulling module 803 is further configured to perform aggregation calculation on the pulled offline behavior feature data through an offline behavior feature data processing module written based on BSQL language, so as to obtain offline real-time feature data of each user in the current time segment; and acquiring portrait feature data of each user, and performing feature fusion processing on the offline real-time feature data and the portrait feature data corresponding to each user to obtain the first training sample data.
EXAMPLE III
Fig. 9 schematically shows a hardware architecture diagram of a computer device suitable for implementing the model training method according to the third embodiment of the present application. The computer device 20 is a device capable of automatically performing numerical calculation and/or information processing according to instructions set or stored in advance. For example, the server may be a workstation, a rack server, a blade server, a tower server, or a rack server (including an independent server or a server cluster composed of a plurality of servers). As shown in fig. 9, the computer device 20 includes at least, but is not limited to: the memory 21, processor 22, and network interface 23 may be communicatively linked to each other by a system bus. Wherein:
the memory 21 includes at least one type of computer-readable storage medium including flash memory, hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, the storage 21 may be an internal storage module of the computer device 20, such as a hard disk or a memory of the computer device 20. In other embodiments, the memory 21 may also be an external storage device of the computer device 20, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the computer device 20. Of course, the memory 21 may also include both internal and external memory modules of the computer device 20. In this embodiment, the memory 21 is generally used for storing an operating system and various application software installed in the computer device 20, such as program codes of the model training method. Further, the memory 21 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 22 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 22 is generally configured to control the overall operation of the computer device 20, such as performing control and processing related to data interaction or communication with the computer device 20. In this embodiment, the processor 22 is configured to execute the program code stored in the memory 21 or process data.
The network interface 23 may comprise a wireless network interface or a wired network interface, and the network interface 23 is typically used to establish a communication connection between the computer device 20 and other computer devices. For example, the network interface 23 is used to connect the computer device 20 with an external terminal through a network, establish a data transmission channel and a communication connection between the computer device 20 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), or Wi-Fi.
It is noted that fig. 9 only shows a computer device with components 21-23, but it is to be understood that not all of the shown components are required to be implemented, and that more or less components may be implemented instead.
In this embodiment, the model training method stored in the memory 21 may be further divided into one or more program modules and executed by one or more processors (in this embodiment, the processor 22) to complete the present application.
Example four
The present embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, realizes the steps of the model training method in the embodiments.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device. Of course, the computer-readable storage medium may also include both internal and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is generally used for storing an operating system and various types of application software installed in the computer device, for example, the program codes of the model training method in the embodiment, and the like. Further, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different from that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all the equivalent structures or equivalent processes that can be directly or indirectly applied to other related technical fields by using the contents of the specification and the drawings of the present application are also included in the scope of the present application.
Claims (10)
1. A method of model training, the method comprising:
acquiring an offline behavior feature data set, and orderly storing the offline behavior feature data set to a target storage unit, wherein the offline behavior feature data set is orderly stored according to a preset time section, and comprises click data and display data of a plurality of users in a historical preset time period;
receiving a stream batch data pulling request of an offline behavior characteristic data set stored in the target storage unit;
sequentially pulling off-line behavior characteristic data of a corresponding time section from the off-line behavior characteristic data set according to a time sequence based on the streaming batch data pulling request, and generating first training sample data according to the pulled off-line behavior characteristic data;
inputting all generated first training sample data into a model to be trained for training to obtain an initial estimation model;
acquiring online behavior characteristic data, and generating second training sample data according to the online behavior characteristic data;
and inputting the second training sample data into the initial prediction model for training to obtain a target prediction model.
2. The model training method of claim 1, wherein the target storage unit is a distributed file system (HDFS), and the obtaining the offline behavior feature data sets and the orderly storing the offline behavior feature data sets to the target storage unit comprise:
identifying a generation time of each piece of offline behavior feature data in the set of offline behavior feature data;
and storing the offline behavior characteristic data belonging to the same time section into the same data block of the HDFS according to the generation time, and sequentially storing the offline behavior characteristic data of different time sections into different data blocks of the HDFS.
3. The model training method according to claim 1, wherein the generating first training sample data according to the pulled offline behavior feature data comprises:
and performing aggregation calculation on the pulled offline behavior characteristic data through an offline behavior characteristic data processing module written based on the BSQL language to obtain the first training sample data.
4. The model training method according to any one of claims 1 to 3, wherein the obtaining online behavior feature data and generating second training sample data according to the online behavior feature data comprises:
acquiring online behavior characteristic data of a plurality of users in a sliding time window mode;
and counting the online behavior data of each user in the time window, and taking the online behavior data of each user as second training sample data.
5. The model training method according to claim 4, wherein the online behavior feature data includes click behavior feature data of users online and presentation behavior feature data for exposing media files of users, the statistics of online behavior data of each user within the time window, and the taking of online behavior data of each user as a second training sample data includes:
and splicing the click behavior characteristic data and the display behavior characteristic data of each user in the time window through an online behavior characteristic data processing module written based on the BSQL language to obtain spliced characteristic data, and taking the spliced characteristic data as the second training sample data.
6. The model training method according to claim 5, wherein the splicing processing is performed on the click behavior feature data and the display behavior feature data of each user in the time window by using an online behavior feature data processing module written based on the BSQL language to obtain spliced feature data, and the using the spliced feature data as the second training sample data includes:
splicing the click behavior characteristic data and the display behavior characteristic data of each user in the time window through an online behavior characteristic data processing module compiled based on the BSQL language to obtain spliced characteristic data;
and acquiring portrait feature data of each user, and performing feature fusion processing on the spliced feature data and the portrait feature data corresponding to each user to obtain the second training sample data.
7. The model training method of claim 3, wherein the offline behavior feature data set comprises offline click behavior feature data of a user and display behavior feature data for a user exposed media file, and the obtaining the first training sample data by performing aggregation calculation on the pulled offline behavior feature data through an offline behavior feature data processing module written based on the BSQL language comprises:
performing aggregation calculation on the pulled offline behavior characteristic data through an offline behavior characteristic data processing module written based on a BSQL language to obtain offline real-time characteristic data of each user in a current time section;
and acquiring portrait feature data of each user, and performing feature fusion processing on the offline real-time feature data and the portrait feature data corresponding to each user to obtain the first training sample data.
8. A model training apparatus, comprising:
the acquisition module is used for acquiring an offline behavior characteristic data set and sequentially storing the offline behavior characteristic data set to a target storage unit, wherein the offline behavior characteristic data set is stored in order according to a preset time section and comprises click data and display data of a plurality of users in a historical preset time period;
the receiving module is used for receiving a stream batch data pulling request of the offline behavior characteristic data set stored in the target storage unit;
the pull module is used for sequentially pulling the off-line behavior characteristic data of the corresponding time section from the off-line behavior characteristic data set according to the time sequence based on the stream batch data pull request and generating first training sample data according to the pulled off-line behavior characteristic data;
the first training module is used for inputting all the generated first training sample data into a model to be trained for training so as to obtain an initial pre-estimated model;
the generating module is used for acquiring online behavior characteristic data and generating second training sample data according to the online behavior characteristic data;
and the second training module is used for inputting the second training sample data into the initial prediction model for training so as to obtain a target prediction model.
9. A computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor is adapted to carry out the steps of the model training method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, is adapted to carry out the steps of the model training method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011459323.9A CN112612768B (en) | 2020-12-11 | 2020-12-11 | Model training method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011459323.9A CN112612768B (en) | 2020-12-11 | 2020-12-11 | Model training method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112612768A CN112612768A (en) | 2021-04-06 |
| CN112612768B true CN112612768B (en) | 2022-09-16 |
Family
ID=75233597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011459323.9A Active CN112612768B (en) | 2020-12-11 | 2020-12-11 | Model training method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112612768B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113220657B (en) * | 2021-05-14 | 2022-12-06 | 上海哔哩哔哩科技有限公司 | Data processing method and device and computer equipment |
| CN113507419B (en) * | 2021-07-07 | 2022-11-01 | 工银科技有限公司 | Training method of traffic distribution model, traffic distribution method and device |
| CN113836411A (en) * | 2021-09-22 | 2021-12-24 | 上海哔哩哔哩科技有限公司 | Data processing method and device and computer equipment |
| CN114661571A (en) * | 2022-03-30 | 2022-06-24 | 北京百度网讯科技有限公司 | Model evaluation method, model evaluation device, electronic equipment and storage medium |
| CN117252112B (en) * | 2023-11-16 | 2024-01-30 | 江苏林洋亿纬储能科技有限公司 | Method for training driving data model and method for estimating remaining life of battery |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110008399A (en) * | 2019-01-30 | 2019-07-12 | 阿里巴巴集团控股有限公司 | A kind of training method and device, a kind of recommended method and device of recommended models |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109816412B (en) * | 2017-11-21 | 2022-11-29 | 腾讯科技（深圳）有限公司 | Training model generation method, device, equipment and computer storage medium |
| CN109299327A (en) * | 2018-11-16 | 2019-02-01 | 广州市百果园信息技术有限公司 | Video recommendation method, device, equipment and storage medium |
| CN110033342A (en) * | 2019-01-30 | 2019-07-19 | 阿里巴巴集团控股有限公司 | A kind of training method and device, a kind of recommended method and device of recommended models |
| CN110532883B (en) * | 2019-07-30 | 2023-09-01 | 平安科技（深圳）有限公司 | Improvement of on-line tracking algorithm by off-line tracking algorithm |
| CN111163072B (en) * | 2019-12-24 | 2022-06-10 | 五八有限公司 | Method and device for determining characteristic value in machine learning model and electronic equipment |
| CN111026971B (en) * | 2019-12-25 | 2023-05-02 | 腾讯科技（深圳）有限公司 | Content pushing method and device and computer storage medium |
| CN111210024B (en) * | 2020-01-14 | 2023-09-15 | 深圳供电局有限公司 | Model training method, device, computer equipment and storage medium |
-
2020
- 2020-12-11 CN CN202011459323.9A patent/CN112612768B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110008399A (en) * | 2019-01-30 | 2019-07-12 | 阿里巴巴集团控股有限公司 | A kind of training method and device, a kind of recommended method and device of recommended models |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112612768A (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112612768B (en) | Model training method and device | |
| US11288142B2 (en) | Recovery strategy for a stream processing system | |
| CN107577805B (en) | Business service system for log big data analysis | |
| US11755452B2 (en) | Log data collection method based on log data generated by container in application container environment, log data collection device, storage medium, and log data collection system | |
| US10395181B2 (en) | Machine learning system flow processing | |
| US10643144B2 (en) | Machine learning system flow authoring tool | |
| US9367601B2 (en) | Cost-based optimization of configuration parameters and cluster sizing for hadoop | |
| CN109344170B (en) | Stream data processing method, system, electronic device and readable storage medium | |
| CN111966943A (en) | Streaming data distribution method and system | |
| CN112559475B (en) | Data real-time capturing and transmitting method and system | |
| US20140379632A1 (en) | Smarter big data processing using collaborative map reduce frameworks | |
| CN112732949B (en) | Service data labeling method and device, computer equipment and storage medium | |
| CN112019605A (en) | Data distribution method and system of data stream | |
| CN109190025A (en) | information monitoring method, device, system and computer readable storage medium | |
| US10127617B2 (en) | System for analyzing social media data and method of analyzing social media data using the same | |
| CN113609374A (en) | Data processing method, device and equipment based on content push and storage medium | |
| CN107391528B (en) | Front-end component dependent information searching method and equipment | |
| US20170193409A1 (en) | Business resource modeling | |
| CN106897313B (en) | Mass user service preference evaluation method and device | |
| US10331693B1 (en) | Filters and event schema for categorizing and processing streaming event data | |
| CN114595473A (en) | Access data processing method and device, electronic equipment and computer readable medium | |
| CN112560938A (en) | Model training method and device and computer equipment | |
| US20230289241A1 (en) | Automatic data pipeline generation | |
| CN116955326A (en) | Method and device for controlling life cycle of data product and electronic equipment | |
| US20240070161A1 (en) | Global status monitoring for open data platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |