CN113128530B - Data classification method and device - Google Patents
Data classification method and device Download PDFInfo
- Publication number
- CN113128530B CN113128530B CN201911397622.1A CN201911397622A CN113128530B CN 113128530 B CN113128530 B CN 113128530B CN 201911397622 A CN201911397622 A CN 201911397622A CN 113128530 B CN113128530 B CN 113128530B
- Authority
- CN
- China
- Prior art keywords
- sample
- data
- category
- preset
- parameters corresponding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000013145 classification model Methods 0.000 claims abstract description 65
- 238000012549 training Methods 0.000 claims abstract description 29
- 238000001514 detection method Methods 0.000 claims description 21
- 238000004891 communication Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 description 15
- 238000012360 testing method Methods 0.000 description 14
- 239000013598 vector Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a data classification method and device, wherein the method comprises the following steps: acquiring a plurality of sample data, wherein the data types of the sample data are sample types; training a preset network by using a plurality of sample data to obtain a preset classification model; extracting first final classifier parameters corresponding to each sample class in a preset classification model; determining a final classifier parameter corresponding to each non-sample category according to a preset association relation between data categories and a first final classifier parameter corresponding to each sample category, wherein the data categories comprise sample categories and non-sample categories; and classifying the data to be classified of each non-sample category by using a preset classification model with the final classifier parameters corresponding to the determined non-sample category. By applying the technical scheme provided by the embodiment of the invention, the data of the data category without sample data is classified, and the application range is enlarged.
Description
Technical Field
The present invention relates to the field of machine learning technologies, and in particular, to a data classification method and apparatus.
Background
In some scenes, data needs to be classified, for example, after a road monitoring video is collected, video images are classified according to targets in the video, one type of video image is an image aiming at pedestrians, and the other type of video image is an image aiming at vehicles, so that different processing methods can be adopted for images of different types, and the road monitoring can be realized. Generally, the data can be classified by using the classification model, so that manual operation is reduced, and efficiency is improved.
In the related art, the data classification method is generally: firstly, a sample training set is obtained, wherein the sample training set comprises sample data of various data types, then the sample training set is trained by utilizing a preset model to obtain a classification model aiming at the various data types, and further data to be classified can be input into the classification model obtained through training to realize classification of the data to be classified.
However, in the above method, the classification model obtained by training can only identify sample data of several categories included in the sample training set, for example, if the data categories in the training set are only vehicle and person, the classification model obtained by training can only identify both vehicle and person data categories. If the class of the data to be classified is the data class which does not exist in the sample training set, the method cannot effectively classify the data to be classified, so that the application range of the data classification method is limited, and the user requirement is difficult to meet.
Disclosure of Invention
The embodiment of the invention aims to provide a data classification method and device, so as to realize classification of data types without sample data and expand application range. The specific technical scheme is as follows:
in order to achieve the above object, an embodiment of the present invention provides a data classification method, including:
acquiring a plurality of sample data, wherein the data types of the sample data are sample types;
training a preset network by utilizing the plurality of sample data to obtain a preset classification model;
extracting first final classifier parameters corresponding to each sample class in the preset classification model;
determining a final classifier parameter corresponding to each non-sample category according to a preset association relation between data categories and a first final classifier parameter corresponding to each sample category, wherein the data categories comprise sample categories and non-sample categories;
and classifying the data to be classified of each non-sample category by using a preset classification model with the final classifier parameters corresponding to the determined non-sample category.
In an optional embodiment, the determining the final classifier parameter corresponding to each non-sample category according to the association relationship between preset data categories and the first final classifier parameter corresponding to each sample category includes:
Determining initial classifier parameters corresponding to each non-sample category and initial classifier parameters corresponding to each sample category;
inputting the association relation among the preset data categories, the initial classifier parameters corresponding to each sample-free category and the initial classifier parameters corresponding to each sample-free category into a preset graph network model to obtain the intermediate classifier parameters corresponding to each sample-free category and the intermediate classifier parameters corresponding to each sample-free category;
determining a loss value according to the first final classifier parameter corresponding to each sample class and the intermediate classifier parameter corresponding to each sample class;
and if the loss value is smaller than a preset loss threshold value, determining the middle classifier parameter corresponding to each current non-sample category as a final classifier parameter.
In an alternative embodiment, the method further comprises:
and if the loss value is greater than or equal to the preset loss threshold value, adjusting the initial classifier parameters corresponding to each non-sample type and the initial classifier parameters corresponding to each sample type, and returning to execute the step of inputting the association relation among preset data types, the initial classifier parameters corresponding to each sample type and the initial classifier parameters corresponding to each non-sample type into a preset graph network model.
In an alternative embodiment, the method further comprises:
if the loss value is smaller than a preset loss threshold value, determining that the middle classifier parameter corresponding to each current sample class is a second final classifier parameter;
and classifying the data to be classified of each non-sample category by using a preset classification model with the determined second final classifier parameters corresponding to the sample category.
In an alternative embodiment, before determining the final classifier parameters corresponding to each non-sample class, the method further comprises:
determining the similarity between every two data categories;
and establishing an association relationship between two data categories with similarity greater than a preset similarity threshold.
In an alternative embodiment, the sample data and the data to be classified are image, document, table or speech data.
In an optional embodiment, when the sample data and the data to be classified are images, before classifying the data to be classified of each non-sample category by using the preset classification model with the final classifier parameters corresponding to the determined non-sample category, the method further includes:
performing target detection on the data to be classified, and determining an image area including the detection target in the data to be classified;
And classifying the image area by using a preset classification model with the final classifier parameters corresponding to the determined sample-free class to obtain the sample-free class of the detection target.
To achieve the above object, an embodiment of the present invention further provides a data classification device, including:
the acquisition module is used for acquiring a plurality of sample data, wherein the data category of the sample data is a sample category;
the training module is used for training a preset network by utilizing the plurality of sample data to obtain a preset classification model;
the extraction module is used for extracting first final classifier parameters corresponding to each sample category in the preset classification model;
the determining module is used for determining the final classifier parameters corresponding to each non-sample category according to the association relation among preset data categories and the first final classifier parameters corresponding to each sample category, wherein the data categories comprise sample categories and non-sample categories;
and the classification module is used for classifying the data to be classified of each non-sample category by using a preset classification model with the determined final classifier parameters corresponding to the non-sample category.
In an alternative embodiment, the determining module includes:
the first determining submodule is used for determining initial classifier parameters corresponding to each non-sample category and initial classifier parameters corresponding to each sample category;
the learning sub-module is used for inputting the association relation among the preset data categories, the initial classifier parameters corresponding to each sample category and the initial classifier parameters corresponding to each no-sample category into the preset graph network model to obtain the intermediate classifier parameters corresponding to each sample category and the intermediate classifier parameters corresponding to each no-sample category;
the second determining submodule is used for determining a loss value according to the first final classifier parameter corresponding to each sample category and the middle classifier parameter corresponding to each sample category;
and the third determining submodule is used for determining the middle classifier parameter corresponding to each current non-sample category as a final classifier parameter if the loss value is smaller than a preset loss threshold value.
In an optional embodiment, the third determining submodule is further configured to adjust an initial classifier parameter corresponding to each no-sample class and an initial classifier parameter corresponding to each sample class if the loss value is greater than or equal to the preset loss threshold, and return to executing the step of inputting the association relationship between preset data classes, the initial classifier parameter corresponding to each sample class and the initial classifier parameter corresponding to each no-sample class into a preset graph network model.
In an optional embodiment, the third determining submodule is further configured to determine an intermediate classifier parameter corresponding to each current sample class as a second final classifier parameter if the loss value is less than a preset loss threshold;
the classification module is further configured to classify data to be classified of each non-sample category by using a preset classification model with the determined second final classifier parameter corresponding to the sample category.
In an alternative embodiment, the apparatus further comprises:
the establishing module is used for determining the similarity between every two data categories before determining the final classifier parameters corresponding to each non-sample category, and establishing the association relationship between the two data categories with the similarity greater than a preset similarity threshold.
In an alternative embodiment, the sample data and the data to be classified are image, document, table or speech data.
In an alternative embodiment, the classification module is specifically configured to:
when the sample data and the data to be classified are images, performing target detection on the data to be classified, and determining an image area including the detection target in the data to be classified;
And classifying the image area by using a preset classification model with the final classifier parameters corresponding to the determined sample-free class to obtain the sample-free class of the detection target.
In order to achieve the above object, an embodiment of the present invention further provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any one of the data classification methods when executing the programs stored in the memory.
To achieve the above object, an embodiment of the present invention further provides a computer readable storage medium, in which a computer program is stored, which when executed by a processor, implements any one of the above-described data classification methods.
The embodiment of the invention also provides a computer program which realizes any one of the data classification methods when being executed by a processor.
In the technical scheme provided by the embodiment of the invention, the sample-free category is a data category without sample data. The characteristics of the data of the two data categories with the association relation have certain similarity, based on the first final classifier parameters corresponding to the sample category and the association relation among the preset data categories, the final classifier parameters corresponding to the sample-free category can be determined under the condition that sample data of the sample-free category is not available, and further the data to be classified of the sample-free category can be classified by utilizing the preset classification model with the final classifier parameters corresponding to the sample-free category, namely, the classification of the data category without the sample data is realized, and the application range of the data classification is enlarged.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a data classification method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a category association relationship according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of another data classification method according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of a data classification method according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a data classification device according to an embodiment of the present invention;
fig. 6 is a schematic diagram of another structure of a data classification device according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the related art, the classification model obtained by training can only identify sample data of several data categories included in the sample training set, for example, if the data categories in the training set are only vehicles and people, the classification model obtained by training can only identify two data categories of vehicles and people. If the class of the data to be classified is the data class which does not exist in the sample training set, the method cannot effectively classify the data to be classified, so that the application range of the data classification method is limited, and the user requirement is difficult to meet.
In order to solve the above technical problems, the present invention provides a data classification method, which can be applied to various electronic devices, such as a computer, a server, a network camera, etc., and the embodiment of the present invention is not limited thereto.
The following generally describes a data classification method provided by an embodiment of the present invention, where the data classification method includes:
acquiring a plurality of sample data, wherein the types of the sample data are sample types;
training a preset network by using a plurality of sample data to obtain a preset classification model;
extracting first final classifier parameters corresponding to each sample class in a preset classification model;
Determining a final classifier parameter corresponding to each non-sample category according to a preset association relation between data categories and a first final classifier parameter corresponding to each sample category, wherein the data categories comprise sample categories and non-sample categories;
and classifying the data to be classified of each non-sample category by using a preset classification model with the final classifier parameters corresponding to the determined non-sample category.
In the technical scheme provided by the embodiment of the invention, the sample-free category is a data category without sample data. The characteristics of the data of the two data categories with the association relation have certain similarity, based on the first final classifier parameters corresponding to the sample category and the association relation among the preset data categories, the final classifier parameters corresponding to the sample-free category can be determined under the condition that sample data of the sample-free category is not available, and further the data to be classified of the sample-free category can be classified by utilizing the preset classification model with the final classifier parameters corresponding to the sample-free category, namely, the classification of the data category without the sample data is realized, and the application range of the data classification is enlarged.
The data classification method provided by the embodiment of the invention is described in detail below through a specific embodiment.
As shown in fig. 1, fig. 1 is a flow chart of a data classification method according to an embodiment of the present invention, which includes the following steps.
S101: and acquiring a plurality of sample data, wherein the data type of the sample data is a sample type.
For example, a plurality of sample data forms a sample training set. The sample data may be picture data, document data, or table or voice data. The data types of the plurality of sample data may be the same or different. The data category of the sample data (i.e. the sample category) can be determined according to the actual application scenario. For example, in a road monitoring scenario where the sample data is a monitoring image, then the sample category may include vehicles and pedestrians; in a work office scenario, where the sample data is document data, then there are sample categories that may include English documents, chinese documents, japanese documents, and so forth.
S102: training a preset network by using a plurality of sample data to obtain a preset classification model.
The preset network may be implemented by a network with a supervised recognition task, for example ResNet, VGGNet, which is not limited in the embodiment of the present invention.
In the embodiment of the invention, the preset network is trained by utilizing a plurality of sample data until the preset network converges, so as to obtain the preset classification model.
For example, the process of presetting the classification model may be: extracting a plurality of characteristic values of each sample data, and forming a characteristic vector of the sample data by the plurality of characteristic values of the sample data aiming at each sample data; respectively inputting the feature vector of each sample data into a preset network to obtain the prediction category of each sample data; determining a loss value based on the predicted class of each sample data and the known class of data for each sample data; if the loss value is smaller than the preset threshold value, determining that the preset network converges, and taking the current preset network as a preset classification model; if the loss value is greater than or equal to the preset threshold, determining that the preset network is not converged, adjusting parameters of the preset network, and returning to the step of respectively inputting the feature vector of each sample data into the preset network to obtain the prediction type of each sample data until the determined loss value is smaller than the preset threshold.
In the embodiment of the present invention, other ways may be used to train the preset classification model, which is not particularly limited.
S103: and extracting a first final classifier parameter corresponding to each sample class in the preset classification model.
In the embodiment of the invention, a preset classification model is obtained through training, and first final classifier parameters corresponding to each sample category in the preset classification model are extracted.
In the embodiment of the invention, the final classifier parameters are in one-to-one correspondence with the sample categories. For example, if there are multiple sample classes, multiple final classifier parameters may be extracted. In addition, the final classifier parameters include a plurality of feature values. Based on which a first final classifier parameter matrix corresponding to the sample class may be determined.
For example, the first final classifier parameter matrix corresponding to the sample class is represented as P' train The number of sample classes is N train Each of the first final classifier parameters corresponding to the sample classThe number of characteristic values included is C, then P' train Can be expressed as N train *C。
S104: according to the association relation among preset data categories and the first final classifier parameters corresponding to each sample-free category, determining the final classifier parameters corresponding to each sample-free category, wherein the data categories comprise sample-free categories and sample-free categories.
In the embodiment of the invention, the association relation among the data categories is preset. If there is similarity between the data of the two data categories, there is an association relationship between the two data categories. For example, if the vehicle X1 and the vehicle X2 are vehicles of the same brand, and there is a certain similarity between the two vehicles, for example, the vehicle lamp shape and the window shape are the same, it is possible to set that there is a correlation between the two vehicles.
For data categories with associations, their classifier parameters are associated and influence each other. Therefore, according to the association relation among the preset data categories and the first final classifier parameters corresponding to each sample-free category, the final classifier parameters corresponding to each sample-free category are determined.
In the embodiment of the invention, the final classifier parameters are in one-to-one correspondence with the sample-free classes. If there are multiple sample classes, for example, multiple final classifier parameters may be determined. In addition, the final classifier parameters include a plurality of feature values. Based on which a final classifier parameter matrix corresponding to the no-sample class may be determined.
For example, the final classifier parameter matrix corresponding to no sample class is denoted as P test The number of sample classes is N test The number of eigenvalues included in the final classifier parameters for each no-sample class is C, then P test Can be expressed as N test *C。
In the embodiment of the invention, the number of the final classifier parameters corresponding to the sample-free class is the same as the number of the feature values included in the first final classifier parameters corresponding to the sample-free class.
In an alternative embodiment, before determining the final classifier parameters corresponding to each non-sample category, determining the similarity between each two data categories, and establishing the association relationship between the two data categories with the similarity greater than the preset similarity threshold.
For example, the current data category has a category 11-15, a similarity between the category 11 and the category 12 is S1, a similarity between the category 11 and the category 13 is S2, a similarity between the category 11 and the category 14 is S3, a similarity between the category 11 and the category 15 is S4, a similarity between the category 12 and the category 13 is S5, a similarity between the category 12 and the category 14 is S6, a similarity between the category 12 and the category 15 is S7, a similarity between the category 13 and the category 14 is S8, a similarity between the category 13 and the category 14 is S9, and a similarity between the category 14 and the category 15 is S10. Wherein, the similarity threshold delta is preset s . If S1, S3, S4, S5, S8, S9 is greater than delta s The association between categories 11-15 is established as shown in fig. 2. Each circle in fig. 2 represents a category, and the numbers in each circle represent the corresponding category.
In the embodiment of the invention, the association relationship between the data categories can also be preset by a user.
S105: and classifying the data to be classified of each non-sample category by using a preset classification model with the final classifier parameters corresponding to the determined non-sample category.
In the embodiment of the invention, the data to be classified can be image, document, table or voice data. The type of the data to be classified is the same as the type of the sample data, if the type of the sample data is an image, the type of the data to be classified is also an image, and if the type of the sample data is a document, the type of the data to be classified is also a document.
In an optional embodiment, when the sample data and the data to be classified are images, before classifying the data to be classified of each non-sample type by using a preset classification model with a final classifier parameter corresponding to the determined non-sample type, target detection may be performed on the data to be classified, and an image area including a detection target in the data to be classified is determined; and classifying the image area by using a preset classification model with the final classifier parameters corresponding to the determined sample-free class to obtain the sample-free class of the detection target. In the embodiment of the invention, the sample-free class of the detection target is determined by classifying the image area comprising the detection target, so that the interference of the background area in the image on the determination of the class of the detection target is reduced, and the accuracy of data classification is improved.
After determining the final classifier parameters corresponding to the no-sample class, classifying the data to be classified of the no-sample class by using a preset classification model with the final classifier parameters corresponding to the no-sample class. At this time, the data which can be classified is not limited to the data to be classified with the sample class, and the application range of data classification is enlarged.
Fig. 3 is a schematic flow chart of a data classification method according to an embodiment of the present invention, as shown in fig. 3. In this method, step S104 may be refined to steps S1041-S1044, specifically as follows.
S101: and acquiring a plurality of sample data, wherein the data type of the sample data is a sample type.
S102: training a preset network by using a plurality of sample data to obtain a preset classification model.
S103: and extracting a first final classifier parameter corresponding to each sample class in the preset classification model.
S1041: initial classifier parameters corresponding to each non-sample category are determined.
In the embodiment of the invention, a plurality of groups of characteristic values are randomly acquired and respectively used as the initial classifier parameters corresponding to each non-sample category and the initial classifier parameters corresponding to each sample category. The initial classifier parameters corresponding to the no-sample class are the same as the number of feature values included in the initial classifier parameters corresponding to the sample class. The number of feature values included in the initial classifier parameters and the final classifier parameters may be the same or different. The initial classifier parameters comprise initial classifier parameters corresponding to the non-sample category and initial classifiers corresponding to the sample category, and the final classification parameters comprise final classifier parameters corresponding to the non-sample category and final classifiers corresponding to the sample category.
S1042: inputting the association relation among the preset data categories, the initial classifier parameters corresponding to each sample-containing category and the initial classifier parameters corresponding to each no-sample category into a preset graph network model to obtain the intermediate classifier parameters corresponding to each sample-containing category and the intermediate classifier parameters corresponding to each no-sample category.
Inputting the association relation among the preset data categories, the initial classifier parameters corresponding to each sample-free category and the initial classifier parameters corresponding to each sample-free category into a preset graph network model, and performing graph parameter learning to obtain the intermediate classifier parameters corresponding to each sample-free category and the intermediate classifier parameters corresponding to each sample-free category.
For example, the following formula (1) may be used to obtain the intermediate classifier parameters corresponding to each sample class and the intermediate classifier parameters corresponding to each no sample class.
P all ＝f(E all ) (1)
Wherein P is all Representing the parameter matrix of the intermediate classifier corresponding to all data categories, P all Is of dimension N all *C，N all For the total number of sampled and non-sampled classes, C is the number of eigenvalues included in the final classifier parameters, P all Includes P train And P' test ，P″ train Representing an intermediate classifier parameter matrix corresponding to a sample class, P test Representing an intermediate classifier parameter matrix corresponding to a sample-free class, f representing a mapping function of a preset graph network model, E all Representing an initial classifier parameter matrix corresponding to all data classes, E all Is of dimension N all *C init ，C init For the number of eigenvalues included in the initial classifier parameters, E all Includes E train And E is test ，E train Representing an initial classifier parameter matrix corresponding to the sample class, E test And representing an initial classifier parameter matrix corresponding to the no-sample class.
And (3) inputting the association relation among the preset data categories, the initial classifier parameters corresponding to each sample category and the initial classifier parameters corresponding to each no-sample category into a preset graph network model by utilizing the formula (1) to obtain the intermediate classifier parameters corresponding to each sample category and the intermediate classifier parameters corresponding to each no-sample category.
S1043: and determining a loss value according to the first final classifier parameter corresponding to each sample class and the intermediate classifier parameter corresponding to each sample class.
In an alternative embodiment, for each sampled class, the similarity of the first final classifier parameter and the intermediate classifier parameter corresponding to the sampled class is calculated. And determining that the calculated target with the similarity smaller than a preset similarity threshold has a sample category. A ratio of the number of target sampled categories to the total number of all sampled categories is calculated as a loss value.
In another alternative embodiment, the similarity between the first final classifier parameter matrix corresponding to the sample class and the intermediate classifier parameter matrix corresponding to the sample class may be calculated, and the reciprocal of the calculated similarity is taken as the loss value.
In the embodiment of the invention, the loss value can be determined in other manners. For example, the following formula (2) may be adopted, and the first classifier parameter and the initial classifier parameter are calculated by using a preset loss function to obtain a loss value:
L＝L mse (P″ train ,P′ train ) (2)
wherein L represents a loss value, L mse Representing a predetermined loss function, P train Representing an intermediate classifier parameter matrix corresponding to a sample class, P' train A first final classifier parameter matrix corresponding to the sample class is represented.
S1044: if the loss value is smaller than the preset loss threshold value, determining the middle classifier parameter corresponding to each current non-sample category as the final classifier parameter.
In the embodiment of the invention, if the loss value is smaller than the preset loss threshold value, the completion of graph parameter learning is determined, and the current intermediate classifier parameter corresponding to each non-sample type is used as the final classifier parameter corresponding to the non-sample type.
For example, the current P ", may be used test Intermediate classifier parameter matrix P corresponding to no-sample class test 。
S105: and classifying the data to be classified of each non-sample category by using a preset classification model with the final classifier parameters corresponding to the determined non-sample category.
For example, the information can be obtained by argmax (P test * x) classifying the data to be classified, wherein x represents the feature vector of the data to be classified.
In an alternative embodiment, if the loss value is smaller than the preset loss threshold, determining the intermediate classifier parameter corresponding to each current sample class as the second final classifier parameter. And classifying the data to be classified of each sample category by using a preset classification model with the determined second final classifier parameters corresponding to the sample category.
In the embodiment of the invention, the data to be classified can be classified by using the second final classifier parameter corresponding to each sample type and the final classifier parameter corresponding to each no-sample type obtained by the preset graph network model, so that the data to be classified with sample type and no-sample type can be classified, namely the data without sample data and the data with sample type can be classified, and the application range of data classification is enlarged.
The following describes a data classification method according to an embodiment of the present invention with reference to a schematic diagram of a graph network shown in fig. 4. In the lower right rectangular box of fig. 4, each circle represents a data category, and the numerical values in the circles represent the corresponding categories. Wherein, there are sample data for category 1-3 and sample data for category 4-5.
In step S401, a feature vector of each sample data of the class 1-3 is acquired.
Step S402, inputting the feature vectors of the acquired sample data into a preset network to obtain the prediction category of each sample data.
In step S403, a loss value L1 is determined by using a loss function according to the predicted class of each sample data and the class of the known sample data.
Step S404, using the loss value L1 to adjust the classifier parameters in the preset network until the preset network converges, and obtaining the preset classification model.
Step S405 extracts a final classifier parameter 1 corresponding to the category 1, a final classifier parameter 2 corresponding to the category 2, and a final classifier parameter 3 corresponding to the category 3 in the preset classification model.
For example, final classifier parameter 1 is [ a1, a2, a3, a4, a5], final classifier parameter 2 is [ b1, b2, b3, b4, b5], and final classifier parameter 3 is [ c1, c2, c3, c4, c5].
At this time, the liquid crystal display device,
step S406, constructing a graph network of the category 1-5, such as a rectangular box at the bottom right in FIG. 4.
In the figure, a connection line between two categories indicates that an association relationship exists between the two categories.
Step S407, performing graph parameter learning by using the initial classifier parameters corresponding to the classes 1-5 in the graph network, and determining a final classifier parameter 11 corresponding to the class 1, a final classifier parameter 12 corresponding to the class 2, a final classifier parameter 13 corresponding to the class 3, a final classifier parameter 14 corresponding to the class 4, and a final classifier parameter 15 corresponding to the class 5.
For example, final classifier parameters 11 are [ a11, a12, a13, a14, a15], final classifier parameters 12 are [ b11, b12, b13, b14, b15], final classifier parameters 13 are [ c11, c12, c13, c14, c15], final classifier parameters 14 are [ d1, d2, d3, d4, d5], and final classifier parameters 15 are [ e1, e2, e3, e4, e5].
At this time, the liquid crystal display device,
step S408, calculateP′ train And P' train A loss value L2 of (2); if the loss value L2 is smaller than the preset loss threshold value, determining the current P test Is P test Current P train Is P train The method comprises the steps of carrying out a first treatment on the surface of the If the loss value L2 is greater than or equal to the preset loss threshold value, determining to adjust P train And P' test And returns to step S407.
The above description of the steps S401-S408 is relatively simple, and reference is made in particular to the description of the parts of fig. 1-3.
Corresponding to the above data classification method, as shown in fig. 5, an embodiment of the present invention further provides a data classification device, where the device includes: an acquisition module 501, a training module 502, an extraction module 503, a determination module 504, and a classification module 505.
An obtaining module 501, configured to obtain a plurality of sample data, where a data class of the sample data is a sample class;
the training module 502 is configured to train the preset network by using a plurality of sample data to obtain a preset classification model;
an extracting module 503, configured to extract a first final classifier parameter corresponding to each sample class in the preset classification model;
the determining module 504 is configured to determine, according to a preset association relationship between data classes and a first final classifier parameter corresponding to each sample class, a final classifier parameter corresponding to each no-sample class, where the data classes include a sample class and a no-sample class;
the classification module 505 is configured to classify the data to be classified of each non-sample category by using a preset classification model with the final classifier parameters corresponding to the determined non-sample category.
In an alternative embodiment, as shown in fig. 6, the determining module 504 may include:
A first determining submodule 5041, configured to determine an initial classifier parameter corresponding to each non-sample category and an initial classifier parameter corresponding to each sample category;
the learning submodule 5042 is configured to input a preset association relationship among data categories, an initial classifier parameter corresponding to each sample category, and an initial classifier parameter corresponding to each no-sample category into a preset graph network model to obtain an intermediate classifier parameter corresponding to each sample category and an intermediate classifier parameter corresponding to each no-sample category;
a second determining submodule 5043, configured to determine a loss value according to the first final classifier parameter corresponding to each sample class and the intermediate classifier parameter corresponding to each sample class;
a third determining submodule 5044 is configured to determine, if the loss value is smaller than the preset loss threshold, an intermediate classifier parameter corresponding to each current non-sample category as a final classifier parameter.
In an alternative embodiment, the third determining submodule 5044 may be further configured to adjust the initial classifier parameter corresponding to each non-sample category and the initial classifier parameter corresponding to each sample category if the loss value is greater than or equal to the preset loss threshold, and return to executing the step of inputting the association relationship between the preset data categories, the initial classifier parameter corresponding to each sample category and the initial classifier parameter corresponding to each non-sample category into the preset graph network model.
In an alternative embodiment, the third determining submodule 5044 is further configured to determine, if the loss value is smaller than the preset loss threshold, an intermediate classifier parameter corresponding to each current sample class as the second final classifier parameter;
the classification module 505 is further configured to classify the data to be classified of each non-sample category by using a preset classification model with the determined second final classifier parameter corresponding to the sample category.
In an alternative embodiment, the data classifying device may further include:
the establishing module is used for determining the similarity between every two data categories before determining the final classifier parameters corresponding to each non-sample category, and establishing the association relationship between the two data categories with the similarity greater than a preset similarity threshold.
In an alternative embodiment, the sample data and the data to be classified are image, document, table or speech data.
In an alternative embodiment, the classification module 505 may specifically be configured to:
when the sample data and the data to be classified are images, carrying out target detection on the data to be classified, and determining an image area including a detection target in the data to be classified;
and classifying the image area by using a preset classification model with the final classifier parameters corresponding to the determined sample-free class to obtain the sample-free class of the detection target.
In the technical scheme provided by the embodiment of the invention, the sample-free category is a data category without sample data. The characteristics of the data of the two data categories with the association relation have certain similarity, based on the first final classifier parameters corresponding to the sample category and the association relation among the preset data categories, the final classifier parameters corresponding to the sample-free category can be determined under the condition that sample data of the sample-free category is not available, and further the data to be classified of the sample-free category can be classified by utilizing the preset classification model with the final classifier parameters corresponding to the sample-free category, namely, the classification of the data category without the sample data is realized, and the application range of the data classification is enlarged.
Corresponding to the above data classification method, the embodiment of the present invention further provides an electronic device, as shown in fig. 7, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, where the processor 701, the communication interface 702 and the memory 703 complete communication with each other through the communication bus 704,
a memory 703 for storing a computer program;
the processor 701 is configured to execute the program stored in the memory 703, and implement the following steps:
Acquiring a plurality of sample data, wherein the data types of the sample data are sample types;
training a preset network by utilizing the plurality of sample data to obtain a preset classification model;
extracting first final classifier parameters corresponding to each sample class in the preset classification model;
determining a final classifier parameter corresponding to each non-sample category according to a preset association relation between data categories and a first final classifier parameter corresponding to each sample category, wherein the data categories comprise sample categories and non-sample categories;
and classifying the data to be classified of each non-sample category by using a preset classification model with the final classifier parameters corresponding to the determined non-sample category.
In the technical scheme provided by the embodiment of the invention, the sample-free category is a data category without sample data. The characteristics of the data of the two data categories with the association relation have certain similarity, based on the first final classifier parameters corresponding to the sample category and the association relation among the preset data categories, the final classifier parameters corresponding to the sample-free category can be determined under the condition that sample data of the sample-free category is not available, and further the data to be classified of the sample-free category can be classified by utilizing the preset classification model with the final classifier parameters corresponding to the sample-free category, namely, the classification of the data category without the sample data is realized, and the application range of the data classification is enlarged.
The communication bus mentioned above for the electronic devices may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; but also digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present application, a computer readable storage medium is provided, in which a computer program is stored, which when executed by a processor implements any of the above-mentioned data classification method steps.
In a further embodiment of the present application, a computer program is provided, which when executed by a processor implements any of the above data classification method steps.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus embodiment, the electronic device embodiment, the computer-readable storage medium embodiment, and the computer program embodiment, since they are substantially similar to the method embodiment, the description is relatively simple, and the relevant points are referred to in the partial description of the method embodiment.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (9)
1. A method of classifying data, the method comprising:
acquiring a plurality of sample data, wherein the data types of the sample data are sample types;
training a preset network by using the plurality of sample data to obtain a preset classification model, wherein the preset network is a network with a supervision and identification task;
extracting first final classifier parameters corresponding to each sample class in the preset classification model;
determining a final classifier parameter corresponding to each non-sample category according to a preset association relation between data categories and a first final classifier parameter corresponding to each sample category, wherein the data categories comprise sample categories and non-sample categories;
classifying the data to be classified of each non-sample category by using a preset classification model with the final classifier parameters corresponding to the determined non-sample category; the sample data and the data to be classified are image, document, table or voice data;
The determining the final classifier parameters corresponding to each non-sample category according to the association relation among the preset data categories and the first final classifier parameters corresponding to each sample category comprises the following steps:
randomly acquiring a plurality of groups of characteristic values as initial classifier parameters corresponding to each non-sample category and initial classifier parameters corresponding to each sample category;
inputting the association relation among the preset data categories, the initial classifier parameters corresponding to each sample-free category and the initial classifier parameters corresponding to each sample-free category into a preset graph network model to obtain the intermediate classifier parameters corresponding to each sample-free category and the intermediate classifier parameters corresponding to each sample-free category;
determining a loss value according to the first final classifier parameter corresponding to each sample class and the intermediate classifier parameter corresponding to each sample class;
if the loss value is smaller than a preset loss threshold value, determining that the middle classifier parameter corresponding to each current non-sample category is a final classifier parameter;
and if the loss value is greater than or equal to the preset loss threshold value, adjusting the initial classifier parameters corresponding to each non-sample type and the initial classifier parameters corresponding to each sample type, and returning to execute the step of inputting the association relation among preset data types, the initial classifier parameters corresponding to each sample type and the initial classifier parameters corresponding to each non-sample type into a preset graph network model.
2. The method according to claim 1, wherein the method further comprises:
if the loss value is smaller than a preset loss threshold value, determining that the middle classifier parameter corresponding to each current sample class is a second final classifier parameter;
and classifying the data to be classified of each non-sample category by using a preset classification model with the determined second final classifier parameters corresponding to the sample category.
3. The method of any of claims 1-2, further comprising, prior to determining the final classifier parameters for each no-sample class:
determining the similarity between every two data categories;
and establishing an association relationship between two data categories with similarity greater than a preset similarity threshold.
4. The method of claim 1, wherein when the sample data and the data to be classified are images, before classifying the data to be classified for each non-sample category using a preset classification model having final classifier parameters corresponding to the determined non-sample category, further comprising:
performing target detection on the data to be classified, and determining an image area including the detection target in the data to be classified;
And classifying the image area by using a preset classification model with the final classifier parameters corresponding to the determined sample-free class to obtain the sample-free class of the detection target.
5. A data sorting apparatus, the apparatus comprising:
the acquisition module is used for acquiring a plurality of sample data, wherein the data category of the sample data is a sample category;
the training module is used for training a preset network by utilizing the plurality of sample data to obtain a preset classification model, wherein the preset network is a network with a supervision and identification task;
the extraction module is used for extracting first final classifier parameters corresponding to each sample category in the preset classification model;
the determining module is used for determining the final classifier parameters corresponding to each non-sample category according to the association relation among preset data categories and the first final classifier parameters corresponding to each sample category, wherein the data categories comprise sample categories and non-sample categories;
the classification module is used for classifying the data to be classified of each non-sample category by using a preset classification model with the determined final classifier parameters corresponding to the non-sample category; the sample data and the data to be classified are image, document, table or voice data;
The determining module includes:
the first determining submodule is used for randomly acquiring a plurality of groups of characteristic values to serve as initial classifier parameters corresponding to each non-sample category and initial classifier parameters corresponding to each sample category;
the learning sub-module is used for inputting the association relation among the preset data categories, the initial classifier parameters corresponding to each sample category and the initial classifier parameters corresponding to each no-sample category into the preset graph network model to obtain the intermediate classifier parameters corresponding to each sample category and the intermediate classifier parameters corresponding to each no-sample category;
the second determining submodule is used for determining a loss value according to the first final classifier parameter corresponding to each sample category and the middle classifier parameter corresponding to each sample category;
a third determining submodule, configured to determine an intermediate classifier parameter corresponding to each current non-sample class as a final classifier parameter if the loss value is less than a preset loss threshold;
and the third determining submodule is further used for adjusting the initial classifier parameters corresponding to each non-sample category and the initial classifier parameters corresponding to each sample category if the loss value is greater than or equal to the preset loss threshold value, and returning to execute the step of inputting the association relation among preset data categories, the initial classifier parameters corresponding to each sample category and the initial classifier parameters corresponding to each non-sample category into a preset graph network model.
6. The apparatus of claim 5, wherein the third determining submodule is further configured to determine an intermediate classifier parameter corresponding to each current sample class as a second final classifier parameter if the loss value is less than a preset loss threshold;
the classification module is further configured to classify data to be classified of each non-sample category by using a preset classification model with the determined second final classifier parameter corresponding to the sample category.
7. The apparatus of claim 5, wherein the classification module is specifically configured to:
when the sample data and the data to be classified are images, performing target detection on the data to be classified, and determining an image area including the detection target in the data to be classified;
and classifying the image area by using a preset classification model with the final classifier parameters corresponding to the determined sample-free class to obtain the sample-free class of the detection target.
8. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
A memory for storing a computer program;
a processor for carrying out the method steps of any one of claims 1-4 when executing a program stored on a memory.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911397622.1A CN113128530B (en) | 2019-12-30 | 2019-12-30 | Data classification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911397622.1A CN113128530B (en) | 2019-12-30 | 2019-12-30 | Data classification method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113128530A CN113128530A (en) | 2021-07-16 |
| CN113128530B true CN113128530B (en) | 2023-11-03 |
Family
ID=76767861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911397622.1A Active CN113128530B (en) | 2019-12-30 | 2019-12-30 | Data classification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113128530B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107273927A (en) * | 2017-06-13 | 2017-10-20 | 西北工业大学 | Sorting technique is adapted to based on the unsupervised field matched between class |
| CN107563444A (en) * | 2017-09-05 | 2018-01-09 | 浙江大学 | A kind of zero sample image sorting technique and system |
| CN108376267A (en) * | 2018-03-26 | 2018-08-07 | 天津大学 | A kind of zero sample classification method based on classification transfer |
| CN110175251A (en) * | 2019-05-25 | 2019-08-27 | 西安电子科技大学 | The zero sample Sketch Searching method based on semantic confrontation network |
| CN110222771A (en) * | 2019-06-10 | 2019-09-10 | 成都澳海川科技有限公司 | A kind of classification recognition methods of zero samples pictures |
-
2019
- 2019-12-30 CN CN201911397622.1A patent/CN113128530B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107273927A (en) * | 2017-06-13 | 2017-10-20 | 西北工业大学 | Sorting technique is adapted to based on the unsupervised field matched between class |
| CN107563444A (en) * | 2017-09-05 | 2018-01-09 | 浙江大学 | A kind of zero sample image sorting technique and system |
| CN108376267A (en) * | 2018-03-26 | 2018-08-07 | 天津大学 | A kind of zero sample classification method based on classification transfer |
| CN110175251A (en) * | 2019-05-25 | 2019-08-27 | 西安电子科技大学 | The zero sample Sketch Searching method based on semantic confrontation network |
| CN110222771A (en) * | 2019-06-10 | 2019-09-10 | 成都澳海川科技有限公司 | A kind of classification recognition methods of zero samples pictures |
Non-Patent Citations (3)
| Title |
|---|
| Semi-supervised classification with graph convolutional networks;Thomas N. Kipf等;《arXiv:1609.02907v4》;20170222;第1-14页 * |
| Zero-shot image categorization by image correlation exploration;LianLi Gao等;《Proceedings of the 5th ACM on International Conference on Multimedia Retrieval》;20150622;第487–490页 * |
| Zero-Shot Recognition via Semantic Embeddings and Knowledge Graphs;Xiaolong Wang等;《2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition》;20181216;第1-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113128530A (en) | 2021-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020239015A1 (en) | Image recognition method and apparatus, image classification method and apparatus, electronic device, and storage medium | |
| JP6994588B2 (en) | Face feature extraction model training method, face feature extraction method, equipment, equipment and storage medium | |
| US20200175062A1 (en) | Image retrieval method and apparatus, and electronic device | |
| CN109389135B (en) | Image screening method and device | |
| WO2020062360A1 (en) | Image fusion classification method and apparatus | |
| Alharbi et al. | Automatic counting of wheat spikes from wheat growth images | |
| CN110969170B (en) | Image theme color extraction method and device and electronic equipment | |
| CN110969066A (en) | Live video identification method and device and electronic equipment | |
| JP2012032370A (en) | Defect detection method, defect detection apparatus, learning method, program, and recording medium | |
| CN110263730B (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN109934216B (en) | Image processing method, device and computer readable storage medium | |
| CN110956615B (en) | Image quality evaluation model training method and device, electronic equipment and storage medium | |
| CN107272899B (en) | VR (virtual reality) interaction method and device based on dynamic gestures and electronic equipment | |
| CN110610123A (en) | Multi-target vehicle detection method and device, electronic equipment and storage medium | |
| CN112434717B (en) | Model training method and device | |
| CN113989626B (en) | Multi-class garbage scene distinguishing method based on target detection model | |
| CN110287361B (en) | Figure picture screening method and device | |
| Dubosclard et al. | Automated visual grading of grain kernels by machine vision | |
| Yang et al. | Individual feral cat identification through deep learning | |
| CN113128530B (en) | Data classification method and device | |
| CN111340140A (en) | Image data set acquisition method and device, electronic equipment and storage medium | |
| CN115879002A (en) | Training sample generation method, model training method and device | |
| CN114626483A (en) | Landmark image generation method and device | |
| Chica et al. | Standard methods for inexpensive pollen loads authentication by means of computer vision and machine learning | |
| CN113298166A (en) | Defect classifier, defect classification method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |