CN109766872A - Image-recognizing method and device - Google Patents
Image-recognizing method and device Download PDFInfo
- Publication number
- CN109766872A CN109766872A CN201910101257.9A CN201910101257A CN109766872A CN 109766872 A CN109766872 A CN 109766872A CN 201910101257 A CN201910101257 A CN 201910101257A CN 109766872 A CN109766872 A CN 109766872A
- Authority
- CN
- China
- Prior art keywords
- model
- image
- training
- obtains
- precision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
Abstract
The invention discloses a kind of image-recognizing method and devices.Wherein, this method comprises: obtaining images to be recognized；Obtain the image recognition model pre-established, wherein, image recognition model is to be trained by multiple training sets to initial model, initial model is the identification model established based on branch's training algorithm, the same training set is to extract to obtain from the same data set, and different training sets are to extract to obtain from different data sets；Images to be recognized is identified using image recognition model, obtains recognition result.The present invention solves the low technical problem of recognition accuracy of image-recognizing method in the prior art.
Description
Technical field
The present invention relates to field of image recognition, in particular to a kind of image-recognizing method and device.
Background technique
In existing field of image recognition, the field of face identification of special mainstream, mainly by image recognition model into
Row identification, image recognition model is all based on deep learning algorithm model and is trained to obtain, deep learning model training it is good
The bad influence to recognition accuracy is most important.And in entire depth learning model training process, for trained data set
It is the most important thing again, can exerts a decisive influence to the final algorithm performance of deep learning model.
It carries out, such as in field of face identification, instructs on individualized training data set currently, deep learning model is substantially
Practice the open face database that data set can be the collected human face data under some scene or download from the Internet.By
Identical people may be covered between different data collection, and due to naming rule disunity between different data collection, so
It is difficult to merge the face picture of identical people according to its filename.And when carrying out recognition of face classification based training, must be requested that phase
The face picture of same people shares identical label classification number, so leading to not be likely to occur personnel's intersection using multiple simultaneously
Human face data collection.It is based only on the deep learning model that the training of individualized training data set obtains, the accuracy in image recognition
It is low, it is unable to satisfy the demand of different application.
For the low problem of the recognition accuracy of image-recognizing method in the prior art, effective solution is not yet proposed at present
Scheme.
Summary of the invention
The embodiment of the invention provides a kind of image-recognizing method and devices, at least to solve image recognition in the prior art
The low technical problem of the recognition accuracy of method.
According to an aspect of an embodiment of the present invention, a kind of image-recognizing method is provided, comprising: obtain figure to be identified
Picture；Obtain the image recognition model that pre-establishes, wherein image recognition model be by multiple training sets to initial model into
Row training obtains, and initial model is the identification model established based on branch's training algorithm, and the same training set is from same
Extraction obtains in data set, and different training sets are to extract to obtain from different data sets；It is treated using image recognition model
Identification image is identified, recognition result is obtained.
Further, the above method further include: obtain multiple data sets；Every image in multiple data sets is divided
Class obtains the label of every image, wherein label is used to characterize the classification results of every image, includes in multiple data sets
The label of at least two images is identical；Sample image is extracted from sorted each data set, obtains multiple training sets.
Further, sample image is being extracted from sorted each data set, it is above-mentioned before obtaining multiple training sets
Method further include: extract the default feature of every image in sorted each data set；Default spy based on every image
Sign carries out alignment operation to every image；Sample image is extracted from each data set after operation, obtains multiple training sets.
Further, in the case where every image is facial image, feature is preset including at least one of following: eyes,
Eyebrow, nose and the corners of the mouth.
Further, sample image is extracted from each data set after operation, obtains multiple training sets, comprising: from behaviour
Sample image is extracted at random in each data set after work；The store path and label for obtaining sample image, obtain multiple training
Collection.
Further, multiple data sets are obtained, comprising: obtain the collected video image of acquisition equipment and preset data
Collection；Video image and preset data collection are detected, multiple data sets are obtained.
Further, the above method further include: initial model is established based on branch's training algorithm, wherein initial model is extremely
It less include: multiple loss functions, multiple loss functions are one-to-one with multiple training sets；Multiple training sets are inputted parallel
In initial model, initial model is trained；Whether the model that training of judgement obtains meets preset condition；If training obtains
Model meet preset condition, it is determined that the obtained model of training is image recognition model.
Further, multiple training sets are inputted in initial model parallel, initial model is trained, comprising: will be more
A training set inputs in initial model parallel, obtains the functional value of multiple loss functions；According to the functional value of multiple loss functions
With chain type derivative algorithms, the gradient value of each parameter in initial model is obtained；According to stochastic gradient descent algorithm to each parameter
Gradient value be updated, obtain the obtained model of training.
Further, whether the model that training of judgement obtains meets preset condition, comprising: obtains verifying collection；Utilize verifying
Collect the model for obtaining training to verify, obtains the precision for the model that training obtains；The precision for the model that training of judgement obtains
It is whether identical as history precision, wherein history precision is model precision obtained in upper primary verification process that training obtains；
If the precision for the model that training obtains is identical as history precision, it is determined that the model that training obtains meets preset condition.
Further, if the precision for the model that training obtains is different from history precision, it is determined that the model that training obtains
Precision be history precision, and continue to be trained initial model.
Further, precision is used to characterize verifying and concentrates the sum of verification result of all verifying samples and all verifying samples
The ratio of sum.
Further, verifying collection is obtained, comprising: obtain other images in multiple data sets except sample image；From it
Image authentication pair is extracted at random in his image, is verified collection.
Further, image authentication to include: positive sample to and negative sample pair, positive sample is to identical comprising two labels
Image, negative sample is to the image different comprising two labels.
Further, loss function is quadratic loss function.
According to another aspect of an embodiment of the present invention, a kind of pattern recognition device is additionally provided, comprising: first obtains mould
Block, for obtaining images to be recognized；Second obtains module, for obtaining the image recognition model pre-established, wherein image
Identification model is to be trained by multiple training sets to initial model, and initial model is built based on branch's training algorithm
Vertical identification model, the same training set are to extract to obtain from the same data set, and different training sets are from different data
Extraction is concentrated to obtain；Identification module obtains identification knot for identifying using image recognition model to images to be recognized
Fruit.
According to another aspect of an embodiment of the present invention, a kind of storage medium is additionally provided, storage medium includes the journey of storage
Sequence, wherein equipment where control storage medium executes above-mentioned image-recognizing method in program operation.
According to another aspect of an embodiment of the present invention, a kind of processor is additionally provided, processor is used to run program,
In, program executes above-mentioned image-recognizing method when running.
In embodiments of the present invention, initial model can be established based on branch's training algorithm, and raw by different data collection
At multiple training sets initial model is trained, obtain image recognition model, further by image recognition model to
The images to be recognized of family input is identified, final recognition result is obtained.Compared with prior art, multiple data sets are combined
Branch training image recognition model than it is existing based on individual data collection training image recognition model accuracy rate it is higher,
The technical effect for improving recognition accuracy is reached, and then the recognition accuracy for solving image-recognizing method in the prior art is low
The technical issues of.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present invention, constitutes part of this application, this hair
Bright illustrative embodiments and their description are used to explain the present invention, and are not constituted improper limitations of the present invention.In the accompanying drawings:
Fig. 1 is a kind of flow chart of image-recognizing method according to an embodiment of the present invention；
Fig. 2 is a kind of schematic diagram of optional face picture according to an embodiment of the present invention；
Fig. 3 is the schematic diagram of the face picture after a kind of optional alignment according to an embodiment of the present invention；
Fig. 4 is a kind of recognition of face depth nerve optionally based on individual data collection input according to an embodiment of the present invention
The schematic diagram of network model；
Fig. 5 is a kind of recognition of face depth nerve optionally based on the input of multiple data sets according to an embodiment of the present invention
The schematic diagram of network model；
Fig. 6 is a kind of flow chart of optional image-recognizing method according to an embodiment of the present invention；And
Fig. 7 is a kind of schematic diagram of pattern recognition device according to an embodiment of the present invention.
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention
Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
It should be noted that description and claims of this specification and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way
Data be interchangeable under appropriate circumstances, so as to the embodiment of the present invention described herein can in addition to illustrating herein or
Sequence other than those of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that cover
Cover it is non-exclusive include, for example, the process, method, system, product or equipment for containing a series of steps or units are not necessarily limited to
Step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, product
Or other step or units that equipment is intrinsic.
Embodiment 1
According to embodiments of the present invention, a kind of embodiment of image-recognizing method is provided, it should be noted that in attached drawing
The step of process illustrates can execute in a computer system such as a set of computer executable instructions, although also,
Logical order is shown in flow chart, but in some cases, it can be to be different from shown by sequence execution herein or retouch
The step of stating.
Fig. 1 is a kind of flow chart of image-recognizing method according to an embodiment of the present invention, as shown in Figure 1, this method includes
Following steps:
Step S102 obtains images to be recognized.
Specifically, above-mentioned images to be recognized can be the image identified, in embodiments of the present invention, with people
It is described in detail for face image.
Step S104 obtains the image recognition model pre-established, wherein image recognition model is by multiple training
What collection was trained initial model, initial model is the identification model established based on branch's training algorithm, the same instruction
Practicing collection is to extract to obtain from the same data set, and different training sets are to extract to obtain from different data sets.
Specifically, in order to improve image recognition accuracy rate, it is more that multiple and different data set buildings can be first passed through in advance
A training set, and initial model is trained by training set, to obtain final image recognition model.
In field of face identification, due to that may include the face picture of identical people, Er Qieyong between different data collection
Family can not determine that different data is concentrated comprising which identical people, it is thus impossible to which different data collection is simply directly closed
And at a single data set.Score value training method can be gathered and establish deep neural network model, obtain initial model, led to
It crosses and different data collection is separately carried out to branch's training, so as to obtain trained image recognition model, and will be trained
Image recognition model is deployed in application scenarios.
Step S106 identifies images to be recognized using image recognition model, obtains recognition result.
Specifically, it in field of face identification, can be carried out by comparing face characteristic feat-ID (using Euclidean distance)
Recognition of face process.
In the above embodiments of the present application, initial model can be established based on branch's training algorithm, and pass through different data collection
The multiple training sets generated are trained initial model, obtain image recognition model, further pass through image recognition model pair
The images to be recognized of user's input identifies, obtains final recognition result.Compared with prior art, multiple data are combined
Collection branch training image recognition model than it is existing based on individual data collection training image recognition model accuracy rate more
Height has reached the technical effect for improving recognition accuracy, and then the identification for solving image-recognizing method in the prior art is accurate
The low technical problem of rate.
Optionally, in the above embodiment of the present invention, this method further include: obtain multiple data sets；To in multiple data sets
Every image classify, obtain the label of every image, wherein label is used to characterize the classification results of every image, more
A data concentrate the label at least two images for including identical；Sample image is extracted from sorted each data set,
Obtain multiple training sets.
Specifically, it in field of face identification, in order to construct multiple training sets, can obtain in advance under different application scene
Face picture, obtain multiple data sets.Since the open face data set downloaded from internet has usually marked
, for the data set not marked, with artificial detection and face picture can be extracted, classified and is marked, phase will be belonged to
The face picture of same people is put together and is marked, and the label of every photo is obtained.Assuming that total number of persons is N, everyone has M
Open face picture.A certain number of face pictures can be randomly selected in each data set being labeled, obtained every
A training set.
Optionally, in the above embodiment of the present invention, sample image is being extracted from sorted each data set, is being obtained more
Before a training set, this method further include: extract the default feature of every image in sorted each data set；Based on every
The default feature for opening image carries out alignment operation to every image；Sample image is extracted from each data set after operation, is obtained
To multiple training sets.
Optionally, in the case where every image is facial image, feature is preset including at least one of following: eyes, eyebrow
Hair, nose and the corners of the mouth.
Specifically, in field of face identification, facial angle and face location in face picture be it is inconsistent, in order to
Guarantee extracts stable feature and obtains preferable recognition of face effect, needs to carry out alignment operation to face picture, to remove
Facial angle influences recognition of face bring.Key point includes the position of eyes, nose and corners of the mouth etc., as shown in Figure 2.Alignment
Face afterwards is as shown in Figure 3.
Optionally, in the above embodiment of the present invention, sample image is extracted from each data set after operation, is obtained multiple
Training set, comprising: extract sample image at random from each data set after operation；Obtain the store path and mark of sample image
Label, obtain multiple training sets.
Specifically, it can be labeled and randomly selected in the face picture of face alignment while including face identity
The face picture of information and verification information, obtains sample image, and each training sample extracted is as follows: face picture img_1,
The identity information (classification number) of img_1 ..., the identity information (classification number) of face picture img_N, img_N.
Wherein, face picture img_1 refers to that the store path of the 1st face picture, classification number refer to us for the people
The label marked in advance, classification number is generally since 0.For different people inside the same data set of different tag representations
Digital code.For example first data is concentrated, and 100 people is shared, then classification number is respectively 1-0,1-1,1-2 ... ..., 1-
99；Second data set or scene cover 50 people, then classification number is respectively 2-0,2-1,2-2 ... ..., 2-49.Two groups
It is not equivalent between classification number, respectively from different data sets.
Optionally, in the above embodiment of the present invention, multiple data sets are obtained, comprising: obtain the acquisition collected view of equipment
Frequency image and preset data collection；Video image and preset data collection are detected, multiple data sets are obtained.
Specifically, in field of face identification, acquisition equipment can be mounted in the camera in different application scene, make
Video pictures are acquired with camera, and in computer systems by network transmission and data line storage, application scenarios can be
The corresponding usage scenario of engineering project, such as bank VTM (Video Teller Machine, remote teller machine) verifying, jewelry
Shop VIP identification etc..Above-mentioned preset data collection can be the open face data set from the Internet download.
Identical people may be covered between the human face data collection obtained by the above method, for example, in bank and jewelry
The customer that shop was photographed with camera, photo may also occur on the internet and be organized in open face data set.And
And disclosed on internet between face data set A and B may also include same person face picture.
Video pictures collected for camera carry out Face datection to collected video pictures, by face picture
It extracts and is stored in computer system hard.
Optionally, in the above embodiment of the present invention, this method further include: initial model is established based on branch's training algorithm,
Wherein, initial model includes at least: multiple loss functions, and multiple loss functions are one-to-one with multiple training sets；It will be more
A training set inputs in initial model parallel, is trained to initial model；It is default whether the model that training of judgement obtains meets
Condition；If the model that training obtains meets preset condition, it is determined that the model that training obtains is image recognition model.
It should be noted that only using a Softmax loss loss function in existing image recognition model as mesh
Mark is trained, and the image recognition model shown in Fig. 4 based on individual data collection input only includes a Classification Loss function,
Loss=SoftmaxLoss 1.
Different data collection can separately be carried out to branch's training, be concurrently input in the same image recognition model, i-th
Face picture after the alignment that a data are concentrated is docked to corresponding loss function after propagated forward obtains feature
SoftmaxLoss i is optimized as independent objective function.As shown in figure 5, concentrated when i-th of human face data of input
When face picture carries out branch's training into initial model, corresponding loss function is Loss=SoftmaxLoss i.
It should be noted that Fig. 4 and image recognition model shown in fig. 5 show showing for simplified general residual error network
It is intended to.
Optionally, loss function is quadratic loss function.
Specifically, in field of face identification, in order to carry out recognition of face process using Euclidean distance, in initial model
Multiple loss functions can be quadratic loss function.
Further, above-mentioned preset condition, which can be training, terminates Rule of judgment, when the model that training obtains meets in advance
If when condition, determining that training terminates, the model that final training obtains is trained image recognition model.
Optionally, in the above embodiment of the present invention, multiple training sets are inputted in initial model parallel, to initial model into
Row training, comprising: multiple training sets are inputted in initial model parallel, obtain the functional value of multiple loss functions；According to multiple
The functional value and chain type derivative algorithms of loss function, obtain the gradient value of each parameter in initial model；According under stochastic gradient
Drop algorithm is updated the gradient value of each parameter, obtains the model that training obtains.
Specifically, it after inputting multiple training sets in initial model parallel, can be lost by branch's training
Then the functional value Loss of function is obtained each in image recognition model as shown in Figure 5 according to Loss and chain type derivative algorithms
The gradient value of a parameter finally updates model parameter according to stochastic gradient descent algorithm, obtains trained model, training
Model meet training terminate Rule of judgment after, can determine trained model be final image recognition model.
Optionally, in the above embodiment of the present invention, whether the model that training of judgement obtains meets preset condition, comprising: obtains
Verifying is taken to collect；It is verified using the model that verifying collection obtains training, obtains the precision for the model that training obtains；Training of judgement
Whether the precision of obtained model is identical as history precision, wherein history precision is to train obtained model in upper primary verifying
Precision obtained in process；If the precision for the model that training obtains is identical as history precision, it is determined that the model that training obtains
Meet preset condition.
It, can will be current every fixed the number of iterations it should be noted that in the training process of image recognition model
Trained model is tested on verifying collection, with the training of model, precision meeting of the trained model on verifying collection
It is constantly promoted, but as model is constantly trained, when model tends to restrain or the phenomenon that over-fitting occur, model collects in verifying
On precision will not stable promotion again, show that model training can stopped.
Optionally, it is total with all verifying samples to be used to characterize the sum of verification results of all verifying samples of verifying concentration for precision
Several ratios.
Specifically, in field of face identification, verifying collection is by the face picture verifying randomly selected to forming.According to the world
The rule of standard faces validation test collection LFW, it is 6000 pairs that the quantity of face picture verifying pair is concentrated in verifying.For including 6000
To the verifying collection of face picture verifying pair, measuring accuracy can be with is defined as:Wherein, xiFor characterizing i-th
The verification result of face picture verifying pair.If the recognition result of model is identical as the physical tags of face picture verifying pair,
It determines and verifies correct namely xi=1；If the recognition result of model is different from the physical tags of face picture verifying pair, really
Determine authentication error namely xi=0.
Further, above-mentioned history precision can be last when verifying to trained model, get
The precision of trained model.If the precision of trained model is identical as history precision, Ye Jixun this time in verification process
The no longer stable promotion of the precision for the model perfected can determine that training terminates, using this trained model as final figure
As identification model.
Optionally, in the above embodiment of the present invention, if the precision for the model that training obtains is different from history precision, really
The precision for the model that fixed training obtains is history precision, and continues to be trained initial model.
In a kind of optional scheme, if the precision for the model that training obtains is different from history precision, that is, trained
To model satisfaction be unsatisfactory for preset condition, it is determined that training be not finished, need to continue to train, using this precision as under
History precision in model verification process.Whether the precision with history precision of the good model of training of judgement are identical again, from
And whether the model for determining that training obtains meets preset condition.
Optionally, in the above embodiment of the present invention, obtain verifying collection, comprising: obtain in multiple data sets sample image it
Other outer images；Image authentication pair is extracted at random from other images, is verified collection.
Optionally, image authentication to include: positive sample to and negative sample pair, positive sample is to including the identical figure of two labels
Picture, negative sample is to the image different comprising two labels.
Specifically, in field of face identification, it is assumed that there is the face picture of K people to be used for the production of training set, then it can be with
The face picture of remaining N-K people is used to verify the production of collection.Verifying collection is verified by the human face photo randomly selected to group
At, extract positive sample to and negative sample pair, the quantity of positive and negative samples pair is identical, for including the verifying pair of 6000 pairs of face pictures
Verifying collection, positive and negative samples are to respectively taking 3000 pairs.Wherein, positive sample is to a picture for n-th of people, the b of n-th of people
Picture；Negative sample is to the c picture for i-th of people, the d picture of j-th of people.Image recognition model is by positive sample pair
In two face pictures when being judged as the same person, can determine that verification result is correct；Image recognition model is by negative sample pair
In two face pictures be judged as when not being a people, can determine verifying structure；No person, verification result mistake.
Fig. 6 is a kind of flow chart of optional image-recognizing method according to an embodiment of the present invention, with field of face identification
For be illustrated, as shown in fig. 6, this method comprises: collecting the face picture under multiple scenes；To the face picture being collected into
Face datection is carried out, face picture is extracted and is stored in hard disc of computer；Manually to the face figure for detecting and extracting
Piece is classified and is marked, and the face picture for belonging to identical people is put together and is marked；Face picture is carried out crucial
Point alignment operation is influenced with removing facial angle to recognition of face bring；In being labeled the photo being aligned with face
It randomly selects while the face picture comprising face identity information and verification information is to being trained, namely extract face identity-
Verify training set；Conjugate branch training algorithm establishes recognition of face deep neural network model, includes multiple losses in model
Function；Recognition of face deep neural network model is trained based on more data sets, obtains trained network model；Judgement
Whether measuring accuracy of the trained network model on verifying collection is constantly promoted, namely judges whether that reaching training terminates item
Part；If conditions are not met, then continuing model training；If it is satisfied, then obtaining face recognition algorithms network model and model ginseng
Number；Trained face recognition algorithms network model is deployed in application scenarios, it can be by comparing face characteristic feat-ID
(using Euclidean distance) carries out recognition of face process.
The scheme provided through the foregoing embodiment can be used in bank VIP identification project, adopt under true application scenarios
Collect face picture, while also downloading to some open face data sets from internet；Then by the face in these data sets
Picture detected, alignment operation, and makes corresponding face identity-verifying training set；Use the training of previously described method
Face recognition algorithms model out is known to obtain the face with high discrimination and recognition effect in bank VIP identification scene
Other algorithm, this method can preferably combine the human face data information in multiple data sets, so as to obtain recognition effect more
Good human face recognition model.Combine multiple data sets branch training face deep neural network model than it is general based on
The face recognition algorithms accuracy rate of the deep learning network of individual data collection training (including gradually being finely tuned on multiple data sets)
It is higher.
Embodiment 2
According to embodiments of the present invention, a kind of embodiment of pattern recognition device is provided.
Fig. 7 is a kind of schematic diagram of pattern recognition device according to an embodiment of the present invention, as shown in fig. 7, the device includes:
First obtains module 72, for obtaining images to be recognized.
Specifically, above-mentioned images to be recognized can be the image identified, in embodiments of the present invention, with people
It is described in detail for face image.
Second obtains module 74, for obtaining the image recognition model pre-established, wherein image recognition model is logical
Cross what multiple training sets were trained initial model, initial model is the identification mould established based on branch's training algorithm
Type, the same training set are to extract to obtain from the same data set, and different training sets are extracted from different data sets
It arrives.
Specifically, in order to improve image recognition accuracy rate, it is more that multiple and different data set buildings can be first passed through in advance
A training set, and initial model is trained by training set, to obtain final image recognition model.
In field of face identification, due to that may include the face picture of identical people, Er Qieyong between different data collection
Family can not determine that different data is concentrated comprising which identical people, it is thus impossible to which different data collection is simply directly closed
And at a single data set.Score value training method can be gathered and establish deep neural network model, obtain initial model, led to
It crosses and different data collection is separately carried out to branch's training, so as to obtain trained image recognition model, and will be trained
Image recognition model is deployed in application scenarios.
Identification module 76 obtains recognition result for identifying using image recognition model to images to be recognized.
Specifically, it in field of face identification, can be carried out by comparing face characteristic feat-ID (using Euclidean distance)
Recognition of face process.
In the above embodiments of the present application, initial model can be established based on branch's training algorithm, and pass through different data collection
The multiple training sets generated are trained initial model, obtain image recognition model, further pass through image recognition model pair
The images to be recognized of user's input identifies, obtains final recognition result.Compared with prior art, multiple data are combined
Collection branch training image recognition model than it is existing based on individual data collection training image recognition model accuracy rate more
Height has reached the technical effect for improving recognition accuracy, and then the identification for solving image-recognizing method in the prior art is accurate
The low technical problem of rate.
Optionally, in the above embodiment of the present invention, the device further include: third obtains module, for obtaining multiple data
Collection；Categorization module obtains the label of every image, wherein label for classifying to every image in multiple data sets
For characterizing the classification results of every image, the label of at least two for including in multiple data sets image is identical；First mentions
Modulus block obtains multiple training sets for extracting sample image from sorted each data set.
Optionally, in the above embodiment of the present invention, the device further include: the second extraction module, it is sorted for extracting
The default feature of every image in each data set；Alignment module schemes every for the default feature based on every image
As carrying out alignment operation；Third extraction module obtains multiple instructions for extracting sample image from each data set after operation
Practice collection.
Optionally, in the case where every image is facial image, feature is preset including at least one of following: eyes, eyebrow
Hair, nose and the corners of the mouth.
Optionally, in the above embodiment of the present invention, third extraction module includes: extraction unit, for from every after operation
A data concentrate random extraction sample image；First acquisition unit is obtained for obtaining the store path and label of sample image
Multiple training sets.
Optionally, in the above embodiment of the present invention, it includes: second acquisition unit that third, which obtains module, for obtaining acquisition
The collected video image of equipment and preset data collection；Detection unit, for being detected to video image and preset data collection,
Obtain multiple data sets.
Optionally, in the above embodiment of the present invention, the device further include: module is established, for being based on branch's training algorithm
Establish initial model, wherein initial model includes at least: multiple loss functions, multiple loss functions and multiple training sets are one
One is corresponding；Training module is trained initial model for inputting multiple training sets in initial model parallel；Judgement
Whether module, the model obtained for training of judgement meet preset condition；Determining module, if the model for training to obtain is full
Sufficient preset condition, it is determined that the model that training obtains is image recognition model.
Optionally, loss function is quadratic loss function.
Optionally, in the above embodiment of the present invention, training module includes: input unit, for multiple training sets are parallel
It inputs in initial model, obtains the functional value of multiple loss functions；Processing unit, for the functional value according to multiple loss functions
With chain type derivative algorithms, the gradient value of each parameter in initial model is obtained；Updating unit, for being calculated according to stochastic gradient descent
Method is updated the gradient value of each parameter, obtains the model that training obtains.
Optionally, in the above embodiment of the present invention, judgment module includes: third acquiring unit, for obtaining verifying collection；It tests
Unit is demonstrate,proved, the model for being obtained using verifying collection to training is verified, and the precision for the model that training obtains is obtained；Judgement is single
Whether member, the precision and history precision of the model obtained for training of judgement are identical, wherein history precision is the mould that training obtains
Type precision obtained in upper primary verification process；Determination unit, if the precision and history essence of the model obtained for training
It spends identical, it is determined that the model that training obtains meets preset condition.
Optionally, it is total with all verifying samples to be used to characterize the sum of verification results of all verifying samples of verifying concentration for precision
Several ratios.
Optionally, in the above embodiment of the present invention, if training module is also used to train the precision of obtained model and go through
History precision is different, it is determined that the precision for the model that training obtains is history precision, and continues to be trained initial model.
Optionally, in the above embodiment of the present invention, third acquiring unit for obtain in multiple data sets sample image it
Other outer images, and image authentication pair is extracted at random from other images, it is verified collection.
Optionally, image authentication to include: positive sample to and negative sample pair, positive sample is to including the identical figure of two labels
Picture, negative sample is to the image different comprising two labels.
Embodiment 3
According to embodiments of the present invention, a kind of embodiment of storage medium is provided, storage medium includes the program of storage,
In, in program operation, equipment where control storage medium executes the image-recognizing method in above-described embodiment 1.
Embodiment 4
According to embodiments of the present invention, a kind of embodiment of processor is provided, processor is for running program, wherein journey
The image-recognizing method in above-described embodiment 1 is executed when sort run.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
In the above embodiment of the invention, it all emphasizes particularly on different fields to the description of each embodiment, does not have in some embodiment
The part of detailed description, reference can be made to the related descriptions of other embodiments.
In several embodiments provided herein, it should be understood that disclosed technology contents can pass through others
Mode is realized.Wherein, the apparatus embodiments described above are merely exemplary, such as the division of the unit, Ke Yiwei
A kind of logical function partition, there may be another division manner in actual implementation, for example, multiple units or components can combine or
Person is desirably integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Between coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or communication link of unit or module
It connects, can be electrical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
On unit.It can some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can for personal computer, server or network equipment etc.) execute each embodiment the method for the present invention whole or
Part steps.And storage medium above-mentioned includes: that USB flash disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), mobile hard disk, magnetic or disk etc. be various to can store program code
Medium.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.
Claims (17)
1. a kind of image-recognizing method characterized by comprising
Obtain images to be recognized；
Obtain the image recognition model pre-established, wherein described image identification model is by multiple training sets to initial
What model was trained, the initial model is the identification model established based on branch's training algorithm, the same training set
It is to extract to obtain from the same data set, different training sets are to extract to obtain from different data sets；
The images to be recognized is identified using described image identification model, obtains recognition result.
2. the method according to claim 1, wherein the method also includes:
Obtain multiple data sets；
Classify to every image in the multiple data set, obtains the label of every image, wherein the label
For characterizing the classification results of every image, the label phase of at least two for including in the multiple data set image
Together；
Sample image is extracted from sorted each data set, obtains the multiple training set.
3. according to the method described in claim 2, it is characterized in that, extracting the sample from sorted each data set
Image, before obtaining the multiple training set, the method also includes:
Extract the default feature of every image in sorted each data set；
Based on the default feature of every image, alignment operation is carried out to every image；
The sample image is extracted from each data set after operation, obtains the multiple training set.
4. according to the method described in claim 3, it is characterized in that, every image be facial image in the case where, institute
Default feature is stated including at least one of following: eyes, eyebrow, nose and the corners of the mouth.
5. according to the method described in claim 3, it is characterized in that, extracting the sample graph from each data set after operation
Picture obtains the multiple training set, comprising:
The sample image is extracted at random from each data set after the operation；
The store path and label for obtaining the sample image, obtain the multiple training set.
6. according to the method described in claim 2, it is characterized in that, obtaining multiple data sets, comprising:
Obtain the collected video image of acquisition equipment and preset data collection；
The video image and the preset data collection are detected, the multiple data set is obtained.
7. according to the method described in claim 2, it is characterized in that, the method also includes:
The initial model is established based on branch's training algorithm, wherein the initial model includes at least: multiple loss letters
Number, the multiple loss function is one-to-one with the multiple training set；
The multiple training set is inputted parallel in the initial model, the initial model is trained；
Whether the model that training of judgement obtains meets preset condition；
If the model that the training obtains meets the preset condition, it is determined that the model that the training obtains is described image
Identification model.
8. the method according to the description of claim 7 is characterized in that the multiple training set is inputted the initial model parallel
In, the initial model is trained, comprising:
The multiple training set is inputted parallel in the initial model, the functional value of the multiple loss function is obtained；
According to the functional value of the multiple loss function and chain type derivative algorithms, the ladder of each parameter in the initial model is obtained
Angle value；
It is updated according to gradient value of the stochastic gradient descent algorithm to each parameter, obtains the mould that the training obtains
Type.
9. the method according to the description of claim 7 is characterized in that whether the model that training of judgement obtains meets preset condition,
Include:
Obtain verifying collection；
It is verified using the model that the verifying collection obtains the training, obtains the precision for the model that the training obtains；
Judge whether precision and the history precision of the model that the training obtains are identical, wherein the history precision is the instruction
The model got precision obtained in upper primary verification process；
If the precision for the model that the training obtains is identical as the history precision, it is determined that the model that the training obtains is full
The foot preset condition.
10. according to the method described in claim 9, it is characterized in that, if the precision for the model that the training obtains with it is described
History precision is different, it is determined that the precision for the model that the training obtains is the history precision, and is continued to the introductory die
Type is trained.
11. according to the method described in claim 10, it is characterized in that, the precision concentrates all test for characterizing the verifying
Demonstrate,prove the ratio of the sum of verification result of sample with all verifying total sample numbers.
12. according to the method described in claim 9, it is characterized in that, obtaining verifying collection, comprising:
Obtain other images in the multiple data set except sample image；
Image authentication pair is extracted at random from other described images, obtains the verifying collection.
13. according to the method for claim 12, which is characterized in that described image verifying to include: positive sample to negative sample
This is right, and the positive sample is to comprising the identical image of two labels, and the negative sample is to the image different comprising two labels.
14. the method according to the description of claim 7 is characterized in that the loss function is quadratic loss function.
15. a kind of pattern recognition device characterized by comprising
First obtains module, for obtaining images to be recognized；
Second obtains module, for obtaining the image recognition model pre-established, wherein described image identification model is to pass through
What multiple training sets were trained initial model, the initial model is the identification mould established based on branch's training algorithm
Type, the same training set are to extract to obtain from the same data set, and different training sets are extracted from different data sets
It arrives；
Identification module obtains recognition result for identifying using described image identification model to the images to be recognized.
16. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein run in described program
When control the storage medium where equipment perform claim require any one of 1 to 14 described in image-recognizing method.
17. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run
Benefit requires any one of 1 to 14 described image recognition methods.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910101257.9A CN109766872B (en) | 2019-01-31 | 2019-01-31 | Image recognition method and device |
| PCT/CN2019/127817 WO2020155939A1 (en) | 2019-01-31 | 2019-12-24 | Image recognition method and device, storage medium and processor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910101257.9A CN109766872B (en) | 2019-01-31 | 2019-01-31 | Image recognition method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109766872A true CN109766872A (en) | 2019-05-17 |
| CN109766872B CN109766872B (en) | 2021-07-09 |
Family
ID=66455816
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910101257.9A Active CN109766872B (en) | 2019-01-31 | 2019-01-31 | Image recognition method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109766872B (en) |
| WO (1) | WO2020155939A1 (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110569911A (en) * | 2019-09-11 | 2019-12-13 | 深圳绿米联创科技有限公司 | Image recognition method, device, system, electronic equipment and storage medium |
| CN110674720A (en) * | 2019-09-18 | 2020-01-10 | 深圳市网心科技有限公司 | Picture identification method and device, electronic equipment and storage medium |
| CN110784465A (en) * | 2019-10-25 | 2020-02-11 | 新华三信息安全技术有限公司 | Data stream detection method and device and electronic equipment |
| CN111141412A (en) * | 2019-12-25 | 2020-05-12 | 深圳供电局有限公司 | Cable temperature and anti-theft dual-monitoring method and system and readable storage medium |
| WO2020155939A1 (en) * | 2019-01-31 | 2020-08-06 | 广州视源电子科技股份有限公司 | Image recognition method and device, storage medium and processor |
| CN112766052A (en) * | 2020-12-29 | 2021-05-07 | 有米科技股份有限公司 | CTC-based image character recognition method and device |
| CN113052561A (en) * | 2021-04-01 | 2021-06-29 | 苏州惟信易量智能科技有限公司 | Flow control system and method based on wearable device |
| WO2022033150A1 (en) * | 2020-08-11 | 2022-02-17 | Oppo广东移动通信有限公司 | Image recognition method, apparatus, electronic device, and storage medium |
| CN114782757A (en) * | 2022-06-21 | 2022-07-22 | 北京远舢智能科技有限公司 | Cigarette defect detection model training method and device, electronic equipment and storage medium |
| CN115019218A (en) * | 2022-08-08 | 2022-09-06 | 阿里巴巴（中国）有限公司 | Image processing method and processor |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112149741B (en) * | 2020-09-25 | 2024-04-16 | 北京百度网讯科技有限公司 | Training method and device for image recognition model, electronic equipment and storage medium |
| CN112529008A (en) * | 2020-11-03 | 2021-03-19 | 浙江大华技术股份有限公司 | Image recognition method, image feature processing method, electronic device and storage medium |
| CN112395439B (en) * | 2020-11-17 | 2024-03-01 | 林铭 | Image database implementation method and system and network communication equipment thereof |
| CN112766162B (en) * | 2021-01-20 | 2023-12-22 | 北京市商汤科技开发有限公司 | Living body detection method, living body detection device, electronic equipment and computer readable storage medium |
| CN112733958A (en) * | 2021-01-22 | 2021-04-30 | 北京农业信息技术研究中心 | Greenhouse ozone concentration control method and system |
| CN112818865A (en) * | 2021-02-02 | 2021-05-18 | 北京嘀嘀无限科技发展有限公司 | Vehicle-mounted field image identification method, identification model establishing method, device, electronic equipment and readable storage medium |
| CN113657406B (en) * | 2021-07-13 | 2024-04-23 | 北京旷视科技有限公司 | Model training and feature extraction method and device, electronic equipment and storage medium |
| CN113743499B (en) * | 2021-09-02 | 2023-09-05 | 广东工业大学 | View angle irrelevant feature dissociation method and system based on contrast learning |
| CN114264361A (en) * | 2021-12-07 | 2022-04-01 | 深圳市博悠半导体科技有限公司 | Object identification method and device combining radar and camera and intelligent electronic scale |
| CN116612358B (en) * | 2023-07-20 | 2023-10-03 | 腾讯科技（深圳）有限公司 | Data processing method, related device, equipment and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080262312A1 (en) * | 2007-04-17 | 2008-10-23 | University Of Washington | Shadowing pipe mosaicing algorithms with application to esophageal endoscopy |
| US20100246980A1 (en) * | 2009-03-31 | 2010-09-30 | General Electric Company | System and method for automatic landmark labeling with minimal supervision |
| CN105404877A (en) * | 2015-12-08 | 2016-03-16 | 商汤集团有限公司 | Human face attribute prediction method and apparatus based on deep study and multi-task study |
| CN107025443A (en) * | 2017-04-06 | 2017-08-08 | 江南大学 | Stockyard smoke monitoring and on-time model update method based on depth convolutional neural networks |
| CN107247947A (en) * | 2017-07-07 | 2017-10-13 | 北京智慧眼科技股份有限公司 | Face character recognition methods and device |
| CN107392164A (en) * | 2017-07-28 | 2017-11-24 | 深圳市唯特视科技有限公司 | A kind of Expression analysis method based on the estimation of Facial action unit intensity |
| CN107633242A (en) * | 2017-10-23 | 2018-01-26 | 广州视源电子科技股份有限公司 | Training method, device, equipment and the storage medium of network model |
| CN107844784A (en) * | 2017-12-08 | 2018-03-27 | 广东美的智能机器人有限公司 | Face identification method, device, computer equipment and readable storage medium storing program for executing |
| CN108509860A (en) * | 2018-03-09 | 2018-09-07 | 西安电子科技大学 | HOh Xil Tibetan antelope detection method based on convolutional neural networks |
| CN108921092A (en) * | 2018-07-02 | 2018-11-30 | 浙江工业大学 | A kind of melanoma classification method based on convolutional neural networks model Two-level ensemble |
| CN109074499A (en) * | 2016-04-12 | 2018-12-21 | 北京市商汤科技开发有限公司 | The method and system identified again for object |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6456991B1 (en) * | 1999-09-01 | 2002-09-24 | Hrl Laboratories, Llc | Classification method and apparatus based on boosting and pruning of multiple classifiers |
| CN104715227B (en) * | 2013-12-13 | 2020-04-03 | 北京三星通信技术研究有限公司 | Method and device for positioning key points of human face |
| CN105975959B (en) * | 2016-06-14 | 2019-09-03 | 广州视源电子科技股份有限公司 | Face characteristic neural network based extracts modeling, face identification method and device |
| CN106503669B (en) * | 2016-11-02 | 2019-12-10 | 重庆中科云丛科技有限公司 | Training and recognition method and system based on multitask deep learning network |
| CN106778684A (en) * | 2017-01-12 | 2017-05-31 | 易视腾科技股份有限公司 | deep neural network training method and face identification method |
| CN109766872B (en) * | 2019-01-31 | 2021-07-09 | 广州视源电子科技股份有限公司 | Image recognition method and device |
-
2019
- 2019-01-31 CN CN201910101257.9A patent/CN109766872B/en active Active
- 2019-12-24 WO PCT/CN2019/127817 patent/WO2020155939A1/en active Application Filing
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080262312A1 (en) * | 2007-04-17 | 2008-10-23 | University Of Washington | Shadowing pipe mosaicing algorithms with application to esophageal endoscopy |
| US20100246980A1 (en) * | 2009-03-31 | 2010-09-30 | General Electric Company | System and method for automatic landmark labeling with minimal supervision |
| CN105404877A (en) * | 2015-12-08 | 2016-03-16 | 商汤集团有限公司 | Human face attribute prediction method and apparatus based on deep study and multi-task study |
| CN109074499A (en) * | 2016-04-12 | 2018-12-21 | 北京市商汤科技开发有限公司 | The method and system identified again for object |
| CN107025443A (en) * | 2017-04-06 | 2017-08-08 | 江南大学 | Stockyard smoke monitoring and on-time model update method based on depth convolutional neural networks |
| CN107247947A (en) * | 2017-07-07 | 2017-10-13 | 北京智慧眼科技股份有限公司 | Face character recognition methods and device |
| CN107392164A (en) * | 2017-07-28 | 2017-11-24 | 深圳市唯特视科技有限公司 | A kind of Expression analysis method based on the estimation of Facial action unit intensity |
| CN107633242A (en) * | 2017-10-23 | 2018-01-26 | 广州视源电子科技股份有限公司 | Training method, device, equipment and the storage medium of network model |
| CN107844784A (en) * | 2017-12-08 | 2018-03-27 | 广东美的智能机器人有限公司 | Face identification method, device, computer equipment and readable storage medium storing program for executing |
| CN108509860A (en) * | 2018-03-09 | 2018-09-07 | 西安电子科技大学 | HOh Xil Tibetan antelope detection method based on convolutional neural networks |
| CN108921092A (en) * | 2018-07-02 | 2018-11-30 | 浙江工业大学 | A kind of melanoma classification method based on convolutional neural networks model Two-level ensemble |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020155939A1 (en) * | 2019-01-31 | 2020-08-06 | 广州视源电子科技股份有限公司 | Image recognition method and device, storage medium and processor |
| CN110569911A (en) * | 2019-09-11 | 2019-12-13 | 深圳绿米联创科技有限公司 | Image recognition method, device, system, electronic equipment and storage medium |
| CN110569911B (en) * | 2019-09-11 | 2022-06-07 | 深圳绿米联创科技有限公司 | Image recognition method, device, system, electronic equipment and storage medium |
| CN110674720A (en) * | 2019-09-18 | 2020-01-10 | 深圳市网心科技有限公司 | Picture identification method and device, electronic equipment and storage medium |
| CN110784465A (en) * | 2019-10-25 | 2020-02-11 | 新华三信息安全技术有限公司 | Data stream detection method and device and electronic equipment |
| CN111141412A (en) * | 2019-12-25 | 2020-05-12 | 深圳供电局有限公司 | Cable temperature and anti-theft dual-monitoring method and system and readable storage medium |
| WO2022033150A1 (en) * | 2020-08-11 | 2022-02-17 | Oppo广东移动通信有限公司 | Image recognition method, apparatus, electronic device, and storage medium |
| CN112766052A (en) * | 2020-12-29 | 2021-05-07 | 有米科技股份有限公司 | CTC-based image character recognition method and device |
| CN113052561A (en) * | 2021-04-01 | 2021-06-29 | 苏州惟信易量智能科技有限公司 | Flow control system and method based on wearable device |
| CN114782757A (en) * | 2022-06-21 | 2022-07-22 | 北京远舢智能科技有限公司 | Cigarette defect detection model training method and device, electronic equipment and storage medium |
| CN115019218A (en) * | 2022-08-08 | 2022-09-06 | 阿里巴巴（中国）有限公司 | Image processing method and processor |
| CN115019218B (en) * | 2022-08-08 | 2022-11-15 | 阿里巴巴（中国）有限公司 | Image processing method and processor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109766872B (en) | 2021-07-09 |
| WO2020155939A1 (en) | 2020-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109766872A (en) | Image-recognizing method and device | |
| CN104992142B (en) | A kind of pedestrian recognition method being combined based on deep learning and attribute study | |
| CN109784186A (en) | A kind of pedestrian recognition methods, device, electronic equipment and computer readable storage medium again | |
| CN104239858B (en) | A kind of method and apparatus of face characteristic checking | |
| CN105809123B (en) | Method for detecting human face and device | |
| CN104915351B (en) | Picture sort method and terminal | |
| CN108009528A (en) | Face authentication method, device, computer equipment and storage medium based on Triplet Loss | |
| CN109815801A (en) | Face identification method and device based on deep learning | |
| CN106951825A (en) | A kind of quality of human face image assessment system and implementation method | |
| CN109711281A (en) | A kind of pedestrian based on deep learning identifies again identifies fusion method with feature | |
| CN108229335A (en) | It is associated with face identification method and device, electronic equipment, storage medium, program | |
| CN106529499A (en) | Fourier descriptor and gait energy image fusion feature-based gait identification method | |
| CN106778684A (en) | deep neural network training method and face identification method | |
| CN108009482A (en) | One kind improves recognition of face efficiency method | |
| US20130250181A1 (en) | Method for face registration | |
| CN105975959A (en) | Face characteristic extraction modeling method based on neural network, face identification method, face characteristic extraction modeling device and face identification device | |
| CN105184260B (en) | A kind of image characteristic extracting method and pedestrian detection method and device | |
| CN104933428B (en) | A kind of face identification method and device based on tensor description | |
| CN109214280A (en) | Shop recognition methods, device, electronic equipment and storage medium based on streetscape | |
| CN109145766A (en) | Model training method, device, recognition methods, electronic equipment and storage medium | |
| CN109934198A (en) | Face identification method and device | |
| CN105740808B (en) | Face identification method and device | |
| CN109063649A (en) | Pedestrian's recognition methods again of residual error network is aligned based on twin pedestrian | |
| CN109902202A (en) | A kind of video classification methods and device | |
| CN108108711A (en) | Face supervision method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |