CN109902190A - Image encrypting algorithm optimization method, search method, device, system and medium - Google Patents
Image encrypting algorithm optimization method, search method, device, system and medium Download PDFInfo
- Publication number
- CN109902190A CN109902190A CN201910161195.0A CN201910161195A CN109902190A CN 109902190 A CN109902190 A CN 109902190A CN 201910161195 A CN201910161195 A CN 201910161195A CN 109902190 A CN109902190 A CN 109902190A
- Authority
- CN
- China
- Prior art keywords
- visual
- image
- characteristic value
- visual dictionary
- dictionary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application discloses a kind of image encrypting algorithm optimization method, search method, device, system and media, this method comprises: obtaining the first combination visual dictionary of training set of images, the first combination visual dictionary includes multiple visual dictionaries, which includes multiple vision words；Dimension-reduction treatment is carried out to the first combination visual dictionary, obtains characteristic value and the corresponding feature vector of this feature value；This feature vector is arranged according to preset rules, obtains dimensionality reduction mapping matrix；Based on the dimensionality reduction mapping matrix and the first combination visual dictionary, multiple target visual dictionaries are generated.The embodiment of the present application is by carrying out dimension-reduction treatment to original combination visual dictionary, and each feature vector is rearranged, so that the variance equiblibrium mass distribution between the corresponding data set of multiple visual dictionaries based on this generation, the optimization for completing figure retrieval model improves the accuracy of image retrieval.
Description
Technical field
General pattern processing technology field of the present invention, and in particular to a kind of image encrypting algorithm optimization method, search method,
Device, system and medium.
Background technique
Content-based image retrieval technical application is more and more extensive.It mainly comprises the processes of building image library in advance, is giving
Out after image to be retrieved, image similar with image to be retrieved or image collection can be retrieved from image library.
Visual dictionary when carrying out image retrieval using the bag of words based on cartesian product, in bag of words therein
It is to be obtained by successively calculating residual error and cluster.It is due to obtaining the data set of cluster by successively calculating residual error, then subsequent
The variance in data set got gradually becomes smaller, so that when carrying out image retrieval using the bag of words, bag of words
In visual dictionary shared energy in visual signature quantizing process is gradually decreased, influence image searching result, reduce retrieval essence
Degree.
Summary of the invention
In view of drawbacks described above in the prior art or deficiency, it is intended to provide a kind of image encrypting algorithm optimization method, retrieval
Method, apparatus, system and medium, arranged again the visual dictionary in total visual dictionary of acquisition by dimension-reduction treatment
Column combination, the visual dictionary optimized improve the accuracy rate of image retrieval.
In a first aspect, image encrypting algorithm optimization method provided by the embodiments of the present application, comprising:
The first combination visual dictionary of training set of images is obtained, which includes multiple visual dictionaries,
The visual dictionary includes multiple vision words；
Dimension-reduction treatment is carried out to the first combination visual dictionary, obtains characteristic value and the corresponding feature vector of this feature value；
This feature vector is arranged according to preset rules, obtains dimensionality reduction mapping matrix；
Based on the dimensionality reduction mapping matrix and the first combination visual dictionary, multiple target visual dictionaries, multiple mesh are generated
Mark equiblibrium mass distribution between the variance of the corresponding data set of visual dictionary.
In one embodiment of the application or multiple embodiments, based on the big of the corresponding this feature value of this feature vector
It is small, this feature vector is arranged, the dimensionality reduction mapping matrix is obtained.
In one embodiment of the application or multiple embodiments, M empty data acquisition system, the data acquisition system of the sky are constructed
Quantity with this first combine visual dictionary in visual dictionary quantity it is equal；
The corresponding feature vector of M characteristic value before row is successively sequentially added in the corresponding M data acquisition systems, it should
Characteristic value is arranged according to the sequence successively decreased；
The corresponding feature vector of M+1 characteristic value is added in m-th data acquisition system；
Determine the corresponding data acquisition system of current the smallest characteristic value total amount, when only include a feature in the data acquisition system to
When amount, this feature value total amount is the corresponding characteristic value of this feature vector being currently included in the data acquisition system；When being wrapped in data set
When including multiple feature vectors, this feature value total amount is the corresponding characteristic value of this feature vector being currently included in the data acquisition system
Product；
Judge whether the number of the feature vector in the corresponding data acquisition system of the smallest characteristic value total amount is less than l/M,
In, the corresponding feature vector of l characteristic value before row is feature vector to be arranged, which is less than or equal to each visual dictionary
The number of interior vision word, and l/M is the integral multiple of M；
If not, indicating that the data acquisition system is full, then the corresponding data acquisition system of current the smallest characteristic value total amount is skipped, and return
It is back to the step of determining current the smallest characteristic value total amount corresponding data set；
If so, the corresponding feature vector of remaining the M+n characteristic value is successively added the smallest characteristic value total amount pair
In the not full data acquisition system answered, and it is back to the step of determining current the smallest characteristic value total amount corresponding data set, made
Obtain includes l/M feature vector, 2≤n≤l in each data set.
In one embodiment of the application or multiple embodiments, the visual signature that training image concentrates training image is obtained；
Multiple visual dictionaries are generated based on the visual signature；
Each vision word in multiple visual dictionaries is successively combined, the first combination visual dictionary is obtained.
In one embodiment of the application or multiple embodiments, based on the dimensionality reduction mapping matrix and the first combination visual word
Allusion quotation generates the second combination visual dictionary；
Cutting is carried out to the second combination visual dictionary, obtains the target visual dictionary
In one embodiment of the application or multiple embodiments, determine that the training image is concentrated based on the target visual dictionary
The index of each visual signature of training image；
Determine the term frequency-inverse document frequency weight of the index of each visual signature of the training image；
The term frequency-inverse document frequency weight of index based on each visual signature generates the first word of the training image
Bag vector.
Second aspect, the embodiment of the present application provide a kind of image search method, comprising:
Obtain the visual signature of image to be retrieved；
The visual signature corresponding second is calculated using target visual dictionary as discussed based on the visual signature
Bag of words vector；
The second bag of words vector is calculated at a distance from the first bag of words vector as described above；
The similarity of the image to be retrieved and the training image is determined based on the distance；
It determines that the similarity is greater than the training image of predetermined threshold, is exported as target image.
The third aspect, image encrypting algorithm provided by the embodiments of the present application optimize device, comprising:
Module is obtained, for obtaining the first combination visual dictionary of training set of images, which includes
Multiple visual dictionaries, the visual dictionary include multiple vision words；
Dimensionality reduction module obtains characteristic value and this feature value pair for carrying out dimension-reduction treatment to the first combination visual dictionary
The feature vector answered；
It arranges module and obtains dimensionality reduction mapping matrix for being arranged according to preset rules this feature vector；
Generation module, for generating multiple target visuals based on the dimensionality reduction mapping matrix and the first combination visual dictionary
Dictionary, equiblibrium mass distribution between the variance of multiple corresponding data sets of target visual dictionary.
In one embodiment of the application or multiple embodiments, arrangement module is specifically used for corresponding based on this feature vector
The size of this feature value arranges this feature vector, obtains the dimensionality reduction mapping matrix.
Fourth aspect, image retrieving apparatus provided by the embodiments of the present application, comprising:
Module is obtained, the visual signature of image to be retrieved is obtained；
First computing module calculates the visual signature using target visual dictionary as described above based on the visual signature
Corresponding second bag of words vector；
Second computing module calculates the second bag of words vector at a distance from the first bag of words vector as described above；
Determining module determines the similarity of the image to be retrieved and the training image based on the distance；
Output module determines that the similarity is greater than the training image of predetermined threshold, exports as target image.
5th aspect, the embodiment of the present application provide a kind of computer system, including memory, processor and are stored in
On reservoir and the computer program that can run on a processor, the side that such as first aspect is somebody's turn to do is realized when which executes the program
The method that method or second aspect are somebody's turn to do.
6th aspect, the embodiment of the present application provide a kind of computer readable storage medium, are stored thereon with computer program,
The computer program is for realizing method described in method or second aspect as described in relation to the first aspect.
To sum up, a kind of image encrypting algorithm optimization method provided by the embodiments of the present application, search method, device, system and
Medium by carrying out dimension-reduction treatment to the first combination visual dictionary for obtaining training set of images, and carries out each feature vector
It rearranges, so that the variance equiblibrium mass distribution between the corresponding data set of multiple visual dictionaries based on this generation, solves view
Feel the technical issues of variance in the corresponding data set of dictionary gradually becomes smaller, completes the optimization of figure retrieval model, improve figure
As the accuracy of retrieval.
Further, by the size of the corresponding characteristic value of feature vector, multiple feature vectors are arranged, so that most
More balanced distribution between the data set variance of the target visual dictionary obtained afterwards.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the application's is other
Feature, objects and advantages will become more apparent upon:
Fig. 1 is the flow diagram of the image encrypting algorithm optimization method of embodiments herein；
Fig. 2 is the flow diagram of the image encrypting algorithm optimization method of another embodiment of the application；
Fig. 3 is the schematic illustration that the first combination visual dictionary of the application generates；
Fig. 4 is the flow diagram of the feature vector aligning method of embodiments herein；
Fig. 5 is the schematic illustration that the feature vector of embodiments herein arranges；
Fig. 6 is the flow diagram of the image search method of embodiments herein；
Fig. 7 is that the image encrypting algorithm of the application optimizes the structural schematic diagram of device；
Fig. 8 is the structural schematic diagram of the image retrieving apparatus of the application；
Fig. 9 is the structural schematic diagram of the server of embodiments herein.
Specific embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that in order to
Convenient for description, part relevant to invention is illustrated only in attached drawing.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
It is appreciated that the optimization of the image encrypting algorithm of the embodiment of the present application, can be applied to the optimization of bag of words, the word
Bag model is the algorithms most in use of image retrieval, and the specific building of the bag of words is as follows:
The visual signature for extracting training image first, then clusters visual signature by clustering algorithm, generates view
Feel dictionary.Specifically, clustering first by clustering algorithm to visual signature, First look dictionary is generated, then by vision
The residual error of feature and vision word is clustered as new visual signature, generates the second visual dictionary, and be repeated as many times, directly
To the visual dictionary for generating preset quantity, such as M is a.
It is possible to further quantify the visual signature of training image by K nearest neighbor algorithm, obtains and weighed by TF-IDF
Again, and according to the TF-IDF weight of acquisition the bag of words vector of each training image is generated.
It is appreciated that since deuterogenic visual dictionary is the cluster centre of residual error amount, corresponding data set
Variance will be gradually reduced.And then utilizing visual dictionary, when being indexed to the visual signature of image to be retrieved, later period shape
At visual dictionary shared by specific gravity will reduce, to influence retrieval accuracy.
The embodiment of the present application is by being combined cutting again to multiple visual dictionaries of generation, so that each visual dictionary exists
Shared balancing energy during image index, to improve the accuracy of image retrieval.
In order to facilitate understanding and illustrate, image retrieval provided by the embodiments of the present application is described in detail below by Fig. 1 to Fig. 9
Model optimization method, search method, device, system and medium.
Fig. 1 show a kind of flow diagram of image encrypting algorithm optimization method provided in an embodiment of the present invention, such as Fig. 1
It is shown, this method comprises:
S110 obtains the first combination visual dictionary of training set of images.
S120 carries out dimension-reduction treatment to the first combination visual dictionary, obtains multiple characteristic values and the corresponding spy of characteristic value
Levy vector.
S130 arranges this feature vector according to preset rules, obtains dimensionality reduction mapping matrix；
S140 generates multiple target visual dictionaries based on the dimensionality reduction mapping matrix and the first combination visual dictionary.
Specifically, the image encrypting algorithm optimization method provided in the embodiment of the present application, obtains training image collection first
First combination visual dictionary.The training image that the training image is concentrated can be to be stored in image in image data base, can wrap
Landscape image, character image, commodity image, Architectural drawing, animal painting and plant image etc. are included, the application does not limit this
System.
The first combination visual dictionary may include multiple visual dictionaries, and such as M visual dictionary, M is just more than or equal to 2
Integer.It may include multiple vision words in each visual dictionary, such as k, k can be the positive integer more than or equal to 1.
Then dimension-reduction treatment can be carried out to the first combination visual dictionary, the first combination can be such as regarded using SVD or PCA
Feel that dictionary carries out dimension-reduction treatment, obtains corresponding multiple characteristic values and feature vector, i.e. k characteristic value and k feature vector.
Further, it is possible to be arranged according to pre-set queueing discipline above-mentioned feature vector, obtains dimensionality reduction and reflect
Penetrate matrix.Finally, operation can be done to obtained dimensionality reduction mapping matrix and the first combination visual dictionary, the view of corresponding number is exported
Dictionary, i.e. target visual dictionary are felt, so that the difference of the variance of the vision word in two neighboring target visual dictionary is smaller, i.e.,
Distributing equilibrium between the variance of the corresponding data set of each target visual dictionary, the distribution being not gradually reduced.
The embodiment of the present application carries out dimension-reduction treatment by combining visual dictionary to obtain training set of images first, and to each
A feature vector is rearranged, so that the variance between the corresponding data set of multiple visual dictionaries based on this generation is balanced
Distribution, solves the technical issues of variance in the corresponding data set of visual dictionary gradually becomes smaller, completes figure retrieval model
Optimization, improves the accuracy of image retrieval.
In order to better understand image encrypting algorithm optimization method provided by the embodiments of the present application, below with reference to Fig. 2 to Fig. 5
Elaborate image encrypting algorithm optimization method provided by the embodiments of the present application.This method comprises:
S210 obtains the visual signature that training image concentrates training image.
Specifically, multiple images can be acquired from image data base, the visual signature of each image is then extracted, that is, is regarded
Feel feature vector.Such as (SIFT) algorithm can be converted based on scale invariant feature, accelerates robust feature (SURF) algorithm or fast
Fast feature extraction and description (ORB) algorithm extract the visual signature of training image.The visual signature can be texture maps feature, side
To histogram of gradients feature, color histogram feature etc..Such as collecting visual feature vector a1, a2, a3 ... ad, d is positive integer
S220, view-based access control model feature generate multiple visual dictionaries.
Specifically, can use clustering algorithm first to each training image of extraction when generating multiple visual dictionaries
Visual signature clustered, the First look dictionary using cluster centre as vision word.It is appreciated that will scheme to each training
The visual signature of picture carries out cluster and obtains the cluster centres of all kinds of clusters as vision word, i.e. code word (vision word vector), more
A vision word vector (code word) forms visual dictionary (subcode book).For example, when cluster centre has 8, i.e., by 8 vision words
Vector then forms a visual dictionary by this 8 vision words, that is, forms the subcode book matrix of a 8 × d.And then it can be true
With above-mentioned visual signature apart from nearest vision word in the fixed visual dictionary, and the vector and visual dictionary of computation vision feature
In each vision word vector distance, with determine apart from nearest vision word.Computation vision feature with it is nearest away from it
The difference of vision word, using calculated visual signature with it is special as new vision away from its difference apart from nearest vision word
Sign, re-starts cluster, and using cluster centre as vision word, and multiple the second visual dictionaries of vision word combination producing.
It goes forward side by side with the distance value away from its second nearest vision word as new visual signature likewise, calculating new visual signature
Row cluster, generates third visual dictionary.
It is appreciated that preset quantity time dictionary, such as M=3 can be generated by the above method.The number of visual dictionary
Amount can be determined according to factors such as scale, the memory sizes of training set of images.
It is further appreciated that is, code word is vision spy due to the vision word in the second visual dictionary and third visual dictionary
The residual error of sign and vision word (cluster centre).Therefore, the corresponding data set of the second visual dictionary (multiple vision word vectors)
Method be less than the variance of the corresponding data set of First look dictionary, the variance of the corresponding data set of third visual dictionary is less than the
The variance of the corresponding data set of two visual dictionaries.
S230 successively combines the vision word in multiple visual dictionaries, generates the first combination visual dictionary.
Specifically, obtaining above-mentioned multiple visual dictionaries, the vector including multiple vision words in each visual dictionary.
Such as 8, i.e., each visual dictionary is the matrix of a k × d.3 above-mentioned matrixes can be then combined, generate first group
It closes visual dictionary (total code book), as shown in Figure 3.
For example, can be by the first of first vector of First look dictionary and the second visual dictionary, third visual dictionary
A vector is sequentially connected, likewise, k-th of vector of M visual dictionary is sequentially connected, so as to form a k × Md
Matrix.
S240 carries out dimension-reduction treatment to the first combination visual dictionary, obtains multiple characteristic values and the corresponding feature of characteristic value
Vector.
Specifically, can obtain the first combination visual dictionary to above-mentioned, i.e., original total code book carries out dimension-reduction treatment, obtains pair
The characteristic value and feature vector answered such as obtain k characteristic value and corresponding k feature vector.
For example, dimension-reduction treatment can be carried out to the first combination visual dictionary using SVD or PCA, the spy of corresponding dimension is obtained
Value indicative and feature vector (feature vector).It is appreciated that in general, Md > k, therefore can be realized using when PCA or SVD
The dimensionality reduction of original matrix.Also, after dimensionality reduction, the characteristic value of output is descending to be arranged successively, and forward biggish feature
Value occupies the main component of population characteristic value.
It, can be only with the corresponding feature of forward multiple characteristic values it is further appreciated that after to original total code book dimensionality reduction
Vector carries out subsequent operation.In this embodiment it is possible to which it is subsequent to acquire the corresponding feature vector progress of l forward characteristic value
Operation, the l are the positive integer less than or equal to k, and are the integral multiple of M.
S250, the size based on characteristic value, arranges feature vector, obtains dimensionality reduction mapping matrix.
Specifically, can according to the size for the characteristic value that dimensionality reduction is got, to the corresponding feature vector of all characteristic values into
Row combination, generates dimensionality reduction mapping matrix.
Optionally, it as shown in figure 4, in combination of eigenvectors group corresponding to all or part of characteristic value, can take
Following methods:
S251 constructs M empty data acquisition system.
The corresponding feature vector of M characteristic value before row is successively sequentially added to corresponding M data acquisition system by S252.
The corresponding feature vector of M+1 characteristic value is added in m-th data acquisition system by S253.
S254 determines the corresponding data acquisition system of current the smallest characteristic value total amount.
S255, judges whether the number of the feature vector in the corresponding data set of current the smallest characteristic value total amount is less than l/
M。
If not, indicating that the data set is full, then the corresponding data acquisition system of current the smallest characteristic value total amount is skipped, and return
It is back to and determines the corresponding data set of current the smallest characteristic value total amount；
S256, if so, the corresponding feature vector of remaining the M+n characteristic value is successively added to the smallest characteristic value
In the corresponding data set of total amount, and it is back to and determines the corresponding data set of current the smallest characteristic value total amount,
Specifically, in conjunction with Fig. 5 the data acquisition system with the same number of M sky of original visual dictionary can be preset first, i.e.,
M empty barrel, the data acquisition system is for being sequentially filled the feature vector that above-mentioned dimension-reduction treatment obtains.
Further, it is possible to characteristic value is arranged successively from big to small according to the size of characteristic value, as shown in figure 5, by greatly to
The characteristic value sequence of small to be followed successively by p1, p2, p3 ... pk.Then the corresponding feature vector of first characteristic value p1 is placed into first
It in a data acquisition system (in first bucket), is arranged successively, the corresponding feature vector of m-th characteristic value can be placed into m-th
In data acquisition system.As shown in figure 5, M is 4, then by the corresponding feature vector addition of p4 in the 4th data acquisition system.At this point, each
There to be a feature vector in data acquisition system.
Then the corresponding data acquisition system of current the smallest characteristic value total amount can be determined.
Herein, characteristic value total amount are as follows: when only including a feature vector in data acquisition system, characteristic value total amount is data set
The corresponding characteristic value of the feature vector being currently included in conjunction；When in data set including multiple feature vectors, characteristic value total amount is
The product of the corresponding characteristic value of the feature vector being currently included in data acquisition system.
It is appreciated that under above situation, due to one and only one feature vector in each data acquisition system, so, it can be with
Determine that characteristic value total amount is the corresponding characteristic value of described eigenvector being currently included in the data acquisition system.And the smallest feature
The corresponding data acquisition system of total amount is m-th.
Therefore, the corresponding feature vector of the M+1 characteristic value can be placed in the data set, in the 4th bucket.Then
Feature vector there are two in M bucket, and one and only one feature vector in remaining bucket.
At this point it is possible to determine that characteristic value total amount is the corresponding spy of described eigenvector being currently included in the data acquisition system
The corresponding characteristic value of described eigenvector being currently included in the product or data acquisition system of value indicative, that is, determine current characteristic value or
The smallest data set of the product of characteristic value can further judge whether feature vector current in the data acquisition system reaches threshold
Value.
Then the corresponding feature vector of remaining M+n (2≤n≤l) a characteristic value is successively added into the smallest characteristic value
In the corresponding not full data acquisition system of total amount, and it is back to the step for determining the corresponding data set of current the smallest characteristic value total amount
Suddenly.
For example, it is assumed that the product (p4 × p5) of the characteristic value in fourth data set is less than in third data acquisition system
Characteristic value further judges whether 2 feature vectors in fourth data set reach preset value.
If preset value is 2, then it represents that the data acquisition system is full, can exclude from all data acquisition systems, i.e. the data
Feature vector will be no longer added in set.At this point it is possible to search the corresponding data set of next currently the smallest characteristic value total amount
It closes.The corresponding feature vector of the M+2 characteristic value can be then placed in the data set by for example third bucket.
If preset value is 3, then it represents that the data acquisition system is not full, at this point it is possible to directly that the M+2 characteristic value is corresponding
Feature vector be placed in the data set.
Further, it is possible to continue to search the corresponding data acquisition system of next current the smallest characteristic value total amount, and by M+3
The corresponding feature vector addition of a characteristic value is in the corresponding not full data acquisition system of the smallest characteristic value total amount.
It is appreciated that as queueing discipline, the selection such as corresponding feature vector of l characteristic value can be placed respectively
In each data acquisition system, so that each data acquisition system finally may include l/M feature vector.Wherein, the l≤k, and l/
M is the integral multiple of M.
Finally, the matrix of above-mentioned M feature vector composition, as dimensionality reduction mapping matrix.
The dimensionality reduction mapping matrix constructed through the above way, the sequence of the corresponding characteristic value of internal feature vector is
It is disturbed, is not according to sequence arrangement from big to small.
S260 generates the second combination visual dictionary based on dimensionality reduction mapping matrix and the first combination visual dictionary.
S270 carries out cutting to the second combination visual dictionary, obtains target visual dictionary.
Specifically, can further combine, obtain with the first combination visual dictionary after obtaining above-mentioned dimensionality reduction mapping matrix
Second combination visual dictionary, i.e., new total code book.And cutting is carried out to obtained new total code book, it can such as be combined according in S230
Mode reverse direction, obtain the new subcode books of M, i.e. target visual dictionary.
It is appreciated that since target visual dictionary is by obtaining to first combination visual dictionary (original total code book) dimensionality reduction
It arrives, so the dimension in each target visual dictionary is less than the dimension of original visual dictionary.
For example, the corresponding matrix of the first combination visual dictionary can be multiplied with dimensionality reduction mapping matrix, the second combination is obtained
Visual dictionary, such as matrix of still available l × Md size.The target visual dictionary of last available M l × d.
It is appreciated that by target visual dictionary obtained by the above method, since new total code book is by original total code book
With obtained by dimensionality reduction mapping matrix product so that according to original dimension cutting, the target visual dictionary of generation comprising multiple numbers
According to the variance of collection by equiblibrium mass distribution, the trend being gradually reduced is avoided, so as to carry out figure using the target visual dictionary
When as retrieval, the accuracy of retrieval can be improved.
Further, this method can also include:
S280 is based on each target visual dictionary, generates the first bag of words vector of each training image.
Specifically, after being optimized by original visual dictionary of the above method to acquisition, after can use optimization
Visual dictionary carries out the further perfect of bag of words.Described image training can be determined based on the target visual dictionary first
The index of each visual signature of concentration training image.Determine the inverse text of the word frequency-of the index of each visual signature of training image
Shelves frequency weight；Finally the instruction can be generated based on the term frequency-inverse document frequency weight of the index of each visual signature
Practice the bag of words vector of image.
That is, determining that the TF-IDF of the index of visual signature is weighed by the cartesian product of M this target visual dictionary
Weight, by the bag of words vector of the TF-IDF weight composition training image of the index of each visual signature of training image.
Further, after optimizing to image encrypting algorithm, the model after can use optimization carries out the embodiment of the present application
Image retrieval, as shown in connection with fig. 6, this method comprises:
S610 obtains the visual signature of image to be retrieved；
S620 is based on the visual signature, using each target visual dictionary, calculate corresponding second bag of words of the visual signature to
Amount；
S630 calculates the second bag of words vector at a distance from the first bag of words vector；
S640 determines the similarity of the image to be retrieved and the training image based on the distance；
S650 determines that the similarity is greater than the training image of predetermined threshold, exports as target image.
Specifically, such as the visual signature of the image to be retrieved, such as scale can be extracted first for some image to be retrieved
Invariant features, acceleration robust feature, color histogram feature or textural characteristics etc..Then according to M this target after above-mentioned optimization
Visual dictionary calculates the TF-IDF weight of the index of the visual signature of image to be retrieved, that is, passing through this visual dictionary of M
Cartesian product determines the TF-IDF weight of visual signature.And then it can be based on the index of each visual signature of image to be retrieved
The bag of words vector for the image to be retrieved that TF-IDF weight obtains.Meanwhile the bag of words vector of available above-mentioned each training image, then
Image to be retrieved can be calculated at a distance from the BoW vector of each training image, can be determined according to calculated distance to be checked
The similitude of rope image and each training image.The training for being greater than preset threshold with the similitude of image to be retrieved can finally be exported
Image, to complete the retrieval of image to be retrieved.
It is appreciated that in the embodiment of the present application, due to by dimension-reduction treatment and the recombination of feature vector to visual dictionary into
Optimization is gone, so that the Energy distribution of visual dictionary is uniform, to calculate by visual dictionary to be checked in image retrieval procedure
When the TF-IDF weight of rope image or training image, the decisive role for the embodiment that each visual dictionary rises is consistent, so that inspection
The accuracy of rope is higher.
Fig. 7 show the structural schematic diagram of image encrypting algorithm optimization device provided by the embodiments of the present application, as shown,
The device includes:
Module 710 is obtained, for obtaining the first combination visual dictionary of training set of images, the first combination visual dictionary packet
Multiple visual dictionaries are included, which includes multiple vision words；
Dimensionality reduction module 720 obtains characteristic value and this feature value for carrying out dimension-reduction treatment to the first combination visual dictionary
Corresponding feature vector；
It arranges module 730 and obtains dimensionality reduction mapping matrix for being arranged according to preset rules this feature vector；
Generation module 740, for generating the second combination view based on the dimensionality reduction mapping matrix and the first combination visual dictionary
Feel dictionary；
Cutting module 750, it is multiple to be somebody's turn to do for obtaining target visual dictionary to the second combination visual dictionary progress cutting
It is in balanced distribution between the variance of the corresponding data set of target visual dictionary.
Optionally, image encrypting algorithm provided by the embodiments of the present application optimizes device, which is specifically used for:
Based on the size of the corresponding this feature value of this feature vector, this feature vector is arranged, the dimensionality reduction is obtained and reflects
Penetrate matrix.
Fig. 8 show the structural schematic diagram of image retrieving apparatus provided by the embodiments of the present application, as shown, the device packet
It includes:
Module 810 is obtained, the visual signature of image to be retrieved is obtained；
It is corresponding to calculate the visual signature using target visual dictionary based on the visual signature for first computing module 820
Second bag of words vector；
Second computing module 830 calculates the second bag of words vector at a distance from such as claim the first bag of words vector；
Determining module 840 determines the similarity of the image to be retrieved and the training image based on the distance；
Output module 850 determines that the similarity is greater than the training image of predetermined threshold, exports as target image.
Below with reference to Fig. 9, it illustrates the computer systems 900 for the server for being suitable for being used to realize the embodiment of the present application
Structural schematic diagram.
As shown in figure 9, computer system 900 includes central processing unit (CPU) 901, it can be read-only according to being stored in
Program in memory (ROM) 902 or be loaded into the program in random access storage device (RAM) 903 from storage section 903 and
Execute various movements appropriate and processing.In RAM 903, also it is stored with system 900 and operates required various programs and data.
CPU 901, ROM 902 and RAM 903 are connected with each other by bus 904.Input/output (I/O) interface 905 is also connected to always
Line 904.
I/O interface 905 is connected to lower component: the importation 906 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 907 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 908 including hard disk etc.；
And the communications portion 909 of the network interface card including LAN card, modem etc..Communications portion 909 via such as because
The network of spy's net executes communication process.Driver 910 is also connected to I/O interface 905 as needed.Detachable media 911, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 910, in order to read from thereon
Computer program be mounted into storage section 908 as needed.
Particularly, the embodiment optimized according to image encrypting algorithm disclosed in the present application, above with reference to Fig. 1, Fig. 2 and Fig. 6
The process of description may be implemented as computer software programs.For example, the implementation of image encrypting algorithm optimization disclosed in the present application
Example includes a kind of computer program product comprising the computer program being tangibly embodied on machine readable media, the calculating
Machine program includes the program code for executing the method for Fig. 2.In such embodiments, which can be by logical
Letter part 909 is downloaded and installed from network, and/or is mounted from detachable media 911.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In this application, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.
And in this application, computer-readable signal media may include passing in a base band or as carrier wave a part
The data-signal broadcast, wherein carrying computer-readable program code.The data-signal of this propagation can use a variety of shapes
Formula, including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media may be used also
To be any computer-readable medium other than computer readable storage medium, which can send, propagate
Either transmission is for by the use of instruction execution system, device or device or program in connection.It is computer-readable
The program code for including on medium can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc.
Deng or above-mentioned any appropriate combination.
Flow chart and block diagram in attached drawing, illustrate and are according to the various image encrypting algorithm optimal enforcement examples of the application
The architecture, function and operation in the cards of system, method and computer program product.In this regard, flowchart or block diagram
In each box can represent a part of a module, program segment or code, the one of aforementioned modules, program segment or code
Part includes one or more executable instructions for implementing the specified logical function.It is replaced it should also be noted that being used as at some
In the realization changed, function marked in the box can also occur in a different order than that indicated in the drawings.For example, two connect
The box even indicated can actually be basically executed in parallel, they can also be executed in the opposite order sometimes, this is according to institute
Depending on the function being related to.It is also noted that in each box and block diagram and or flow chart in block diagram and or flow chart
Box combination, can the dedicated hardware based systems of the functions or operations as defined in executing realize, or can be with
It realizes using a combination of dedicated hardware and computer instructions.
Being described in the embodiment of the present application involved unit or module can be realized by way of software, can also be with
It is realized by way of hardware.Described unit or module also can be set in the processor, for example, can be described as:
A kind of processor includes obtaining module, dimensionality reduction module, arrangement module, generation module and cutting module.Wherein, these units or mould
The title of block does not constitute the restriction to the unit or module itself under certain conditions, for example, arrangement module can also be retouched
It states as " for being arranged according to preset rules described eigenvector, obtaining the module of dimensionality reduction mapping matrix ".
As on the other hand, present invention also provides a kind of computer readable storage medium, the computer-readable storage mediums
Matter can be computer readable storage medium included in aforementioned device in above-described embodiment；It is also possible to individualism, not
The computer readable storage medium being fitted into equipment.Computer-readable recording medium storage has one or more than one journey
Sequence, foregoing routine be used to execute by one or more than one processor be described in the application determination image encrypting algorithm it is excellent
The method of change is specific to execute:
The first combination visual dictionary of training set of images is obtained, the first combination visual dictionary includes multiple visual words
Allusion quotation, the visual dictionary include multiple vision words；
To it is described first combination visual dictionary carry out dimension-reduction treatment, obtain characteristic value and the corresponding feature of the characteristic value to
Amount；
Described eigenvector is arranged according to preset rules, obtains dimensionality reduction mapping matrix；
Based on the dimensionality reduction mapping matrix and the first combination visual dictionary, the second combination visual dictionary is generated；
Cutting is carried out to the second combination visual dictionary, obtains target visual dictionary, multiple target visual dictionaries
It is in balanced distribution between the variance of corresponding data set.
In conclusion a kind of image encrypting algorithm optimization method provided by the embodiments of the present application, search method, device, being
System and medium, by carrying out dimension-reduction treatment to the first combination visual dictionary for obtaining training set of images, and to each feature vector
It is rearranged, so that the equiblibrium mass distribution between the data set variance between multiple visual dictionaries based on this generation, avoids
The distributions of successively decreasing of multiple multiple variances of target visual dictionary, completes the optimization of figure retrieval model, improves image retrieval
Accuracy.
Above description is only the preferred embodiment of the application and the explanation to institute's application technology principle.Those skilled in the art
Member is it should be appreciated that invention scope involved in the application, however it is not limited to technology made of the specific combination of above-mentioned technical characteristic
Scheme, while should also cover in the case where not departing from the inventive concept, it is carried out by above-mentioned technical characteristic or its equivalent feature
Any combination and the other technical solutions formed.Such as features described above has similar function with (but being not limited to) disclosed herein
Can technical characteristic replaced mutually and the technical solution that is formed.
Claims (11)
1. a kind of image encrypting algorithm optimization method, which is characterized in that the described method includes:
The first combination visual dictionary of training set of images is obtained, the first combination visual dictionary includes multiple visual dictionaries, institute
Stating visual dictionary includes multiple vision words；
To it is described first combination visual dictionary carry out dimension-reduction treatment, obtain multiple characteristic values and the corresponding feature of the characteristic value to
Amount；
Described eigenvector is arranged according to preset rules, obtains dimensionality reduction mapping matrix；
Based on the dimensionality reduction mapping matrix and the first combination visual dictionary, multiple target visual dictionaries are generated, it is multiple described
Equiblibrium mass distribution between the variance of the corresponding data set of target visual dictionary.
2. image encrypting algorithm optimization method according to claim 1, it is characterised in that described to be pressed to described eigenvector
It is arranged according to preset rules, obtaining dimensionality reduction mapping matrix includes:
Based on the size of the corresponding characteristic value of described eigenvector, described eigenvector is arranged, obtains the drop
Tie up mapping matrix.
3. image encrypting algorithm optimization method according to claim 2, which is characterized in that described to be based on described eigenvector
The size of corresponding characteristic value, carrying out arrangement to the feature vector in described eigenvector includes:
M empty data acquisition system of building, the quantity of the data acquisition system of the sky combine the vision in visual dictionary with described first
The quantity of dictionary is equal；
The corresponding feature vector of M characteristic value before row is successively sequentially added in the corresponding M data acquisition systems, it is described
Characteristic value is arranged according to the sequence successively decreased；
The corresponding feature vector of M+1 characteristic value is added in m-th data acquisition system；
The corresponding data acquisition system of current the smallest characteristic value total amount is determined, when only including a feature vector in the data acquisition system
When, the characteristic value total amount is the corresponding characteristic value of described eigenvector being currently included in the data acquisition system；Work as data set
Interior when including multiple feature vectors, the characteristic value total amount is that the described eigenvector that is currently included is corresponding in the data acquisition system
Characteristic value product；
Judge whether the number of the feature vector in the corresponding data acquisition system of the smallest characteristic value total amount is less than l/M, wherein
The corresponding feature vector of l characteristic value before row is feature vector to be arranged, and the l is less than or equal to each visual dictionary
The number of interior vision word, and l/M is the integral multiple of M；
If not, indicating that the data acquisition system is full, then the corresponding data acquisition system of current the smallest characteristic value total amount is skipped, and return
To the step of determining current the smallest characteristic value total amount corresponding data set；
If so, it is corresponding that the corresponding feature vector of remaining the M+n characteristic value is successively added the smallest characteristic value total amount
In not full data acquisition system, and it is back to the step of determining current the smallest characteristic value total amount corresponding data set, so that often
It include l/M feature vector in a data set, wherein 2≤n≤l.
4. image encrypting algorithm optimization method according to claim 1, which is characterized in that described to obtain the of training image collection
One, which combines visual dictionary, includes:
Obtain the visual signature that training image concentrates training image；
Multiple visual dictionaries are generated based on the visual signature；
The vision word in multiple visual dictionaries is successively combined, the first combination visual dictionary is obtained；
Described to be based on the dimensionality reduction mapping matrix and the first combination visual dictionary, generating multiple target visual dictionaries includes:
The second combination visual dictionary is generated based on the dimensionality reduction mapping matrix and the first combination visual dictionary；
Cutting is carried out to the second combination visual dictionary, obtains the target visual dictionary.
5. image encrypting algorithm optimization method according to claim 1, which is characterized in that the method also includes:
Determine that the training image concentrates the index of each visual signature of training image based on the target visual dictionary；
Determine the term frequency-inverse document frequency weight of the index of each visual signature of the training image；
The term frequency-inverse document frequency weight of index based on each visual signature generates the first word of the training image
Bag vector.
6. a kind of image search method, which is characterized in that the described method includes:
Obtain the visual signature of image to be retrieved；
It is special to calculate the vision using target visual dictionary as described in any one in claim 1-5 based on the visual signature
Levy corresponding second bag of words vector；
The second bag of words vector is calculated at a distance from the first bag of words vector as claimed in claim 5；
The similarity of the image to be retrieved and the training image is determined based on the distance；
It determines that the similarity is greater than the training image of predetermined threshold, is exported as target image.
7. a kind of image encrypting algorithm optimizes device, which is characterized in that described device includes:
Module is obtained, for obtaining the first combination visual dictionary of training set of images, the first combination visual dictionary includes more
A visual dictionary, the visual dictionary include multiple vision words；
Dimensionality reduction module obtains multiple characteristic values and the feature for carrying out dimension-reduction treatment to the first combination visual dictionary
It is worth corresponding feature vector；
It arranges module and obtains dimensionality reduction mapping matrix for being arranged according to preset rules described eigenvector；
Generation module, for generating multiple target visuals based on the dimensionality reduction mapping matrix and the first combination visual dictionary
Dictionary, equiblibrium mass distribution between the variance of the corresponding data set of the multiple target visual dictionaries.
8. image encrypting algorithm according to claim 7 optimizes device, which is characterized in that the arrangement module is specifically used
In:
Based on the size of the corresponding characteristic value of described eigenvector, described eigenvector is arranged, obtains the drop
Tie up mapping matrix.
9. a kind of image retrieving apparatus, which is characterized in that described device includes:
Module is obtained, the visual signature of image to be retrieved is obtained；
First computing module is based on the visual signature, using target visual dictionary as described in any one in claim 1-5,
Calculate the corresponding second bag of words vector of the visual signature；
Second computing module calculates the second bag of words vector at a distance from the first bag of words vector as claimed in claim 5；
Determining module determines the similarity of the image to be retrieved and the training image based on the distance；
Output module determines that the similarity is greater than the training image of predetermined threshold, exports as target image.
10. a kind of computer system, can run on a memory and on a processor including memory, processor and storage
Computer program, which is characterized in that the processor realizes side as described in any one in claim 1-5 when executing described program
Method or method as claimed in claim 6.
11. a kind of computer readable storage medium is stored thereon with computer program, the computer program is for realizing such as power
Benefit requires described in any item methods of 1-5 or method as claimed in claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910161195.0A CN109902190B (en) | 2019-03-04 | 2019-03-04 | Image retrieval model optimization method, retrieval method, device, system and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910161195.0A CN109902190B (en) | 2019-03-04 | 2019-03-04 | Image retrieval model optimization method, retrieval method, device, system and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109902190A true CN109902190A (en) | 2019-06-18 |
| CN109902190B CN109902190B (en) | 2021-04-27 |
Family
ID=66946278
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910161195.0A Active CN109902190B (en) | 2019-03-04 | 2019-03-04 | Image retrieval model optimization method, retrieval method, device, system and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109902190B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111522795A (en) * | 2020-04-23 | 2020-08-11 | 北京互金新融科技有限公司 | Method and device for processing data |
| CN111986089A (en) * | 2020-08-28 | 2020-11-24 | 计易数据科技(上海)有限公司 | Image storage and comparison method, device, equipment and medium with characteristic value being integer |
| CN114722226A (en) * | 2022-06-09 | 2022-07-08 | 深圳大学 | Adaptive retrieval method and device capable of matching images and storage medium |
| US11620829B2 (en) * | 2020-09-30 | 2023-04-04 | Snap Inc. | Visual matching with a messaging application |
| CN116506775A (en) * | 2023-05-22 | 2023-07-28 | 广州市声讯电子科技股份有限公司 | Distributed loudspeaker array arrangement point selection and optimization method and system |
| CN116932802A (en) * | 2023-07-10 | 2023-10-24 | 上海鱼微阿科技有限公司 | Image retrieval method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110093794A1 (en) * | 2008-04-30 | 2011-04-21 | Mark Grace | Mobile electronic communications dialog using sequenced digital images |
| CN102982165A (en) * | 2012-12-10 | 2013-03-20 | 南京大学 | Large-scale human face image searching method |
| CN103020265A (en) * | 2012-12-25 | 2013-04-03 | 深圳先进技术研究院 | Image retrieval method and system |
| CN103838864A (en) * | 2014-03-20 | 2014-06-04 | 北京工业大学 | Visual saliency and visual phrase combined image retrieval method |
| CN104680179A (en) * | 2015-03-09 | 2015-06-03 | 西安电子科技大学 | Data dimension reduction method based on neighborhood similarity |
| CN108647307A (en) * | 2018-05-09 | 2018-10-12 | 京东方科技集团股份有限公司 | Image processing method, device, electronic equipment and storage medium |
-
2019
- 2019-03-04 CN CN201910161195.0A patent/CN109902190B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110093794A1 (en) * | 2008-04-30 | 2011-04-21 | Mark Grace | Mobile electronic communications dialog using sequenced digital images |
| CN102982165A (en) * | 2012-12-10 | 2013-03-20 | 南京大学 | Large-scale human face image searching method |
| CN103020265A (en) * | 2012-12-25 | 2013-04-03 | 深圳先进技术研究院 | Image retrieval method and system |
| CN103838864A (en) * | 2014-03-20 | 2014-06-04 | 北京工业大学 | Visual saliency and visual phrase combined image retrieval method |
| CN104680179A (en) * | 2015-03-09 | 2015-06-03 | 西安电子科技大学 | Data dimension reduction method based on neighborhood similarity |
| CN108647307A (en) * | 2018-05-09 | 2018-10-12 | 京东方科技集团股份有限公司 | Image processing method, device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 张运超: "面向海量图像检索的视觉编码方法分析与优化", 《中国博士学位论文全文数据库信息科技辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111522795A (en) * | 2020-04-23 | 2020-08-11 | 北京互金新融科技有限公司 | Method and device for processing data |
| CN111986089A (en) * | 2020-08-28 | 2020-11-24 | 计易数据科技(上海)有限公司 | Image storage and comparison method, device, equipment and medium with characteristic value being integer |
| US11620829B2 (en) * | 2020-09-30 | 2023-04-04 | Snap Inc. | Visual matching with a messaging application |
| US11823456B2 (en) | 2020-09-30 | 2023-11-21 | Snap Inc. | Video matching with a messaging application |
| CN114722226A (en) * | 2022-06-09 | 2022-07-08 | 深圳大学 | Adaptive retrieval method and device capable of matching images and storage medium |
| CN116506775A (en) * | 2023-05-22 | 2023-07-28 | 广州市声讯电子科技股份有限公司 | Distributed loudspeaker array arrangement point selection and optimization method and system |
| CN116506775B (en) * | 2023-05-22 | 2023-10-10 | 广州市声讯电子科技股份有限公司 | Distributed loudspeaker array arrangement point selection and optimization method and system |
| CN116932802A (en) * | 2023-07-10 | 2023-10-24 | 上海鱼微阿科技有限公司 | Image retrieval method |
| CN116932802B (en) * | 2023-07-10 | 2024-05-14 | 玩出梦想(上海)科技有限公司 | Image retrieval method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109902190B (en) | 2021-04-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902190A (en) | Image encrypting algorithm optimization method, search method, device, system and medium | |
| US11967152B2 (en) | Video classification model construction method and apparatus, video classification method and apparatus, device, and medium | |
| CN110503192A (en) | The effective neural framework of resource | |
| CN111414953B (en) | Point cloud classification method and device | |
| CN108647307A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN109117781A (en) | Method for building up, device and the more attribute recognition approaches of more attribute Recognition Models | |
| CN110490203A (en) | Image partition method and device, electronic equipment and computer readable storage medium | |
| CN109740674A (en) | A kind of image processing method, device, equipment and storage medium | |
| CN117616424A (en) | Systems and methods for balancing sparsity in weights for accelerating deep neural networks | |
| CN109359727A (en) | Structure determination methodology, device, equipment and the readable medium of neural network | |
| CN109446379A (en) | Method and apparatus for handling information | |
| WO2021062219A1 (en) | Clustering data using neural networks based on normalized cuts | |
| WO2021012691A1 (en) | Method and device for image retrieval | |
| CN116883740A (en) | Similar picture identification method, device, electronic equipment and storage medium | |
| CN110209895A (en) | Vector index method, apparatus and equipment | |
| CN110826726B (en) | Target processing method, target processing device, target processing apparatus, and medium | |
| CN115035408A (en) | Unmanned aerial vehicle image tree species classification method based on transfer learning and attention mechanism | |
| CN112016599A (en) | Neural network training method and device for image retrieval and electronic equipment | |
| US11313694B2 (en) | Method and apparatus for recommending travel way | |
| US20230368030A1 (en) | Block-wise pruning of weights in deep neural network | |
| US20230376765A1 (en) | Performing operation in neural network with storage pointer and sparsity map | |
| EP4357978A1 (en) | Deep neural network (dnn) accelerator facilitating quantized inference | |
| CN115827876B (en) | Method and device for determining unlabeled text and electronic equipment | |
| CN109739980A (en) | Emotion classifiers are carried out with the method, apparatus and terminal of tuning | |
| US20230325665A1 (en) | Sparsity-based reduction of gate switching in deep neural network accelerators |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |