CN109766468A - A kind of implementation method and device retrieved in appearance patent image based on iamge description algorithm with management - Google Patents
A kind of implementation method and device retrieved in appearance patent image based on iamge description algorithm with management Download PDFInfo
- Publication number
- CN109766468A CN109766468A CN201910007692.5A CN201910007692A CN109766468A CN 109766468 A CN109766468 A CN 109766468A CN 201910007692 A CN201910007692 A CN 201910007692A CN 109766468 A CN109766468 A CN 109766468A
- Authority
- CN
- China
- Prior art keywords
- image
- appearance patent
- library
- patent image
- appearance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The implementation method and device that the invention discloses a kind of to be retrieved based on iamge description algorithm and be managed in appearance patent image, belong to the technical field of artificial intelligence.Method of the invention collects appearance patent image first, then manually carries out high-level semantics mark to image；It is then based on iamge description algorithm, convolutional neural networks are trained using appearance patent image, the vector coding generated after training is input to Recognition with Recurrent Neural Network decoding, so that whole network study is marked to image with corresponding high-level semantics, realizes search function with this.The device of the invention, including appearance patent image library, image description data collection, CNN study module, RNN decoder module, pictograph describe library, searching, managing module.Present invention utilizes the iamge description algorithms of current deep learning to realize the retrieval to appearance patent image based on high-level semantic on the basis of completing computer to the description of appearance patent image generative semantics.
Description
Technical field
The present invention relates to the technical field of artificial intelligence, particularly a kind of application image identification technology retrieval and management
Implementation method and device.
Technical background
At present for the technology of image retrieval, it is broadly divided into two kinds of retrieval schemes: based on text and based on content.
Text based image retrieval technologies (TBIR) are to be infused and explained the hand of keyword to image by artificial mark
Section, so by text and picture establish it is corresponding contact, thus the problem of image retrieval turns during database retrieval
The problem of turning to text key search, due to not needing to do a large amount of contrast conting in retrieving, this retrieval
Method speed is fast, but this method is manually marked manually one by one and contacted with data volume foundation, institute since height relies on
To be not suitable for the application of the appearance patent image retrieval of large data sets, the application that the database images suitable for small data quantity are retrieved
Scene does not have semantic relation, therefore can not meet semantic inspection simultaneously because what is explained is relatively independent keyword between words
The accurate demand of rope, so can occur the case where retrieval is not desired image often, therefore for the appearance patent of big data quantity
Image, TBIR are unable to satisfy the Search Requirement of big data quantity, simultaneously for intellectual property data library it is efficient management and classification not
It is too applicable universal.
Content-based image retrieval technology (CBIR) be current image retrieval major technique and current appearance patent
The most important application technology of image retrieval.It is the inspection of text keyword that TBIR, which is totally different from, by indirect reformer the problem of image retrieval
The method of rope, CBIR directly using image as ' foundation ' of retrieval, are realized really in the form of scheming to search figure, while without to figure
Piece is labeled, therefore avoids the cost problem that repetition largely manually marks；And what CBIR utilized is image content
The visions low-level image feature such as color, texture, shape, therefore compared with TBIR, the mankind in retrieving have been bypassed on certain depth have been managed
Existing semantic gap between solution and machine reading retrieves data content with data content, therefore the accuracy of CBIR mentions greatly
It rises, is widely used in a variety of applications.But CBIR technology is limited by picture photo angle, looks after intensity, circumstance of occlusion and deformation
The influence of the factors such as degree, there are biggish otherness, easy duplicity for the result of retrieval.CBIR is the view based on low level simultaneously
Feel that characteristic similarity determines, therefore do not have the high-level ability for having perception judgement to picture material, so that system is deposited
In a bottleneck, that is, the similitude that it and the mankind obtain from advanced image, semantic feature judges there are huge gap,
Namely semantic gap problem.Although saving the time that mark needs in a manner of " searching figure to scheme " based on CBIR, but
It is difficult to overcome real semantic gap.Although can satisfy accurate efficient image retrieval problem, for a large amount of of data
Management does not have too big advantage.
Summary of the invention
In order to solve the problems, such as one or more in the presence of background technique, the present invention provides one kind to be retouched based on image
State implementation method and device that algorithm is retrieved in appearance patent image with managed, iamge description of this method based on deep learning
Technology.Specific technical solution is as follows.
A kind of implementation method retrieved in appearance patent image based on iamge description algorithm with management, including walk as follows
It is rapid:
S1, appearance patent image is collected to form image library, the method manually marked is by each appearance patent image
Advanced sentence label is carried out, then whole sentences label is put together to form appearance patent image descriptor data set；
S2, it is based on iamge description algorithm, using convolutional neural networks from the image library in S1 to each appearance patent figure
As extracting foundation characteristic, vector coding is obtained after being learnt, each coding vector is directed to appearance patent iamge description in S1
The advanced sentence mark of one of data set；
S3, whole vector codings input Recognition with Recurrent Neural Network that S2 is obtained is decoded, the figure that generation is described with sentence
As verbal description library；
S4, it is marked using the advanced sentence of appearance patent image description data collection in S1, compares pictograph description in S3
The sentence in library is calibrated, and search function module is generated.
Further, in S2, from the beginning training convolutional neural networks obtain sub-neural network frame first；Then with migration
The mode of study continues training convolutional neural networks on the basis of sub-neural network frame, obtains coding vector.
Further, library is described from the image data in S3, generates two functional modules of data management and property right protection.
Compared with the prior art, the technical program has the technical effect that
The technical program is different from TBIR or CBIR and is only applicable to relatively independent retrieval application, and current depth is utilized
The iamge description algorithm of habit is realized on the basis of completing computer to the description of appearance patent image generative semantics to appearance
Retrieval of the patent image based on high-level semantic, while meeting except opposite search function, go back what benefit in turn was generated algorithmically by
Semantic description is improved to the efficient classification and management of appearance patent image and appearance patent image in intellectual property big data
Protection and maintenance needs during, provide effectively credible available data according to and support.
A kind of realization device that iamge description algorithm is retrieved and managed in appearance patent image, including including appearance patent
Image library, image description data collection, CNN study module, RNN decoder module, pictograph describe library, searching, managing module；Outside
See the raw data base that patent image library is whole device；Each appearance patent image in appearance patent image library is carried out
After advanced sentence mark, image description data collection, the advanced sentence database that image description data integrates as whole device are formed；
CNN study module is based on iamge description algorithm, obtains material from appearance patent image library, is extracted by the way of transfer learning outer
Convolution is carried out after seeing the foundation characteristic of patent image, then exports coding vector corresponding with appearance patent image foundation characteristic；
RNN decoder module obtains the vector coding of CNN study module output, and is decoded to vector coding, and text is generated after decoding
It describes and compares image descriptor data set to be calibrated；Pictograph describes library and collects image after calibration by RNN decoder module
Descriptive statement is established；Searching, managing module describes library as intermediate match data source, with appearance patent image library using pictograph
For search result, intermediate match data source is directed toward corresponding search result.
Compared with the prior art, the technical program has the technical effect that
Compared to the gopher under TBIR or CBIR independent retrieval mode, the technical program is carried out using advanced sentence
The retrieval for carrying out appearance patent image avoids and key search or low to scheme to search the accuracy of figure is used alone, retrieval knot
The big disadvantage of fruit range.
Detailed description of the invention
The content of Figure of description is tentatively illustrated below.
Fig. 1 is the frame for the realization device that the iamge description algorithm of the technical program is retrieved and managed in appearance patent image
Structure schematic diagram；
Fig. 2 is the work for the implementation method that the iamge description algorithm of the technical program is retrieved and managed in appearance patent image
Make flow chart；
In figure, CNN is the abbreviation of convolutional neural networks, and RNN is the abbreviation of Recognition with Recurrent Neural Network.
Specific embodiment
The content that book attached drawing 1 and attached drawing 2 is described below is combined together, and does the embodiment of the technical program into one
Step illustrates.
A kind of implementation method and device retrieved in appearance patent image based on iamge description algorithm with management, the realization
The step of method and corresponding realization device are as follows:
First, establish the appearance patent image descriptor data set of appropriate amount；It is special to form appearance to collect appearance patent image
Sharp image library, manually carries out high-level semantic tagger to each appearance patent image, and every mark is comprising basis view
Feel the sentence of feature, after carrying out high-level semantic tagger, all marks are saved as into descriptor data set.
Second, according to iamge description algorithm, use appearance patent image library training convolutional neural networks module from the beginning；
Instruction is treated after extracting feature to each appearance patent image using the convolutional neural networks (CNN) based on iamge description algorithm
Experienced CNN network layer carries out suitable random initializtion, designs suitable loss function, inputs foundation characteristic, utilizes migration
The mode of study first learns sub-neural network out in CNN network, then continues training convolutional neural networks with sub-neural network, connects
The corresponding coding vector of output, each coding vector be directed to a high-level semantic tagger；
Coding vector input Recognition with Recurrent Neural Network module (RNN) that training obtains is decoded, obtains after decoding by third
High-level descriptive statement；The appearance patent image descriptor data set completed before control both maps to every mark corresponding
It decodes and carries out semantic fine tuning on text, form pictograph and describe library；
4th, decoded high-level descriptive statement in library is described using pictograph, establishes high-level descriptive statement and outer
The mapping relations of patent image basis feature are seen, search function module is generated.
5th, it describes to call appearance patent image in library from pictograph, generates two data management, property right protection function
It can module.
Effect of this embodiment is that the iamge description algorithm of current deep learning is utilized, computer is being completed
On the basis of the description of appearance patent image generative semantics, the retrieval to appearance patent image based on high-level semantic is realized, simultaneously
Meeting except opposite search function, also and then the sharp semantic description being generated algorithmically by is improved to the efficient of appearance patent image
Classification during protection and maintenance needs in intellectual property big data, has been provided with management and appearance patent image
Imitate credible available data foundation and support.Wherein semantic retrieval principle is different similar to text based keyword retrieval
In key search, the iamge description that algorithm generates can be realized the Search Requirement of the high-level semantic of image, will be more accurate.
Above embodiments are the basic principles for illustrating the technical program, are not exactly that the whole of the technical program implement
Mode.To those skilled in the art, any based on the limited content of claims, original according to the technical program
Made conventional displacement improves, and each falls within protection scope of the present invention.
Claims (4)
1. a kind of implementation method retrieved in appearance patent image based on iamge description algorithm with management, which is characterized in that packet
Include following steps:
S1, appearance patent image is collected to form image library, the method manually marked carries out each appearance patent image
Advanced sentence label, then whole sentences label is put together to form appearance patent image descriptor data set；
S2, it is based on iamge description algorithm, each appearance patent image is mentioned from the image library in S1 using convolutional neural networks
Foundation characteristic is taken, vector coding is obtained after being learnt, each coding vector is directed to appearance patent image description data in S1
The advanced sentence mark of one of collection；
S3, whole vector codings input Recognition with Recurrent Neural Network that S2 is obtained is decoded, the image text that generation sentence describes
Word description library；
S4, it is marked using the advanced sentence of appearance patent image description data collection in S1, compares pictograph in S3 and describe library
Sentence is calibrated, and search function module is generated.
2. the realization side retrieved and managed in appearance patent image based on iamge description algorithm described according to claim 1
Method, it is characterised in that: in S2, from the beginning training convolutional neural networks obtain sub-neural network frame first；Then it is learned with migration
The mode of habit continues training convolutional neural networks on the basis of sub-neural network frame, obtains coding vector.
3. the realization side retrieved and managed in appearance patent image based on iamge description algorithm according to claim 2
Method, it is characterised in that: describe library from the image data in S3, generate two functional modules of data management and property right protection.
4. a kind of iamge description algorithm according to implementation method described in claim 3 is retrieved and management in appearance patent image
Realization device, it is characterised in that: decode mould including appearance patent image library, image description data collection, CNN study module, RNN
Block, pictograph describe library, searching, managing module；Appearance patent image library is the raw data base of whole device；It is special to appearance
After each appearance patent image in sharp image library carries out advanced sentence mark, image description data collection, iamge description are formed
Data set is the advanced sentence database of whole device；CNN study module is based on iamge description algorithm, from appearance patent image library
Material is obtained, carries out convolution after the foundation characteristic of appearance patent image is extracted by the way of transfer learning, is then exported and outer
See the corresponding coding vector of patent image basis feature；RNN decoder module obtains the vector coding of CNN study module output, and
Vector coding is decoded, verbal description is generated after decoding and compares image descriptor data set and is calibrated；Pictograph is retouched
It states library and iamge description sentence is collected after calibration to establish by RNN decoder module；Searching, managing module describes library with pictograph
For intermediate match data source, using appearance patent image library as search result, intermediate match data source is directed toward corresponding search result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910007692.5A CN109766468A (en) | 2019-01-04 | 2019-01-04 | A kind of implementation method and device retrieved in appearance patent image based on iamge description algorithm with management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910007692.5A CN109766468A (en) | 2019-01-04 | 2019-01-04 | A kind of implementation method and device retrieved in appearance patent image based on iamge description algorithm with management |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109766468A true CN109766468A (en) | 2019-05-17 |
Family
ID=66452588
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910007692.5A Withdrawn CN109766468A (en) | 2019-01-04 | 2019-01-04 | A kind of implementation method and device retrieved in appearance patent image based on iamge description algorithm with management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109766468A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111782853A (en) * | 2020-06-23 | 2020-10-16 | 西安电子科技大学 | Semantic image retrieval method based on attention mechanism |
| WO2021008213A1 (en) * | 2019-07-12 | 2021-01-21 | 智慧芽信息科技(苏州)有限公司 | Image database establishing method, searching method, electronic device, and storage medium |
-
2019
- 2019-01-04 CN CN201910007692.5A patent/CN109766468A/en not_active Withdrawn
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021008213A1 (en) * | 2019-07-12 | 2021-01-21 | 智慧芽信息科技(苏州)有限公司 | Image database establishing method, searching method, electronic device, and storage medium |
| CN111782853A (en) * | 2020-06-23 | 2020-10-16 | 西安电子科技大学 | Semantic image retrieval method based on attention mechanism |
| CN111782853B (en) * | 2020-06-23 | 2022-12-02 | 西安电子科技大学 | Semantic image retrieval method based on attention mechanism |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110298037B (en) | Convolutional neural network matching text recognition method based on enhanced attention mechanism | |
| US9965717B2 (en) | Learning image representation by distilling from multi-task networks | |
| CN112163122B (en) | Method, device, computing equipment and storage medium for determining label of target video | |
| WO2016095487A1 (en) | Human-computer interaction-based method for parsing high-level semantics of image | |
| CN114840747B (en) | News recommendation method based on contrast learning | |
| CN112257661A (en) | Identification method, device and equipment of vulgar image and computer readable storage medium | |
| Zhang et al. | A comprehensive survey on computational aesthetic evaluation of visual art images: Metrics and challenges | |
| CN112836702B (en) | Text recognition method based on multi-scale feature extraction | |
| CN116975615A (en) | Task prediction method and device based on video multi-mode information | |
| Nie et al. | Perceptual attributes optimization for multivideo summarization | |
| CN112115253A (en) | Depth text ordering method based on multi-view attention mechanism | |
| Valenzise et al. | Advances and challenges in computational image aesthetics | |
| Takimoto et al. | Image aesthetics assessment based on multi-stream CNN architecture and saliency features | |
| CN109766468A (en) | A kind of implementation method and device retrieved in appearance patent image based on iamge description algorithm with management | |
| Han et al. | 3d shape temporal aggregation for video-based clothing-change person re-identification | |
| CN111680190A (en) | Video thumbnail recommendation method fusing visual semantic information | |
| CN114168773A (en) | Semi-supervised sketch image retrieval method based on pseudo label and reordering | |
| Spyrou et al. | Semantic multimedia analysis and processing | |
| Adnan et al. | Automated image annotation with novel features based on deep ResNet50-SLT | |
| Abdulmunem et al. | Semantic based video retrieval system: survey | |
| CN117009570A (en) | Image-text retrieval method and device based on position information and confidence perception | |
| CN117173715A (en) | Attention visual question-answering method and device, electronic equipment and storage medium | |
| CN116049434A (en) | Construction method and device of power construction safety knowledge graph and electronic equipment | |
| Huang et al. | Modeling multiple aesthetic views for series photo selection | |
| CN118155214B (en) | Prompt learning method, image classification method and related devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20190517 |
|
| WW01 | Invention patent application withdrawn after publication |