CN110517328A - It is a kind of based on related double application methods of the self-encoding encoder in zero degree in study - Google Patents

It is a kind of based on related double application methods of the self-encoding encoder in zero degree in study Download PDF

Info

Publication number
CN110517328A
CN110517328A CN201910629751.2A CN201910629751A CN110517328A CN 110517328 A CN110517328 A CN 110517328A CN 201910629751 A CN201910629751 A CN 201910629751A CN 110517328 A CN110517328 A CN 110517328A
Authority
CN
China
Prior art keywords
semantic feature
feature
self
visual signature
model needs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910629751.2A
Other languages
Chinese (zh)
Other versions
CN110517328B (en
Inventor
姜明
刘志勇
张旻
汤景凡
吴俊磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN201910629751.2A priority Critical patent/CN110517328B/en
Publication of CN110517328A publication Critical patent/CN110517328A/en
Application granted granted Critical
Publication of CN110517328B publication Critical patent/CN110517328B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/48—Extraction of image or video features by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention discloses a kind of based on related double application methods of the self-encoding encoder in zero degree in study.The present invention establishes self-encoding encoder to visual signature and semantic feature respectively.But the two self-encoding encoders be not it is independent, they be it is associated, the feature that we obtain after encoding visual signature be added to coding after semantic feature on, then this semantic feature after being added is decoded again.Finally by this decode after obtained semantic feature is added acquisition more preferably more complete semantic feature with former semantic feature.Semantic feature after this is optimized, which re-maps, does Classification and Identification on visual signature.The present invention optimizes semantic feature using related double self-encoding encoder models, and acquisition more has discrimination, more fine-grained semantic feature.Semantic feature after the optimization obtained in this way re-maps visual signature spatially, can obtain better Classification and Identification accuracy rate.

Description

It is a kind of based on related double application methods of the self-encoding encoder in zero degree in study
Technical field
The present invention has used related double self-encoding encoders in zero degree in study, belongs to zero degree learning art field, and in particular to It is a kind of based on related double application methods of the self-encoding encoder in zero degree in study.
Background technique
In the research of existing related zero degree study, attention be more placed on image visual signature and each class In mapping relations between semantic feature.However, feature itself can also have important influence to final Classification and Identification.Especially Semantic feature, somewhat like classification, their character representation is very close, and the discrimination between classification is little.In addition to this, If classification span is bigger, the case where there is also feature redundancy.Therefore, construction more has discrimination, more fine-grained semanteme Feature has very important significance.
Summary of the invention
The present invention more has discrimination to construct, more fine-grained semantic feature, proposes a kind of based on related double self-editing Application method of the code device in zero degree in study, a kind of side optimizing semantic feature based on the model framework of related double self-encoding encoders Method.
For technical purpose more than realization, the present invention will take technical solution below to include the following steps:
1. a kind of based on related double application methods of the self-encoding encoder in zero degree in study, it is characterised in that including walking as follows It is rapid:
Step (1) obtains the visual signature after coding；
Step (2) obtains the semantic feature after optimization；
Semantic feature after optimization is mapped to visual signature by step (3)；
Wherein step (2) is realized by following process:
Step A establishes self-encoding encoder to semantic feature, and the visual signature after encoding obtained in step (1) is added to coding In semantic feature afterwards, then this semantic feature after being added is decoded again；Step B by this decode after obtained language Adopted feature is added the more preferable more complete semantic feature of acquisition with former semantic feature.
Step (1) is implemented as follows:
Visual signature is handled by a self-encoding encoder, the feature V after being encodede；Formula expression are as follows:
Ve=σ (VW1+b1)
Wherein, VeWhat is indicated is the feature after visual signature coding, and what V was indicated is original visual feature, and what σ was indicated is Relu activation primitive；W1It is the parameter matrix that model needs training；b1It is the offset parameter that model needs training；
Decoding process is indicated with following formula:
Vd=σ (VeW2+b2)
Wherein, VdThat indicate is VeVisual signature after decoding and rebuilding, W2It is the parameter matrix that model needs training；b2It is Model needs the offset parameter of training.
Step A specifically:
First semantic feature is encoded:
Ae=σ (AW3+b3)
AeThat indicate is the attribute spy after semantic feature coding, and what A was indicated is original semantic feature, and what σ was indicated is Relu activation primitive；W3It is the parameter matrix that model needs training；b3It is the offset parameter that model needs training；
Then by the visual signature V after obtained codingeIt is mapped to obtain Vae,VaeDimension and coding after attribute it is special Levy AeDimension it is the same；Two full articulamentum+Relu activation primitive layers are used in mapping process, are formulated as follows:
B=σ (VeW4+b4)
Vae=σ (BW5+b5)
Wherein B is VeThe value that first Relu+FC block obtains, VaeIt is VeMap obtained end value；W4And W5Be model Need trained parameter matrix；b4And b5It is the offset parameter that model needs training；Then the V that will be obtainedaeWith the category after coding Property feature AeIt is added, obtains a new attributive character A 'e, this new attributive character A 'eThe influence of visual signature is received, Include more information；What σ was indicated is Relu activation primitive；
A′e=Ae+Vae
Decoded process is indicated with following formula:
Ad=σ (A 'eW6+b6)
Wherein AdThat indicate is the attributive character after rebuilding, W6It is the parameter matrix that model needs training；b6It is model needs Trained offset parameter.
Step B specifically: original semantic feature is added the semanteme after being optimized with decoded semantic feature Feature；
Aall=A+Ad
AallSemantic feature after indicating optimization.
Step (3) specifically: the semantic feature after the optimization for obtaining step (2) is swashed by two full articulamentum+Relu Function layer living is mapped to visual signature space, and mapping relations formula is expressed as follows:
C=σ (AallW7+b7)
Vall=σ (CW8+b8)
Wherein, C is AallThe value that first Relu+FC block obtains, VallIt is AallMap obtained end value；
W7And W8With the parameter matrix for being model needs training；b7And b8It is the offset parameter that model needs training.
There is three constraint conditions for model of the present invention: firstly the need of establish two about the two from coding constraint items Part；That is:
min||V-Vd||2
min||A-Ad||2
The last one constraint condition is exactly that the semantic feature after finally obtained optimization is mapped to initial visual signature； That is:
min||Vall-V||2
The parameter of entire model establishes loss function under the action of these constraint conditions, and constantly training obtains optimal Training result.
The present invention has the beneficial effect that:
The present invention optimizes semantic feature using related double self-encoding encoder models, and acquisition more has discrimination, more particulate The semantic feature of degree.Semantic feature after the optimization obtained in this way re-maps visual signature spatially, can obtain and preferably divide Class recognition accuracy.
Detailed description of the invention
Fig. 1 is flow chart of the present invention.
Fig. 2 is present invention building visual signature self-encoding encoder illustraton of model.
Fig. 3 is the semantic feature illustraton of model after the present invention is optimized.
Fig. 4 is mapping relations figure of the present invention.
Specific embodiment
Present invention will be further explained below with reference to the attached drawings and examples.
As shown in Figure 1, it is a kind of based on related double application methods of the self-encoding encoder in zero degree in study, include the following steps:
Step (1) obtains the visual signature after coding；
Step (2) obtains the semantic feature after optimization；
Semantic feature after optimization is mapped to visual signature by step (3)；
Step (2) obtains as follows: as shown in Fig. 2, step A establishes self-encoding encoder to semantic feature, by step (1) visual signature after encoding obtained in is added in the semantic feature after encoding, then after being added semantic special to this again Sign is decoded；Step B by this decode after obtained semantic feature is added acquisition more preferably more complete language with former semantic feature Adopted feature.
The technical solution that the present invention further limits are as follows:
Step (1) specifically: visual signature is handled by a self-encoding encoder, the feature V after being encodede。 This process can be expressed simply with following formula:
Ve=σ (VW1+b1)
Wherein VeWhat is indicated is after visual signature encodes as a result, V expression is original visual signature.Decoding process can To be indicated with following formula:
Vd=σ (VeW2+b2)
Wherein VdThat indicate is VeVisual signature after decoding and rebuilding.
Further, step A specifically:
First semantic feature is encoded:
Ae=σ (AW3+b3)
AeWhat is indicated is the result after semantic feature coding, and what A was indicated is original semantic feature.
Next, we are by the visual signature V after coding obtained in the previous stepeIt is mapped to obtain Vae,VaeDimension with Attributive character A after codingeDimension it is the same.We used two full articulamentum+Relu to activate letter during experiment Several layers.This process can be indicated with following simple formula:
B=σ (VeW4+b4)
Vae=σ (BW5+b5)
Wherein B is VeThe value that first Relu+FC block obtains, VaeIt is VeMap obtained end value.Then we incite somebody to action The V arrivedaeWith the attributive character A after codingeIt is added, obtains a new Ae, this AeThe influence of visual signature is received, includes More information.
A′e=Ae+Vae
Decoded process is indicated with following formula:
Ad=σ (A 'eW6+b6)
Wherein AdThat indicate is the attributive character after rebuilding, W6It is the parameter matrix that model needs training；b6It is model needs Trained offset parameter.
Further, as shown in figure 3, step B specifically: initial semantic feature to be added with decoded semantic feature Semantic feature after being optimized.
Aall=A+Ad
AallSemantic feature after indicating optimization.
Further, as shown in figure 4, step (3) specifically: pass through the semantic feature after optimization that step (2) obtains Two full articulamentum+Relu activation primitive layers are mapped to visual signature space.Here mapping relations can use following formula It indicates:
C=σ (AallW7+b7)
Vall=σ (CW8+b8)
Wherein C is AallThe value that first Relu+FC block obtains, VallIt is AallMap obtained end value.
There is three constraint conditions for this model, it is intended that these three constraint conditions reach as much as possible.This three Under the collective effect of a constraint condition, entire model is optimized.Firstly, since visual signature and semantic feature do it is self-editing The operation of code device, it is intended that the feature after reconstruction is similar to primitive character as far as possible, so we need to establish two passes In the constraint condition that the two are encoded certainly.That is:
min||V-Vd||2
min||A-Ad||2
The last one constraint condition is exactly that the semantic feature after finally obtained optimization is mapped to initial visual signature. That is:
min||Vall-V||2
The parameter of entire model establishes loss function under the action of these constraint conditions, and constantly training obtains optimal Training result.

Claims (6)

1. a kind of based on related double application methods of the self-encoding encoder in zero degree in study, it is characterised in that include the following steps:
Step (1) obtains the visual signature after coding；
Step (2) obtains the semantic feature after optimization；
Semantic feature after optimization is mapped to visual signature by step (3)；
Wherein step (2) is realized by following process:
Step A establishes self-encoding encoder to semantic feature, after the visual signature after encoding obtained in step (1) is added to coding In semantic feature, then this semantic feature after being added is decoded again；Step B by this decode after obtained semanteme it is special Sign is added the more preferable more complete semantic feature of acquisition with former semantic feature.
2. according to claim 1 a kind of based on related double application methods of the self-encoding encoder in zero degree in study, feature It is that step (1) is implemented as follows:
Visual signature is handled by a self-encoding encoder, the feature V after being encodede；Formula expression are as follows:
Ve=σ (VW1+b1)
Wherein, VeWhat is indicated is the feature after visual signature coding, and what V was indicated is original visual feature, and what σ was indicated is that Relu swashs Function living；W1It is the parameter matrix that model needs training；b1It is the offset parameter that model needs training；
Decoding process is indicated with following formula:
Vd=σ (VeW2+b2)
Wherein, VdThat indicate is VeVisual signature after decoding and rebuilding, W2It is the parameter matrix that model needs training；b2It is that model needs The offset parameter to be trained.
3. according to claim 2 a kind of based on related double application methods of the self-encoding encoder in zero degree in study, feature It is step A specifically:
First semantic feature is encoded:
Ae=σ (AW3+b3)
AeThat indicate is the attribute spy after semantic feature coding, and what A was indicated is original semantic feature, and what σ was indicated is Relu activation Function；W3It is the parameter matrix that model needs training；b3It is the offset parameter that model needs training；
Then by the visual signature V after obtained codingeIt is mapped to obtain Vae, VaeDimension and coding after attributive character Ae Dimension it is the same；Two full articulamentum+Relu activation primitive layers are used in mapping process, are formulated as follows:
B=σ (VeW4+b4)
Vae=σ (BW5+b5)
Wherein B is VeThe value that first Relu+FC block obtains, VaeIt is VeMap obtained end value；W4And W5Be model needs Trained parameter matrix；b4And b5It is the offset parameter that model needs training；Then the V that will be obtainedaeIt is special with the attribute after coding Levy AeIt is added, obtains a new attributive character A 'e, this new attributive character A 'eThe influence of visual signature is received, includes More information；What σ was indicated is Relu activation primitive；
A′e=Ae+Vae
Decoded process is indicated with following formula:
Ad=σ (A 'eW6+b6)
Wherein AdThat indicate is the attributive character after rebuilding, W6It is the parameter matrix that model needs training；b6It is that model needs to train Offset parameter.
4. according to claim 3 a kind of based on related double application methods of the self-encoding encoder in zero degree in study, feature It is step B specifically: original semantic feature is added the semantic feature after being optimized with decoded semantic feature；
Aall=A+Ad
AallSemantic feature after indicating optimization.
5. according to claim 4 a kind of based on related double application methods of the self-encoding encoder in zero degree in study, feature It is step (3) specifically: the semantic feature after the optimization for obtaining step (2) activates letter by two full articulamentum+Relu Several layers are mapped to visual signature space, and mapping relations formula is expressed as follows:
C=σ (AallW7+b7)
Vall=σ (CW8+b8)
Wherein, C is AallThe value that first Relu+FC block obtains, VallIt is AallMap obtained end value；
W7And W8With the parameter matrix for being model needs training；b7And b8It is the offset parameter that model needs training.
6. it is according to claim 4 or 5 a kind of based on related double application methods of the self-encoding encoder in zero degree in study, it is special Sign is model there is three constraint conditions: firstly the need of establishing two constraint conditions about the two from coding；That is:
min||V-Vd||2
min||A-Ad||2
The last one constraint condition is exactly that the semantic feature after finally obtained optimization is mapped to initial visual signature；That is:
min||Vall-V||2
The parameter of entire model establishes loss function under the action of these constraint conditions, and constantly training obtains optimal training As a result.
CN201910629751.2A 2019-07-12 2019-07-12 Application method based on relevant double-self-encoder in zero-time learning Active CN110517328B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910629751.2A CN110517328B (en) 2019-07-12 2019-07-12 Application method based on relevant double-self-encoder in zero-time learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910629751.2A CN110517328B (en) 2019-07-12 2019-07-12 Application method based on relevant double-self-encoder in zero-time learning

Publications (2)

Publication Number Publication Date
CN110517328A true CN110517328A (en) 2019-11-29
CN110517328B CN110517328B (en) 2020-08-25

Family

ID=68622880

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910629751.2A Active CN110517328B (en) 2019-07-12 2019-07-12 Application method based on relevant double-self-encoder in zero-time learning

Country Status (1)

Country Link
CN (1) CN110517328B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113378851A (en) * 2020-02-25 2021-09-10 阿里巴巴集团控股有限公司 Visual recognition method and device for image data, storage medium and processor
WO2023071123A1 (en) * 2021-10-29 2023-05-04 广东坚美铝型材厂（集团）有限公司 Self-learning method for semantic feature with maximum gap, and computer device and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104166982A (en) * 2014-06-30 2014-11-26 复旦大学 Image optimization clustering method based on typical correlation analysis
CN107679556A (en) * 2017-09-18 2018-02-09 天津大学 The zero sample image sorting technique based on variation autocoder
CN108537257A (en) * 2018-03-26 2018-09-14 天津大学 The zero sample classification method based on identification dictionary matrix pair
CN109597998A (en) * 2018-12-20 2019-04-09 电子科技大学 A kind of characteristics of image construction method of visual signature and characterizing semantics joint insertion
CN109598279A (en) * 2018-09-27 2019-04-09 天津大学 Based on the zero sample learning method for generating network from coding confrontation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104166982A (en) * 2014-06-30 2014-11-26 复旦大学 Image optimization clustering method based on typical correlation analysis
CN107679556A (en) * 2017-09-18 2018-02-09 天津大学 The zero sample image sorting technique based on variation autocoder
CN108537257A (en) * 2018-03-26 2018-09-14 天津大学 The zero sample classification method based on identification dictionary matrix pair
CN109598279A (en) * 2018-09-27 2019-04-09 天津大学 Based on the zero sample learning method for generating network from coding confrontation
CN109597998A (en) * 2018-12-20 2019-04-09 电子科技大学 A kind of characteristics of image construction method of visual signature and characterizing semantics joint insertion

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
林克正等: ""语义自编码结合关系网络的零样本图像识别算法"", 《模式识别与人工智能》 *
陈祥凤等: ""度量学习改进语义自编码零样本分类算法"", 《北京邮电大学学报》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113378851A (en) * 2020-02-25 2021-09-10 阿里巴巴集团控股有限公司 Visual recognition method and device for image data, storage medium and processor
WO2023071123A1 (en) * 2021-10-29 2023-05-04 广东坚美铝型材厂（集团）有限公司 Self-learning method for semantic feature with maximum gap, and computer device and storage medium

Also Published As

Publication number Publication date
CN110517328B (en) 2020-08-25

Similar Documents

Publication Publication Date Title
WO2023077816A1 (en) Boundary-optimized remote sensing image semantic segmentation method and apparatus, and device and medium
CN109543667B (en) Text recognition method based on attention mechanism
JP6584477B2 (en) Skip architecture neural network device and method for improved semantic segmentation
CN106295245B (en) Method of the storehouse noise reduction based on Caffe from coding gene information feature extraction
CN111160008A (en) Entity relationship joint extraction method and system
CN109657051A (en) Text snippet generation method, device, computer equipment and storage medium
CN111462896B (en) Real-time intelligent auxiliary ICD coding system and method based on medical record
AU2020333132A1 (en) Method and system for disease classification coding based on deep learning, and device and medium
CN110473529B (en) Stream type voice transcription system based on self-attention mechanism
CN109785833A (en) Human-computer interaction audio recognition method and system for smart machine
CN113283244B (en) Pre-training model-based bidding data named entity identification method
CN110517328A (en) It is a kind of based on related double application methods of the self-encoding encoder in zero degree in study
CN113033534A (en) Method and device for establishing bill type identification model and identifying bill type
CN111476023A (en) Method and device for identifying entity relationship
CN106022363A (en) Method for recognizing Chinese characters in natural scene
CN114091450B (en) Judicial domain relation extraction method and system based on graph convolution network
CN114944213A (en) Memory-driven Transformer medical endoscope image report generation method
CN113822026A (en) Multi-label entity labeling method
CN114610935A (en) Method and system for synthesizing semantic image of text control image style
CN116051549A (en) Method, system, medium and equipment for dividing defects of solar cell
CN110263304A (en) Statement coding method, sentence coding/decoding method, device, storage medium and equipment
CN113919811B (en) Hot line event distribution method based on strengthened correlation
CN115758758A (en) Inverse synthesis prediction method, medium, and apparatus based on similarity feature constraint
CN110175588B (en) Meta learning-based few-sample facial expression recognition method and system
CN107452374A (en) Based on unidirectional from the various visual angles language identification method for marking auxiliary information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant