CN109740688A - A kind of terahertz image information interpretation method, network and storage medium - Google Patents
A kind of terahertz image information interpretation method, network and storage medium Download PDFInfo
- Publication number
- CN109740688A CN109740688A CN201910019673.4A CN201910019673A CN109740688A CN 109740688 A CN109740688 A CN 109740688A CN 201910019673 A CN201910019673 A CN 201910019673A CN 109740688 A CN109740688 A CN 109740688A
- Authority
- CN
- China
- Prior art keywords
- resolution
- obtains
- super
- terahertz image
- carried out
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
The invention discloses a kind of terahertz image information interpretation method, network and storage medium, method is the following steps are included: obtain low resolution terahertz image；Super-resolution reconstruction is carried out to the low resolution terahertz image of acquisition, obtains super-resolution terahertz image；Feature extraction is carried out to low resolution terahertz image using at least one layer of convolutional layer, obtains low resolution characteristic pattern；Feature extraction is carried out to super-resolution terahertz image using at least one layer of convolutional layer, obtains super-resolution characteristic pattern；Low resolution characteristic pattern and super-resolution characteristic pattern are subjected to feature level image co-registration, obtain Fusion Features figure；Region proposal is carried out to Fusion Features figure, obtains candidate region；Pool area is carried out to candidate region, obtains pool area result；Target detection and/or semantic segmentation are carried out to pool area result, complete the information interpretation to terahertz image.The present invention can efficiently and in high quality complete the information interpretation to terahertz image.
Description
Technical field
The present invention relates to field of image processing more particularly to a kind of terahertz image information interpretation methods, network and storage
Medium.
Background technique
Terahertz Technology can carry out non-contact automatic identification to carry-on concealment danger, can solve and pacify under public environment
The technical problem that full inspection is surveyed.For a long time, the poor imaging definition of the terahertz image of low resolution and contrast, edge wheel
Wide unintelligible, coherent speckle noise interference becomes one of principal element for hindering terahertz image application.
Image processing techniques is combined with terahertz imaging principle, terahertz image can be identified, but made
Semantic segmentation is carried out with low-resolution image, although recognition rate is high, is weighed with respect to losing many small for high-definition picture
The detailed information wanted.
In order to show that information that image itself needs to express, the classification information including scene and some object need one
The new terahertz image information interpretation method of kind, to meet the interpretation requirement to the high quality of low resolution terahertz image.With
The functions such as real-time, intellectualized detection are realized, so that Terahertz safety check technology is in urban track traffic field of safety check with good
Application scenarios.
Summary of the invention
The purpose of the present invention is to provide a kind of terahertz image information interpretation method, network and storage mediums, to low point
Resolution terahertz image is interpreted in high quality.
To achieve this purpose, the present invention adopts the following technical scheme:
A kind of terahertz image information interpretation method, comprising the following steps:
Super-resolution reconstruction is carried out to the low resolution terahertz image of acquisition, obtains super-resolution terahertz image；
Feature extraction is carried out to the low resolution terahertz image using at least one layer of convolutional layer, it is special to obtain low resolution
Sign figure；Feature extraction is carried out to the super-resolution terahertz image using at least one layer of convolutional layer, obtains super-resolution feature
Figure；
The low resolution characteristic pattern and the super-resolution characteristic pattern are subjected to feature level image co-registration, obtain Fusion Features
Figure；
Region proposal is carried out to the Fusion Features figure, obtains candidate region；Pool area is carried out to the candidate region,
Obtain pool area result；
Target detection and/or semantic segmentation are carried out to the pool area result, complete the information solution to terahertz image
It translates.
Optionally, the step: super-resolution reconstruction is carried out to the low resolution terahertz image of acquisition, obtains super-resolution
Rate terahertz image, specifically includes:
Super-resolution reconstruction is carried out using low resolution terahertz image of the super-resolution reconstruction sub-network to acquisition, is obtained
Super-resolution terahertz image；
Wherein, the super-resolution reconstruction sub-network includes at least one recurrence block, and each recurrence block includes at least one
A residual unit；The output valve of the last one recurrence block is superimposed after carrying out convolution with the low resolution terahertz image,
Obtain the super-resolution terahertz image.
Optionally, the low resolution characteristic pattern and the super-resolution characteristic pattern step: are subjected to feature level image
Fusion obtains Fusion Features figure, specifically includes:
Deconvolution, cavity convolution sum L2 normalization calculating are successively carried out to the low resolution characteristic pattern, obtain low resolution
Rate processing figure；
The convolution sum L2 normalization for successively carrying out convolution kernel 1*1 to the super-resolution characteristic pattern calculates, and obtains super-resolution
Rate processing figure；
The low resolution processing figure and the super-resolution processing figure are merged, carry out ReLu activation primitive later
It calculates, obtains Fusion Features figure.
Optionally, the step: region proposal is carried out to the Fusion Features figure, candidate region is obtained, specifically includes:
For the Fusion Features figure, using anchor mechanism on the sliding window of x*y, generate it is a variety of it is different size of can
It can region；
Classified calculating is carried out to a variety of Probability Areas and frame is returned and calculated, classification results is obtained and frame returns knot
Fruit；
According to classification results preliminary screenings are carried out to a variety of Probability Areas, according to frame regression result to filtering out
Probability Area is tentatively deviated, and candidate region is obtained.
Optionally, the step: carrying out pool area to the candidate region, obtains pool area as a result, specifically including:
The candidate region is mapped to the corresponding position of Fusion Features figure, obtains mapping area；
The mapping area is divided into the section of multiple same sizes, maximum Chi Huaji is carried out to each section
It calculates, obtains pool area result.
Optionally, the step: target detection and/or semantic segmentation are carried out to the pool area result, completed to too
The information interpretation of hertz image, specifically includes:
Deconvolution is carried out to the pool area result；
The deconvolution result of the pool area result is inputted into full articulamentum conversion and obtains one-dimensional vector；
Device is returned using region classifier and frame, and classified calculating and frame recurrence meter are carried out to the one-dimensional vector respectively
It calculates, obtains classification results and frame regression result；
Preliminary screening acquisition is carried out to the one-dimensional vector according to classification results and belongs to the other candidate frame of predetermined class, according to side
Frame regression result deviates the position of the candidate frame, obtains target detection figure；
And/or
The deconvolution result of the pool area result is up-sampled；Semantic classifiers are utilized to the result of up-sampling
Classify, up-sampled again later, obtains semantic segmentation figure；
Optionally, the step: deconvolution is successively carried out to the low resolution characteristic pattern, cavity convolution sum L2 is normalized
It calculates, obtains low resolution processing figure, later further include a step:
The deconvolution result of the low resolution characteristic pattern is successively carried out to the convolution sum Softmax function of convolution kernel 1*1
It calculates, obtains penalty values L1；Wherein, Softmax function calculates the low resolution label using cascade label instructions；
The step: device is returned using region classifier and frame, classified calculating and side are carried out to the one-dimensional vector respectively
Frame, which returns, to be calculated, and is obtained classification results and frame regression result, is later further included a step:
Region classifier is trained using Softmax Loss function, obtains penalty values L21；Utilize Smooth
L1Loss returns device to frame and is trained, and obtains penalty values L22；By penalty values L21And L22Superposition obtains penalty values L2；
The step: classifying to the result of up-sampling using semantic classifiers, later further includes a step:
Classification based training is carried out to semantic classifiers using the high-resolution label of cascade label instructions, obtains penalty values L3；
The step: the information interpretation to terahertz image is completed, later further includes a step:
Calculate total losses function L；
If only carrying out target detection, the total losses function L=λ to the pool area result1L1+λ2L2；
If only carrying out semantic segmentation, the total losses function L=λ to the pool area result1L1+λ3L3；
If carrying out target detection and semantic segmentation, the total losses function L=λ to the pool area result1L1+λ2L2+
λ3L3；
Minimize total losses function L.
A kind of terahertz image information interpretation network, comprising:
Super-resolution reconstruction sub-network, for carrying out super-resolution reconstruction to the low resolution terahertz image of acquisition；
Feature extraction sub-network, for carrying out feature to the low resolution terahertz image using at least one layer of convolutional layer
It extracts, obtains low resolution characteristic pattern；Feature is carried out to the super-resolution terahertz image using at least one layer of convolutional layer to mention
It takes, obtains super-resolution characteristic pattern；
Fusion Features sub-network, for the low resolution characteristic pattern and the super-resolution characteristic pattern to be carried out feature level figure
As fusion, Fusion Features figure is obtained；
Sub-network is proposed in region, for carrying out region proposal to the Fusion Features figure, obtains candidate region；
Pool area beggar's network obtains pool area result for carrying out pool area to the candidate region；
Target detection and semantic segmentation sub-network, for target detection and/or language will to be carried out to the pool area result
Justice segmentation, completes the information interpretation to terahertz image；
The super-resolution reconstruction sub-network connects the feature extraction sub-network, and the feature extraction sub-network connects institute
Fusion Features sub-network is stated, the Fusion Features sub-network connects the region and proposes sub-network, and sub-network is proposed in the region
Connect the pool area beggar network.
A kind of storage medium is stored with computer executable instructions on the storage medium, which is characterized in that the calculating
When machine executable instruction is subsequently can by computer device and executes, the step of realizing above-mentioned terahertz image information interpretation method.
Compared with prior art, the invention has the following advantages:
The present invention realizes the super-resolution reconstruction of low resolution terahertz image, the super-resolution terahertz that reconstruct is obtained
Hereby image and low resolution terahertz image carry out feature level image co-registration, to the Fusion Features figure carry out target detection and/
Or semantic segmentation, complete the information interpretation to low resolution terahertz image.Low resolution Terahertz can be effectively reduced
The calculating cost of the identification of image, while the accuracy rate of final output will not be reduced, to efficiently complete to terahertz image
Information interpretation.And the features such as can effectively overcoming terahertz imaging low resolution, strong noise.Information interpretation method of the invention
It is very beneficial for the research of terahertz image processing technique and the application of terahertz image.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention without any creative labor, may be used also for those of ordinary skill in the art
To obtain other attached drawings according to these attached drawings.
Fig. 1 is the flow diagram for the terahertz image information interpretation method that the embodiment of the present invention one provides.
Fig. 2 is the schematic diagram for the super-resolution reconstruction sub-network that the embodiment of the present invention one provides.
Fig. 3 is the schematic diagram for the recurrence block that the embodiment of the present invention one provides.
Fig. 4 is the schematic diagram of terahertz image information interpretation network provided by Embodiment 2 of the present invention.
Fig. 5 is the schematic diagram of terahertz image information interpretation network provided by Embodiment 2 of the present invention.
Fig. 6 is the schematic diagram of Fusion Features sub-network provided by Embodiment 2 of the present invention.
It illustrates: 11, super-resolution reconstruction sub-network；12, feature extraction sub-network；13, Fusion Features sub-network；
14, sub-network is proposed in region；15, pool area beggar network；16, target detection and semantic segmentation sub-network.
Specific embodiment
In order to make the invention's purpose, features and advantages of the invention more obvious and easy to understand, below in conjunction with the present invention
Attached drawing in embodiment, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that disclosed below
Embodiment be only a part of the embodiment of the present invention, and not all embodiment.Based on the embodiments of the present invention, this field
Those of ordinary skill's all other embodiment obtained without making creative work, belongs to protection of the present invention
Range.
In the description of the present invention, it is to be understood that, term " on ", "lower", "top", "bottom", "inner", "outside" etc. indicate
Orientation or positional relationship be based on the orientation or positional relationship shown in the drawings, be merely for convenience of description the present invention and simplification retouch
It states, rather than the device or element of indication or suggestion meaning must have a particular orientation, be constructed and operated in a specific orientation,
Therefore it is not considered as limiting the invention.It should be noted that when a component is considered as " connection " another component,
It can be directly to another component or may be simultaneously present the component being centrally located.
To further illustrate the technical scheme of the present invention below with reference to the accompanying drawings and specific embodiments.
Embodiment one
The present embodiment one provides a kind of terahertz image information interpretation method, referring to FIG. 1, itself the following steps are included:
S1, super-resolution reconstruction is carried out to the low resolution terahertz image of acquisition, obtains super-resolution terahertz image；
S2, feature extraction is carried out to low resolution terahertz image using at least one layer of convolutional layer, it is special obtains low resolution
Sign figure；Feature extraction is carried out to super-resolution terahertz image using at least one layer of convolutional layer, obtains super-resolution characteristic pattern；
S3, low resolution characteristic pattern and super-resolution characteristic pattern are subjected to feature level image co-registration, obtain Fusion Features figure；
S4, region proposal is carried out to Fusion Features figure, obtains candidate region；Pool area is carried out to candidate region, is obtained
Pool area result；
S5, target detection and/or semantic segmentation are carried out to pool area result, completes the information solution to terahertz image
It translates.
In the present embodiment, S1 step specifically includes:
Super-resolution reconstruction is carried out to low resolution terahertz image using super-resolution reconstruction sub-network 11, obtains oversubscription
Resolution terahertz image；
Wherein, super-resolution reconstruction sub-network 11 includes at least one recurrence block, and each recurrence block includes at least one residual error
Unit；The output valve of the last one recurrence block is superimposed after carrying out convolution with low resolution terahertz image, obtains super-resolution too
Hertz image.
Fig. 2 be include n recurrence block super-resolution reconstruction sub-network 11 schematic diagram.Fig. 3 be include two residual error lists
The schematic diagram of the recurrence block of member.Wherein, n >=1.
As shown in figure 3, the output valve of previous recurrence block, inputs the 1st residual unit after carrying out convolution.In the 1st residual unit,
Input value is superimposed after carrying out convolution twice with input value, output to the 2nd residual unit；In the 2nd residual unit, the 1st residual unit
Output valve be superimposed twice with the input value of the 1st residual unit after convolution, export to next recurrence block.
If in a recurrence block including m residual unit, carried out in the output valve of m residual unit, m-1 residual unit
It is superimposed after convolution with the input value of the 1st residual unit twice, output to next recurrence block.Wherein, m >=1.
As m=1, super-resolution reconstruction sub-network 11 can be indicated are as follows:
Y=D (x)=fconv(Rn(Rn-1(...(R1(x))...)))+x
Wherein, R indicates the function of recurrence block, fconvIndicate the last layer convolution layer functions.
S3 step specifically includes:
Deconvolution, cavity convolution sum L2 normalization calculating are successively carried out to low resolution characteristic pattern, are obtained at low resolution
Reason figure；
The convolution sum Softmax function for successively carrying out convolution kernel 1*1 to the deconvolution result of low resolution characteristic pattern calculates,
Obtain penalty values L1；Wherein, Softmax function calculates the low resolution label using cascade label instructions；
The convolution sum L2 normalization that convolution kernel 1*1 is successively carried out to super-resolution characteristic pattern calculates, and obtains at super-resolution
Reason figure；
Low resolution processing figure and super-resolution processing figure are merged, the calculating of ReLu activation primitive is carried out later, obtains
Obtain Fusion Features figure.
The present embodiment is according to the penalty values L of acquisition1It is trained, can be enhanced from low resolution characteristic pattern to low resolution
Handle the study of the intermediate step of figure.In order to combine the characteristic pattern of different resolution, the present embodiment first leads to low resolution characteristic pattern
It crosses deconvolution to be up-sampled, receptive field is increased with it；And super-resolution characteristic pattern is carried out by the convolution of convolution kernel 1*1
Dimensionality reduction obtains the dimension as low resolution characteristic pattern；Then the characteristic pattern of high-resolution and low resolution is returned by L2
One changes, and is merged；Using ReLu activation primitive, so that it may obtain and melt with the feature of super-resolution characteristic pattern equal resolution
Close figure.
In S4 step, region proposal is carried out to Fusion Features figure, candidate region is obtained, specifically includes:
For Fusion Features figure, using anchor mechanism on the sliding window of x*y, a variety of different size of possible areas are generated
Domain；
Classified calculating is carried out to a variety of Probability Areas and frame is returned and calculated, obtains classification results and frame regression result；
Preliminary screening is carried out to a variety of Probability Areas according to classification results, according to frame regression result to the possibility filtered out
Region is tentatively deviated, and candidate region is obtained.
Wherein, Probability Area is generally 300.
In S4 step, pool area is carried out to candidate region, obtains pool area as a result, specifically including:
It is mapped to the corresponding position of Fusion Features figure to candidate region, obtains mapping area；
Mapping area is divided into the section of multiple same sizes, maximum pondization is carried out to each section and is calculated, area is obtained
Domain pond result.
S5 step specifically includes:
S51, deconvolution is carried out to pool area result；
S52, full articulamentum conversion acquisition one-dimensional vector is inputted to the deconvolution result of pool area result；
Device is returned using region classifier and frame, classified calculating and frame recurrence calculating are carried out to one-dimensional vector respectively, obtain
Obtain classification results and frame regression result；
Preliminary screening acquisition is carried out to one-dimensional vector according to classification results and belongs to the other candidate frame of predetermined class, is returned according to frame
Sum up fruit to deviate the position of candidate frame, obtains target detection figure；
Wherein, region classifier is trained using Softmax Loss function, obtains penalty values L21Utilize Smooth
L1Loss returns device to frame and is trained, and obtains penalty values L22, by penalty values L21And L22Superposition obtains penalty values L2；
And/or
S53, the deconvolution result of pool area result is up-sampled；Semantic classifiers are utilized to the result of up-sampling
It after being classified, then is up-sampled, obtains semantic segmentation figure；
Classification based training is carried out to semantic classifiers using the high-resolution label of cascade label instructions, obtains penalty values L3。
After realizing pool area, target detection is obtained by target detection sub-network and semantic segmentation sub-network respectively
Figure and semantic segmentation figure, it is comprehensive for the interpretation of terahertz image.
Further include a step S6 after S5 step:
Calculate total losses function L；
If only carrying out target detection, the total losses function L=λ to the pool area result1L1+λ2L2；
If only carrying out semantic segmentation, the total losses function L=λ to the pool area result1L1+λ3L3；
If carrying out target detection and semantic segmentation, the total losses function L=λ to the pool area result1L1+λ2L2+
λ3L3；
Minimize total losses function L.
The interpretation of terahertz image can be made more accurate by minimizing total losses function.
The embodiment of the present invention one carries out Super-resolution reconstruction to low-resolution image using the subnetwork with multiple recurrence blocks
Structure, the super-resolution image of acquisition carry out feature extraction by least one layer of convolutional layer, at the same low-resolution image also pass through to
Few one layer of convolutional layer carries out feature extraction, the characteristic pattern of the Resolutions after feature extraction is merged, in fusion process
Obtain loss function L1.Is carried out by deconvolution, is then carried out respectively after region proposal and pool area for Fusion Features figure again
Target detection and semantic segmentation complete the information interpretation to low resolution terahertz image；Wherein, target detection process is damaged
Lose function L2, semantic segmentation process acquisition loss function L3.According to L1、L2And L3, calculate total losses function and minimized.
Above procedure significantly reduces the calculating cost of the identification of low resolution terahertz image, while will not reduce most
The accuracy rate exported eventually efficiently completes the information interpretation to terahertz image.The experimental results showed that this network structure is effectively
The wisp correct classification rate of low resolution terahertz image is improved, while obtaining finer segmentation effect, can also be reached
To satisfactory splitting speed.
The present invention realizes super-resolution reconstruction, target detection and the semantic segmentation of low resolution terahertz image, completes
To the information interpretation of low resolution terahertz image.Meanwhile the convolution layer number of super-resolution branch is much smaller than low resolution
Branch, using the input data of a small amount of convolutional layer processing super-resolution, very short time constructs fusion feature figure.So as to reality
The target detection and semantic segmentation of existing fast hi-resolution, embody more detailed information, can effectively overcome terahertz imaging
The features such as low resolution, strong noise.Information interpretation method of the invention is very beneficial for the research of terahertz image processing technique
And the application of terahertz image.
Embodiment two
Present embodiments provide a kind of Terahertz figure of terahertz image information interpretation method for realizing embodiment one
As information interpretation network, specifically include:
Super-resolution reconstruction sub-network 11, for carrying out super-resolution reconstruction to the low resolution terahertz image of acquisition；
Feature extraction sub-network 12 is mentioned for carrying out feature to low resolution terahertz image using at least one layer of convolutional layer
It takes, obtains low resolution characteristic pattern, feature extraction is carried out to super-resolution terahertz image using at least one layer of convolutional layer, is obtained
Super-resolution characteristic pattern；
Fusion Features sub-network 13 is melted for low resolution characteristic pattern and super-resolution characteristic pattern to be carried out feature level image
It closes, obtains Fusion Features figure；
Sub-network 14 is proposed in region, for carrying out region proposal to Fusion Features figure, obtains candidate region；
Pool area beggar network 15 is completed for carrying out target detection and/or semantic segmentation to pool area result to too
The information interpretation of hertz image；
Target detection and semantic segmentation sub-network 16, for will to the pool area result carry out target detection and/or
Semantic segmentation completes the information interpretation to terahertz image.
11 connection features of super-resolution reconstruction sub-network extract sub-network 12, the fusion of 12 connection features of feature extraction sub-network
Sub-network 13,13 join domain of Fusion Features sub-network propose sub-network 14, and 14 join domain pond beggar of sub-network is proposed in region
Network 15, the detection of 15 linking objective of pool area beggar network and semantic segmentation sub-network 16.
Fig. 2 be include n recurrence block super-resolution reconstruction sub-network 11 schematic diagram.Fig. 3 be include two residual error lists
The schematic diagram of the recurrence block of member.Wherein, n >=1.
As shown in Fig. 2, super-resolution reconstruction sub-network 11 includes at least one recurrence block, each recurrence block includes at least one
Residual unit；The output valve of the last one recurrence block is superimposed after carrying out convolution with low resolution terahertz image, obtains super-resolution
Rate terahertz image.
As shown in figure 3, the output valve of previous recurrence block, inputs the 1st residual unit after carrying out convolution.In 1st residual error, input
Value is superimposed after carrying out convolution twice with input value, output to the 2nd residual unit；In 2nd residual unit, the 1st residual unit it is defeated
Value is superimposed after carrying out convolution twice with the input value of the 1st residual unit out, output to next recurrence block.
If in recurrence block including m residual unit, in m residual unit, the output valve of m-1 residual unit carries out two
It is superimposed after secondary convolution with the input value of the 1st residual unit, output to next recurrence block.Wherein, m >=1.
As m=1, super-resolution reconstruction sub-network 11 can be indicated are as follows:
Y=D (x)=fconv(Rn(Rn-1(...(R1(x))...)))+x
Wherein, R indicates the function of recurrence block, fconvIndicate the last layer convolution layer functions.
Referring to FIG. 5, Fig. 5 is the schematic diagram of terahertz image information interpretation network.
As shown, feature extraction sub-network 12 is specifically used for:
To low resolution terahertz image by least one layer of convolutional layer processing, low resolution characteristic pattern is obtained；
To super-resolution terahertz image by least one layer of convolutional layer processing, high-resolution features figure is obtained.
I.e. feature extraction sub-network 12 includes the i convolutional layers and j for being handled low resolution terahertz image
A convolutional layer for being handled super-resolution terahertz image；Wherein, i >=1, j >=1.
Referring to FIG. 6, Fig. 5 is characterized the schematic diagram of fusion sub-network 13.As shown, Fusion Features sub-network 13 is specific
For:
Deconvolution is carried out to low resolution characteristic pattern；
Cavity convolution sum L2 normalization is successively carried out to the deconvolution result of low resolution characteristic pattern to calculate, and obtains low resolution
Rate processing figure；
The convolution sum Softmax function for successively carrying out convolution kernel 1*1 to the deconvolution result of low resolution characteristic pattern calculates,
Obtain penalty values L1；Wherein, Softmax function calculates the low resolution label using cascade label instructions；
The convolution sum L2 normalization that convolution kernel 1*1 is successively carried out to super-resolution characteristic pattern calculates, and obtains at super-resolution
Reason figure；
Low resolution processing figure and super-resolution processing figure are merged, the calculating of ReLu activation primitive is carried out later, obtains
Obtain Fusion Features figure.
In the present embodiment, region proposes that sub-network 14 is specifically used for:
For Fusion Features figure, using anchor mechanism on the sliding window of x*y, a variety of different size of possible areas are generated
Domain；
Classified calculating is carried out to a variety of Probability Areas and frame is returned and calculated, obtains classification results and frame regression result；
Preliminary screening is carried out to a variety of Probability Areas according to classification results, according to frame regression result to the possibility filtered out
Region is tentatively deviated, and candidate region is obtained.
Wherein, Probability Area is generally 300.
Pool area beggar network 15 is specifically used for:
It is mapped to the corresponding position of Fusion Features figure to candidate region, obtains mapping area；
Mapping area is divided into the section of multiple same sizes, maximum pondization is carried out to each section and is calculated, area is obtained
Domain pond result.
Referring to FIG. 5, target detection and semantic segmentation sub-network 16 are specifically used for:
Deconvolution is carried out to pool area result；
Full articulamentum conversion is inputted to the deconvolution result of pool area result and obtains one-dimensional vector；
Device is returned using region classifier and frame, classified calculating and frame recurrence calculating are carried out to one-dimensional vector respectively, obtain
Obtain classification results and frame regression result；
Preliminary screening acquisition is carried out to one-dimensional vector according to classification results and belongs to the other candidate frame of predetermined class, is returned according to frame
Sum up fruit to deviate the position of candidate frame, obtains target detection figure；
Wherein, region classifier is trained using Softmax Loss function, obtains penalty values L21Utilize Smooth
L1Loss returns device to frame and is trained, and obtains penalty values L22, by penalty values L21And L22Superposition obtains penalty values L2；
The deconvolution result of pool area result is up-sampled；The result of up-sampling is carried out using semantic classifiers
It after classification, then is up-sampled, obtains semantic segmentation figure；
Wherein, classification based training is carried out to semantic classifiers using the high-resolution label of cascade label instructions, is lost
Value L3。
In the present embodiment, terahertz image information interpretation network further includes a training unit, the training unit by: based on
Calculate total losses function L；
If only carrying out target detection, the total losses function L=λ to the pool area result1L1+λ2L2；
If only carrying out semantic segmentation, the total losses function L=λ to the pool area result1L1+λ3L3；
If carrying out target detection and semantic segmentation, the total losses function L=λ to the pool area result1L1+λ2L2+
λ3L3；
Minimize total losses function L.
Training unit is trained the full convolution cascade network of entire super-resolution by minimizing total losses function L, to increase
Its strong accuracy.
Embodiment three
The embodiment of the invention provides a kind of storage medium comprising computer executable instructions, computer executable instructions
When being executed by computer processor for executing the terahertz image information interpretation method such as the embodiment of the present invention one.
The computer storage medium of the embodiment of the present invention, can be using any of one or more computer-readable media
Combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It is computer-readable
Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or
Device, or any above combination.The more specific example (non exhaustive list) of computer readable storage medium includes: tool
There are electrical connection, the portable computer diskette, hard disk, random access memory (RAM), read-only memory of one or more conducting wires
(ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-
ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.In this document, computer-readable storage
Medium can be any tangible medium for including or store program, which can be commanded execution system, device or device
Using or it is in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including --- but it is unlimited
In wireless, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof
Program code, programming language include object oriented program language-such as Java, Smalltalk, C++, are also wrapped
Include conventional procedural programming language-such as " C " language or similar programming language.Program code can be complete
Ground executes on the user computer, partly executes on the user computer, executing as an independent software package, partially existing
Part executes on the remote computer or executes on a remote computer or server completely on subscriber computer.It is being related to
In the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or wide area
Net (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as utilize ISP
To be connected by internet).
More than, the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although referring to aforementioned reality
Applying example, invention is explained in detail, those skilled in the art should understand that: it still can be to aforementioned each
Technical solution documented by embodiment is modified or equivalent replacement of some of the technical features；And these are modified
Or replacement, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.
Claims (9)
1. a kind of terahertz image information interpretation method, which comprises the following steps:
Super-resolution reconstruction is carried out to the low resolution terahertz image of acquisition, obtains super-resolution terahertz image；
Feature extraction is carried out to the low resolution terahertz image using at least one layer of convolutional layer, obtains low resolution feature
Figure；Feature extraction is carried out to the super-resolution terahertz image using at least one layer of convolutional layer, obtains super-resolution characteristic pattern；
The low resolution characteristic pattern and the super-resolution characteristic pattern are subjected to feature level image co-registration, obtain Fusion Features figure；
Region proposal is carried out to the Fusion Features figure, obtains candidate region；Pool area is carried out to the candidate region, is obtained
Pool area result；
Target detection and/or semantic segmentation are carried out to the pool area result, complete the information interpretation to terahertz image.
2. terahertz image information interpretation method according to claim 1, which is characterized in that the step: to acquisition
Low resolution terahertz image carries out super-resolution reconstruction, obtains super-resolution terahertz image, specifically includes:
Super-resolution reconstruction is carried out using low resolution terahertz image of the super-resolution reconstruction sub-network to acquisition, obtains oversubscription
Resolution terahertz image；
Wherein, the super-resolution reconstruction sub-network includes at least one recurrence block, and each recurrence block includes that at least one is residual
Poor unit；The output valve of the last one recurrence block is superimposed after carrying out convolution with the low resolution terahertz image, is obtained
The super-resolution terahertz image.
3. terahertz image information interpretation method according to claim 1, which is characterized in that the step: will be described low
It differentiates characteristic pattern and the super-resolution characteristic pattern carries out feature level image co-registration, obtain Fusion Features figure, specifically include:
Deconvolution, cavity convolution sum L2 normalization calculating are successively carried out to the low resolution characteristic pattern, obtained at low resolution
Reason figure；
The convolution sum L2 normalization for successively carrying out convolution kernel 1*1 to the super-resolution characteristic pattern calculates, and obtains at super-resolution
Reason figure；
The low resolution processing figure and the super-resolution processing figure are merged, carry out ReLu activation primitive meter later
It calculates, obtains Fusion Features figure.
4. terahertz image information interpretation method according to claim 1, which is characterized in that the step: to the spy
Sign fusion figure carries out region proposal, obtains candidate region, specifically includes:
For the Fusion Features figure, using anchor mechanism on the sliding window of x*y, a variety of different size of possible areas are generated
Domain；
Classified calculating is carried out to a variety of Probability Areas and frame is returned and calculated, obtains classification results and frame regression result；
Preliminary screening is carried out to a variety of Probability Areas according to classification results, according to frame regression result to the possibility filtered out
Region is tentatively deviated, and candidate region is obtained.
5. terahertz image information interpretation method according to claim 1, which is characterized in that the step: to the time
Favored area carries out pool area, obtains pool area as a result, specifically including:
The candidate region is mapped to the corresponding position of Fusion Features figure, obtains mapping area；
The mapping area is divided into the section of multiple same sizes, maximum pondization is carried out to each section and is calculated, is obtained
Obtain pool area result.
6. terahertz image information interpretation method according to claim 3, which is characterized in that the step: to the area
Domain pond result carries out target detection and/or semantic segmentation, completes to specifically include the information interpretation of terahertz image:
Deconvolution is carried out to the pool area result；
The deconvolution result of the pool area result is inputted into full articulamentum conversion and obtains one-dimensional vector；
Device is returned using region classifier and frame, classified calculating and frame recurrence calculating are carried out to the one-dimensional vector respectively, obtain
Obtain classification results and frame regression result；
Preliminary screening acquisition is carried out to the one-dimensional vector according to classification results and belongs to the other candidate frame of predetermined class, is returned according to frame
Sum up fruit to deviate the position of the candidate frame, obtains target detection figure；
And/or
The deconvolution result of the pool area result is up-sampled；The result of up-sampling is carried out using semantic classifiers
Classification, is up-sampled again later, obtains semantic segmentation figure.
7. terahertz image information interpretation method according to claim 6, which is characterized in that the step: to described low
Resolution characteristics figure successively carries out deconvolution, convolution sum L2 normalization in cavity calculates, and obtains low resolution processing figure, also wraps later
Include a step:
The convolution sum Softmax function that the deconvolution result of the low resolution characteristic pattern successively carries out convolution kernel 1*1 is calculated,
Obtain penalty values L1；Wherein, Softmax function calculates the low resolution label using cascade label instructions；
The step: device is returned using region classifier and frame, classified calculating and frame time are carried out to the one-dimensional vector respectively
Return calculating, obtain classification results and frame regression result, later further include a step:
Region classifier is trained using Softmax Loss function, obtains penalty values L21；Utilize Smooth L1 Loss
Device is returned to frame to be trained, and obtains penalty values L22；By penalty values L21And L22Superposition obtains penalty values L2；
The step: classifying to the result of up-sampling using semantic classifiers, later further includes a step:
Classification based training is carried out to semantic classifiers using the high-resolution label of cascade label instructions, obtains penalty values L3；
The step: the information interpretation to terahertz image is completed, later further includes a step:
Calculate total losses function L；
If only carrying out target detection, the total losses function L=λ to the pool area result1L1+λ2L2；
If only carrying out semantic segmentation, the total losses function L=λ to the pool area result1L1+λ3L3；
If carrying out target detection and semantic segmentation, the total losses function L=λ to the pool area result1L1+λ2L2+λ3L3；
Minimize total losses function L.
8. a kind of terahertz image information interpretation network characterized by comprising
Super-resolution reconstruction sub-network, for carrying out super-resolution reconstruction to the low resolution terahertz image of acquisition；
Feature extraction sub-network is mentioned for carrying out feature to the low resolution terahertz image using at least one layer of convolutional layer
It takes, obtains low resolution characteristic pattern；Feature extraction is carried out to the super-resolution terahertz image using at least one layer of convolutional layer,
Obtain super-resolution characteristic pattern；
Fusion Features sub-network is melted for the low resolution characteristic pattern and the super-resolution characteristic pattern to be carried out feature level image
It closes, obtains Fusion Features figure；
Sub-network is proposed in region, for carrying out region proposal to the Fusion Features figure, obtains candidate region；
Pool area beggar's network obtains pool area result for carrying out pool area to the candidate region；
Target detection and semantic segmentation sub-network, for target detection and/or semantic point will to be carried out to the pool area result
It cuts, completes the information interpretation to terahertz image；
The super-resolution reconstruction sub-network connects the feature extraction sub-network, and the feature extraction sub-network connects the spy
Sign fusion sub-network, the Fusion Features sub-network connect the region and propose sub-network, and sub-network connection is proposed in the region
The pool area beggar network, the pool area beggar are connected to the network the target detection and semantic segmentation sub-network.
9. a kind of storage medium, computer executable instructions are stored on the storage medium, which is characterized in that the computer
When executable instruction is subsequently can by computer device execution, terahertz image described in any claim in claim 1-7 is realized
The step of information interpretation method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910019673.4A CN109740688B (en) | 2019-01-09 | 2019-01-09 | Terahertz image information interpretation method, network and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910019673.4A CN109740688B (en) | 2019-01-09 | 2019-01-09 | Terahertz image information interpretation method, network and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109740688A true CN109740688A (en) | 2019-05-10 |
| CN109740688B CN109740688B (en) | 2023-04-18 |
Family
ID=66364095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910019673.4A Active CN109740688B (en) | 2019-01-09 | 2019-01-09 | Terahertz image information interpretation method, network and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109740688B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111123266A (en) * | 2019-11-22 | 2020-05-08 | 中国电子科技集团公司第四十一研究所 | Terahertz wave large-area uniform illumination device and imaging method |
| CN111353940A (en) * | 2020-03-31 | 2020-06-30 | 成都信息工程大学 | Image super-resolution reconstruction method based on deep learning iterative up-down sampling |
| CN111784573A (en) * | 2020-05-21 | 2020-10-16 | 昆明理工大学 | Passive terahertz image super-resolution reconstruction method based on transfer learning |
| CN112435162A (en) * | 2020-11-13 | 2021-03-02 | 中国科学院沈阳自动化研究所 | Terahertz image super-resolution reconstruction method based on complex field neural network |
| US20210209732A1 (en) * | 2020-06-17 | 2021-07-08 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Face super-resolution realization method and apparatus, electronic device and storage medium |
| CN116309274A (en) * | 2022-12-12 | 2023-06-23 | 湖南红普创新科技发展有限公司 | Method and device for detecting small target in image, computer equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104778671A (en) * | 2015-04-21 | 2015-07-15 | 重庆大学 | Image super-resolution method based on SAE and sparse representation |
| US20150213579A1 (en) * | 2012-08-06 | 2015-07-30 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Method and device for reconstructing super-resolution images |
| US20160358337A1 (en) * | 2015-06-08 | 2016-12-08 | Microsoft Technology Licensing, Llc | Image semantic segmentation |
| CN108428212A (en) * | 2018-01-30 | 2018-08-21 | 中山大学 | A kind of image magnification method based on double laplacian pyramid convolutional neural networks |
| CN108830225A (en) * | 2018-06-13 | 2018-11-16 | 广东工业大学 | The detection method of target object, device, equipment and medium in terahertz image |
| CN108876792A (en) * | 2018-04-13 | 2018-11-23 | 北京迈格威科技有限公司 | Semantic segmentation methods, devices and systems and storage medium |
| CN108875659A (en) * | 2018-06-26 | 2018-11-23 | 上海海事大学 | A kind of sea chart culture zone recognition methods based on multi-spectrum remote sensing image |
-
2019
- 2019-01-09 CN CN201910019673.4A patent/CN109740688B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150213579A1 (en) * | 2012-08-06 | 2015-07-30 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Method and device for reconstructing super-resolution images |
| CN104778671A (en) * | 2015-04-21 | 2015-07-15 | 重庆大学 | Image super-resolution method based on SAE and sparse representation |
| US20160358337A1 (en) * | 2015-06-08 | 2016-12-08 | Microsoft Technology Licensing, Llc | Image semantic segmentation |
| CN108428212A (en) * | 2018-01-30 | 2018-08-21 | 中山大学 | A kind of image magnification method based on double laplacian pyramid convolutional neural networks |
| CN108876792A (en) * | 2018-04-13 | 2018-11-23 | 北京迈格威科技有限公司 | Semantic segmentation methods, devices and systems and storage medium |
| CN108830225A (en) * | 2018-06-13 | 2018-11-16 | 广东工业大学 | The detection method of target object, device, equipment and medium in terahertz image |
| CN108875659A (en) * | 2018-06-26 | 2018-11-23 | 上海海事大学 | A kind of sea chart culture zone recognition methods based on multi-spectrum remote sensing image |
Non-Patent Citations (1)
| Title |
|---|
| 邸志刚 等: "太赫兹成像技术在无损检测中的实验研究", 《激光与红外》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111123266A (en) * | 2019-11-22 | 2020-05-08 | 中国电子科技集团公司第四十一研究所 | Terahertz wave large-area uniform illumination device and imaging method |
| CN111123266B (en) * | 2019-11-22 | 2023-05-16 | 中国电子科技集团公司第四十一研究所 | Terahertz wave large-area uniform illumination device and imaging method |
| CN111353940A (en) * | 2020-03-31 | 2020-06-30 | 成都信息工程大学 | Image super-resolution reconstruction method based on deep learning iterative up-down sampling |
| CN111784573A (en) * | 2020-05-21 | 2020-10-16 | 昆明理工大学 | Passive terahertz image super-resolution reconstruction method based on transfer learning |
| US20210209732A1 (en) * | 2020-06-17 | 2021-07-08 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Face super-resolution realization method and apparatus, electronic device and storage medium |
| US11710215B2 (en) * | 2020-06-17 | 2023-07-25 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Face super-resolution realization method and apparatus, electronic device and storage medium |
| CN112435162A (en) * | 2020-11-13 | 2021-03-02 | 中国科学院沈阳自动化研究所 | Terahertz image super-resolution reconstruction method based on complex field neural network |
| CN112435162B (en) * | 2020-11-13 | 2024-03-05 | 中国科学院沈阳自动化研究所 | Terahertz image super-resolution reconstruction method based on complex domain neural network |
| CN116309274A (en) * | 2022-12-12 | 2023-06-23 | 湖南红普创新科技发展有限公司 | Method and device for detecting small target in image, computer equipment and storage medium |
| CN116309274B (en) * | 2022-12-12 | 2024-01-30 | 湖南红普创新科技发展有限公司 | Method and device for detecting small target in image, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109740688B (en) | 2023-04-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109740688A (en) | A kind of terahertz image information interpretation method, network and storage medium | |
| Yan et al. | RigNet: Repetitive image guided network for depth completion | |
| Li et al. | Deep learning for remote sensing image classification: A survey | |
| Qingyun et al. | Cross-modality fusion transformer for multispectral object detection | |
| Abdollahi et al. | Building footprint extraction from high resolution aerial images using generative adversarial network (GAN) architecture | |
| Wang et al. | DDU-Net: Dual-decoder-U-Net for road extraction using high-resolution remote sensing images | |
| CN113222835B (en) | Remote sensing full-color and multi-spectral image distributed fusion method based on residual error network | |
| Tian et al. | Instance and panoptic segmentation using conditional convolutions | |
| WO2021086673A1 (en) | Object detection based on pixel differences | |
| CN111985374B (en) | Face positioning method and device, electronic equipment and storage medium | |
| CN110533004A (en) | A kind of complex scene face identification system based on deep learning | |
| CN113901909B (en) | Video-based target detection method and device, electronic equipment and storage medium | |
| Hamdi et al. | A New Image Enhancement and Super Resolution technique for license plate recognition | |
| CN112183649A (en) | Algorithm for predicting pyramid feature map | |
| Wang et al. | Improved YOLOX-X based UAV aerial photography object detection algorithm | |
| Peng et al. | Dynamic fusion network for RGBT tracking | |
| Makarov et al. | Depth inpainting via vision transformer | |
| JP6830742B2 (en) | A program for pixel-based image segmentation | |
| CN115018750A (en) | Medium-wave infrared hyperspectral and multispectral image fusion method, system and medium | |
| Dubey et al. | Transformer-based generative adversarial networks in computer vision: A comprehensive survey | |
| Huang et al. | Srnet: scale-aware representation learning network for dense crowd counting | |
| Pang et al. | TransCAR: Transformer-based camera-and-radar fusion for 3D object detection | |
| CN117474781A (en) | High spectrum and multispectral image fusion method based on attention mechanism | |
| Ren et al. | A lightweight object detection network in low-light conditions based on depthwise separable pyramid network and attention mechanism on embedded platforms | |
| Li et al. | A deep learning approach for face hallucination guided by facial boundary responses |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |