CN109584142A - Image Intensified System and method, training method, medium and electronic equipment - Google Patents
Image Intensified System and method, training method, medium and electronic equipment Download PDFInfo
- Publication number
- CN109584142A CN109584142A CN201811483510.3A CN201811483510A CN109584142A CN 109584142 A CN109584142 A CN 109584142A CN 201811483510 A CN201811483510 A CN 201811483510A CN 109584142 A CN109584142 A CN 109584142A
- Authority
- CN
- China
- Prior art keywords
- image
- access
- convolutional layer
- path part
- present
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000012549 training Methods 0.000 title claims abstract description 32
- 230000002708 enhancing effect Effects 0.000 claims abstract description 24
- 238000004364 calculation method Methods 0.000 claims abstract description 6
- 238000010276 construction Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 230000007423 decrease Effects 0.000 claims description 2
- 238000003860 storage Methods 0.000 abstract description 17
- 238000012545 processing Methods 0.000 description 19
- 238000010586 diagram Methods 0.000 description 14
- 230000000694 effects Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000945 filler Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000002844 melting Methods 0.000 description 1
- 230000008018 melting Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000003389 potentiating effect Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
Embodiments of the present invention provide a kind of Image Intensified System, including input terminal, multi-path part, unipath part and output end.Input terminal, for obtaining image to be processed.Multi-path part is connected with the input terminal, wherein the multi-path part includes multiple accesses, and each access includes at least one convolutional layer.Unipath part, including at least one convolutional layer, are connected with the multiple access, for the output of multiple accesses to be fused to the image enhancement result of the image to be processed by convolutional calculation.Output end is connected with the unipath part, for exporting described image enhancing result.In addition, embodiments of the present invention additionally provide a kind of image enchancing method, a kind of training method, a kind of computer readable storage medium and a kind of electronic equipment.
Description
Technical field
Embodiments of the present invention are related to field of image processing, more specifically, embodiments of the present invention are related to a kind of figure
Image intensifying system and method, training method, medium and electronic equipment.
Background technique
Background that this section is intended to provide an explanation of the embodiments of the present invention set forth in the claims or context.Herein
Description recognizes it is the prior art not because not being included in this section.
The method of image enhancement can be divided into two major classes at present, and one is traditional image enchancing methods, pass through various skies
Operator under domain and frequency domain enhances picture.Another kind is to enhance picture by the method for deep neural network.It sees at present, the
Two kinds of performance will be significantly better than the performance of the first.For the network EDSR to be won the championship with NTIRE2017, which is to have 32
Layer convolutional layer, while the network structure of residual error is introduced, which obtains good effect on super-resolution reconstruction.
Summary of the invention
But effect of the current deep neural network on normal data is preferable, but the effect on general data collection
It substantially reduces, the effect that model is able to ascend picture is again smaller, or even the picture generated the situation more worse than original image, nothing occurs
Method is used in production environment.
Thus, it is also very desirable to a kind of improved Image Intensified System, to overcome the problems, such as to apply in production environment.
In the present context, embodiments of the present invention are intended to provide a kind of Image Intensified System, by reducing network mould
The depth of type, to improve application effect in production environment.
In the first aspect of embodiment of the present invention, a kind of Image Intensified System, including input terminal, multi-path are provided
Partially, unipath part and output end.Input terminal, for obtaining image to be processed.Multi-path part, with the input terminal phase
Even, wherein the multi-path part includes multiple accesses, and each access includes at least one convolutional layer.Unipath part,
Including at least one convolutional layer, it is connected with the multiple access, for melting the output of multiple accesses by convolutional calculation
It is combined into the image enhancement result of the image to be processed.Output end is connected, for exporting described image with the unipath part
Enhance result.
In one embodiment of the invention, the multi-path part includes the different access of multiple constructions.
In another embodiment of the present invention, the quantity of the convolutional layer of any two access in the multi-path part
It is different.
In another embodiment of the invention, the corresponding volume of convolutional layer of any two access in the multi-path part
The width and/or height of product core are different.
In another embodiment of the present invention, the input terminal obtains image to be processed, and by the image to be processed
It is transmitted in each access of multi-path part respectively.
In another embodiment of the present invention, in the individual channel of the multi-path part, from the input terminal to
On the direction of the output end, the size of the corresponding convolution kernel of the convolutional layer is sequentially reduced or remains unchanged.
In another embodiment of the present invention, the plurality of different access includes the first access and alternate path, institute
The output for stating at least one convolutional layer of the first access is connected with the input of at least one convolutional layer of the alternate path.
In another embodiment of the present invention, each convolutional layer of each access exports in the multi-path part
Output image width and height it is identical as the width of the input picture that inputs the convolutional layer and highly difference.
In another embodiment of the present invention, each convolutional layer of each access exports in the multi-path part
The output port number of port number convolution kernel corresponding with the convolutional layer of image it is identical.
In yet another embodiment of the present invention, with each convolutional layer pair of each access in the multi-path part
The port number for the convolution kernel answered is identical.
In yet another embodiment of the present invention, the multi-path part includes 4 different accesses.
In the second aspect of embodiment of the present invention, a kind of image enchancing method is provided, including by image to be processed
The input terminal of Image Intensified System as described above is inputted, the processing result of the image to be processed is obtained.
In the third aspect of embodiment of the present invention, a kind of training method is provided, including obtain training set, the instruction
Practicing collection includes generating picture and true picture, and use the above-described Image Intensified System of training set training, wherein
Use the Euclidean distance of the true picture and the enhancing picture obtained based on the generation picture as loss function, by with
The mode of machine gradient decline optimizes loss function, with the training Image Intensified System.
In one embodiment of the invention, the method also includes in the output end access of described image enhancing system
Judgement system, forms countermeasure system, and the judgement system is used to differentiate that the image of input to be true picture or enhancing picture, makes
With the training set training countermeasure system, wherein the judgement system optimizes described image enhancing system by back-propagation algorithm
The parameter of system, and the separation countermeasure system, obtain trained Image Intensified System.
In the fourth aspect of embodiment of the present invention, a kind of medium is provided, is stored thereon with executable instruction, it is described
Unit processed is instructed to realize Image Intensified System as described above when executing.
In the 5th aspect of embodiment of the present invention, a kind of electronic equipment is provided, including, processing unit, Yi Jicun
Storage unit, is stored thereon with executable instruction, and described instruction realizes that image as described above increases when being executed by the processing unit
Strong system.
The Image Intensified System of embodiment handles image to be processed by multi-path part respectively according to the present invention, then
It is merged, reduces the depth of network model, to reduce trained difficulty, obtain preferable image enhancement effects.
Detailed description of the invention
The following detailed description is read with reference to the accompanying drawings, above-mentioned and other mesh of exemplary embodiment of the invention
, feature and advantage will become prone to understand.In the accompanying drawings, if showing by way of example rather than limitation of the invention
Dry embodiment, in which:
Fig. 1 schematically shows the schematic diagram of image enhancement according to an exemplary embodiment of the present invention；
Fig. 2 schematically shows the block diagrams of Image Intensified System according to an exemplary embodiment of the present invention；
Fig. 3 schematically shows the schematic diagram one of Image Intensified System according to an exemplary embodiment of the present invention；
Fig. 4 schematically shows the signals of an access in multi-path part according to an exemplary embodiment of the present invention
Figure；
Fig. 5 schematically shows the schematic diagram two of the Image Intensified System of another exemplary embodiment according to the present invention；
Fig. 6 schematically shows the schematic diagram three of the Image Intensified System of another exemplary embodiment according to the present invention；
Fig. 7 schematically shows the flow chart of training method according to an exemplary embodiment of the present invention；
Fig. 8 schematically shows the schematic diagram of countermeasure system according to an exemplary embodiment of the present invention；
Fig. 9 schematically shows the schematic diagram of computer readable storage medium according to an exemplary embodiment of the present invention；
And
Figure 10 schematically shows the block diagram of electronic equipment according to an exemplary embodiment of the present invention.
In the accompanying drawings, identical or corresponding label indicates identical or corresponding part.
Specific embodiment
The principle and spirit of the invention are described below with reference to several illustrative embodiments.It should be appreciated that providing this
A little embodiments are used for the purpose of making those skilled in the art can better understand that realizing the present invention in turn, and be not with any
Mode limits the scope of the invention.On the contrary, these embodiments are provided so that this disclosure will be more thorough and complete, and energy
It is enough that the scope of the present disclosure is completely communicated to those skilled in the art.
One skilled in the art will appreciate that embodiments of the present invention can be implemented as a kind of system, device, equipment, method
Or computer program product.Therefore, the present disclosure may be embodied in the following forms, it may be assumed that complete hardware, complete software
The form that (including firmware, resident software, microcode etc.) or hardware and software combine.
Embodiment according to the present invention proposes a kind of Image Intensified System and method, training method, medium and electricity
Sub- equipment.
In addition, any number of elements in attached drawing is used to example rather than limitation and any name are only used for distinguishing,
Without any restrictions meaning.
Below with reference to several representative embodiments of the invention, the principle and spirit of the present invention are explained in detail.
Summary of the invention
Field of image enhancement at present, deep neural network all use narrow and deep network, the network won the championship with NTIRE2017
For EDSR, which has 32 layers CNN layers, while introducing residual error network structure.But the inventors discovered that these it is narrow and
Effect of the deep network on normal data is good, but since the image of general data collection can pass through various compressions, is turned
Code, recompression, watermarking etc., for actual effect with regard to very poor, the picture of generation, can not be in life it is easy to appear the situation poorer than original image
Environment is produced to use.It has been recognised by the inventors that the main reason is that being that network is too deep, so that the inadequate robust of model.In order to solve this
Problem, exemplary embodiment of the present provide a kind of Image Intensified System, handle figure to be processed respectively by multi-path part
Picture is then merged, and the depth of network model is reduced, to reduce trained difficulty, can be obtained preferable image and be increased
Potent fruit.
After introduced the basic principles of the present invention, lower mask body introduces various non-limiting embodiment party of the invention
Formula.
Application scenarios overview
The signal of image enhancement according to an exemplary embodiment of the present invention is schematically shown referring initially to Fig. 1, Fig. 1
Figure.
As shown in Figure 1, image 110 to be processed can be the image obtained from various actual scenes, due to actual scene
It is difficult to obtain ideal image, therefore clarity is not high.This image is directly used in various application scenarios (such as image recognition
Deng) be difficult to obtain good effect.Image Intensified System and method obtain enhanced image for handling such image
120。
It should be noted which is shown only for the purpose of facilitating an understanding of the spirit and principles of the present invention for above-mentioned application scenarios, this
The embodiment of invention is unrestricted in this regard.On the contrary, embodiments of the present invention can be applied to it is applicable any
Scene.
Exemplary system
Below with reference to the application scenarios of Fig. 1, the image of illustrative embodiments according to the present invention is described with reference to Fig. 2~Fig. 4
Enhancing system.
Fig. 2 schematically shows the block diagrams of Image Intensified System 200 according to an exemplary embodiment of the present invention.
As shown in Fig. 2, the system includes input terminal 210, multi-path part 220, unipath part 230 and output end
240。
Input terminal 210, for obtaining image to be processed.
Multi-path part 220 is connected with the input terminal, wherein the multi-path part includes multiple accesses, Mei Gesuo
Stating access includes at least one convolutional layer.
Unipath part 230, including at least one convolutional layer, are connected with the multiple access, and being used for will be multiple described logical
The output on road is fused to the image enhancement result of the image to be processed by convolutional calculation.
Output end 240 is connected with the unipath part, for exporting described image enhancing result.
It is introduced below with reference to Image Intensified System 200 of the Fig. 3 and Fig. 4 to the embodiment of the present disclosure.
Fig. 3 schematically shows the schematic diagram one of Image Intensified System 200 according to an exemplary embodiment of the present invention.
As shown in figure 3, the input terminal 210 obtains image to be processed, such as image to be processed 110 shown in FIG. 1, and will
The image to be processed is transmitted to respectively in each access of multi-path part 220.According to an exemplary embodiment of the present, described
Multi-path part for example may include 4 different accesses 221,222,223 and 224.The image to be processed is through multiple and different
After access processing, the processing result of individual channel will be transferred into unipath part 230.Those processing results are through unipath part
Fusion and processing, generate enhanced image, such as enhancing image 120 shown in FIG. 1, and be sent to output end, by output end
Export the enhanced image.
Fig. 4 schematically shows an accesses in multi-path part according to an exemplary embodiment of the present invention (with logical
For road 221) schematic diagram.
As shown in figure 4, the front end of the access 221 is connected with input terminal 210, rear end is connected with unipath part 230.This is logical
Road 221 includes at least one convolutional layer, such as shown in figure 2211,2212,2213 and 2214 etc., and each convolutional layer corresponds to
One convolution kernel, each convolutional layer are used to handle the input information received by corresponding convolution kernel, obtain output letter
It ceases and transmits backward.
According to an exemplary embodiment of the present, the construction of the individual channel of the multi-path part can be identical, or
Person, the multi-path part may include the different access of multiple constructions.For example, multi-path part can have two constructions not
Same or identical access.For another example, multi-path part can have three or more accesses, wherein can include at least two structures
Any two construction made between different access or those accesses is different from.
According to an exemplary embodiment of the present, it may include again a variety of situations that the construction of two paths is different.For example, two
The convolution layer number for including in access is different, although alternatively, convolution layer number is identical, each convolutional layer in two paths
The size of convolution kernel is different, such as is in width, height or the port number difference of the convolution kernel of two convolutional layers of corresponding position.
According to an exemplary embodiment of the present, the three dimensional convolution kernel used has width, height and port number, and it is, for example, possible to use 3*
The convolution kernel of the size of 3*96, width are 3 pixels, are highly 3 pixels, have 96 channels.The multi-pass of the present embodiment
The quantity of the convolutional layer of any two access in the part of road can be different, and/or, it is any in the multi-path part
The width and/or height of the corresponding convolution kernel of the convolutional layer of two paths can be different.
The Image Intensified System of illustrative embodiments handles figure to be processed by multi-path part respectively according to the present invention
Picture is then merged, and the depth of network model is reduced, to reduce trained difficulty, obtains preferable image enhancement effect
Fruit.
According to an exemplary embodiment of the present, in the individual channel of the multi-path part, from the input terminal to institute
It states on the direction of output end, the size of the corresponding convolution kernel of the convolutional layer is sequentially reduced or remains unchanged.For example, from input
It holds on the direction of output end, an access, which may include, successively has 9*9*96,7*7*96,5*5*96,3*3*96,3*3*96
Convolution kernel five convolutional layers, size is sequentially reduced.
According to an exemplary embodiment of the present, the plurality of different access includes the first access and alternate path, described
The output of at least one convolutional layer of the first access is connected with the input of at least one convolutional layer of the alternate path.Below
It is illustrated in conjunction with Image Intensified System of the Fig. 5 to exemplary embodiment of the present.
Fig. 5 schematically shows the schematic diagram of the Image Intensified System 500 of another exemplary embodiment according to the present invention
Two.
As shown in figure 5, in the Image Intensified System 500, including multiple accesses, wherein access 222 and access 223 are simultaneously
It is non-fully independent.One convolutional layer 2223 of access 222 is connected with a convolutional layer 2233 of access 223, convolutional layer 2223
Output can be used as the input of convolutional layer 2233.According to an exemplary embodiment of the present, convolutional layer 2233 is in addition in channel 223
Interior one convolutional layer of the past obtains outside information, also obtains information from convolutional layer 2223, is used as convolutional layer after two parts information is merged
2233 input.For example, two outputs are the image data of 256*256*96 size, 256*256* can be merged into
192 image data, the input as convolutional layer 2233.
The Image Intensified System that exemplary embodiment of the present provides can reduce meter by the multiplexing between access significantly
Calculation amount improves reinforcing effect.
According to an exemplary embodiment of the present, each convolutional layer output of each access in the multi-path part
Width and the height for exporting image are identical as the width of the input picture of the input convolutional layer and height difference.The multi-path
The port number convolution kernel corresponding with the convolutional layer of the output image of each convolutional layer output of each access in part
Port number it is identical.
According to an exemplary embodiment of the present, corresponding with each convolutional layer of each access in the multi-path part
Convolution kernel port number it is identical.
In the present embodiment, in order to guarantee that the size for exporting image is consistent with the size of input picture, the image can be made
The output of each convolutional layer in enhancing system is consistent with the size of input.On the other hand, due to having connection between multiple accesses,
Such as convolutional layer 2233 shown in fig. 5, the port number of input image data is 192, in order to make to pass in multi-path part
The port number for the image data passed is constant, needs that its port number is made to be reduced to 96 after convolution.In order to achieve the goal above, this hair
Each convolutional layer of bright exemplary embodiment carries out process of convolution using following convolution mode.
The convolution mode that exemplary embodiment of the present uses is referred to alternatively as " same " mode.It is any channel for input
The image data (having a size of m × n × k) of quantity, can first be handled the image data.Exemplary reality according to the present invention
Example is applied, can be by the data processing in multiple channels, such as the data processing in multiple channels is a number by the mode that is averaged
Value, and image is expanded into (x-1) row (y-1) column (setting the size of convolution kernel as x × y × p), it obtains having a size of (m+x-1) × (n+
Y-1 two-dimensional image data).Right and lower section filler pixels of the image completion technology in image can be for example used when expansion
Point.It according to an exemplary embodiment of the present, can be in the region of extension all with the filling of 0 value.After treatment, using having a size of x
The convolution kernel processing of × y × p carries out convolution algorithm to treated the image data, and the step-length of convolution is 1 pixel, obtains ruler
Very little is m × n × p output result, wherein the calculating for each channel, the two-dimensional image data and one that is all that treated
The convolutional calculation of a two-dimensional convolution core, wherein the two-dimensional convolution core is logical at some having a size of x × y × p three dimensional convolution kernel
Part in road.As it can be seen that the length and width dimensions of image are constant, and port number is defined as after the convolution operation of " same "
It is identical as the port number of convolution kernel.
Fig. 6 schematically shows the schematic diagram of the Image Intensified System 600 of another exemplary embodiment according to the present invention
Three.
As shown in fig. 6, the Image Intensified System 600 includes input terminal 210, multi-path part 220, unipath part 230
And output end 240.Wherein, multi-path part 220 includes 4 accesses, herein by being from top to bottom sequentially referred to as the in figure
One access, alternate path, third path and fourth passage.The convolution kernel size difference of each access is as follows:
First access: 3 3*3*96 convolution kernels；
Alternate path: the convolution kernel of 1 5*5*96, the convolution kernel of 3 3*3*96；
Third path: the convolution kernel of 1 7*7*96, the convolution kernel of 1 5*5*96, the convolution kernel of 3 3*3*96；
Fourth passage: the convolution kernel of 1 9*9*96, the convolution kernel of 1 7*7*96, the convolution kernel of 1 5*5*96,3 3*
The convolution kernel of 3*96.
Before the layer and layer of network, the first access third layer exports the input that can be transferred to alternate path third layer.The
The output of two access third layer can be transferred to the input of third path third layer, and the output of third path third layer can be transferred to
The input of four access third layer.
It is transported using the convolution kernel of a 3*3*3 according to the convolution of above-described same mode in unipath part 230
It calculates, by the channel map of 96 features at 3 channels, which may be used as 3 color channels of RGB image, generates and increases
Strong color image.The Image Intensified System of the embodiment of the present invention can obtain preferable effect.
Illustrative methods
After describing the system of exemplary embodiment of the invention, next, to exemplary embodiment of the invention
Training method be illustrated.
Fig. 7 schematically shows the flow chart of training method according to an exemplary embodiment of the present invention.
As shown in fig. 7, this method includes operation S710 and S720.
In operation S710, training set is obtained, the training set includes generating picture and true picture.Wherein, the generation figure
Piece can be true picture it is down-sampled after picture.According to an exemplary embodiment of the present, down-sampled method for example can wrap
Include Gaussian Blur, JEPG compression, direct down-sampling etc..The input of the generation picture as Image Intensified System, it is available defeated
Enhancing picture out.
In operation S720, the above-described Image Intensified System of training set training is used, wherein using described true
Picture, as loss function, passes through stochastic gradient descent with the Euclidean distance for enhancing picture obtained based on the generation picture
Mode optimizes loss function, with the training Image Intensified System.
According to an exemplary embodiment of the present, by calculate true picture with based on generating enhancing picture that picture obtains
Euclidean distance has quantified the difference for enhancing picture and true picture, by minimizing the difference, reaches training image enhancing system
The purpose of system.
Fig. 8 schematically shows the schematic diagram of countermeasure system according to an exemplary embodiment of the present invention.
As shown in figure 8, according to an exemplary embodiment of the present, the method also includes in described image enhancing system
Output end accesses judgement system, forms countermeasure system, the judgement system be used to differentiate the image of input for true picture or
Enhance picture, uses the training set training countermeasure system, wherein the judgement system passes through described in back-propagation algorithm optimization
The parameter of Image Intensified System, and the separation countermeasure system, obtain trained Image Intensified System.
According to an exemplary embodiment of the present, which fights neural network (GAN).Wherein, judgement system is used
In identification enhancing picture and true picture, if it is possible to which correct identification then returns to the parameter of adjustment Image Intensified System, improves figure
The ability of image intensifying system enhancing image；If identifying mistake, just adjustment differentiates the parameter of network, improves and differentiates Network Recognition figure
The ability of picture.By constantly training, the ability of Image Intensified System enhancing image can be effectively improved.
Exemplary embodiment of the present additionally provides a kind of image enchancing method, including image to be processed is inputted institute as above
The input terminal for the Image Intensified System stated obtains the processing result of the image to be processed.
Exemplary media
After describing the system of exemplary embodiment of the invention, next, with reference to Fig. 9 to the exemplary reality of the present invention
A kind of computer readable storage medium for applying mode is illustrated.Exemplary embodiment of the invention provides a kind of computer can
Storage medium is read, executable instruction is stored thereon with, described instruction unit processed realizes image as described above when executing
Enhancing system.
In some possible embodiments, various aspects of the invention are also implemented as a kind of shape of program product
Formula comprising program code, when described program product is run on an electronic device, said program code is for making the electronics
Equipment realizes described in above-mentioned " exemplary system " part of this specification the figure of various illustrative embodiments according to the present invention
Image intensifying system, for example, Image Intensified System 200 as shown in Figure 2 may be implemented in the electronic equipment.
Described program product can be using any combination of one or more readable mediums.Readable medium can be readable letter
Number medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example may be-but not limited to-electricity, magnetic, optical, electromagnetic, red
The system of outside line or semiconductor, device or device, or any above combination.The more specific example of readable storage medium storing program for executing
(non exhaustive list) includes: the electrical connection with one or more conducting wires, portable disc, hard disk, random access memory
(RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc
Read memory (CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
As shown in figure 9, describing the image enhancement program product 900 of embodiment according to the present invention, can use
Portable compact disc read only memory (CD-ROM) and including program code, and can be on electronic equipment, such as PC
Operation.However, program product of the invention is without being limited thereto, in this document, readable storage medium storing program for executing, which can be, any to be included or deposits
The tangible medium of program is stored up, which can be commanded execution system, device or device use or in connection.
Readable signal medium may include in a base band or as the data-signal that carrier wave a part is propagated, wherein carrying
Readable program code.The data-signal of this propagation can take various forms, including --- but being not limited to --- electromagnetism letter
Number, optical signal or above-mentioned any appropriate combination.Readable signal medium can also be other than readable storage medium storing program for executing it is any can
Read medium, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or
Program in connection.
The program code for including on readable medium can transmit with any suitable medium, including --- but being not limited to ---
Wirelessly, wired, optical cable, RF etc. or above-mentioned any appropriate combination.It can be with one or more programming languages
Any combination writes the program code for executing operation of the present invention, and described program design language includes the program of object-oriented
Design language --- such as Java, C++ etc. further includes conventional procedural programming language --- such as " C ", language or class
As programming language.Program code can be executed fully on consumer electronic devices, partially on consumer electronic devices
Part executes in devices in remote electronic or executes in devices in remote electronic or server completely.It is being related to electronic remote
In the situation of equipment, devices in remote electronic can pass through the network of any kind --- including local area network (LAN) or wide area network
(WAN) one consumer electronic devices are connected to, or, it may be connected to external electronic device (such as provided using Internet service
Quotient is connected by internet).
Example electronic device
After the method, system and medium for describing exemplary embodiment of the invention, next, with reference to Figure 10 to this
The electronic equipment of invention illustrative embodiments is illustrated.
The embodiment of the invention also provides a kind of electronic equipment.Person of ordinary skill in the field is it is understood that this hair
Bright various aspects can be implemented as system, method or program product.Therefore, various aspects of the invention can be implemented as
Following form, it may be assumed that complete hardware embodiment, complete Software Implementation (including firmware, microcode etc.) or hardware and
The embodiment that software aspects combine, may be collectively referred to as circuit, " module " or " system " here.
In some possible embodiments, image enhancement electronic equipment according to the present invention can include at least at least one
A processing unit and at least one storage unit.Wherein, the storage unit is stored with program code, when described program generation
When code is by the processing unit execution processing unit is realized in above-mentioned " exemplary system " part of this specification to describe
Illustrative embodiments various according to the present invention Image Intensified System.For example, the processing unit may be implemented as in Fig. 2
Shown in Image Intensified System 200.
The image enhancement electronic equipment 1000 of this embodiment according to the present invention is described referring to Figure 10.Such as figure
Electronic equipment 1000 shown in 1000 is only an example, should not function to the embodiment of the present invention and use scope bring and appoint
What is limited.
As shown in Figure 10, electronic equipment 1000 is showed in the form of universal electronic device.The component of electronic equipment 1000 can
To include but is not limited to: at least one above-mentioned processing unit 1010, connects not homologous ray at least one above-mentioned storage unit 1020
The bus 1030 of component (including storage unit 1020 and processing unit 1010).
Bus 1030 includes data/address bus, address bus and control bus.
Storage unit 1020 may include volatile memory, such as random access memory (RAM) 1021 and/or high speed
Buffer memory 1022 can further include read-only memory (ROM) 1023.
Storage unit 1020 can also include program/utility with one group of (at least one) program module 1024
1025, such program module 1024 includes but is not limited to: operating system, one or more application program, other program moulds
It may include the realization of network environment in block and program data, each of these examples or certain combination.
Electronic equipment 1000 can also be with one or more external equipments 1040 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communicate, this communication can be carried out by input/output (I/O) interface 1050.Also, electronic equipment 1000 can also lead to
Cross network adapter 1060 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network,
Such as internet) communication.As shown, network adapter 1060 is logical by bus 1030 and other modules of electronic equipment 1000
Letter.It should be understood that although not shown in the drawings, can in conjunction with electronic equipment 1000 use other hardware and/or software module, including
But it is not limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive
And data backup storage system etc..
It should be noted that although being referred to several units/modules or son list of Image Intensified System in the above detailed description
Member/module, but it is this division be only exemplary it is not enforceable.In fact, embodiment according to the present invention, on
The feature and function of two or more units/modules of text description can embody in a units/modules.Conversely, above
The feature and function of one units/modules of description can be to be embodied by multiple units/modules with further division.
In addition, although describing the operation of the method for the present invention in the accompanying drawings with particular order, this do not require that or
Hint must execute these operations in this particular order, or have to carry out shown in whole operation be just able to achieve it is desired
As a result.Additionally or alternatively, it is convenient to omit multiple steps are merged into a step and executed by certain steps, and/or by one
Step is decomposed into execution of multiple steps.
Although detailed description of the preferred embodimentsthe spirit and principles of the present invention are described by reference to several, it should be appreciated that, this
It is not limited to the specific embodiments disclosed for invention, does not also mean that the feature in these aspects cannot to the division of various aspects
Combination is benefited to carry out, this to divide the convenience merely to statement.The present invention is directed to cover appended claims spirit and
Included various modifications and equivalent arrangements in range.
Claims (10)
1. a kind of Image Intensified System, comprising:
Input terminal, for obtaining image to be processed；
Multi-path part is connected with the input terminal, wherein the multi-path part includes multiple accesses, each access
Including at least one convolutional layer；
Unipath part, including at least one convolutional layer, are connected with the multiple access, for by the output of multiple accesses
The image enhancement result of the image to be processed is fused to by convolutional calculation；
Output end is connected with the unipath part, for exporting described image enhancing result.
2. Image Intensified System according to claim 1, wherein the multi-path part, which includes that multiple constructions are different, to be led to
Road, for any two access in the multi-path part:
The quantity of the convolutional layer of described two accesses is different；And/or
The width and/or height of convolution kernel corresponding from the convolutional layer of described two accesses are different.
3. Image Intensified System according to claim 1, wherein the input terminal obtains image to be processed, and will be described
Image to be processed is transmitted to respectively in each access of multi-path part.
4. Image Intensified System according to claim 1, wherein in the individual channel of the multi-path part, from institute
It states on input terminal to the direction of the output end, the size of the corresponding convolution kernel of the convolutional layer is sequentially reduced or remains unchanged.
5. Image Intensified System according to claim 1, wherein the plurality of different access includes the first access and the
Two accesses, the output and the input of at least one convolutional layer of the alternate path of at least one convolutional layer of first access
It is connected.
6. Image Intensified System according to claim 1, in which:
The width of the output image of each convolutional layer output of each access and height and input in the multi-path part
The width of the input picture of the convolutional layer is identical respectively with height；
The port number of the output image of each convolutional layer output of each access and the convolution in the multi-path part
The port number of the corresponding convolution kernel of layer is identical.
7. Image Intensified System according to claim 6, wherein in the multi-path part each access it is each
The port number of the corresponding convolution kernel of a convolutional layer is identical.
8. Image Intensified System according to claim 1, wherein the multi-path part includes 4 different accesses.
9. a kind of training method, comprising:
Training set is obtained, the training set includes generating picture and true picture；
Use Image Intensified System of the training set training as described in any one of claim 1~8, wherein use institute
True picture is stated with the Euclidean distance of the enhancing picture obtained based on the generation picture as loss function, passes through stochastic gradient
The mode of decline optimizes loss function, with the training Image Intensified System.
10. according to the method described in claim 9, further include:
Judgement system is accessed in the output end of described image enhancing system, forms countermeasure system, the judgement system is for differentiating
The image of input is true picture or enhancing picture；
Use the training set training countermeasure system, wherein the judgement system optimizes described image by back-propagation algorithm
The parameter of enhancing system；And
The countermeasure system is separated, trained Image Intensified System is obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811483510.3A CN109584142A (en) | 2018-12-05 | 2018-12-05 | Image Intensified System and method, training method, medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811483510.3A CN109584142A (en) | 2018-12-05 | 2018-12-05 | Image Intensified System and method, training method, medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109584142A true CN109584142A (en) | 2019-04-05 |
Family
ID=65927568
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811483510.3A Pending CN109584142A (en) | 2018-12-05 | 2018-12-05 | Image Intensified System and method, training method, medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109584142A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378419A (en) * | 2019-07-19 | 2019-10-25 | 广东浪潮大数据研究有限公司 | A kind of image set extending method, device, equipment and readable storage medium storing program for executing |
| CN111681177A (en) * | 2020-05-18 | 2020-09-18 | 腾讯科技（深圳）有限公司 | Video processing method and device, computer readable storage medium and electronic equipment |
| CN111445392B (en) * | 2020-03-20 | 2023-09-15 | Oppo广东移动通信有限公司 | Image processing method and device, computer readable storage medium and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101949768A (en) * | 2010-08-20 | 2011-01-19 | 中国科学院光电技术研究所 | Processor of Hartmann -Shack front slope relative to point target and manufacture method |
| CN103984959A (en) * | 2014-05-26 | 2014-08-13 | 中国科学院自动化研究所 | Data-driven and task-driven image classification method |
| CN104281858A (en) * | 2014-09-15 | 2015-01-14 | 中安消技术有限公司 | Three-dimensional convolutional neutral network training method and video anomalous event detection method and device |
| CN107464210A (en) * | 2017-07-06 | 2017-12-12 | 浙江工业大学 | A kind of image Style Transfer method based on production confrontation network |
| CN107784654A (en) * | 2016-08-26 | 2018-03-09 | 杭州海康威视数字技术股份有限公司 | Image partition method, device and full convolutional network system |
| CN107886474A (en) * | 2017-11-22 | 2018-04-06 | 北京达佳互联信息技术有限公司 | Image processing method, device and server |
| CN108875752A (en) * | 2018-03-21 | 2018-11-23 | 北京迈格威科技有限公司 | Image processing method and device, computer readable storage medium |
| CN108921822A (en) * | 2018-06-04 | 2018-11-30 | 中国科学技术大学 | Image object method of counting based on convolutional neural networks |
-
2018
- 2018-12-05 CN CN201811483510.3A patent/CN109584142A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101949768A (en) * | 2010-08-20 | 2011-01-19 | 中国科学院光电技术研究所 | Processor of Hartmann -Shack front slope relative to point target and manufacture method |
| CN103984959A (en) * | 2014-05-26 | 2014-08-13 | 中国科学院自动化研究所 | Data-driven and task-driven image classification method |
| CN104281858A (en) * | 2014-09-15 | 2015-01-14 | 中安消技术有限公司 | Three-dimensional convolutional neutral network training method and video anomalous event detection method and device |
| CN107784654A (en) * | 2016-08-26 | 2018-03-09 | 杭州海康威视数字技术股份有限公司 | Image partition method, device and full convolutional network system |
| CN107464210A (en) * | 2017-07-06 | 2017-12-12 | 浙江工业大学 | A kind of image Style Transfer method based on production confrontation network |
| CN107886474A (en) * | 2017-11-22 | 2018-04-06 | 北京达佳互联信息技术有限公司 | Image processing method, device and server |
| CN108875752A (en) * | 2018-03-21 | 2018-11-23 | 北京迈格威科技有限公司 | Image processing method and device, computer readable storage medium |
| CN108921822A (en) * | 2018-06-04 | 2018-11-30 | 中国科学技术大学 | Image object method of counting based on convolutional neural networks |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110378419A (en) * | 2019-07-19 | 2019-10-25 | 广东浪潮大数据研究有限公司 | A kind of image set extending method, device, equipment and readable storage medium storing program for executing |
| CN110378419B (en) * | 2019-07-19 | 2021-07-16 | 广东浪潮大数据研究有限公司 | Image set expansion method, device, equipment and readable storage medium |
| CN111445392B (en) * | 2020-03-20 | 2023-09-15 | Oppo广东移动通信有限公司 | Image processing method and device, computer readable storage medium and electronic equipment |
| CN111681177A (en) * | 2020-05-18 | 2020-09-18 | 腾讯科技（深圳）有限公司 | Video processing method and device, computer readable storage medium and electronic equipment |
| CN111681177B (en) * | 2020-05-18 | 2022-02-25 | 腾讯科技（深圳）有限公司 | Video processing method and device, computer readable storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110263913A (en) | A kind of deep neural network compression method and relevant device | |
| CN109584142A (en) | Image Intensified System and method, training method, medium and electronic equipment | |
| CN109558832A (en) | A kind of human body attitude detection method, device, equipment and storage medium | |
| CN110599492A (en) | Training method and device for image segmentation model, electronic equipment and storage medium | |
| CN108765481A (en) | A kind of depth estimation method of monocular video, device, terminal and storage medium | |
| CN108027885A (en) | Space transformer module | |
| CN110059728B (en) | RGB-D image visual saliency detection method based on attention model | |
| CN108121931A (en) | two-dimensional code data processing method, device and mobile terminal | |
| WO2023174098A1 (en) | Real-time gesture detection method and apparatus | |
| CN109902723A (en) | Image processing method and device | |
| CN110458011A (en) | Character recognition method and device, computer equipment and readable medium end to end | |
| CN115311720B (en) | Method for generating deepfake based on transducer | |
| Chen et al. | StereoEngine: An FPGA-based accelerator for real-time high-quality stereo estimation with binary neural network | |
| CN109003297A (en) | A kind of monocular depth estimation method, device, terminal and storage medium | |
| CN112233012A (en) | Face generation system and method | |
| CN114332573A (en) | Multi-mode information fusion recognition method and system based on attention mechanism | |
| CN109064407A (en) | Intensive connection network image super-resolution method based on multi-layer perception (MLP) layer | |
| CN114187165A (en) | Image processing method and device | |
| JP2022101645A (en) | Encryption mask determination method, image recognition method, model training method, apparatus, electronic device, storage medium, and computer program | |
| CN116569218A (en) | Image processing method and image processing apparatus | |
| CN111294614B (en) | Method and apparatus for digital image, audio or video data processing | |
| CN112435197A (en) | Image beautifying method and device, electronic equipment and storage medium | |
| CN115984949B (en) | Low-quality face image recognition method and equipment with attention mechanism | |
| CN110110775A (en) | A kind of matching cost calculation method based on hyper linking network | |
| CN108460335A (en) | The recognition methods of video fine granularity, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190405 |
|
| RJ01 | Rejection of invention patent application after publication |