CN108898557A - Image recovery method and device, electronic equipment, computer program and storage medium - Google Patents
Image recovery method and device, electronic equipment, computer program and storage medium Download PDFInfo
- Publication number
- CN108898557A CN108898557A CN201810541974.9A CN201810541974A CN108898557A CN 108898557 A CN108898557 A CN 108898557A CN 201810541974 A CN201810541974 A CN 201810541974A CN 108898557 A CN108898557 A CN 108898557A
- Authority
- CN
- China
- Prior art keywords
- image
- currently pending
- prior model
- pending image
- recovery
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000011084 recovery Methods 0.000 title claims abstract description 140
- 238000000034 method Methods 0.000 title claims abstract description 111
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 238000004590 computer program Methods 0.000 title claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 134
- 230000015556 catabolic process Effects 0.000 claims abstract description 91
- 238000006731 degradation reaction Methods 0.000 claims abstract description 91
- 238000000605 extraction Methods 0.000 claims description 19
- 238000003709 image segmentation Methods 0.000 claims description 14
- 238000013528 artificial neural network Methods 0.000 description 31
- 239000011159 matrix material Substances 0.000 description 29
- 238000013527 convolutional neural network Methods 0.000 description 26
- 230000000694 effects Effects 0.000 description 20
- 230000006854 communication Effects 0.000 description 12
- 230000009466 transformation Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 11
- 238000009434 installation Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000010801 machine learning Methods 0.000 description 9
- 238000005520 cutting process Methods 0.000 description 8
- 238000012549 training Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000005304 joining Methods 0.000 description 4
- 238000003672 processing method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000005036 nerve Anatomy 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000007850 degeneration Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a kind of image recovery method and device, electronic equipment, computer program and storage mediums, wherein method includes：Prior model is constructed according to the characteristic pattern of currently pending image, the prior model is used to learn the information of the currently pending image, and the currently pending image includes at least one image degradation phenomena；Based on the prior model, image recovery processing is carried out for the described image degradation phenomena of the currently pending image, obtains target image.The embodiment of the present invention can obtain good image restoration result.
Description
Technical field
The present invention relates to computer vision technique, especially a kind of image recovery method and device, electronic equipment, computer
Program and storage medium.
Background technique
Image is in formation, record, processing, transmission process since imaging system, recording equipment, processing method and transmission are situated between
Matter it is not perfect, the degradation phenomenas such as noise, fuzzy, partial pixel missing can be generated, cause image quality decrease.
Image recovery is a kind of the problem of being widely noticed in computer vision, it is proposed for image degradation phenomena
, it is intended to it is rebuild or is restored to obtain the original image for not generating degeneration in the image of quality decline.
Summary of the invention
The embodiment of the present invention provides a kind of image-recovery technique scheme.
According to an aspect of an embodiment of the present invention, a kind of image recovery method is provided, including：
According to the characteristic pattern of currently pending image construct prior model, the prior model for learn it is described currently to
The information of image is handled, the currently pending image includes at least one image degradation phenomena；
Based on the prior model, image recovery is carried out for the described image degradation phenomena of the currently pending image
Processing, obtains corresponding target image.
Optionally, in above method embodiment of the present invention, described image degradation phenomena is comprised at least one of the following：Image
Noise, image are fuzzy and image pixel lacks；
Described image recovery processing comprises at least one of the following：Image denoising processing, image deblurring processing and image slices
Plain completion processing.
Optionally, in any of the above-described embodiment of the method for the present invention, the characteristic pattern structure according to currently pending image
It builds before prior model, further includes：
Feature extraction is carried out to the currently pending image, obtains that there is identical size with the currently pending image
One group of characteristic pattern；
It is described that prior model is constructed according to the characteristic pattern of currently pending image, including：
According to one group of characteristic pattern building prior model with the currently pending image with identical size.
Optionally, in any of the above-described embodiment of the method for the present invention, the basis has with the currently pending image
Before one group of characteristic pattern of identical size constructs the prior model, further include：
Image point is carried out to each characteristic pattern in one group of characteristic pattern with the currently pending image with identical size
Processing is cut, one group of part that wherein each characteristic pattern corresponds respectively to each predeterminable area in the currently pending image is obtained
Characteristic pattern；
The basis with the currently pending image there is one group of characteristic pattern of identical size to construct the prior model,
Including：
According to the local feature for corresponding to identical predeterminable area in the currently pending image in each group local feature figure
Figure determines one group of prior model for corresponding respectively to each predeterminable area in the currently pending image；
It is described to be based on the prior model, image is carried out for the described image degradation phenomena of the currently pending image
Recovery is handled, and before obtaining corresponding target image, further includes：
Image dividing processing is carried out to the currently pending image, obtains corresponding to each in the currently pending image
The local image to be processed of one group of predeterminable area；
It is described to be based on the prior model, image is carried out for the described image degradation phenomena of the currently pending image
Recovery processing, obtains corresponding target image, including：
To each predeterminable area in the currently pending image, it is based respectively on the corresponding prior model, for right
The described image degradation phenomena for the part image to be processed answered carries out image recovery processing, obtains corresponding respectively to described work as
One group of localized target image of each predeterminable area in preceding image to be processed；
It is described to be based on the prior model, image is carried out for the described image degradation phenomena of the currently pending image
Recovery is handled, and after obtaining corresponding target image, further includes：
One group of localized target image for corresponding respectively to each predeterminable area in the currently pending image is spliced
Processing, obtains the target image.
Optionally, described according to corresponding to institute in each group local feature figure in any of the above-described embodiment of the method for the present invention
The local feature figure of identical predeterminable area in currently pending image is stated, determination corresponds respectively in the currently pending image
One group of prior model of each predeterminable area, including：
Correspond to the part of identical predeterminable area in the currently pending image in the local feature figure according to each group
Characteristic pattern, building corresponds to one group of part non-directed graph of each predeterminable area in the currently pending image respectively；
According to one group of part non-directed graph for corresponding respectively to each predeterminable area in the currently pending image, difference is determined
One group of prior model corresponding to each predeterminable area in the currently pending image.
Optionally, corresponding in the local feature figure according to each group in any of the above-described embodiment of the method for the present invention
The local feature figure of identical predeterminable area in the currently pending image, building corresponds to the currently pending figure respectively
One group of part non-directed graph of each predeterminable area as in, including：
To each predeterminable area in the currently pending image, according to each office for corresponding to each predeterminable area
The distance between pixel determines the weight in local non-directed graph between respective pixel in portion's characteristic pattern；
The weight between pixel determined according to each local feature figure for corresponding to each predeterminable area, building pair
The local non-directed graph of predeterminable area described in Ying Yu.
Optionally, in any of the above-described embodiment of the method for the present invention, the basis corresponds to each of each predeterminable area
The distance between pixel determines the weight in local non-directed graph between respective pixel in the local feature figure, including：
According to being less than or equal to pre-determined distance threshold value in each local feature figure for corresponding to each predeterminable area
The distance between pixel determines the weight in the non-directed graph of part between respective pixel.
Optionally, described to be based on the prior model in any of the above-described embodiment of the method for the present invention, for described current
The described image degradation phenomena of image to be processed carries out image recovery processing：
Preliminary recovery processing carried out to the described image degradation phenomena of the currently pending image, at the preliminary recovery
Image after reason is of the same size with the currently pending image；
It is described to be based on the prior model, image is carried out for the described image degradation phenomena of the currently pending image
Recovery processing, obtains corresponding target image, including：
Based on the prior model, figure is carried out for the described image degradation phenomena of the preliminary image that restores that treated
As recovery processing, the target image is obtained.
Optionally, in any of the above-described embodiment of the method for the present invention, the prior model includes：Figure Laplacian Matrix.
Optionally, described to be based on the prior model in any of the above-described embodiment of the method for the present invention, for described current
The described image degradation phenomena of image to be processed carries out image recovery processing, obtains corresponding target image, including：
By solving quadratic programming of the currently pending image using the figure Laplacian Matrix as regularization term
Problem obtains the target image.
Optionally, described to be based on the prior model in any of the above-described embodiment of the method for the present invention, for described current
The described image degradation phenomena of image to be processed carries out image recovery processing：
Coefficient of the figure Laplacian Matrix as regularization term is determined according to the currently pending image.
Optionally, described to be based on the prior model in any of the above-described embodiment of the method for the present invention, for described current
The described image degradation phenomena of image to be processed carries out image recovery processing, after obtaining corresponding target image, further includes：
Iteration executes：Using the target image as currently pending image, according to the spy of the currently pending image
Sign figure building prior model；
Based on the prior model, image recovery is carried out for the described image degradation phenomena of the currently pending image
Processing obtains corresponding target image.
Optionally, in any of the above-described embodiment of the method for the present invention, it is applied to mobile terminal and/or DAS (Driver Assistant System).
Other side according to an embodiment of the present invention provides a kind of image recovery device, including：
Modeling unit, for constructing prior model according to the characteristic pattern of currently pending image, the prior model is used for
Learn the information of the currently pending image, the currently pending image includes at least one image degradation phenomena；
Processing unit is degenerated now for being based on the prior model for the described image of the currently pending image
As carrying out image recovery processing, corresponding target image is obtained.
Optionally, in above-mentioned apparatus embodiment of the present invention, described image degradation phenomena is comprised at least one of the following：Image
Noise, image are fuzzy and image pixel lacks；
Described image recovery processing comprises at least one of the following：Image denoising processing, image deblurring processing and image slices
Plain completion processing.
Optionally, in any of the above-described Installation practice of the present invention, further include：
Extraction unit obtains and the currently pending figure for carrying out feature extraction to the currently pending image
As having one group of characteristic pattern of identical size；
The modeling unit, for according to one group of characteristic pattern building with the currently pending image with identical size
The prior model.
Optionally, in any of the above-described Installation practice of the present invention, further include：
Cutting unit, for each feature in one group of characteristic pattern with the currently pending image with identical size
Figure carries out image dividing processing, obtains wherein each characteristic pattern and corresponds respectively to each preset areas in the currently pending image
One group of local feature figure in domain；And image dividing processing is carried out to the currently pending image, it obtains corresponding to described work as
The local image to be processed of one group of each predeterminable area in preceding image to be processed；
The modeling unit, for identical pre- in the currently pending image according to corresponding in each group local feature figure
If the local feature figure in region, one group of priori mould for corresponding respectively to each predeterminable area in the currently pending image is determined
Type；
The processing unit, for being based respectively on corresponding institute to each predeterminable area in the currently pending image
Prior model is stated, image recovery processing is carried out for the described image degradation phenomena of the corresponding part image to be processed, obtains
To one group of localized target image for corresponding respectively to each predeterminable area in the currently pending image；
Described device further includes：
Concatenation unit, for one group of localized target for corresponding respectively to each predeterminable area in the currently pending image
Image carries out splicing, obtains the target image.
Optionally, in any of the above-described Installation practice of the present invention, the modeling unit, for the part according to each group
Correspond to the local feature figure of identical predeterminable area in the currently pending image in characteristic pattern, building corresponds to described respectively
One group of part non-directed graph of each predeterminable area in currently pending image；And according to corresponding respectively to the currently pending figure
One group of part non-directed graph of each predeterminable area as in, determination correspond respectively to each predeterminable area in the currently pending image
One group of prior model.
Optionally, in any of the above-described Installation practice of the present invention, the modeling unit, for described currently pending
Each predeterminable area in image, according to the distance between pixel in each local feature figure for corresponding to each predeterminable area
Determine the weight in the non-directed graph of part between respective pixel；And it is special according to each part for corresponding to each predeterminable area
Weight between the determining pixel of sign figure, building correspond to the local non-directed graph of the predeterminable area.
Optionally, in any of the above-described Installation practice of the present invention, the modeling unit, for according to corresponding to each
It is less than or equal to the distance between the pixel of pre-determined distance threshold value in each local feature figure of predeterminable area, determines part nothing
Weight into figure between respective pixel.
Optionally, in any of the above-described Installation practice of the present invention, further include：
Pretreatment unit carries out at preliminary recovery for the described image degradation phenomena to the currently pending image
Reason, it is described preliminary to restore that treated that image is of the same size with the currently pending image；
The processing unit preliminary restores described in treated image for being based on the prior model for described
Image degradation phenomena carries out image recovery processing, obtains the target image.
Optionally, in any of the above-described Installation practice of the present invention, the prior model includes：Figure Laplacian Matrix.
Optionally, in any of the above-described Installation practice of the present invention, the processing unit, for described current by solving
Quadratic programming problem of the image to be processed using the figure Laplacian Matrix as regularization term, obtains the target image.
Optionally, in any of the above-described Installation practice of the present invention, further include：
Determination unit, for determining the figure Laplacian Matrix as regularization term according to the currently pending image
Coefficient.
Optionally, in any of the above-described Installation practice of the present invention, including：At least two cascade recovery modules, it is described
Recovery module includes：The modeling unit and the processing unit；
First recovery module, for constructing prior model according to the characteristic pattern of currently pending image；And base
In the prior model, image recovery processing is carried out for the described image degradation phenomena of the currently pending image, is obtained
Corresponding target image；
At least one described recovery module in addition to first recovery module, for using the target image as working as
Preceding image to be processed constructs prior model according to the characteristic pattern of the currently pending image；And it is based on the prior model,
Image recovery processing is carried out for the described image degradation phenomena of the currently pending image, obtains corresponding target image.
Optionally, in any of the above-described Installation practice of the present invention, it is applied to mobile terminal and/or DAS (Driver Assistant System).
Another aspect according to an embodiment of the present invention, a kind of electronic equipment provided, including any of the above-described embodiment institute
The device stated.
Another aspect according to an embodiment of the present invention, a kind of electronic equipment provided, including：
Memory, for storing executable instruction；And
Processor completes method described in any of the above-described embodiment for executing the executable instruction.
Another aspect according to an embodiment of the present invention, a kind of computer program provided, including computer-readable code,
When the computer-readable code is run in equipment, the processor in the equipment is executed for realizing any of the above-described implementation
The instruction of example the method.
Another aspect according to an embodiment of the present invention, a kind of computer storage medium provided, for storing computer
Readable instruction, described instruction, which is performed, realizes method described in any of the above-described embodiment.
The image recovery method and device that are there is provided based on the above embodiment of the present invention, computer program and are deposited electronic equipment
Storage media is based on prior model, for currently pending by constructing prior model according to the characteristic pattern of currently pending image
The image degradation phenomena of image carries out image recovery processing, corresponding target image is obtained, wherein currently pending image includes
At least one image degradation phenomena, prior model are used to learn the information of currently pending image, are learnt using characteristic pattern current
The information architecture prior model of image to be processed, can make prior model more really react the image of currently pending image
Recovery demand, thus by that in conjunction with prior model, can obtain machine learning to the good image of currently pending image
Recovery effects.
Below by drawings and examples, technical scheme of the present invention will be described in further detail.
Detailed description of the invention
The attached drawing for constituting part of specification describes the embodiment of the present invention, and together with description for explaining
The principle of the present invention.
The present invention can be more clearly understood according to following detailed description referring to attached drawing, wherein：
Fig. 1 is the flow chart of the image recovery method of some embodiments of the invention；
Fig. 2 is the flow chart of the image recovery method of other embodiments of the invention；
Fig. 3 is the schematic diagram for realizing the network structure of image recovery method of other embodiments of the invention；
Fig. 4 is the schematic diagram for realizing the network structure of image recovery method of other embodiments of the invention；
Fig. 5 is the structural schematic diagram of the image recovery device of some embodiments of the invention；
Fig. 6 is the structural schematic diagram of the image recovery device of other embodiments of the invention；
Fig. 7 is the structural schematic diagram for the electronic equipment that some embodiments of the invention provide.
Specific embodiment
Carry out the various exemplary embodiments of detailed description of the present invention now with reference to attached drawing.It should be noted that：Unless in addition having
Body explanation, the unlimited system of component and the positioned opposite of step, numerical expression and the numerical value otherwise illustrated in these embodiments is originally
The range of invention.
Simultaneously, it should be appreciated that for ease of description, the size of various pieces shown in attached drawing is not according to reality
Proportionate relationship draw.
Be to the description only actually of at least one exemplary embodiment below it is illustrative, never as to the present invention
And its application or any restrictions used.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable
In the case of, the technology, method and apparatus should be considered as part of specification.
It should be noted that：Similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
The embodiment of the present invention can be applied to computer system/server, can be with numerous other general or specialized calculating
System environments or configuration operate together.Suitable for be used together with computer system/server well-known computing system, ring
The example of border and/or configuration includes but is not limited to：Personal computer system, server computer system, thin client, thick client
Machine, hand-held or laptop devices, microprocessor-based system, set-top box, programmable consumer electronics, NetPC Network PC,
Little type Ji calculates machine Xi Tong ﹑ large computer system and the distributed cloud computing technology environment including above-mentioned any system, etc..
Computer system/server can be in computer system executable instruction (such as journey executed by computer system
Sequence module) general context under describe.In general, program module may include routine, program, target program, component, logic, number
According to structure etc., they execute specific task or realize specific abstract data type.Computer system/server can be with
Implement in distributed cloud computing environment, in distributed cloud computing environment, task is long-range by what is be linked through a communication network
Manage what equipment executed.In distributed cloud computing environment, it includes the Local or Remote meter for storing equipment that program module, which can be located at,
It calculates in system storage medium.
Fig. 1 is the flow chart of the image recovery method of some embodiments of the invention.It should be understood that example shown in FIG. 1 is only
It is to help those skilled in the art and more fully understands technical solution of the present invention, and be not construed as to limit of the invention
It is fixed.Those skilled in the art can carry out various transformation on the basis of Fig. 1, and this transformation is it is also contemplated that at the technology of the present invention
A part of scheme.
As shown in Figure 1, this method includes：
102, prior model is constructed according to the characteristic pattern of currently pending image.
The image to be processed of the embodiment of the present invention can be the original image obtained from image capture device, be also possible to
The image obtained after a certain image procossing in image processing process, the embodiment of the present invention do not make the acquisition modes of image to be processed
It limits.In embodiments of the present invention, prior model is used to learn the information of image to be processed, and image to be processed includes at least one
Image degradation phenomena, for example, image degradation phenomena comprises at least one of the following：Picture noise, image are fuzzy and image pixel lacks
Lose etc., the embodiment of the present invention is not construed as limiting the type of image degradation phenomena in image to be processed.
It optionally, can be to currently pending before constructing prior model according to the characteristic pattern of currently pending image
Image carry out feature extraction, obtain one group of characteristic pattern of currently pending image, wherein each characteristic pattern in this group of characteristic pattern with
Currently pending image has identical size, then has the characteristic pattern structure of identical size with currently pending image according to the group
Build prior model.It is alternatively possible to be carried out using neural network or the method for other machine learning to currently pending image
Feature extraction obtains one group of characteristic pattern for having identical size with currently pending image.In an optional example, nerve
Network can use convolutional neural networks.The embodiment of the present invention does not make the mode for obtaining characteristic pattern according to currently pending image
It limits.
The embodiment of the present invention is not construed as limiting the quantity of the characteristic pattern of building prior model.For example, by will be currently wait locate
It manages image and inputs convolutional neural networks, obtain channel there are three one and image to be processed length and width having the same and tools
Three-dimensional array can export three characteristic patterns using each of this three-dimensional array channel as a characteristic pattern, can be with
Prior model is constructed based on three characteristic patterns of output.
Optionally, the embodiment of the present invention is not construed as limiting the form of prior model, such as：Prior model can be become using complete
Sub-model, sparse expression model etc..In an optional example, prior model can use Tula Laplace model, that is, scheme
Laplace model (Graph Laplacians).
104, it is based on prior model, image recovery processing is carried out for the image degradation phenomena of currently pending image, obtains
To corresponding target image.
The method that the embodiment of the present invention carries out image recovery processing to the image degradation phenomena of image to be processed can basis
The type of prior model determines that the embodiment of the present invention carries out the image degradation phenomena based on prior model for image to be processed
The method that image restores processing is not construed as limiting.In an optional example, prior model can use Tula Laplace model,
Can by solving currently pending image to scheme Laplacian Matrix as the quadratic programming problem of regularization term, to currently to
The image degradation phenomena for handling image carries out image recovery processing, obtains target image.
Optionally, corresponding to the image degradation phenomena for including in currently pending image, image recovery processing can be with
The corresponding image procossing of image degradation phenomena, for example, when image degradation phenomena comprises at least one of the following：Picture noise, image
Whens fuzzy and image pixel missing etc., corresponding image recovery processing is comprised at least one of the following：Image denoising processing, image are gone
Fuzzy Processing and image pixel completion processing etc..
Optionally, it is being based on prior model, the image degradation phenomena for currently pending image carries out at image recovery
Reason, before obtaining corresponding target image, can also image degradation phenomena to currently pending image carry out at preliminary recovery
Reason, is then based on prior model, carries out image recovery processing for the image degradation phenomena of preliminary recovery treated image, obtains
To target image, wherein preliminary to restore that treated image is of the same size with currently pending image.Optionally, may be used
The image degradation phenomena of currently pending image is carried out just with restoring processing method using pretreatment network or other images
Walk recovery processing.In an optional example, pretreatment network can use convolutional neural networks.The embodiment of the present invention is to first
The method that step restores processing is not construed as limiting.
Based on the image recovery method that the above embodiment of the present invention provides, pass through the characteristic pattern according to currently pending image
Prior model is constructed, prior model is based on, image recovery processing is carried out for the image degradation phenomena of currently pending image, obtains
To corresponding target image, wherein currently pending image includes at least one image degradation phenomena, prior model is for learning
The information of currently pending image is learnt the information architecture prior model of currently pending image using characteristic pattern, can make elder generation
The image recovery demand that model more really reacts currently pending image is tested, is moved back especially for image in practical application scene
Change more complicated situation, it is good to currently pending image by that in conjunction with prior model, can obtain machine learning
Image recovery effects.
Compared to the simple method for carrying out image recovery by neural network, need to be completely dependent on the quantity of training data with
Quality is easy to happen over-fitting and can not obtain very when the inadequate robust of quality and quantity of the neural network to training data
Good image restoration result, the embodiment of the present invention is by conjunction with neural network, limiting mind using prior model for prior model
Freedom degree through network makes neural network have better robustness to the quality and quantity of training data, it is possible to prevente effectively from
The generation of over-fitting, while good image restoration result can be obtained.It is extensive that image is carried out using prior model compared to merely
Multiple method needs to do the property of image itself some it is assumed that lacking flexibility, can shadow when assumed condition is unable to satisfy
Ring image restore effect, the embodiment of the present invention by by prior model in conjunction with neural network, data neural network based
Driving building prior model, influence caused by can completely avoid the inflexible recovery effects image of prior model, obtain compared with
Good image recovery effects.
The image that the embodiment of the present invention can be applied to the mobile terminals such as mobile phone, DAS (Driver Assistant System) etc. restores.
Fig. 2 is the flow chart of the image recovery method of other embodiments of the invention.It should be understood that example shown in Fig. 2 is only
Technical solution of the present invention is more fully understood only to assist in those skilled in the art, and is not construed as to limit of the invention
It is fixed.Those skilled in the art can carry out various transformation on the basis of Fig. 2, and this transformation is it is also contemplated that at the technology of the present invention
A part of scheme.
As shown in Fig. 2, this method includes：
202, feature extraction is carried out to currently pending image, obtains there is identical size with currently pending image one
Group characteristic pattern.
It is alternatively possible to carry out feature to currently pending image using neural network or the method for other machine learning
It extracts, obtains one group of characteristic pattern that there is identical size with currently pending image.In an optional example, neural network
Convolutional neural networks can be used.The embodiment of the present invention does not limit the mode for obtaining characteristic pattern according to currently pending image
It is fixed.
204, image point is carried out to each characteristic pattern in one group of characteristic pattern with currently pending image with identical size
Processing is cut, one group of local feature that wherein each characteristic pattern corresponds respectively to each predeterminable area in currently pending image is obtained
Figure.
It is alternatively possible to carry out image dividing processing to each characteristic pattern according to pre-set image dividing method, make each Zhang Te
One group of local feature figure that sign figure obtains after image segmentation corresponds respectively to each predeterminable area in currently pending image,
And whole currently pending image can be completely covered, wherein adjacent local feature figure can have identical region, i.e.,
Overlapping region.In an optional example, image dividing processing can be carried out using neural network, it can also be using except nerve
Other methods outside network carry out image dividing processing, and the embodiment of the present invention is not construed as limiting the method for image dividing processing.
The quantity and size for the local feature figure that the embodiment of the present invention obtains image dividing processing are not construed as limiting, and are usually schemed
The size of the local feature figure obtained as dividing processing is bigger, and the recovery effects of the image finally obtained can be better, but to every
The processing time of one local feature figure also can be longer, can comprehensively consider the effect and effect of image recovery in practical applications
Rate determines the size and number for the local feature figure that image dividing processing obtains.For example, each characteristic pattern can be divided into
The local feature figure of one group of 26x26 pixel.
206, according to the local feature for corresponding to identical predeterminable area in currently pending image in each group local feature figure
Figure determines one group of prior model for corresponding respectively to each predeterminable area in currently pending image.
It is alternatively possible to according to the office for corresponding to identical predeterminable area in currently pending image in each group local feature figure
Portion's characteristic pattern, building corresponds to one group of part non-directed graph of each predeterminable area in currently pending image respectively, then basis point
Not Dui Yingyu in currently pending image each predeterminable area one group of part non-directed graph, determination corresponds respectively to currently pending figure
One group of prior model of each predeterminable area as in.
It optionally, can be according to corresponding to each predeterminable area to each predeterminable area in currently pending image
The distance between pixel determines the weight in local non-directed graph between respective pixel in each local feature figure, and then basis corresponds to
The weight between pixel that each local feature figure of each predeterminable area determines, building correspond to the local nothing of the predeterminable area
Xiang Tu.
In an optional example, can according to correspond to each predeterminable area each local feature figure in be less than or
Equal to the distance between the pixel of pre-determined distance threshold value, the weight in the non-directed graph of part between respective pixel is determined.For example, part
Characteristic pattern uses eight connectivity figure, and in each local feature figure, each pixel is connected with eight pixels around it, when
When the distance between each pixel and four pixels of its upper and lower, left and right are 1, each pixel and its upper left, upper right, a left side
Under, bottom right, i.e., the distance between pixel on diagonal line be 2 square root, then at this time pre-determined distance threshold value can be 2 square
Root.Usual preset threshold is bigger, and obtained non-directed graph can have more connections, and the recovery effects of the image finally obtained can be got over
It is good, but the complexity handled also can be higher.
In an optional example, prior model can use Tula Laplace model, basis can correspond to respectively
One group of part non-directed graph of each predeterminable area in currently pending image, determination correspond respectively to each pre- in currently pending image
If a group picture laplace model in region.Such as：Each office can be constructed according to the weight in each local non-directed graph between pixel
Non-directed graph corresponding adjacency matrix in portion's constructs corresponding degree matrix further according to each adjacency matrix, then according to each local non-directed graph
Corresponding adjacency matrix and degree matrix obtain a group picture Laplacian Matrix.
208, image dividing processing is carried out to currently pending image, obtains corresponding to each default in currently pending image
The local image to be processed of one group of region.
Optionally, operation 208 can use and operate 204 identical pre-set image dividing methods, make currently pending figure
As obtained after image segmentation one group locally image to be processed correspond respectively to each preset areas in currently pending image
Domain, and whole currently pending image can be completely covered, wherein adjacent part image to be processed also can have identical
Region, i.e. overlapping region, image to be processed obtained after image segmentation one group locally image to be processed and each characteristic pattern
The one group of local feature figure obtained after image segmentation is of the same size and quantity.It, can in an optional example
To carry out image dividing processing using neural network, can also be carried out at image segmentation using other methods in addition to neural network
Reason, the embodiment of the present invention are not construed as limiting the method for image dividing processing.
210, to each predeterminable area in currently pending image, it is based respectively on corresponding prior model, for corresponding
The image degradation phenomena of part image to be processed carries out image recovery processing, obtains corresponding respectively to each in currently pending image
One group of localized target image of predeterminable area.
The method that the embodiment of the present invention carries out image recovery processing to the image degradation phenomena of each part image to be processed can
To be determined according to the type of prior model, the embodiment of the present invention is to corresponding prior model is based on, for corresponding part wait locate
The image degradation phenomena of reason image carries out the method that image restores to handle and is not construed as limiting.In an optional example, priori mould
Type can use Tula Laplace model, can be by solving one corresponding to predeterminable area each in currently pending image respectively
Group quadratic programming problem obtains one group of localized target image, wherein each quadratic programming problem is respectively to correspond to currently pending figure
The figure Laplacian Matrix of identical predeterminable area is as regularization term as in.
It optionally, can be with before the image degradation phenomena to each part image to be processed carries out image recovery processing
Coefficient of the figure Laplacian Matrix as regularization term, i.e. a group picture Laplace regularization are determined according to currently pending image
Term coefficient, wherein figure Laplce's regularization term is will to correspond in currently pending image according in each group local feature figure
The local feature figure of identical predeterminable area, the one group of Tula for corresponding respectively to each predeterminable area in currently pending image determined
This matrix of pula is as regularization term.
It is alternatively possible to be handled currently pending image to obtain one group of work using neural network or other methods
For the data for scheming Laplce's regularization term coefficient, the embodiment of the present invention obtains figure Laplce to according to currently pending image
The method of regularization term coefficient is not construed as limiting.
212, one group of localized target image for corresponding respectively to each predeterminable area in currently pending image is spliced
Processing, obtains target image.
It is alternatively possible to carry out splicing to each localized target image by pre-set image joining method, obtain final
Target image, wherein target image is of the same size with currently pending image, for same area, i.e., heavy
The localized target image in folded region, can by overlapping region is averaged mode determines the image of the overlapping region.It can
Selection of land, pre-set image joining method can be reciprocal with pre-set image dividing method.It, can be using mind in an optional example
Splicing is carried out to image through network, splicing can also be carried out to image using other methods in addition to neural network.
The method that the embodiment of the present invention handles image mosaic is not construed as limiting.
Based on the above embodiment of the present invention provide image recovery method, by by currently pending image segmentation at multiple
Part image to be processed, and multiple prior models are constructed according to the local feature figure for corresponding to each part image to be processed respectively,
It can be quantified as restoring problem to the image of multiple parts image to be processed by problem is restored to the image of currently pending image,
When the image degradation phenomena to multiple parts image to be processed carries out image recovery, by by machine learning and prior model knot
It closes, the information architecture prior model of part image to be processed is practised using local feature graphics, prior model can be made truer
The image of reaction part image to be processed restore demand, it is hereby achieved that restoring to currently pending image better image
Effect.
Fig. 3 is the schematic diagram for realizing the network structure of image recovery method of some embodiments of the invention.It should be understood that Fig. 3
Shown in example just for the sake of helping those skilled in the art to more fully understand technical solution of the present invention, and be not construed as
Limitation of the invention.Those skilled in the art can carry out various transformation on the basis of Fig. 3, and this transformation it is also contemplated that
At a part of technical solution of the present invention.
As described in Figure 3, which includes：Three convolutional neural networks CNNF、CNNYAnd CNNμ, image segmentation module,
Non-directed graph constructs module, Quadratic Programming Solution module and image mosaic module, wherein non-directed graph building module and quadratic programming are asked
It solves module and forms figure Laplace regularization layer.The input of the network structure is currently pending image Y, the currently pending figure
The image for having image degradation phenomena as can be noise image or blurred picture etc., the output of the network structure are corresponding mesh
Logo image X*, which can be network degradation phenomena is restored after eliminate the image of network degradation phenomena.
Firstly, currently pending image Y is inputted first convolutional neural networks CNNFFeature extraction is carried out, output N opens
Characteristic pattern F identical with currently pending image Y size1,F2,…,FN.It at the same time, can be defeated by currently pending image Y
Enter second convolutional neural networks CNNY, preliminary recovery processing carried out to currently pending image Y, output one with currently wait locate
Manage the identical image Y` of image Y size.
It is then possible to by N characteristic pattern F1,F2,…,FNTreated that image Y` input picture divides mould with preliminary recovery
Each characteristic pattern and preliminary recovery treated image are divided into K according to identical predetermined manner by block, image segmentation module
A square figure, wherein by FnK-th of square seal that (1≤n≤N) is split makees fn k, split by Y` k-th
Square seal makees yk, fn kAnd ykIt can be indicated using vectorization, i.e., by each Leie time head and the tail phase in the corresponding matrix of image
It is connected into a vector repeatedly, the length of these vectors is set as m herein.
Next, can for preliminary recovery treated each part of image Y` by image segmentation module output to
Handle image yk(1≤k≤K) establishes a non-directed graph and seeks corresponding figure Laplce matrix Lk, for ykIt carries out
" figure Laplace regularization ".Wherein, by N characteristic pattern F1,F2,…,FNInput picture divides each local feature of module output
Figure input non-directed graph constructs module, can use and corresponds to identical part image y to be processed in each local feature figurekIt is N number of
Local feature figure f1 k,f2 k,…,fN kThe non-directed graph for having m node is established jointly, wherein each of non-directed graph node
Represent a pixel.The method for building up of non-directed graph is as follows：
Wherein, wijFor the weight in local feature figure between ith pixel and j-th of pixel, in addition, if ith pixel with
The position of jth pixel in the picture is separated by far, such as more than one distance threshold d of distance, then wij=0, that is to say, that the
It is not connected between i pixel and jth pixel.After obtaining non-directed graph, then its corresponding figure Laplce matrix L can be calculatedk。
Next, currently pending image Y can be inputted third convolutional neural networks CNNμIt is handled, exports K
A data mu1,μ2,…,μK, wherein μkCorresponding to part image y to be processedk(1≤k≤K) will be used as subsequent solution quadratic programming
The weight of Laplce's regularization term, i.e. coefficient are schemed when problem.
It later, can be to each part image y to be processedkCarry out the solution of following quadratic programming problem：
Wherein, restore that treated that localized target image is denoted as Xk *, the Section 2 in above-mentioned quadratic programming problem is Tula
This regularization term of pula.
Finally, can restore that treated that localized target image is denoted as X for allk *Input picture splicing module, splices
To have obtained the final goal image after image restores.
In above process, currently pending image Y only passes through the second convolutional neural networks CNNYJust with figure Laplce
Then change layer carried out image recovery operation twice just obtained image restore after target image.
Wherein, non-directed graph building module and Quadratic Programming Solution module constitute figure Laplace regularization layer, at this
Layer in two modules be guidable, therefore can the network structure to above-mentioned image recovery method can carry out end to end
Training.
In practical applications, first convolutional neural networks CNNFCan using compared with multilayer structure, allow to energy
More efficiently study is to currently pending image to construct suitable non-directed graph, second convolutional neural networks CNNYThe number of plies can
With less, for example, using the structure such as 4 layers to 6 layers a possibility that network itself brings over-fitting can be reduced in this way, simultaneously
The effect of subsequent figure Laplace regularization layer can be played as much as possible.First convolutional neural networks CNNμThe number of plies can also
With less.
In the various embodiments described above of the present invention, the method that iterative processing can also be used, i.e., according to currently pending figure
The characteristic pattern of picture constructs prior model, is based on prior model, carries out image for the image degradation phenomena of currently pending image
Recovery processing after obtaining corresponding target image, can execute following operation with iteration：Using target image as currently wait locate
Image is managed, prior model is constructed according to the characteristic pattern of currently pending image, prior model is based on, for currently pending figure
Picture, needle carry out image recovery processing, obtain corresponding target image.Restored by being iterated formula to the image to be processed of input
Processing, can make image be further enhanced in the quality after iteration each time, can be extensive after successive ignition
It appears again and carrys out the target image of high quality.
Fig. 4 is the schematic diagram for realizing the network structure of image recovery method of other embodiments of the invention.It should be understood that figure
Example shown in 4 should not be understood just for the sake of helping those skilled in the art to more fully understand technical solution of the present invention
At limitation of the invention.Those skilled in the art can carry out various transformation on the basis of fig. 4, and this transformation should also be managed
Solution at technical solution of the present invention a part.
As shown in figure 4, the network structure includes：Cascade multiple images recovery module, the input of the network structure are to work as
Preceding image Y to be processed, the currently pending image can be the figure that noise image or blurred picture etc. have image degradation phenomena
Picture, the output of the network structure are target image XT, by cascading T image-restoration module, to the currently pending figure of input
It is handled as Y carries out multiple image recovery, better image recovery effects can be obtained.Wherein, every level-one image-restoration module can
To use identical network structure, for example, every level-one image-restoration module can use the structure in Fig. 3, and cascade each
Image-restoration module can share the parameter of neural network, i.e., corresponding neural network is using identical in image-restoration modules at different levels
Parameter, to reduce the quantity of network parameter and the generation of over-fitting can be prevented.
Fig. 5 is the structural schematic diagram of the image recovery device of some embodiments of the invention.It should be understood that example shown in fig. 5
Just for the sake of helping those skilled in the art to more fully understand technical solution of the present invention, and it is not construed as to of the invention
It limits.Those skilled in the art can carry out various transformation on the basis of Fig. 5, and this transformation is it is also contemplated that at skill of the present invention
A part of art scheme.
As shown in figure 5, the device includes：Modeling unit 510 and processing unit 520.Wherein,
Modeling unit 510, for constructing prior model according to the characteristic pattern of currently pending image.
The image to be processed of the embodiment of the present invention can be the original image obtained from image capture device, be also possible to
The image obtained after a certain image procossing in image processing process, the embodiment of the present invention do not make the acquisition modes of image to be processed
It limits.In embodiments of the present invention, prior model is used to learn the information of image to be processed, and image to be processed includes at least one
Image degradation phenomena, for example, image degradation phenomena comprises at least one of the following：Picture noise, image are fuzzy and image pixel lacks
Lose etc., the embodiment of the present invention is not construed as limiting the type of image degradation phenomena in image to be processed.
Optionally, which can also include extraction unit, and extraction unit can carry out feature to currently pending image
It extracts, obtains one group of characteristic pattern of currently pending image, wherein each characteristic pattern in this group of characteristic pattern and currently pending figure
As having identical size, modeling unit 510 with currently pending image there is the characteristic pattern of identical size to construct first according to the group
Test model.Optionally, extraction unit can be using neural network or the method for other machine learning to currently pending image
Feature extraction is carried out, one group of characteristic pattern that there is identical size with currently pending image is obtained.In an optional example,
Neural network can use convolutional neural networks.The embodiment of the present invention obtains feature according to currently pending image to extraction unit
The mode of figure is not construed as limiting.
The embodiment of the present invention is not construed as limiting the quantity of the characteristic pattern of building prior model.For example, by will be currently wait locate
It manages image and inputs convolutional neural networks, obtain channel there are three one and image to be processed length and width having the same and tools
Three-dimensional array can export three characteristic patterns using each of this three-dimensional array channel as a characteristic pattern, can be with
Prior model is constructed based on three characteristic patterns of output.
Optionally, the embodiment of the present invention is not construed as limiting the form of prior model, such as：Prior model can be become using complete
Sub-model, sparse expression model etc..In an optional example, prior model can use Tula Laplace model, that is, scheme
Laplace model (Graph Laplacians).
Processing unit 520 carries out image for the image degradation phenomena of currently pending image for being based on prior model
Recovery processing, obtains corresponding target image.
Processing unit of the embodiment of the present invention 520 carries out the side that image restores processing to the image degradation phenomena of image to be processed
Method can determine that the embodiment of the present invention is based on prior model to processing unit 520, for be processed according to the type of prior model
The image degradation phenomena of image carries out the method that image restores to handle and is not construed as limiting.In an optional example, prior model
Tula Laplace model can be used, processing unit 520 can be by solving currently pending image to scheme Laplacian Matrix
As the quadratic programming problem of regularization term, image recovery processing is carried out to the image degradation phenomena of currently pending image, is obtained
Obtain target image.
Optionally, corresponding to the image degradation phenomena for including in currently pending image, the image of processing unit 520 restores
Processing can be image procossing corresponding with image degradation phenomena, for example, when image degradation phenomena comprises at least one of the following：Figure
As noise, whens image is fuzzy and image pixel missing etc., corresponding image recovery processing is comprised at least one of the following：Image denoising
Processing, image deblurring processing and image pixel completion processing etc..
Optionally, which can also include pretreatment unit, and pretreatment unit can be to the figure of currently pending image
As degradation phenomena carries out preliminary recovery processing, processing unit 520 is based on prior model, for preliminary recovery treated image
Image degradation phenomena carries out image recovery processing, obtains target image, wherein the preliminary image that restores that treated with currently wait locate
Reason image is of the same size.Optionally, pretreatment unit can be using pretreatment network or other image recoveries processing
Method carries out preliminary recovery processing to the image degradation phenomena of currently pending image.In an optional example, pretreatment
Network can use convolutional neural networks.The method that the embodiment of the present invention tentatively restores processing to pretreatment unit is not construed as limiting.
Based on the image recovery device that the above embodiment of the present invention provides, pass through the characteristic pattern according to currently pending image
Prior model is constructed, prior model is based on, image recovery processing is carried out for the image degradation phenomena of currently pending image, obtains
To corresponding target image, wherein currently pending image includes at least one image degradation phenomena, prior model is for learning
The information of currently pending image is learnt the information architecture prior model of currently pending image using characteristic pattern, can make elder generation
The image recovery demand that model more really reacts currently pending image is tested, is moved back especially for image in practical application scene
Change more complicated situation, it is good to currently pending image by that in conjunction with prior model, can obtain machine learning
Image recovery effects.
Compared to the simple device for carrying out image recovery by neural network, need to be completely dependent on the quantity of training data with
Quality is easy to happen over-fitting and can not obtain very when the inadequate robust of quality and quantity of the neural network to training data
Good image restoration result, the embodiment of the present invention is by conjunction with neural network, limiting mind using prior model for prior model
Freedom degree through network makes neural network have better robustness to the quality and quantity of training data, it is possible to prevente effectively from
The generation of over-fitting, while good image restoration result can be obtained.It is extensive that image is carried out using prior model compared to merely
Multiple method needs to do the property of image itself some it is assumed that lacking flexibility, can shadow when assumed condition is unable to satisfy
Ring image restore effect, the embodiment of the present invention by by prior model in conjunction with neural network, data neural network based
Driving building prior model, influence caused by can completely avoid the inflexible recovery effects image of prior model, obtain compared with
Good image recovery effects.
The image that the embodiment of the present invention can be applied to the mobile terminals such as mobile phone, DAS (Driver Assistant System) etc. restores.
Fig. 6 is the structural schematic diagram of the image recovery device of other embodiments of the invention.It should be understood that example shown in fig. 6
Son is not construed as just for the sake of helping those skilled in the art to more fully understand technical solution of the present invention to the present invention
Restriction.Those skilled in the art can carry out various transformation on the basis of Fig. 6, and this transformation is it is also contemplated that at the present invention
A part of technical solution.
As shown in fig. 6, the device includes：Extraction unit 610, cutting unit 620, modeling unit 630, processing unit 640
With concatenation unit 650.Wherein,
Extraction unit 610 obtains having with currently pending image for carrying out feature extraction to currently pending image
One group of characteristic pattern of identical size.
Optionally, extraction unit 610 can be using neural network or the method for other machine learning to currently pending
Image carries out feature extraction, obtains one group of characteristic pattern for having identical size with currently pending image.In an optional example
In son, neural network can use convolutional neural networks.The embodiment of the present invention is to extraction unit 610 according to currently pending image
The mode for obtaining characteristic pattern is not construed as limiting.
Cutting unit 620, for each feature in one group of characteristic pattern with currently pending image with identical size
Figure carries out image dividing processing, obtains wherein each characteristic pattern and corresponds respectively to each predeterminable area in currently pending image
One group of local feature figure；And image dividing processing is carried out to currently pending image, it obtains corresponding to currently pending image
In one group of each predeterminable area local image to be processed.
Optionally, cutting unit 620 can carry out image dividing processing to each characteristic pattern according to pre-set image dividing method,
One group of local feature figure for obtaining each characteristic pattern after image segmentation corresponds respectively in currently pending image
Each predeterminable area, and whole currently pending image can be completely covered, wherein adjacent local feature figure can have phase
Same region, i.e. overlapping region.In an optional example, cutting unit 620 can carry out image point using neural network
Processing is cut, image dividing processing can also be carried out using other methods in addition to neural network, the embodiment of the present invention is single to segmentation
The method that member 620 carries out image dividing processing is not construed as limiting.
The embodiment of the present invention carries out the quantity and size of the local feature figure of image dividing processing acquisition to cutting unit 620
It is not construed as limiting, the size for the local feature figure that usual image dividing processing obtains is bigger, the recovery effects of the image finally obtained
Can be better, but also can be longer to the processing time of each local feature figure, image can be comprehensively considered in practical applications
The effect and efficiency of recovery determine the size and number for the local feature figure that image dividing processing obtains.For example, can will be each
Open the local feature figure that characteristic pattern is divided into one group of 26x26 pixel.
Optionally, cutting unit 620 can use identical pre-set image dividing method, pass through currently pending image
Locally image to be processed corresponds respectively to each predeterminable area in currently pending image to one group obtained after image segmentation, and
Whole currently pending image can be completely covered, wherein adjacent part image to be processed also can have identical region,
That is overlapping region, image to be processed obtained after image segmentation one group locally image to be processed and each characteristic pattern by image
The one group of local feature figure obtained after segmentation is of the same size and quantity.
Modeling unit 630, for according in each group local feature figure correspond to currently pending image in identical preset areas
The local feature figure in domain determines one group of prior model for corresponding respectively to each predeterminable area in currently pending image.
Optionally, modeling unit 630 can be identical in currently pending image according to corresponding in each group local feature figure
The local feature figure of predeterminable area constructs locally undirected corresponding to one group of each predeterminable area in currently pending image respectively
Figure, then according to one group of part non-directed graph for corresponding respectively to each predeterminable area in currently pending image, determination is respectively corresponded
One group of prior model of each predeterminable area in currently pending image.
Optionally, modeling unit 630, can be according to corresponding to each to each predeterminable area in currently pending image
The distance between pixel determines the weight in local non-directed graph between respective pixel in each local feature figure of predeterminable area, then
The weight between pixel determined according to each local feature figure for corresponding to each predeterminable area, building correspond to the preset areas
The local non-directed graph in domain.
In an optional example, modeling unit 630 can be special according to each part for corresponding to each predeterminable area
The distance between the pixel for being less than or equal to pre-determined distance threshold value in figure is levied, determines the power in the non-directed graph of part between respective pixel
Weight.For example, local feature figure uses eight connectivity figure, in each local feature figure, each pixel and eight around it
A pixel is connected, when the distance between four pixels of each pixel and its upper and lower, left and right are 1, each pixel with
The square root that its upper left, upper right, lower-left, bottom right, i.e. the distance between pixel on diagonal line are 2, then pre-determined distance threshold at this time
Value can be 2 square root.Usual preset threshold is bigger, and obtained non-directed graph can have more connections, the figure finally obtained
The recovery effects of picture can be better, but the complexity handled also can be higher.
In an optional example, prior model can use Tula Laplace model, and modeling unit 630 can be distinguished
According to one group of part non-directed graph for corresponding to each predeterminable area in currently pending image, determination corresponds respectively to currently pending
A group picture laplace model of each predeterminable area in image.Such as：Modeling unit 630 can be according to picture in each local non-directed graph
The corresponding adjacency matrix of each part non-directed graph of weight building between element, constructs corresponding degree matrix further according to each adjacency matrix,
Then a group picture Laplacian Matrix is obtained according to the corresponding adjacency matrix of each part non-directed graph and degree matrix.
Processing unit 640, for being based respectively on corresponding priori mould to each predeterminable area in currently pending image
Type carries out image recovery processing for the image degradation phenomena of corresponding part image to be processed, obtains corresponding respectively to current
One group of localized target image of each predeterminable area in image to be processed.
Processing unit of the embodiment of the present invention 640 carries out at image recovery the image degradation phenomena of each part image to be processed
The method of reason can determine that the embodiment of the present invention is based on corresponding priori mould to processing unit 640 according to the type of prior model
Type, the method for carrying out image recovery processing for the image degradation phenomena of corresponding part image to be processed are not construed as limiting.One
In a optional example, prior model can use Tula Laplace model, and processing unit 640 can be by solving correspondence respectively
One group of quadratic programming problem of each predeterminable area obtains one group of localized target image in currently pending image, wherein each secondary
Planning problem is respectively using the figure Laplacian Matrix corresponding to identical predeterminable area in currently pending image as regularization term.
Optionally, which can also include determination unit, and determination unit can be determined according to currently pending image to be schemed
Coefficient of the Laplacian Matrix as regularization term, i.e. a group picture Laplace regularization term coefficient, wherein figure Laplce is just
Then change item be by according in each group local feature figure correspond to currently pending image in identical predeterminable area local feature figure,
The determining group picture Laplacian Matrix for corresponding respectively to each predeterminable area in currently pending image is as regularization term.
Optionally it is determined that unit handle to currently pending image using neural network or other methods
To one group of data as figure Laplce's regularization term coefficient, the embodiment of the present invention is schemed to according to currently pending image
The method of Laplace regularization term coefficient is not construed as limiting.
Concatenation unit 650, for one group of localized target for corresponding respectively to each predeterminable area in currently pending image
Image carries out splicing, obtains target image.
Optionally, concatenation unit 650 can carry out stitching portion to each localized target image by pre-set image joining method
Reason, obtains final target image, wherein target image is of the same size with currently pending image, for phase
Same region, i.e. the localized target image of overlapping region, concatenation unit 650 can by overlapping region is averaged mode is true
The image of the fixed overlapping region.Optionally, the pre-set image joining method of concatenation unit 650 can be pre- with cutting unit 620
If image partition method is reciprocal.In an optional example, concatenation unit 650 can be spelled image using neural network
Processing is connect, splicing can also be carried out to image using other methods in addition to neural network.The embodiment of the present invention is to splicing
The method that unit 650 carries out image mosaic processing is not construed as limiting.
Based on the above embodiment of the present invention provide image recovery device, by by currently pending image segmentation at multiple
Part image to be processed, and multiple prior models are constructed according to the local feature figure for corresponding to each part image to be processed respectively,
It can be quantified as restoring problem to the image of multiple parts image to be processed by problem is restored to the image of currently pending image,
When the image degradation phenomena to multiple parts image to be processed carries out image recovery, by by machine learning and prior model knot
It closes, the information architecture prior model of part image to be processed is practised using local feature graphics, prior model can be made truer
The image of reaction part image to be processed restore demand, it is hereby achieved that restoring to currently pending image better image
Effect.
In the various embodiments described above of the present invention, image recovery device can also include at least two cascade recovery modules,
The recovery module includes that modeling unit and processing unit etc. are used for the unit that image restores.Wherein, first recovery module, is used for
Prior model is constructed according to the characteristic pattern of currently pending image；And it is based on prior model, for currently pending image
Image degradation phenomena carries out image recovery processing, obtains corresponding target image.At least one in addition to first recovery module
Recovery module, for constructing priori according to the characteristic pattern of currently pending image using target image as currently pending image
Model；And it is based on prior model, the image degradation phenomena image recovery carried out for currently pending image is handled, and is obtained pair
The target image answered.Formula recovery processing is iterated to the image to be processed of input by using the mode of recovery module base even,
Image can be made to be further enhanced in the quality after iteration each time, after successive ignition, can be recovered
Carry out the target image of high quality.
The embodiment of the invention also provides a kind of electronic equipment, such as can be mobile terminal, personal computer (PC), put down
Plate computer, server etc..Below with reference to Fig. 7, it illustrates the terminal device or the services that are suitable for being used to realize the embodiment of the present application
The structural schematic diagram of the electronic equipment 700 of device：As shown in fig. 7, electronic equipment 700 includes one or more processors, communication unit
Deng one or more of processors are for example：One or more central processing unit (CPU) 701, and/or one or more figures
As processor (GPU) 713 etc., processor can according to the executable instruction being stored in read-only memory (ROM) 702 or from
Executable instruction that storage section 708 is loaded into random access storage device (RAM) 703 and execute various movements appropriate and place
Reason.Communication unit 712 may include but be not limited to network interface card, and the network interface card may include but be not limited to IB (Infiniband) network interface card,
Processor can with communicate in read-only memory 702 and/or random access storage device 730 to execute executable instruction,
It is connected by bus 704 with communication unit 712 and is communicated through communication unit 712 with other target devices, to completes the application implementation
The corresponding operation of any one method that example provides, for example, prior model is constructed according to the characteristic pattern of currently pending image, it is described
Prior model is used to learn the information of the currently pending image, and the currently pending image includes that at least one image moves back
Change phenomenon；Based on the prior model, image recovery is carried out for the described image degradation phenomena of the currently pending image
Processing, obtains corresponding target image.
In addition, in RAM 703, various programs and data needed for being also stored with device operation.CPU701,ROM702
And RAM703 is connected with each other by bus 704.In the case where there is RAM703, ROM702 is optional module.RAM703 storage
Executable instruction, or executable instruction is written into ROM702 at runtime, executable instruction executes central processing unit 701
The corresponding operation of above-mentioned communication means.Input/output (I/O) interface 705 is also connected to bus 704.Communication unit 712 can integrate
Setting, may be set to be with multiple submodule (such as multiple IB network interface cards), and in bus link.
I/O interface 705 is connected to lower component：Importation 706 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 707 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 708 including hard disk etc.；
And the communications portion 709 of the network interface card including LAN card, modem etc..Communications portion 709 via such as because
The network of spy's net executes communication process.Driver 710 is also connected to I/O interface 705 as needed.Detachable media 711, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 710, in order to read from thereon
Computer program be mounted into storage section 708 as needed.
It should be noted that framework as shown in Figure 7 is only a kind of optional implementation, it, can root during concrete practice
The component count amount and type of above-mentioned Fig. 7 are selected, are deleted, increased or replaced according to actual needs；It is set in different function component
It sets, separately positioned or integrally disposed and other implementations, such as the separable setting of GPU713 and CPU701 or can also be used
GPU713 is integrated on CPU701, the separable setting of communication unit, can also be integrally disposed on CPU701 or GPU713, etc..
These interchangeable embodiments each fall within protection scope disclosed by the invention.
Particularly, according to an embodiment of the invention, may be implemented as computer above with reference to the process of flow chart description
Software program.For example, the embodiment of the present invention includes a kind of computer program product comprising be tangibly embodied in machine readable
Computer program on medium, computer program include the program code for method shown in execution flow chart, program code
It may include the corresponding instruction of corresponding execution method and step provided by the embodiments of the present application, for example, according to currently pending image
Characteristic pattern constructs prior model, and the prior model is used to learn the information of the currently pending image, described currently wait locate
Managing image includes at least one image degradation phenomena；Based on the prior model, for described in the currently pending image
Image degradation phenomena carries out image recovery processing, obtains corresponding target image.In such embodiments, the computer program
It can be downloaded and installed from network by communications portion 709, and/or be mounted from detachable media 711.In the computer
When program is executed by central processing unit (CPU) 701, the above-mentioned function of limiting in the present processes is executed.
In one or more optional embodiments, the embodiment of the invention also provides a kind of productions of computer program program
Product, for storing computer-readable instruction, which is performed so that computer executes any of the above-described possible implementation
In image recovery method.
The computer program product can be realized especially by hardware, software or its mode combined.In an alternative embodiment
In son, which is embodied as computer storage medium, in another optional example, the computer program
Product is embodied as software product, such as software development kit (Software Development Kit, SDK) etc..
In one or more optional embodiments, the embodiment of the invention also provides a kind of image recovery methods and its right
Device, system and electronic equipment, computer storage medium, computer program and the computer program product answered, wherein the party
Method includes：First device sends image to second device and restores instruction, and the instruction is so that second device executes any of the above-described possibility
Embodiment in image recovery method；First device receives the result that the image that second device is sent restores.
In some embodiments, it can be specially call instruction which, which restores instruction, and first device can pass through calling
Mode indicate second device execute image restore, accordingly, in response to call instruction is received, second device can be executed
State the step and/or process in any embodiment in image recovery method.
It should be understood that the terms such as " first " in the embodiment of the present invention, " second " are used for the purpose of distinguishing, and be not construed as
Restriction to the embodiment of the present invention.
It should also be understood that in the present invention, " multiple " can refer to two or more, "at least one" can refer to one,
Two or more.
It should also be understood that clearly being limited or no preceding for the either component, data or the structure that are referred in the present invention
In the case where opposite enlightenment given hereinlater, one or more may be generally understood to.
It should also be understood that the present invention highlights the difference between each embodiment to the description of each embodiment,
Same or similar place can be referred to mutually, for sake of simplicity, no longer repeating one by one.
Methods and apparatus of the present invention may be achieved in many ways.For example, can by software, hardware, firmware or
Software, hardware, firmware any combination realize methods and apparatus of the present invention.The said sequence of the step of for the method
Merely to be illustrated, the step of method of the invention, is not limited to sequence described in detail above, special unless otherwise
It does not mentionlet alone bright.In addition, in some embodiments, also the present invention can be embodied as to record program in the recording medium, these programs
Including for realizing machine readable instructions according to the method for the present invention.Thus, the present invention also covers storage for executing basis
The recording medium of the program of method of the invention.
Description of the invention is given for the purpose of illustration and description, and is not exhaustively or will be of the invention
It is limited to disclosed form.Many modifications and variations are obvious for the ordinary skill in the art.It selects and retouches
It states embodiment and is to more preferably illustrate the principle of the present invention and practical application, and those skilled in the art is enable to manage
The solution present invention is to design various embodiments suitable for specific applications with various modifications.
Claims (10)
1. a kind of image recovery method, which is characterized in that including：
Prior model is constructed according to the characteristic pattern of currently pending image, the prior model is described currently pending for learning
The information of image, the currently pending image include at least one image degradation phenomena；
Based on the prior model, the described image degradation phenomena for the currently pending image is carried out at image recovery
Reason, obtains corresponding target image.
2. the method according to claim 1, wherein described image degradation phenomena comprises at least one of the following：Figure
As noise, image is fuzzy and image pixel lacks；
Described image recovery processing comprises at least one of the following：Image denoising processing, image deblurring processing and image pixel are mended
Full processing.
3. method according to claim 1 or 2, which is characterized in that the characteristic pattern structure according to currently pending image
It builds before prior model, further includes：
Feature extraction is carried out to the currently pending image, obtains there is identical size with the currently pending image one
Group characteristic pattern；
It is described that prior model is constructed according to the characteristic pattern of currently pending image, including：
According to one group of characteristic pattern building prior model with the currently pending image with identical size.
4. according to the method described in claim 3, it is characterized in that, the basis is with the currently pending image with identical
Before one group of characteristic pattern of size constructs the prior model, further include：
Each characteristic pattern in one group of characteristic pattern with the currently pending image with identical size is carried out at image segmentation
Reason, obtains one group of local feature that wherein each characteristic pattern corresponds respectively to each predeterminable area in the currently pending image
Figure；
The basis constructs the prior model, packet with one group of characteristic pattern that the currently pending image has identical size
It includes：
According to the local feature figure for corresponding to identical predeterminable area in the currently pending image in each group local feature figure, really
Surely one group of prior model of each predeterminable area in the currently pending image is corresponded respectively to；
It is described to be based on the prior model, image recovery is carried out for the described image degradation phenomena of the currently pending image
It handles, before obtaining corresponding target image, further includes：
Image dividing processing is carried out to the currently pending image, obtains corresponding to each default in the currently pending image
The local image to be processed of one group of region；
It is described to be based on the prior model, image recovery is carried out for the described image degradation phenomena of the currently pending image
Processing, obtains corresponding target image, including：
To each predeterminable area in the currently pending image, it is based respectively on the corresponding prior model, for corresponding
The described image degradation phenomena of the part image to be processed carries out image recovery processing, obtain corresponding respectively to it is described currently to
Handle one group of localized target image of each predeterminable area in image；
It is described to be based on the prior model, image recovery is carried out for the described image degradation phenomena of the currently pending image
It handles, after obtaining corresponding target image, further includes：
Splicing is carried out to one group of localized target image for corresponding respectively to each predeterminable area in the currently pending image,
Obtain the target image.
5. method as claimed in any of claims 1 to 4, which is characterized in that described to be based on the prior model, needle
Image recovery processing is carried out to the described image degradation phenomena of the currently pending image, obtain corresponding target image it
Before, further include：
Preliminary recovery processing is carried out to the described image degradation phenomena of the currently pending image, after the preliminary recovery processing
Image be of the same size with the currently pending image；
It is described to be based on the prior model, image recovery is carried out for the described image degradation phenomena of the currently pending image
Processing, obtains corresponding target image, including：
Based on the prior model, for the described image degradation phenomena of the preliminary image that restores that treated, to carry out image extensive
Multiple processing, obtains the target image.
6. a kind of image recovery device, which is characterized in that including：
Modeling unit, for constructing prior model according to the characteristic pattern of currently pending image, the prior model is for learning
The information of the currently pending image, the currently pending image include at least one image degradation phenomena；
Processing unit, for be based on the prior model, for the currently pending image described image degradation phenomena into
Row image recovery processing, obtains corresponding target image.
7. a kind of electronic equipment, which is characterized in that including device as claimed in claim 6.
8. a kind of electronic equipment, which is characterized in that including：
Memory, for storing executable instruction；And
Processor completes method described in any one of claim 1 to 5 for executing the executable instruction.
9. a kind of computer program, including computer-readable code, which is characterized in that when the computer-readable code is in equipment
When upper operation, the processor in the equipment executes the instruction for realizing any one of claim 1 to 5 the method.
10. a kind of computer storage medium, for storing computer-readable instruction, which is characterized in that described instruction is held
Method described in any one of claim 1 to 5 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810541974.9A CN108898557B (en) | 2018-05-30 | 2018-05-30 | Image restoration method and apparatus, electronic device, computer program, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810541974.9A CN108898557B (en) | 2018-05-30 | 2018-05-30 | Image restoration method and apparatus, electronic device, computer program, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108898557A true CN108898557A (en) | 2018-11-27 |
| CN108898557B CN108898557B (en) | 2021-08-06 |
Family
ID=64343551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810541974.9A Active CN108898557B (en) | 2018-05-30 | 2018-05-30 | Image restoration method and apparatus, electronic device, computer program, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108898557B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111179158A (en) * | 2019-12-30 | 2020-05-19 | 深圳市商汤科技有限公司 | Image processing method, image processing apparatus, electronic device, and medium |
| CN111754435A (en) * | 2020-06-24 | 2020-10-09 | Oppo广东移动通信有限公司 | Image processing method, image processing device, terminal equipment and computer readable storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102243758A (en) * | 2011-07-14 | 2011-11-16 | 浙江大学 | Fog-degraded image restoration and fusion based image defogging method |
| CN104317902A (en) * | 2014-10-24 | 2015-01-28 | 西安电子科技大学 | Image retrieval method based on local locality preserving iterative quantization hash |
| US20150030231A1 (en) * | 2013-07-23 | 2015-01-29 | Mitsubishi Electric Research Laboratories, Inc. | Method for Data Segmentation using Laplacian Graphs |
| US20160173736A1 (en) * | 2014-12-11 | 2016-06-16 | Mitsubishi Electric Research Laboratories, Inc. | Method and System for Reconstructing Sampled Signals |
| CN107369134A (en) * | 2017-06-12 | 2017-11-21 | 上海斐讯数据通信技术有限公司 | A kind of image recovery method of blurred picture |
| CN107403201A (en) * | 2017-08-11 | 2017-11-28 | 强深智能医疗科技（昆山）有限公司 | Tumour radiotherapy target area and jeopardize that organ is intelligent, automation delineation method |
| CN107680043A (en) * | 2017-09-29 | 2018-02-09 | 杭州电子科技大学 | Single image super-resolution output intent based on graph model |
| CN107705249A (en) * | 2017-07-19 | 2018-02-16 | 苏州闻捷传感技术有限公司 | Image super-resolution method based on depth measure study |
| CN107833182A (en) * | 2017-11-20 | 2018-03-23 | 西安建筑科技大学 | The infrared image super resolution ratio reconstruction method of feature based extraction |
-
2018
- 2018-05-30 CN CN201810541974.9A patent/CN108898557B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102243758A (en) * | 2011-07-14 | 2011-11-16 | 浙江大学 | Fog-degraded image restoration and fusion based image defogging method |
| US20150030231A1 (en) * | 2013-07-23 | 2015-01-29 | Mitsubishi Electric Research Laboratories, Inc. | Method for Data Segmentation using Laplacian Graphs |
| WO2015012136A1 (en) * | 2013-07-23 | 2015-01-29 | Mitsubishi Electric Corporation | Method for segmenting data |
| CN104317902A (en) * | 2014-10-24 | 2015-01-28 | 西安电子科技大学 | Image retrieval method based on local locality preserving iterative quantization hash |
| US20160173736A1 (en) * | 2014-12-11 | 2016-06-16 | Mitsubishi Electric Research Laboratories, Inc. | Method and System for Reconstructing Sampled Signals |
| CN107369134A (en) * | 2017-06-12 | 2017-11-21 | 上海斐讯数据通信技术有限公司 | A kind of image recovery method of blurred picture |
| CN107705249A (en) * | 2017-07-19 | 2018-02-16 | 苏州闻捷传感技术有限公司 | Image super-resolution method based on depth measure study |
| CN107403201A (en) * | 2017-08-11 | 2017-11-28 | 强深智能医疗科技（昆山）有限公司 | Tumour radiotherapy target area and jeopardize that organ is intelligent, automation delineation method |
| CN107680043A (en) * | 2017-09-29 | 2018-02-09 | 杭州电子科技大学 | Single image super-resolution output intent based on graph model |
| CN107833182A (en) * | 2017-11-20 | 2018-03-23 | 西安建筑科技大学 | The infrared image super resolution ratio reconstruction method of feature based extraction |
Non-Patent Citations (2)
| Title |
|---|
| J. PANG等: ""Graph Laplacian Regularization for Image Denoising: Analysis in the Continuous Domain"", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| 苏振明: ""基于结构稀疏表达的图像恢复方法研究"", 《中国博士学位论文全文数据库 信息科技辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111179158A (en) * | 2019-12-30 | 2020-05-19 | 深圳市商汤科技有限公司 | Image processing method, image processing apparatus, electronic device, and medium |
| CN111179158B (en) * | 2019-12-30 | 2023-07-18 | 深圳市商汤科技有限公司 | Image processing method, device, electronic equipment and medium |
| CN111754435A (en) * | 2020-06-24 | 2020-10-09 | Oppo广东移动通信有限公司 | Image processing method, image processing device, terminal equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108898557B (en) | 2021-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yeh et al. | Semantic image inpainting with deep generative models | |
| Wu et al. | Fast end-to-end trainable guided filter | |
| US11301719B2 (en) | Semantic segmentation model training methods and apparatuses, electronic devices, and storage media | |
| Kobler et al. | Variational networks: connecting variational methods and deep learning | |
| KR102302725B1 (en) | Room Layout Estimation Methods and Techniques | |
| Liu et al. | Learning recursive filters for low-level vision via a hybrid neural network | |
| Lu et al. | Patch match filter: Efficient edge-aware filtering meets randomized search for fast correspondence field estimation | |
| US10339421B2 (en) | RGB-D scene labeling with multimodal recurrent neural networks | |
| CN109035319A (en) | Monocular image depth estimation method and device, equipment, program and storage medium | |
| Yin et al. | Fast and efficient implementation of image filtering using a side window convolutional neural network | |
| Singh et al. | Single image dehazing for a variety of haze scenarios using back projected pyramid network | |
| CN108230243B (en) | Background blurring method based on salient region detection model | |
| Lu et al. | Deep texture and structure aware filtering network for image smoothing | |
| US11783500B2 (en) | Unsupervised depth prediction neural networks | |
| CN108121931A (en) | two-dimensional code data processing method, device and mobile terminal | |
| CN113160296B (en) | Three-dimensional reconstruction method and device for vibration liquid drop based on micro-rendering | |
| US10832095B2 (en) | Classification of 2D images according to types of 3D arrangement | |
| US20230153946A1 (en) | System and Method for Image Super-Resolution | |
| WO2022100490A1 (en) | Methods and systems for deblurring blurry images | |
| US20230252605A1 (en) | Method and system for a high-frequency attention network for efficient single image super-resolution | |
| CN108898557A (en) | Image recovery method and device, electronic equipment, computer program and storage medium | |
| CN111489394B (en) | Object posture estimation model training method, system, device and medium | |
| Xia et al. | Single image rain removal via a simplified residual dense network | |
| CN114170290A (en) | Image processing method and related equipment | |
| WO2019219949A1 (en) | Image processing using generative graphical models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |