CN1985274A - Methods, system and program modules for restoration of color components in an image model - Google Patents
Methods, system and program modules for restoration of color components in an image model Download PDFInfo
- Publication number
- CN1985274A CN1985274A CNA2005800231062A CN200580023106A CN1985274A CN 1985274 A CN1985274 A CN 1985274A CN A2005800231062 A CNA2005800231062 A CN A2005800231062A CN 200580023106 A CN200580023106 A CN 200580023106A CN 1985274 A CN1985274 A CN 1985274A
- Authority
- CN
- China
- Prior art keywords
- image
- color component
- iteration
- regularization
- function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 114
- 230000015556 catabolic process Effects 0.000 claims abstract description 31
- 238000006731 degradation reaction Methods 0.000 claims abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 27
- 238000004590 computer program Methods 0.000 claims abstract description 10
- 238000004422 calculation algorithm Methods 0.000 claims description 34
- 238000011084 recovery Methods 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 19
- 238000012360 testing method Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 abstract description 14
- 230000006870 function Effects 0.000 description 45
- 230000003287 optical effect Effects 0.000 description 9
- 230000002950 deficient Effects 0.000 description 8
- 230000007246 mechanism Effects 0.000 description 8
- 238000001914 filtration Methods 0.000 description 7
- 230000007547 defect Effects 0.000 description 6
- 230000007850 degeneration Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000003321 amplification Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 230000006872 improvement Effects 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000003199 nucleic acid amplification method Methods 0.000 description 4
- 229920006395 saturated elastomer Polymers 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 230000003412 degenerative effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000873 masking effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 208000030984 MIRAGE syndrome Diseases 0.000 description 1
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
- G06T2207/20012—Locally adaptive
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2209/00—Details of colour television systems
- H04N2209/04—Picture signal generators
- H04N2209/041—Picture signal generators using solid-state devices
- H04N2209/042—Picture signal generators using solid-state devices having a single pick-up sensor
- H04N2209/045—Picture signal generators using solid-state devices having a single pick-up sensor using mosaic colour filter
- H04N2209/046—Colour interpolation to calculate the missing colour values
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
- H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Facsimile Image Signal Circuits (AREA)
- Color Television Image Signal Generators (AREA)
Abstract
This invention relates to a method for improving image quality of a digital image captured with an imaging module comprising at least imaging optics and an image sensor, where the image is formed through the imaging optics, the image consisting of at least one colour component. In the method, the degradation information of each colour component of the image is found and is used for improving image quality. The degradation information of each colour component is specified by a point-spread function. Each colour component is restored by said degradation function. The image can be unprocessed image data. The invention also relates to several alternatives for implementing the restoration, and for controlling and regularizing the inverse process independently of the image degradation. The invention also relates to a device, to a module, to a system and to a computer program products and to a program modules.
Description
Technical field
The present invention relates to Flame Image Process, the recovery of the color component in digital picture storage in particular or the capture systems.
Background technology
Image blurring or degeneration might be caused by multiple factor, for example, and the optical devices that defocus, or owing to use wide-angle lens or other aberration that the combination of locating owing to inappropriate f-number, focal length and camera lens causes.In the image capturing process, if use the long time shutter, moving of camera or imaging object may cause the picture motion blur so.In addition, if use the short time shutter, the photon numbers of being caught will reduce so, will produce very high noise level thus, and will cause the contrast of captured images very poor.
In association area, the method that much is used to rebuild the image that comprises defective all is known.For the defect block in the image, these defect block can be replaced with some piece around it or the mean value of all pieces.This one of them example is to use three pieces that are positioned at the defective top.The method that also has a kind of by name " best adjacent coupling " in addition, this method are to have the slide block of identical size and make it to move through image with defect area and restore this image by obtaining one.On each position except this slide block and defective position overlapped, the pixel value around this slide block border all is placed in the vector.Pixel value around the defective border then is placed in another vector, and calculates the square error between these two vectors.Then, this defect area piece that will be had a minimum boundary pixel is replaced.
For example, the space error concealing technology is attempted concealing defects by the good reconstruction that forms omission or contaminated pixel.It is to find out pixel average in the zone around the defective that a kind of method is arranged, and uses this average pixel value to replace this defective.Can add requirement, the variance of this reconstruction is equated with the variance of defective peripheral region the variance of rebuilding.
In image reconstruction process, can also use different interpolating methods.For example, bilinear interpolation can be applied to those pixels that is in four jiaos in defective rectangle.Do like this and can make the pixel value that passes defect area have linear and level and smooth transition.Bilinear interpolation is to be defined by pixel on the pixel value that will rebuild, the reconstruction pixel corner and the level and the vertical range of rebuilding between pixel and the corner pixels.Another kind method is the non-linear filtration that interpolation is omitted the edge sensitive of sampling in image.
The purpose of image restoration is to remove these to degenerate, so that the image that is restored looks as far as possible near original scene.In general, if degenerative process is known; The image that the is restored inverse process that can be used as degeneration is caught so.In the prior art, it is known several methods that are used to solve this reverse mathematical problem being arranged.But these most of technology wherein are in the modeling process of problem and reckon without image reconstruction process, and these technology hypothesis is the linear model of oversimplification.In general, finding the solution in the implementation process is very complicated, and its demand aspect calculating also is very high.
Image restoration has comprised two important step usually, i.e. deblurring step and noise filtering step.In the prior art, some method that is used for deblurring is known.These methods can be classified into non-iterative technique and iterative technique.In non-alternative manner, find the solution by a kind of disposable Processing Algorithm and obtain for example contrary filtration of Laplce's high-pass filter, anti-sharpening mask or frequency domain.In alternative manner, in some times processing procedure, the result is carried out refinement.Deblurring is handled by a cost function that sets up standard for thinning processing and is controlled, for example least square method or self-adaptation Landweber algorithm.Usually, through after the iteration several times, between adjacent step, there is not very big improvement.After having exceeded certain point, continue to carry out the deblurring algorithm and might in the image that restores, introduce bothersome pseudomorphism (artefact).The another kind of method that solves the deblurring problem then is to implement to have the single step deblurring method of running parameter repeatedly and keep optimum (blind deconvoluting).
Usually, these methods in the association area are implemented in the image restoration of the high-end applications of uranology and imaging of medical and so on for example usually.Since image acquisition be difficult to quantize and implement required complexity of these algorithms and computing power, therefore in consumer products, the application of these methods is very limited.In the equipment with limited computational resource and storage resources, some method has obtained use.Usually, the method in the association area has been designed to a post-processing operation, this means to restore at image just to be applied to this image after the obtained and storage.In post-processing operation, each color component all has different point spread functions, and this function then is the important rule that can be used to assess the imaging system performance.If restore as after-treatment applications, so will be no longer relevant about the fuzzy information of the difference in each color component.The accurate modeling of image acquisition and processing will be more difficult, and (in most of the cases) will not be linear.Therefore, " contrary " finds the solution more out of true.In addition, the output of digital camera tends to be compressed into the .jpeg form.If recovery applies afterwards in compression (this compression normally diminishes), its result will amplify undesirable blocking artefacts so.
Summary of the invention
The purpose of this invention is to provide a kind of improvement method that is used for restored image.This purpose can realize by a kind of method, model, model use, deblurring method, equipment, module, system, program module and computer program.
According to the present invention, a kind of method that is used to form the model of the picture quality that can improve the digital picture that image-forming module catches is provided, this image-forming module comprises image optics device and imageing sensor at least, wherein image forms by the image optics device, described image has comprised at least one color component, wherein find the degradation information of each color component, obtain the image degradation function and restore described each color component by described degenrate function.
According to the present invention, a kind of model that is used to improve the picture quality of digital picture also is provided, described model can obtain by method required for protection.According to the present invention, also provide the use of this model here.
In addition, according to the present invention, a kind of method that is used to improve the picture quality of the digital picture that image-forming module catches is provided, wherein this image-forming module comprises image optics device and imageing sensor at least, wherein image forms by the image optics device, and described image comprises a color component at least, wherein find the degradation information of each color component in the image, obtain a degenrate function according to this degradation information, and restore described each color component by described degenrate function.
In addition,, provide a kind of method that is used for restored image, wherein restored and implement, and in iteration each time, implemented deblurring method by regularization by the iteration recovery function according to the present invention.
In addition, according to the present invention, a kind of system of model of the picture quality that is used for determining improving the digital picture that image-forming module catches is provided, described module comprises image optics device and imageing sensor at least, wherein image forms by the image optics device, described image comprises a color component at least, wherein this system comprises: first device that is used for finding the degradation information of each color component of image, be used for obtaining second device of degenrate function according to degradation information, and the 3rd device that is used for restoring by described degenrate function described each color component.
In addition, according to the present invention, a kind of image-forming module is provided, this image-forming module comprises image optics device and imageing sensor, so that form image by the image optics device in photosensitive image sensor, the model that wherein is used to improve picture quality is relevant with described image-forming module.In addition, according to the present invention, also provide a kind of equipment that comprises image module.
In addition, according to the present invention, a kind of program module that is used to improve picture quality in an equipment is provided, this equipment comprises image-forming module, the device that described program module has comprised the degradation information of each color component that is used to find image, obtained degenrate function and restore described each color component by described degenrate function according to this degradation information.According to the present invention, other program module that is used for restored image here also is provided, implement the device of handling by the deblurring of regularization comprising the iteration each time that is used for restoring in iteration.
In addition, provide a kind of computer program, this computer program has comprised the degradation information that is used to find each color component, the instruction that obtains the image degradation function and restore described each color component by described degenrate function.According to the present invention, the computer program that is used for restored image also is provided, this computer program comprises that the iteration each time that is used for restoring in iteration implements the computer-readable instruction of handling by the deblurring of regularization.
Further feature of the present invention is described in additional dependent claims.
In instructions, what term " first iconic model " was corresponding is such image, and promptly this image is caught by the imageing sensor of for example CCD (charge-coupled image sensor) or CMOS (complementary metal oxide semiconductor (CMOS)), but does not carry out any processing as yet.This first iconic model is a raw image data.Second iconic model then is a view data of having determined its degradation information.Be understandable that other sensor type except that CMOS or CCD can be applied to the present invention equally.
First iconic model is used to determine the fuzzy of image, and restores second iconic model according to the present invention.This recovery can also be according to the present invention and by regularization.After finishing these steps, can implement other image reconstruction function.Rebuild chain if consider entire image, thought so of the present invention is used recovery as pretreatment operation, and follow-up thus image reconstruction operation will be benefited from this restores.With restore as pretreatment operation be applied mean restore algorithm directly at be the primitive color view data, and so, each color component is all handled independently.
By the present invention, fuzzy can obviously minimizing that optical device produced.Especially, if use is fixed focal length, this process will be very effective so.The present invention is equally applicable to zoom system, and in the case, this processing is considered a plurality of deblurring functions from a look-up table according to the focal position of camera lens.Described deblurring function can also obtain by the interpolation from look-up table.Wherein a kind ofly define may being to use wherein of deblurring function with the continuous calculating of focal length as the deblurring function parameters.Resulting image will be more sharp keen and has a better spatial resolution.What deserves to be mentioned is that the processing of being advised is different from traditional sharpening algorithm, it can also produce has the sharper keen image that high frequency is exaggerated.In fact, the present invention proposes is a kind ofly to reply degenerative process and will be for example fuzzyly by optical device produced reduce to minimum method, make it look more sharp keen thereby sharpening algorithm then is to use common high pass filter to add pseudomorphism in image.
Concerning the sensors of various types that can in the product in future, use, (because linear image form model have better fidelity) feasible especially according to model of the present invention.In current approach, resolution that subsequent step in the image reconstruction chain and algorithm will promote from solution and contrast and be benefited.
Image restoration is applied as pretreatment operation, can be minimized in the nonlinearity of accumulating in the image capturing process.In addition, the present invention can also prevent from excessively to amplify colouring information.
Data recovery comes the sharpening image by the contrary filtration of iteration.This contrary filtration can be controlled by a kind of control method provided by the invention.Since this control method, when image is enough sharp keen, iteration stopping.This control method provides a kind of mechanism of those pixels that enter image in different positions being carried out different disposal.According to this processing, the toning in the restored image (overshooting) can reduce, and can produce the visual quality of better final image thus.In addition, the pixel that is positioned at observed image border is to restore in the mode that is different from the pixel on the smooth region.This control method can also solve the problem of the point spread function of spatial variations.For instance, if the point spread function of optical system is different to different pixel coordinates, using independently so, the image restoration of processes pixel can address this problem.In addition, this control method can be implemented by several deblurring algorithms, thereby improves its performance.
The present invention also is applicable to video restoration.
Description of drawings
The present invention be with reference to the accompanying drawings with subsequent descriptions in example describe.
Fig. 1 has described an example according to system of the present invention;
Fig. 2 has described another example according to system of the present invention;
Fig. 3 has described an example according to equipment of the present invention;
Fig. 4 has described an example according to arrangement of the present invention; And
Fig. 5 has described an example according to iteration restored method of the present invention and control method.
Embodiment
The present invention relates to a kind of method that is used to improve the picture quality of utilizing the digital picture that image-forming module catches, wherein this image-forming module comprises image optics device and imageing sensor at least, image forms by the image optics device, and this image has comprised at least one color component.In the method, the degradation information of each color component is found in the image, and is used to improve picture quality.The degradation information of each color component is then stipulated by point spread function.Each color component is restored by described degenrate function.This image can be untreated view data.In addition, the invention still further relates to and be used to implement restore and the contrary a plurality of substitute modes controlled with regularization of handling.
To according to the description of image restoration of the present invention at three main points, at first, for example the point spread function (PSF) of at least one primitive color component is determined fuzzy degenrate function by surveyingpin.Secondly, this recovery algorithm is at least one primitive color component design.The 3rd, regularization mechanism can be incorporated into wherein, so that alleviate the effect of high-pass filter.In this manual, the optical device in the mobile device uses as an example, because they are confined to very wide focusing range usually.But it will be apparent to those skilled in the art that mobile device is not unique suitable device.For example, the present invention can also use for digital camera, web camera or similar equipment, and can be used by high-end applications.The purpose of this algorithm is to remove or weaken the degeneration processing (bluring) that optical device produces.By this algorithm, resulting image will be more sharp keen, and have the resolution of improvement.
When using term " color component ", what this term related to is different color systems.Example in this example is RGB system (red, green, blue), but those skilled in the art are the systems that can expect other, for example HSV (colourity, saturation degree, purity), YUV (brightness and aberration) or CMYK (blue or green, pinkish red, yellow, black) or the like.
In spatial domain, iconic model can be described as follows:
g
i(m，n)＝h
i(u，v)*f
i(m，n)+n
i(m，n) (1)
G wherein
iBe the color component images that measures, f
iBe the primitive color component, h
iBe fuzzy, the n of corresponding linearity in the color component
iIt then is additive noise term.g
i, f
i, n
iBe that (m n) goes up definition, and h at the pel array of having crossed over image-region
iBe that (u v) upward defines in the pixel of crossing over fuzzy (point spread function) support (support).Index i={1,2,3,4} represents respectively is data about color component, for example red, green 1, blue and green 2 color components.
By Fig. 1 and 2 the present invention is described in more detail, wherein each accompanying drawing has all been described the block diagram according to image restoration system of the present invention.
Fuzzy regulation
What next will describe is the process that the degeneration (Fig. 1,110) in the image that optical element (100) is caught is estimated.In Fig. 2 as can be seen, this degeneration can by with three Color Channels (being R, G, B in this example) in fuzzy (raw data) corresponding point spread function 210 estimate.Described point spread function is used to show the different qualities of each Color Channel.And this point spread function is a kind of major criterion that can be used to assess the imaging system performance.
Described point spread function is as the function of the position in the wavelength and the camera visual field and change.Thus, it might be difficult seeking good point spread function.In this manual, employing is to defocus closely imaging and space invariance Fuzzy Processing.Be used to estimate the point spread function (h that is associated with each color component
i) real process also can be used as an independent utility and use, thereby help to implement evaluation process in the camera system.
Under situation about having provided with a corresponding blurred picture of color component of check-out console pattern, four outside intersecting points manually locate, and what at first determine is the roughly estimation of angle position.Accurate position (being accurate to sub-pixel) is calculated once more by the described search of refinement in the square window of for example 10 * 10 pixels.By using these intersecting points, can average by core and rebuild original mesh image f by the constant luminance value of concluding these squares to each square
iApproximate value.
It is spatially constant that point spread function is assumed to, and can calculate fuzzy by pseudoinverse filter method (for example in Fourier domain) thus.Because the pseudoinverse technology is highstrung to noise, therefore, the frequency low pass filter can be used for limit noise, and can use this process in conjunction with several images, so that obtain the average estimation (the normalization cutoff frequency of the low-pass filter of being mentioned is about 0.6, but any value of from 0.4 to 0.9 also all is suitable at least) of point spread function.
For the fog-level that quantizes to occur on each Color Channel, defined a simple statistical information here, this statistical information is confirmed as the average of the Weighted distance (is unit with the pixel) at and function center, described weight correspondence be the value of the normalization point spread function of this point:
M wherein
IAnd N
IBe the support of point spread function filtrator.S
PsfWhat describe is the degree of bluring.By testing identifiable is that these passages have different fuzzy patterns.For example, when research Mirage-1 camera, the S that is obtained
PsfValue is:
As can be seen, red component is the fuzzyyest and noise is maximum from the result, and least fuzzy be blue component, and this component also has minimum contrast.
Restore algorithm
Data about color component are recorded by sensor 120, and for example the Bayer sensor 220 (Fig. 2) by CMOS or ccd sensor and so on records.Color component can be red component (R), green component 1 (G1), blue component (B) and the green component 2 (G2) among Fig. 2.This each color " image " wherein all is 1/4 size of final output image.
Second iconic model provides (130 for recovery; 250).This image is arranged in the vector according to lexicographic order, and point spread function h
iBe arranged into square-Toeplitz circular matrix H
iIn.Then, second iconic model can be as follows:
Obtaining H
iThe situation of reasonable approximation under, the purpose of image restoration is the observations from degenerating
Middle recovery optimum estimate
Ambiguity function H
iBe irreversible (it is defined on limited support, and its inverse function will have infinite support thus), therefore can't obtain directly contrary separating.What the typical direct method that addresses this problem was considered is that the energy between fuzzy again image input and that simulated is reduced to minimum, and this processing obtains by following norm:
Provide least squares fitting for data thus.When noise is known when being Gaussian noise, the minimizing of described norm also can obtain maximum likelihood and separate.It also obtains the contrary filtrator of broad sense, and described contrary filtrator is following providing:
Want it is found the solution, common way is that the determinacy iterative technique is used in combination with the continuous approximation method, will produce following iteration like this:
If
This iteration convergence, wherein λ so
MaxIt is matrix H
TThe dominant eigenvalue of H.The normalization of energy become very little before, this iteration will be carried out always.
From Fig. 1 and 2, as can be seen, restore (130; 250) carry out independently for each color component R, G, B.
The major advantage of iterative technique is not need dominance ground to implement the inverse operation of fuzzy operator, and restores and handle and can carry out obtaining in the process monitoring at it.
Least square can be extended to classical least square (CLS) technology.In theory, the problem of image restoration is failed to see fixed (ill-posed), that is to say, the small upset in the output such as noise all might cause occurring upsetting greatly in the direct least square solution that as above provides.Therefore, constraint least square side's method only takes in documents and materials usually.These algorithms can reduce to minimum with those that are under the jurisdiction of (smoothly) regular terms in the equation (4), and described regular terms then is made up of the form of the high-pass filter of output.Described regular terms allows to comprise the prior imformation about image.
An example of regularization mechanism
In practice, the image sensing electronic device such as CCD and cmos sensor might be introduced nonlinearity in image, and wherein saturation degree is very serious one.Because form in the model at image, nonlinearity can't be held, therefore, it is painted that the independent processing of Color Channel might cause the edge grave error to occur on every side.Thus, the present invention has introduced regularization mechanism (Fig. 2 of a kind of improvement that is applied to restore; 240).Saturated or under-exposed pixel region is used to design a smooth change coefficient of alleviating the high-pass filter effect in the peripheral region.It is linear (1) that the formulate of image acquisition process is assumed to all the time.Because have sensitivity difference and the fuzzy control that exposes between three Color Channels, in each Color Channel, pixel intensity might occur in unorganized mode.Concerning near the autonomous channel recovery of these zones of saturation, this recovery will cause this color component excessively to be amplified individually, can produce the painted of artificial picture color mismatch and mistake thus near these zones.For fear of this situation occurring, propose a kind of here according to regularization mechanism of the present invention.This regularization mechanism integration is in the iterative of equation (6).Its thought is spatially μ to be regulated, so that near the recovery effect the restriction zone of saturation.Step sizes through overregulating is following given:
μ
adap(m，n)＝β
sat(u，m)μ (9)
Wherein μ is previously discussed overall step sizes, β
SatThen be the local saturation control that step sizes is adjusted.And β
SatBe to use following algorithm to obtain:
To each Color Channel image g
i, i={1..4},
Consider around location of pixels g
i(m, window n) (value of w * w),
To the saturated pixel S in this window
i(m, quantity n) is counted,
Saturated control is provided by following equation:
β
SatChange between 0 and 1 according to the quantity of the saturated pixel in any Color Channel.Another example of iteration restored method and regularization mechanism
Previous data recovery comes the sharpening image by the contrary filtration of iteration.This contrary filtration can be controlled by a kind of control method, can stop this iteration thus when image is enough sharp keen.In Fig. 5, be described with the form of block diagram basic thought to this control method.At the beginning of this method, wherein image initial is changed into to observe image identical, and the parameter of deblurring algorithm has been carried out being provided with (510).After this, the deblurring algorithm is applied to observing image.This processing can be any in the existing disposable algorithm, and for example unsharp masking method, fuzzy field deblurring, difference are filtered or the like (520).In iteration each time, deblurring all is significant, and this is because if deblurring does not have good performance, and the overall performance of system can be not fine so.In following step (530), can check pixel, so that detect toning such as the edge of excessive amplification from de-blurred image.In subsequent step (540), restored image will obtain upgrading.If the toning edge, location of pixels territory in the de-blurred image is corresponding, in iterative processing, can further not upgrade so.Otherwise, will be updated usually from the pixel of restored image.In addition, also masked with the corresponding pixel of toning, so that in ensuing iterative process, make corresponding recovery pixel remain unchanged (concerning those pixels, restore and can stop) at this point.In following step (550), middle output image is scanned, and will detect the pixel that still comprises toning.If detect constant toning (560), then stop global iterative and handle and turn back to restored image.Otherwise the parameter of deblurring algorithm will change (570), and ensuing iteration is to be beginning with the deblurring of observing image.Last step (560-570) will make algorithm be suitable for carrying out blind deconvoluting.Algorithm disclosed herein will prevent that memory image from forming the toning that occurs because of the excessive amplification at edge.This processing is finished by dual mode.At first, in iteration each time, pixel all is independently updated, and thus, the pixel of those degenerations is can not be updated in the restored image.Secondly, if there is the excessive pixel of degenerating in restored image, so whole deblurring is handled and all can be stopped.Next will the embodiment of deblurring method be described in detail.
The method step of Fig. 5 is to implement for the some components among color component R, G, the B.Other two components then are individual processing fully in an identical manner.If what use is the YUV color system, so only need handle component Y.
In step 510, image initial is changed into to observe image identical, and the parameter of deblurring algorithm is set.Here, the observation image of input is represented with I, and final restored image is represented with Ir.At the beginning, restored image is initialized with I (Ir=I).The parameter of deblurring method equally also is initialised.For instance, handle, then the quantity and the parameter thereof of employed blurred picture are selected if the unsharp masking method is used for deblurring.If what implement is another kind of algorithm, its parameter will be provided with at this point so.In addition, here also can size of initialization and image equal and opposite in direction and matrix with identity element.This matrix is represented with mask.
In step 520, wherein can be with the deblurring algorithm application in observing image and obtaining de-blurred image Idb.In step 530, each pixel from de-blurred image is checked, so that detect toning such as excessive amplification edge.Pixel from de-blurred image Idb will be scanned, and the level between the neighbor and vertical range are calculated as follows:
dh1(x，y)＝ldb(x，y)-ldb(x，y-l)
dh2(x，y)＝ldb(x，y)-ldb(x，y+l)
dv1(x，y)＝ldb(x，y)-ldb(x-l，y)
dv2(x，y)＝ldb(x，y)-ldb(x+l，y)
X wherein, what y represented respectively is vertical and horizontal pixel coordinate.In addition, will be scanned equally from the pixel of observing image, and the level between the neighbor can be calculated as follows with vertical difference:
dh3(x，y)＝l(x，y)-l(x，y-l)
dh4(x，y)＝l(x，y)-l(x，y+l)
dv3(x，y)＝l(x，y)-l(x-l，y)
dv4(x，y)＝l(x，y)-l(x+l，y)
For each pixel, will check here whether corresponding difference dh1 is different with the symbol of dv3 and dv2 and dv4 with dh4, dv1 with dh3, dh2 from de-blurred image.If their differences mean then to be positioned at coordinate x that the pixel on the y has comprised toning.This inspection is implemented by following algorithm:
ifNOT[sign(dh1(x，y))＝sign(dh3(x，y))]OR?NOT[sign(dh2(x，y))＝sign(dh4(x，y))]
if[abs(adl(x，y))＞＝th1*MAX]AND[abs(dh2(x，y))＞＝th1*MAX]
mh＝0；
end
end
ifNOT[sign(dv1(x，y))＝sign(dv3(x，y))]OR?NOT[sign(dv2(x，y))＝sign(dv4(x，y))]
if[abs(dv1(x，y)0＞＝th1*MAX]AND[abs(dv2(x，y))＞＝th1*MAX]
mv＝0；
end
end
if(mh＝0)OR(mv＝0)
mask(x，y)＝0；
end
Basically, thought above is to the comparison between the local shape of the local shape that carries out described de-blurred image from each pixel of de-blurred image and described observation image.This processing is to finish by the symbol that compares the corresponding difference of these two images in the horizontal and vertical directions.When the shape of finding these two images there are differences (no matter being level or vertical direction), then mean the respective pixel that excessively to have emphasized from de-blurred image.Concerning these pixels, here estimated value and certain threshold value (th1) to toning compares.If the amount of toning is greater than threshold value (th1), then corresponding element marking being become is (mask value is equalled zero) of distortion.It is the peaked percentage of pixel (value MAX is the maximal value of I) from observing image that this threshold value (th1) then is defined by.By selecting this threshold calculations, we can guarantee that the value and the image range of this threshold value (th1) is suitable.
In step 540, restored image is upgraded.The pixel that forms restored image is to use simply from the pixel of the de-blurred image that is not marked as distortion to upgrade.This step can be implemented as follows:
for?every?pixel?from?Idb(x，y)
if?mask(x，y)＝1
Ir(x，y)＝Idb(x，y)；
end
end
In step 550, middle output image is scanned and detects the pixel that still comprises toning.In the scan reconstruction image, the level between the neighbor can be calculated as follows with vertical difference:
dh5(x，y)＝lr(x，y)，y)-lr(x，y-l)
dh6(x，y)＝lr(x，y)-lr(x，y+l)
dv5(x，y)＝lr(x，y)-lr(x-1，y)
dv6(x，y)＝lr(x，y)-lr(x+1，y)
Symbol to corresponding difference dh5 and dh3, dh6 and dh4, dy5 and dy3 and dy6 and dv4 compares.If these symbols there are differences, overtravel can be calculated as follows so:
If?NOT[sign(dh5(x，y))＝sign(dh3(x，y))]OR?NOT[sign(dh6(x，y))＝sign(dh4(x，y))]
H(x，y)＝min(abs(dh5(x，y))，abs(dh6(x，y)))；
end
If?NOT[sign(dv5(x，y))＝sign(dv3(x，y))]OR?NOT[sign(dv6(x，y))＝sign(dv4(x，y))]
V(x，y)＝min(abs(dh5(x，y))，abs(dh6(x，y)))；
end
By comparing the symbol of the difference that on restored image and original image, calculates, can the local shape of two images be compared.Concerning the variform pixel in part, the toning in the restored image is to estimate by the least absolute value that adopts two adjacent differences.This value calculates on vertical and horizontal direction.
In step 560, toning is checked.If maximum toning then stops recuperation greater than predetermined step-length, and in output, turn back to restored image Ir.If in restored image, do not have pixel, then change the parameter of deblurring method, and this process will begin to continue to carry out from step 520 greater than the toning of threshold value.This step can be implemented as follows:
if?max(max(H(x，y))，max(V(x，y)))＞＝th2*MAX
return?the?image?Ir?and?stop?the?restoration?process
else
modify?the?parameters?of?the?de-blurring?method?and?go?to?step?520.
end
The threshold value th2 that toning detects is defined as the number percent of the max pixel value of original image I.
Regularization method (Fig. 5 530,550 and 560) can also combine with the above-mentioned iteration restored method of equation (6).After the said method in conjunction with local or overall regularization, the non-iteration of other such as high-pass filter is restored algorithm and also can be implemented.The iteration recovery technique that both can be applied to other is simultaneously handled in part defined above and overall regularization, also can be applied to other iteration recovery technique separately.
The image reconstruction chain
The description of restoring about each color component before is to use as the operation of first in the image reconstruction chain.For example Automatic white balance, colour filter array interpolation (CFAI), color gamut conversion, geometric distortion and shadow correction, noise reduce, other operation (140 of sharpening; 260) then follow thereafter.Be understandable that final picture quality (270) depends on the effective and optimization use of all these operations of rebuilding in the chain.The most effective embodiment of image reconstruction algorithm is non-linear.In Fig. 1, for example or/and handling, down-sampling/shake (160) carries out Flame Image Process by compression of images (150).Image both can be checked by view finder of camera (180) or display, also can store (170) with compressed format in storer.
In rebuilding chain, will restore as first operation and use, can guarantee that the linear imaging model presents best fidelity.To the algorithm of back, especially colour filter array interpolation and noise reduce algorithm, and what these algorithms served as is the regularization mechanism of adding, to prevent because of the excessive excessive amplification that recovery was caused.
Embodiment
Can be installed in the equipment such as portable terminal, web camera, digital camera or be used for the miscellaneous equipment of imaging according to system of the present invention.This system can be mounted in the part of the digital signal processing in the camera model of one of described equipment.An example of these equipment is the imaging portable terminals that show with simplified block diagram among Fig. 3.Equipment 300 has comprised optical device 310 or has been used to catch the similar devices of image, this equipment can be operationally with optical device or the digital camera that is used to catch image communicates.In addition, equipment 300 can also comprise the communicator 320 with transmitter 321 and receiver 322.Also can there be other communicator 380 with transmitter 381 and receiver 382.First communicator 320 can be adapted to the execution telecommunication, and another communicator 380 then can be certain short-range communication means, for example Bluetooth
TMSystem, wlan system (WLAN (wireless local area network)) or other system that is fit to local use and communicates with other equipment.Equipment 300 according to Fig. 3 also comprises the display 340 that is used for display visual information.In addition, equipment 300 also comprises the keypad 350 that is used to import operations such as data and control image capturing.In addition, equipment 300 can also comprise the audio frequency apparatus such as earphone 361 and microphone 362, and comprises alternatively and being used for the encode codec of (words of needs also can be decoded) of audio-frequency information.And this equipment 300 comprises also and is used for control module 330 that the function of equipment 300 is controlled that for instance, described function for example is according to recovery algorithm of the present invention.This control module 330 can comprise one or more processors (CPU, DSP).And this equipment also further comprises the storer 370 that is used to store data, program or the like.
Image-forming module according to the present invention comprises image optics device and imageing sensor, and be used for finding the degradation information of each color component of image and determine the device of degenrate function and other device that restores each color component by described degenrate function according to this degradation information.This image-forming module can be installed in the previously described equipment.In addition, this image-forming module can be installed in also in the autonomous device 410 that shown in Figure 4 and imaging device 400 and display device communicate wherein that this display device also can be the miscellaneous equipment of described imaging device 400 or personal computer and so on.Described autonomous device 410 comprises restoration module 411 and has comprised other image-forming module 412 alternatively, and this equipment can independently be used for image reconstruction process.Communicating by letter and to handle by wired or wireless network between imaging device 400 and the autonomous device 410.Example about this type of network can be the Internet, WLAN, Bluetooth or the like.
Above detailed description is only understood the present invention and is provided for clear, and not to be construed as be that the claim to here limits.
Claims (41)
1. an exploitation is used to improve the method for model of the picture quality of the digital picture that image-forming module catches, wherein this image-forming module comprises image optics device and imageing sensor at least, this image forms by described image optics device, described image has comprised at least one color component, and the exploitation of wherein said model comprises the following steps: at least
Find the degradation information of each color component in the described image,
Obtain degenrate function according to described degradation information, and
Restore described each color component by described degenrate function.
2. according to the process of claim 1 wherein that using regularization for the color component that restores controls.
3. according to the process of claim 1 wherein that the described degradation information of each color component finds by point spread function.
4. according to the method for claim 3, wherein restore by the iteration recovery function and implement, this iteration recovery function is to determine from the point spread function of each color component.
5. implement according to the process of claim 1 wherein to restore, wherein in iteration each time, use single step deblurring method by regularization by the iteration recovery function.
6. according to the process of claim 1 wherein that described image is a raw image data, and the color component of wherein said recovery is further handled by other image reconstruction algorithm.
7. according to the process of claim 1 wherein that employed is a kind of in the following color system: RGB, HSV, CMYK, YUV.
8. according to the method for claim 2, wherein said regularization control is implemented in the deblurring method, so that obtain de-blurred image.
9. method according to Claim 8 is wherein by the first threshold and the second threshold test toning pixel.
10. model that is used to improve the picture quality of digital picture, described model can obtain by the method for claim 1.
11. one kind to the use according to the module of claim 10, is used to improve the picture quality of digital picture.
12. method that is used to improve the picture quality of the digital picture that image-forming module catches, wherein this image-forming module comprises image optics device and imageing sensor at least, this image forms by described image optics device, and described image has comprised at least one color component, wherein:
Find the degradation information of each color component in the described image,
Obtain degenrate function according to described degradation information, and
Restore described each color component by described degenrate function.
13., wherein use regularization control for the color component that restores according to the method for claim 12.
14. according to the method for claim 12, wherein the described degradation information of each color component is found by point spread function.
15. according to the method for claim 14, wherein restore by the iteration recovery function and implement, this iteration recovery function is to determine from the point spread function of each color component.
16. according to the method for claim 12, wherein restore by the iteration recovery function and implement, wherein in iteration each time, implement single step deblurring method by regularization.
17. according to the method for claim 12, wherein said image is a raw image data, and the color component of wherein said recovery is further handled by other image reconstruction algorithm.
18. according to the method for claim 12, wherein employed is a kind of in the following color system: RGB, HSV, CMYK, YUV.
19. according to the method for claim 13, wherein said regularization control is implemented in the deblurring method, so that obtain de-blurred image.
20. according to the method for claim 19, wherein by the first threshold and the second threshold test toning pixel.
21. a method that is used for restored image is wherein restored by the iteration recovery function and is implemented, and wherein implements the deblurring method by regularization in iteration each time.
22., wherein use regularization control for the color component that restores according to the method for claim 21.
23. according to the method for claim 21, wherein said regularization control is implemented in the described deblurring method, so that obtain de-blurred image.
24. according to the method for claim 21, wherein by the first threshold and the second threshold test toning pixel.
25. the system of the model of a picture quality that is identified for improving the digital picture that image-forming module catches, wherein this image-forming module comprises image optics device and imageing sensor at least, this image forms by described image optics device, described image has comprised at least one color component, and wherein this system comprises:
Be used for finding first device of the degradation information of described each color component of image,
Be used for obtaining second device of degenrate function according to described degradation information, and
Be used for restoring the 3rd device of described each color component by described degenrate function.
26., also comprise the 4th device that is used for using regularization control in recuperation according to the system of claim 25.
27., also comprise being used for coming the 5th device that described image is further handled by other image reconstruction algorithm according to the system of claim 25.
28. according to the system of claim 25, wherein this system can use a kind of in the following color system: RGB, HSV, CMYK, YUV.
29. according to the system of claim 26, wherein for this regularization control, described system has comprised the device that is used for the image that is restored is carried out deblurring.
30. image-forming module, comprise image optics device and imageing sensor, so that form image by this image optics device in this photosensitive image sensor, wherein the described model that is used to improve picture quality of claim 10 is relevant with described image-forming module.
31., also comprise the device that is used for implementing regularization control in recuperation according to the image-forming module of claim 30.
32. an equipment comprises image-forming module as claimed in claim 30.
33. according to the equipment of claim 32, wherein this equipment is the mobile device with communication capacity.
34. a program module that is used for improving at the equipment that has comprised image-forming module picture quality, described program module has comprised the device that is used for following processing:
Find the degradation information of each color component in the described image,
Obtain degenrate function according to described degradation information, and
Restore described each color component by described degenrate function.
35., also comprise the instruction that is used for implementing regularization control in recuperation according to the program module of claim 34.
36. a program module that is used for restored image comprises that the iteration each time that is used for restoring in iteration implements the device of handling by the deblurring of regularization.
37., also comprise the device that detects the toning pixel by first threshold and second threshold value according to the program module of claim 36.
38. a computer program that is used to improve picture quality comprises by computer implemented instructions, so that
Find the degradation information of each color component in the described image,
Obtain degenrate function according to described degradation information, and
Restore described each color component by described degenrate function.
39., also comprise the instruction that is used for using regularization control in recuperation according to the computer program of claim 38.
40. a computer program that is used for restored image comprises that the iteration each time that is used for restoring in iteration implements the computer-readable instruction of handling by the deblurring of regularization.
41., also comprise the instruction that detects the toning pixel by first threshold and second threshold value according to the computer program of claim 40.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US10/888,534 | 2004-07-09 | ||
| US10/888,534 US7728844B2 (en) | 2004-07-09 | 2004-07-09 | Restoration of color components in an image model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN1985274A true CN1985274A (en) | 2007-06-20 |
Family
ID=35599485
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2005800231062A Pending CN1985274A (en) | 2004-07-09 | 2005-01-04 | Methods, system and program modules for restoration of color components in an image model |
Country Status (6)
| Country | Link |
|---|---|
| US (2) | US7728844B2 (en) |
| EP (1) | EP1766569A1 (en) |
| JP (1) | JP4571670B2 (en) |
| KR (1) | KR100911890B1 (en) |
| CN (1) | CN1985274A (en) |
| WO (1) | WO2006005798A1 (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102326379A (en) * | 2008-12-31 | 2012-01-18 | 浦项工科大学校产学协力团 | Method for removing blur from image and recording medium on which the method is recorded |
| CN101388952B (en) * | 2007-09-12 | 2012-03-28 | 三星电子株式会社 | Apparatus and method for restoring image |
| CN103035185A (en) * | 2011-09-30 | 2013-04-10 | 伊姆普斯封闭式股份有限公司 | Method for brightness correction of defective pixels of digital monochrome image |
| TWI512682B (en) * | 2014-09-30 | 2015-12-11 | Quanta Comp Inc | Image processing system and saturation compensation method |
| CN105933595A (en) * | 2015-02-26 | 2016-09-07 | 诺基亚技术有限公司 | Method, Apparatus And Computer Program Product For Reducing Chromatic Aberrations In Deconvolved Images |
| CN106572285A (en) * | 2015-10-08 | 2017-04-19 | 三星电机株式会社 | Camera module, electronic device, and method of operating the same |
Families Citing this family (60)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7627193B2 (en) * | 2003-01-16 | 2009-12-01 | Tessera International, Inc. | Camera with image enhancement functions |
| JP4250583B2 (en) * | 2004-09-30 | 2009-04-08 | 三菱電機株式会社 | Image capturing apparatus and image restoration method |
| JP2007060457A (en) * | 2005-08-26 | 2007-03-08 | Hitachi Ltd | Image signal processor and processing method |
| KR101292458B1 (en) * | 2005-11-10 | 2013-07-31 | 테세라 인터내셔널, 인크 | Image enhancement in the mosaic domain |
| WO2007072477A2 (en) * | 2005-12-21 | 2007-06-28 | D-Blur Technologies Ltd. | Image enhancement using hardware-based deconvolution |
| US20070165961A1 (en) * | 2006-01-13 | 2007-07-19 | Juwei Lu | Method And Apparatus For Reducing Motion Blur In An Image |
| JP2007304525A (en) | 2006-05-15 | 2007-11-22 | Ricoh Co Ltd | Image input device, electronic equipment, and image input method |
| WO2007141368A1 (en) * | 2006-06-09 | 2007-12-13 | Nokia Siemens Networks Oy | A method, a device, a module and a computer program product for determining the quality of an image |
| JP2008021271A (en) * | 2006-07-14 | 2008-01-31 | Eastman Kodak Co | Image processing apparatus, image restoration method, and program |
| TWI419079B (en) * | 2006-11-07 | 2013-12-11 | Digitaloptics Corp Internat | Image enhancement in the mosaic domain |
| JP5012135B2 (en) * | 2007-03-28 | 2012-08-29 | コニカミノルタアドバンストレイヤー株式会社 | Ultra-deep image generator |
| US7830428B2 (en) * | 2007-04-12 | 2010-11-09 | Aptina Imaging Corporation | Method, apparatus and system providing green-green imbalance compensation |
| US7876363B2 (en) * | 2007-04-19 | 2011-01-25 | Aptina Imaging Corporation | Methods, systems and apparatuses for high-quality green imbalance compensation in images |
| WO2008135995A2 (en) * | 2007-05-07 | 2008-11-13 | D-Blur Technologies Ltd. | Image restoration with enhanced filtering |
| US7983509B1 (en) * | 2007-05-31 | 2011-07-19 | Hewlett-Packard Development Company, L.P. | Estimating a point spread function of an image capture device |
| WO2008149363A2 (en) | 2007-06-05 | 2008-12-11 | Dblur Technologies Ltd. | Non-linear transformations for enhancement of images |
| US8023758B2 (en) * | 2007-08-07 | 2011-09-20 | Qualcomm Incorporated | Surface mesh matching for lens roll-off correction |
| KR101341101B1 (en) * | 2007-09-11 | 2013-12-13 | 삼성전기주식회사 | Apparatus and Method for restoring image |
| US8131072B2 (en) * | 2007-11-26 | 2012-03-06 | Aptina Imaging Corporation | Method and apparatus for reducing image artifacts based on aperture-driven color kill with color saturation assessment |
| JP5071188B2 (en) * | 2008-03-24 | 2012-11-14 | 富士通株式会社 | Image encryption / decryption device and program |
| US8135233B2 (en) * | 2008-05-22 | 2012-03-13 | Aptina Imaging Corporation | Method and apparatus for the restoration of degraded multi-channel images |
| US8131097B2 (en) * | 2008-05-28 | 2012-03-06 | Aptina Imaging Corporation | Method and apparatus for extended depth-of-field image restoration |
| US20100011044A1 (en) * | 2008-07-11 | 2010-01-14 | James Vannucci | Device and method for determining and applying signal weights |
| US20100011041A1 (en) | 2008-07-11 | 2010-01-14 | James Vannucci | Device and method for determining signals |
| US8325801B2 (en) | 2008-08-15 | 2012-12-04 | Mediatek Inc. | Adaptive restoration for video coding |
| CN101666637B (en) * | 2008-09-03 | 2012-06-13 | 鸿富锦精密工业（深圳）有限公司 | Roundness calculation and display system and method |
| JP2010087614A (en) * | 2008-09-29 | 2010-04-15 | Toshiba Corp | Image processing apparatus, image processing method and program |
| EP2175416A1 (en) * | 2008-10-13 | 2010-04-14 | Sony Corporation | Method and system for image deblurring |
| WO2010049006A1 (en) * | 2008-10-31 | 2010-05-06 | Nokia Siemens Networks Oy | Carrier selection for accessing a cellular system |
| JP5213688B2 (en) * | 2008-12-19 | 2013-06-19 | 三洋電機株式会社 | Imaging device |
| US8199248B2 (en) * | 2009-01-30 | 2012-06-12 | Sony Corporation | Two-dimensional polynomial model for depth estimation based on two-picture matching |
| JP5743384B2 (en) * | 2009-04-14 | 2015-07-01 | キヤノン株式会社 | Image processing apparatus, image processing method, and computer program |
| US8587703B2 (en) * | 2009-12-01 | 2013-11-19 | Aptina Imaging Corporation | Systems and methods for image restoration |
| JP2011134204A (en) * | 2009-12-25 | 2011-07-07 | Sony Corp | Image processing device, image processing method and program |
| WO2011132279A1 (en) * | 2010-04-21 | 2011-10-27 | キヤノン株式会社 | Image processing device, method, and recording medium |
| CN102156968B (en) * | 2011-04-11 | 2012-06-13 | 合肥工业大学 | Color cubic priori based single image visibility restoration method |
| TWI549096B (en) * | 2011-05-13 | 2016-09-11 | 華晶科技股份有限公司 | Image processing device and processing method thereof |
| CN102158731B (en) * | 2011-05-26 | 2014-03-12 | 威盛电子股份有限公司 | Image processing system and method |
| CN102158730B (en) * | 2011-05-26 | 2014-04-02 | 威盛电子股份有限公司 | Image processing system and method |
| US9245328B2 (en) | 2012-03-29 | 2016-01-26 | Nikon Corporation | Algorithm for minimizing latent sharp image cost function and point spread function with a spatial mask in a fidelity term |
| US9262815B2 (en) | 2012-03-29 | 2016-02-16 | Nikon Corporation | Algorithm for minimizing latent sharp image cost function and point spread function cost function with a spatial mask in a regularization term |
| TWI563471B (en) * | 2012-04-24 | 2016-12-21 | Altek Corp | Image processing device and processing method thereof |
| KR101382921B1 (en) | 2012-06-28 | 2014-04-08 | 엘지이노텍 주식회사 | Camera, image sensor thereof, and driving method thereof |
| TWI605418B (en) * | 2013-04-15 | 2017-11-11 | 晨星半導體股份有限公司 | Image editing method and apparatus |
| JP6221330B2 (en) * | 2013-05-01 | 2017-11-01 | 富士通株式会社 | Image processing program, image processing apparatus, and image processing method |
| JP5701942B2 (en) * | 2013-07-10 | 2015-04-15 | オリンパス株式会社 | Imaging apparatus, camera system, and image processing method |
| US10165263B2 (en) | 2013-09-30 | 2018-12-25 | Nikon Corporation | Point spread function estimation of optics blur |
| CN103606130A (en) * | 2013-10-22 | 2014-02-26 | 中国电子科技集团公司第二十八研究所 | Infrared degraded image adaptive restoration method |
| CN103679652B (en) * | 2013-11-29 | 2017-04-19 | 北京空间机电研究所 | Image restoration system capable of improving imaging quality greatly |
| JP6071974B2 (en) * | 2014-10-21 | 2017-02-01 | キヤノン株式会社 | Image processing method, image processing apparatus, imaging apparatus, and image processing program |
| JP6350205B2 (en) * | 2014-10-21 | 2018-07-04 | 富士通株式会社 | Processing apparatus, processing method, and processing program |
| JP2017028583A (en) * | 2015-07-24 | 2017-02-02 | キヤノン株式会社 | Image processor, imaging apparatus, image processing method, image processing program, and storage medium |
| US10593291B2 (en) * | 2015-09-17 | 2020-03-17 | Apple Inc. | Methods for color sensing ambient light sensor calibration |
| KR102172634B1 (en) * | 2015-10-08 | 2020-11-03 | 삼성전기주식회사 | Camera module, electronic device, and method for operating the same |
| US9858653B2 (en) * | 2016-02-02 | 2018-01-02 | Motorola Mobility Llc | Deblurring an image |
| US10846829B1 (en) * | 2018-01-30 | 2020-11-24 | Ambarella International Lp | Image sharpening with edge direction based undershoot and overshoot reduction |
| EP3855387A4 (en) * | 2018-09-18 | 2022-02-23 | Zhejiang Uniview Technologies Co., Ltd. | Image processing method and apparatus, electronic device, and readable storage medium |
| CN111123538B (en) * | 2019-09-17 | 2022-04-05 | 印象认知（北京）科技有限公司 | Image processing method and method for adjusting diffraction screen structure based on point spread function |
| CN114339447B (en) * | 2020-09-29 | 2023-03-21 | 北京字跳网络技术有限公司 | Method, device and equipment for converting picture into video and storage medium |
| CN116456098A (en) * | 2022-01-05 | 2023-07-18 | 南宁富联富桂精密工业有限公司 | Video compression method, terminal and computer readable storage medium |
Family Cites Families (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0516477B1 (en) * | 1991-05-30 | 1998-01-07 | Canon Kabushiki Kaisha | Compression enhancement in graphics systems |
| JPH0628469A (en) * | 1992-07-06 | 1994-02-04 | Olympus Optical Co Ltd | Deteriorated image restoring system |
| JP3166462B2 (en) * | 1992-12-28 | 2001-05-14 | ミノルタ株式会社 | Image recording / reproducing system and image reproducing apparatus having blur correction function |
| US5790709A (en) | 1995-02-14 | 1998-08-04 | Ben-Gurion, University Of The Negev | Method and apparatus for the restoration of images degraded by mechanical vibrations |
| US5852675A (en) * | 1995-04-14 | 1998-12-22 | Kiyoshi Matsuo | Color chart for image correction and method of color correction |
| SE9601229D0 (en) * | 1996-03-07 | 1996-03-29 | B Ulf Skoglund | Apparatus and method for providing reconstruction |
| JP3964042B2 (en) * | 1998-04-08 | 2007-08-22 | 株式会社リコー | Color image processing apparatus and color image processing method |
| US6822758B1 (en) | 1998-07-01 | 2004-11-23 | Canon Kabushiki Kaisha | Image processing method, system and computer program to improve an image sensed by an image sensing apparatus and processed according to a conversion process |
| US6414760B1 (en) * | 1998-10-29 | 2002-07-02 | Hewlett-Packard Company | Image scanner with optical waveguide and enhanced optical sampling rate |
| US6288798B1 (en) * | 1998-11-30 | 2001-09-11 | Xerox Corporation | Show-through compensation apparatus and method |
| JP2001197354A (en) * | 2000-01-13 | 2001-07-19 | Minolta Co Ltd | Digital image pickup device and image restoring method |
| US20010008418A1 (en) * | 2000-01-13 | 2001-07-19 | Minolta Co., Ltd. | Image processing apparatus and method |
| JP2001197355A (en) * | 2000-01-13 | 2001-07-19 | Minolta Co Ltd | Digital image pickup device and image restoring method |
| JP2001197356A (en) | 2000-01-13 | 2001-07-19 | Minolta Co Ltd | Device and method for restoring picture |
| AU3047801A (en) | 2000-02-03 | 2001-08-14 | Alst Technical Excellence Center | Image resolution improvement using a color mosaic sensor |
| JP2002290830A (en) * | 2001-03-27 | 2002-10-04 | Minolta Co Ltd | Imaging apparatus with image restoring function |
| JP2002300461A (en) | 2001-03-30 | 2002-10-11 | Minolta Co Ltd | Image restoring device, image restoring method and program thereof and recording medium |
| JP2002300459A (en) * | 2001-03-30 | 2002-10-11 | Minolta Co Ltd | Image restoring device through iteration method, image restoring method and its program, and recording medium |
| JP2002300384A (en) * | 2001-03-30 | 2002-10-11 | Minolta Co Ltd | Image recovery device, image recovery method, program and recording medium |
| JP2003060916A (en) | 2001-08-16 | 2003-02-28 | Minolta Co Ltd | Image processor, image processing method, program and recording medium |
| KR100444329B1 (en) | 2002-02-16 | 2004-08-16 | 주식회사 성진씨앤씨 | Digital video processing device eliminating the noise generated under insufficient illulmination |
-
2004
- 2004-07-09 US US10/888,534 patent/US7728844B2/en not_active Expired - Fee Related
-
2005
- 2005-01-04 KR KR1020077000576A patent/KR100911890B1/en not_active IP Right Cessation
- 2005-01-04 EP EP05701756A patent/EP1766569A1/en not_active Withdrawn
- 2005-01-04 CN CNA2005800231062A patent/CN1985274A/en active Pending
- 2005-01-04 US US11/632,093 patent/US20090046944A1/en not_active Abandoned
- 2005-01-04 WO PCT/FI2005/050001 patent/WO2006005798A1/en active Application Filing
- 2005-01-04 JP JP2007519821A patent/JP4571670B2/en not_active Expired - Fee Related
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101388952B (en) * | 2007-09-12 | 2012-03-28 | 三星电子株式会社 | Apparatus and method for restoring image |
| CN102326379A (en) * | 2008-12-31 | 2012-01-18 | 浦项工科大学校产学协力团 | Method for removing blur from image and recording medium on which the method is recorded |
| CN102326379B (en) * | 2008-12-31 | 2014-11-05 | 浦项工科大学校产学协力团 | Method for removing blur from image |
| CN103035185A (en) * | 2011-09-30 | 2013-04-10 | 伊姆普斯封闭式股份有限公司 | Method for brightness correction of defective pixels of digital monochrome image |
| TWI512682B (en) * | 2014-09-30 | 2015-12-11 | Quanta Comp Inc | Image processing system and saturation compensation method |
| CN105933595A (en) * | 2015-02-26 | 2016-09-07 | 诺基亚技术有限公司 | Method, Apparatus And Computer Program Product For Reducing Chromatic Aberrations In Deconvolved Images |
| CN106572285A (en) * | 2015-10-08 | 2017-04-19 | 三星电机株式会社 | Camera module, electronic device, and method of operating the same |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2008506174A (en) | 2008-02-28 |
| US20090046944A1 (en) | 2009-02-19 |
| JP4571670B2 (en) | 2010-10-27 |
| US7728844B2 (en) | 2010-06-01 |
| KR20070036773A (en) | 2007-04-03 |
| EP1766569A1 (en) | 2007-03-28 |
| KR100911890B1 (en) | 2009-08-11 |
| WO2006005798A1 (en) | 2006-01-19 |
| US20060013479A1 (en) | 2006-01-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1985274A (en) | Methods, system and program modules for restoration of color components in an image model | |
| CN108259774B (en) | Image synthesis method, system and equipment | |
| CN111402146B (en) | Image processing method and image processing apparatus | |
| US9883125B2 (en) | Imaging systems and methods for generating motion-compensated high-dynamic-range images | |
| US8553091B2 (en) | Imaging device and method, and image processing method for imaging device | |
| JP4317134B2 (en) | Method for interpolating and sharpening an image | |
| US9007488B2 (en) | Systems and methods for generating interpolated high-dynamic-range images | |
| US8520081B2 (en) | Imaging device and method, and image processing method for imaging device | |
| US8582911B2 (en) | Image restoration device, image restoration method and image restoration system | |
| EP2263373A2 (en) | Generalized assorted pixel camera systems and methods | |
| DE112013004507T5 (en) | Image processing apparatus, image capturing apparatus, image processing method, program and recording medium | |
| CN105453540B (en) | Image processing apparatus, photographic device, image processing method | |
| CN105453534B (en) | Image processing apparatus and method | |
| Tico et al. | Image enhancement method via blur and noisy image fusion | |
| Kane | Signal detection theory and automotive imaging | |
| CN109672810B (en) | Image processing apparatus, image processing method, and storage medium | |
| Kim et al. | Real-time bayer-domain image restoration for an extended depth of field (EDoF) camera | |
| Baboshina et al. | Combined use of a bilateral and median filter to suppress gaussian noise in images | |
| Lan et al. | Joint Demosaicking/Rectification Of Fisheye Camera Images Using Multi-Color Graph Laplacian Regularization | |
| CN112767291B (en) | Visible light image and infrared image fusion method, device and readable storage medium | |
| CN112767291A (en) | Visible light image and infrared image fusion method and device and readable storage medium | |
| CN116703734A (en) | Image enhancement method and device based on vehicle-mounted looking-around system | |
| Kwon et al. | Restoration for out-of-focus color image based on gradient profile sharpness | |
| Trongtirakul et al. | Color restoration of multispectral images: near-infrared (NIR) filter-to-color (RGB) image | |
| Tektonidis et al. | Evaluation of a HDR image sensor with logarithmic response for mobile video-based applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20070620 |