CN101305598B - Image processing apparatus and method - Google Patents
Image processing apparatus and method Download PDFInfo
- Publication number
- CN101305598B CN101305598B CN2006800415164A CN200680041516A CN101305598B CN 101305598 B CN101305598 B CN 101305598B CN 2006800415164 A CN2006800415164 A CN 2006800415164A CN 200680041516 A CN200680041516 A CN 200680041516A CN 101305598 B CN101305598 B CN 101305598B
- Authority
- CN
- China
- Prior art keywords
- data
- image
- dwindling
- restored
- original image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000012545 processing Methods 0.000 title claims abstract description 127
- 238000000034 method Methods 0.000 title claims description 91
- 238000003672 processing method Methods 0.000 claims abstract description 68
- 230000009467 reduction Effects 0.000 claims abstract description 7
- 238000012546 transfer Methods 0.000 claims description 88
- 238000011084 recovery Methods 0.000 claims description 66
- 230000008859 change Effects 0.000 claims description 48
- 230000006866 deterioration Effects 0.000 claims description 46
- 230000008569 process Effects 0.000 claims description 26
- 238000012958 reprocessing Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 description 74
- 239000011435 rock Substances 0.000 description 17
- 238000011282 treatment Methods 0.000 description 17
- 230000014509 gene expression Effects 0.000 description 13
- 230000005484 gravity Effects 0.000 description 11
- 230000015572 biosynthetic process Effects 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000004075 alteration Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 230000007704 transition Effects 0.000 description 4
- 230000000295 complement effect Effects 0.000 description 3
- 238000012937 correction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000012163 sequencing technique Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000004321 preservation Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- TWDJIKFUVRYBJF-UHFFFAOYSA-N Cyanthoate Chemical compound CCOP(=O)(OCC)SCC(=O)NC(C)(C)C#N TWDJIKFUVRYBJF-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000009931 harmful effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Studio Circuits (AREA)
Abstract
The invention relates to an image processor and an image processing method capable of displaying images approximating completely restored images, in a short time after photographing, and then confirming contents of the image to be processed, thereafter in a short period of time. The image processor comprises a processing section for applying a restoration processing to an image before being changed from an original image being a processing object, or an image having to be substantially photographed, or an approximate image (original image) to them, wherein the processor includes 1) a module for obtaining original image reduction data resulting from reducing number of pixels from the original image, 2) a module for restoring reduced restoration data approximate to a reduced original image, before being changed on the basis of the original image reproduction data, 3) a module for displaying the reproduction restoration data to a display section, and 4) a module for restoring the original image by utilizing the data obtained at restoration. Further, the display section is, e.g. a monitor or the like of a camera.
Description
Technical field
The present invention relates to image processing apparatus and image processing method.
Background technology
Deterioration takes place in regular meeting in the time of on the image that the well-known image processing apparatus that utilizes camera etc. under prior art is recorded when taking pictures.As the main cause of image deterioration, there be rocking etc. when taking pictures.Rectification is the processing of circuit method because of first method of rocking the image deterioration that causes.This processing of circuit method is, obtains the transfer function of the deterioration state of expression photographic images, and photographic images is carried out the inverse transformation of obtained transfer function, thus the method (with reference to patent documentation 1) that photographic images is corrected.
In addition, rectification is the method for moving lens because of second method of rocking the image deterioration that causes.For example, as the method for moving lens, known having by detecting rocking of camera makes the camera lens of regulation move corresponding to this detected rocking, thus the method for correcting (with reference to patent documentation 2).
Patent documentation 1: day disclosure communique, spy open flat 11-24122 number (with reference to specification digest)
Patent documentation 2: day disclosure communique, spy open flat 6-317824 number (with reference to specification digest)
Summary of the invention
If adopt first method, have and correct short advantage of needed time, but also have following disadvantage.That is, obtained transfer function for noise or rock information error etc. very a little less than, these have change slightly, very big change just takes place in transfer function values.Therefore, differ greatly, in fact can't utilize by inverse transformation correcting image that obtains and the image of under the state that does not shake, taking.In addition, under the situation of the inverse transformation of having considered noise etc., also can adopt to decompose to wait and calculate the method for separating, still by the singular value of separating (particular value) of simultaneous equations, the calculated value that is used for this reckoning has astronomical figure so big, and that in fact can't answer is dangerous high.In addition, adopt the camera of second method, owing to the configuration space of the hardware that needs driving camera lens such as motor causes maximizing.In addition, owing to need this hardware self or drive the drive circuit of this hardware, so cost increases.
On the other hand, the operator of camera etc. can want to confirm by the display part (watch-dog) that digital camera etc. has the record case of photographic images sometimes immediately after shooting.Its reason is, after just having taken, under this image condition of poor that can confirm by watch-dog, has considered that the possibility that can take again after the correction of its undesirable level is big.
At this, it is a kind of that problem to be solved by this invention is to provide, and can show with the image of the image approximate corrected fully after the shooting at short notice and can confirm the image processing apparatus and the image processing method of processed thereafter picture material at short notice.
In order to solve above-mentioned problem, image processing apparatus of the present invention is to be provided with handling part, and handling part restores from the original image that becomes process object and changes image before, or the image that should be taken originally, or the approximate image of above-mentioned image (below, be called base image) image processing apparatus, wherein, handling part has: the module that 1) obtains dwindling from the original image that original image cuts pixel count data, 2) dwindle data recovery and the module of dwindling restored data that base image is similar to of dwindling that changes before according to this original image, 3) with the resulting module that restored data is shown in display part, 4 of dwindling) utilize to restore the above-mentioned module of the above-mentioned base image of resulting data recovery when dwindling restored data.
Adopt this invention, by cutting pixel count from original image, the image data amount that should correct tails off.Therefore, can shorten to rock to correct to wait and restore the required time, can after shooting, finish in the short time to restore.And, be shown on the display part by the restored data of dwindling after will restoring, can after shooting, confirm processed thereafter picture material in the short time.
In addition, display part is a watch-dog, is very suitable.This is because be shown in the image that restored data becomes the part that comparing with base image only diminishes cuts pixel count that dwindles after the recovery on the display part.But, for example, the watch-dog that digital camera or Digital Video etc. is had, few with respect to the actual photographic images pixel count that is recorded usually, the display part area is also little.Therefore, when the watch-dog that utilizes various cameras confirms that captured image is which kind of image and situation about how to be recorded, be desirable.
In addition, if from original image to the recuperation of base image, utilize and to dwindle data resulting data to the recuperation of dwindling restored data from original image, can carry out effectively from the recovery of original image to base image.
And then, other inventions are on the basis of foregoing invention, dwindling data recovery according to original image has with the approximate module of dwindling restored data of base image of dwindling that changes before: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate the module of relatively using data, 2) original image is dwindled data and this and relatively make the module that difference obtains differential data with data, 3) utilize resulting differential data to generate and dwindle restored data, and use this to dwindle restored data to replace the specified image data, replace the dwindling restored data of last time and repeat same processing, the module that when reaching the reprocessing number of times of prior setting, stops to handle with the resulting restored data of dwindling afterwards.
Adopt this invention, only by the essential factor information of utilizing image change generate predetermined data generate with base image approximate dwindle restored data, therefore almost do not have the increase on the hardware, device can not maximize.In addition, repeat to generate relatively with data and this is relatively dwindled data with the original image of data and process object ask poor processing from dwindling restored data, thereby obtain little by little with dwindle base image approaching dwindle restored data, the therefore comeback job of becoming a reality property.Therefore, when carrying out the recovery of image, can form the image processing apparatus of processing of circuit method with actuality.
And then if just adopt handling part number of repetition when carrying out reprocessing to reach under the situation of the formation that stipulated number stops to handle, whether difference becomes " 0 " all stops to handle, and therefore can prevent long-timeization of handling.In addition, continue to stipulated number, to dwindle base image before the therefore approximate generation deterioration of dwindling the basis that restored data more approaches to become original image etc. owing to make to handle.And then, having under the situation of noise etc., the situation that difference does not become " 0 " taking place in fact easily, still,, therefore can unrestrictedly not repeat to handle even finish under these circumstances with stipulated number yet.
In addition, other inventions are on the basis of the image processing apparatus of foregoing invention, dwindling data recovery according to original image has with the approximate module of dwindling restored data of base image of dwindling that changes before: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate the module of relatively using data, 2) original image is dwindled data and this and relatively make the module that difference obtains differential data with data, 3) at resulting differential data during greater than setting, utilize resulting differential data to generate and dwindle restored data, and dwindle restored data with this and replace the specified image data, replace and last to dwindle restored data and repeat same processing with the resulting restored data of dwindling afterwards, when resulting differential data is the module that setting stops to handle when following.
In addition, other inventions are on the basis of the image processing apparatus of foregoing invention, dwindling data recovery according to original image has with the approximate module of dwindling restored data of base image of dwindling that changes before: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate the module of relatively using data, 2) original image is dwindled data and this and relatively make the module that difference obtains differential data with data, 3) be that setting is when above at resulting differential data, utilize resulting differential data to generate and dwindle restored data, and dwindle restored data with this and replace the specified image data, replace and last to dwindle restored data and repeat same processing with the resulting restored data of dwindling afterwards, the module that stops to handle during less than setting when resulting differential data.
Adopt this invention and since only by the essential factor information of utilizing image change generate predetermined data generate with base image approximate dwindle restored data, therefore almost do not have the increase on the hardware, device can not maximize.In addition, repeat to generate relatively with data and this is relatively dwindled data with the original image of data and process object ask poor processing from dwindling restored data, thereby obtain little by little with dwindle base image approaching dwindle restored data, the therefore comeback job of becoming a reality property.Therefore, when carrying out the recovery of image, can form the image processing apparatus of processing of circuit method with actuality.In addition, because the words that differential data diminishes just stop to handle, therefore can stop to handle with number of processes to a certain degree.
And then, if just adopt handling part differential data when carrying out reprocessing to become to stop below the setting under the situation of the formation of handling, even, therefore can prevent long-timeization of handling because difference does not become " 0 " does not also stop to handle.In addition, owing to be set at below the setting, (before the deterioration etc.) dwindles base image before the therefore approximate variation of dwindling the basis that restored data more approaches to become original image.And then, having under the situation of noise etc., often in fact to become the situation of " 0 " be impossible take place to difference, even but also can unrestrictedly not repeat under these circumstances to handle.
In addition, handling part also can be when carrying out reprocessing, and the differential data when number of repetition reaches stipulated number is that setting stops when following handling, and when surpassing setting, carries out the reprocessing of stipulated number once more; In addition, the differential data when number of repetition reaches stipulated number stops during less than setting handling, and being setting when above, carries out the reprocessing of stipulated number once more.
Adopt the words of this formation, owing to the value combination of the number of times that will handle and difference is handled, therefore, and only number of processes is increased restriction or the situation that the value of difference limits is compared, can become the processing that can keep the balance of good and processing time of image between short.
And then, other invention is on the basis of foregoing invention, dwindle data recovery and the module of dwindling restored data that base image is similar to of dwindling that changes before according to original image, have by dwindling restored data and original image and dwindle the module that data computation goes out transfer function, utilize the module of restoring resulting data recovery base image when dwindling restored data, have with this transfer function form original image dwindle data the original image reduction magnification inverse doubly after, the interval that is exaggerated is carried out interpolation and obtained the new transfer function of transfer function as the deterioration state of expression original image, and use this new transfer function and the module of formation base image.Adopt this invention, can promptly restore base image according to appropriate transfer function.In addition, make up because the deconvolution that will be used to obtain to dwindle the reprocessing of restored data and utilize transfer function is handled, therefore device can not maximize, and in addition, on the basis of the comeback job of becoming a reality property, processing is by high speed.
Other invention is on the basis of foregoing invention, dwindle data recovery and the module of dwindling restored data that base image is similar to of dwindling that changes before according to original image, have by original image and dwindle data and dwindle the module that restored data is obtained transfer function, utilize the module of restoring resulting data recovery base image when dwindling restored data, have this transfer function transfer function of image as a whole intactly, and by the contrary module that generates the base image before original image changes of calculating.By adopting this formation, device can not maximize, and in addition, on the basis of the comeback job of becoming a reality property, handles by high speed.
And then other invention is on the basis of foregoing invention, and it is to form by the data of original image are taken a sample that original image dwindles data.Adopt this formation, can access transfer function corresponding to general image.
In addition, other invention is on the basis of foregoing invention, and it is to form by intactly take out part zone from the data of original image that original image dwindles data.Adopt this formation, can access corresponding to the subregion and also go for the transfer function of general image.
And then, other invention is on the basis of foregoing invention, the recovery of base image is handled, (1) with will dwindle that restored data is shown in that processing on the display part is carried out simultaneously, carry out (2) when power supply for shooting is disconnected or (3) are carrying out carrying out after restored data is shown in processing on the display part dwindling.
Adopting the invention of above-mentioned (1), is desirable when the recovery of carrying out base image is as early as possible handled.In addition, adopt the words of the invention of above-mentioned (2), (3), even in the recovery of base image is handled, need under the long situation, beyond during handling by the recovery of dwindling data at original image during in carry out base image recovery handle, and can alleviate the burden of image processing apparatus.In addition, even the processing in the carrying out period that postpones like this that the recovery of base image handles, be shown in image on the display part also can be after shooting in the short time as dwindling restored data and being shown, therefore for the operator of the image processing apparatus with shoot function of image processing apparatus, particularly camera etc., do not have inconvenience or harmful effect.
And then, image processing method of the present invention, utilize above-mentioned image processing apparatus, it may further comprise the steps: the original image that 1) obtains cutting from original image pixel count dwindles data, 2) before dwindling data recovery and change according to this original image dwindle base image approximate dwindle restored data, 3) the resulting restored data of dwindling is shown in display part, 4) utilize and restore resulting data recovery base image when dwindling restored data.
Adopt words of the present invention, can access and to show after a kind of take with the image of the image approximate of being corrected fully at short notice and can confirm the image processing apparatus and the image processing method of processed thereafter picture material at short notice.
In addition, other image processing methods of the present invention are on the basis of foregoing invention, the approximate step of dwindling restored data of base image of dwindling before dwindling data recovery and change according to original image may further comprise the steps: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate and relatively use data, 2) original image being dwindled data and this relatively makes difference with data and obtains differential data, 3) utilize resulting differential data to generate and dwindle restored data, and use this to dwindle restored data to replace the specified image data, replace the dwindling restored data of last time and repeat same processing with the resulting restored data of dwindling afterwards, when reaching the reprocessing number of times of prior setting, stop to handle.
And then, other image processing methods of the present invention are on the basis of foregoing invention, the approximate step of dwindling restored data of base image of dwindling before dwindling data recovery and change according to original image may further comprise the steps: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate and relatively use data, 2) original image being dwindled data and this relatively makes difference with data and obtains differential data, 3) at resulting differential data during greater than setting, utilize resulting differential data to generate and dwindle restored data, and dwindle restored data with this and replace the afore mentioned rules view data, replace and last to dwindle restored data and repeat same processing with the resulting restored data of dwindling afterwards, when resulting differential data is that setting stops to handle when following.
And then, other image processing methods of the present invention are on the basis of foregoing invention, the approximate step of dwindling restored data of base image of dwindling before dwindling data recovery and change according to original image may further comprise the steps: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate and relatively use data, 2) original image being dwindled data and this relatively makes difference with data and obtains differential data, 3) be that setting is when above at resulting differential data, utilize resulting differential data to generate and dwindle restored data, and dwindle restored data with this and replace the specified image data, replace and last to dwindle restored data and repeat same processing with the resulting restored data of dwindling afterwards, when resulting differential data stops to handle during less than setting.
And then, other image processing methods of the present invention are on the basis of foregoing invention, dwindle data recovery and the step of dwindling restored data that base image is similar to of dwindling that changes before according to original image, comprise by dwindling restored data and original image and dwindle the step that data computation goes out transfer function; Utilize the step of restoring resulting data recovery base image when dwindling restored data, comprise with this transfer function form original image dwindle data the original image reduction magnification inverse doubly after, the interval that is exaggerated is carried out interpolation and is obtained new transfer function as the transfer function of the deterioration state of representing original image, and use this new transfer function and the step of formation base image.
And then, other image processing methods of the present invention are on the basis of foregoing invention, dwindle data recovery and the step of dwindling restored data that base image is similar to of dwindling that changes before according to original image, comprise by original image and dwindle data and dwindle the step that restored data is obtained transfer function; Utilize to restore the step of resulting data recovery base image when dwindling restored data, comprise this transfer function transfer function of image as a whole intactly, and by the contrary step that generates the base image before original image changes of calculating.
Description of drawings
Fig. 1 is the block diagram of the main composition of the image processing apparatus that relates to of expression embodiment of the present invention.
Fig. 2 is the stereoscopic figure of the summary of expression image processing apparatus shown in Figure 1, is the figure that is used to illustrate the allocation position of angular-rate sensor.
Fig. 3 is the process chart that is used to illustrate the processing method (handling procedure) that the handling part that utilizes image processing apparatus shown in Figure 1 carries out.
Fig. 4 is the figure that is used to illustrate the notion of processing method shown in Figure 3.
Fig. 5 is to be that example specifically describes the figure that processing method shown in Figure 3 is used to rock, and is the table of the energy concentration when representing not shake.
Fig. 6 is to be that example specifically describes the figure that processing method shown in Figure 3 is used to rock, and is the schematic diagram of the view data when not shaking.
Fig. 7 is to be that example specifically describes the figure that processing method shown in Figure 3 is used to rock, and is the schematic diagram of the dispersion of the energy when shaking.
Fig. 8 is to be that example specifically describes the figure that processing method shown in Figure 3 is used to rock, and is to be used to illustrate the figure that is generated the situation of relatively using data by image arbitrarily.
Fig. 9 is to be that example specifically describes the figure that processing method shown in Figure 3 is used to rock, and is to be used to illustrate the figure that generates the situation of differential data to relatively asking difference with data and the fuzzy original image that becomes process object.
Figure 10 is to be that example specifically describes the figure that processing method shown in Figure 3 is used to rock, and is to be used for illustrating by differential data being distributed and being appended to the figure that image arbitrarily generates the situation of restored data.
Figure 11 is to be that example specifically describes the figure that processing method shown in Figure 3 is used to rock, and is to be used to illustrate generated by the restored data that is generated new relatively ask difference and the figure of the situation of generation differential data with data and to these data and the fuzzy original image that becomes process object.
Figure 12 is to be that example specifically describes the figure that processing method shown in Figure 3 is used to rock, and is to be used to illustrate the figure that newly-generated differential data is distributed and generates the situation of new restored data.
Figure 13 is used to illustrate as figure first processing method of having utilized processing method shown in Figure 3, that utilize the processing that changes the essential factor center of gravity, (A) being the schematic diagram of state of a pixel in the data of image of being accurate in one's observation, (B) is the schematic diagram of state of the data expansion of the pixel of being observed in the figure of the data of expression original image.
Figure 14 is used to specifically describe as figure first processing method shown in Figure 13, that utilize the processing that changes the essential factor center of gravity.
Figure 15 is the figure that is used to illustrate second processing method of having utilized processing method shown in Figure 3 and seeks first method of high speed, (A) expression becomes the data of the original image of process object, (B) is the schematic diagram of the data of (rejecting at interval) behind the data sampling with (A).
Figure 16 is the flow chart of second processing method shown in Figure 15.
Figure 17 is the figure that is used to illustrate the 3rd processing method of having utilized processing method shown in Figure 3 and seeks second method of high speed, (A) expression becomes the data of original image of process object, (B) is the schematic diagram of the data after the part of the data of (A) is taken out.
Figure 18 is the flow chart of the 3rd processing method shown in Figure 17.
Figure 19 is the figure that is used to illustrate the variation of Figure 17, the 3rd processing method shown in Figure 180, and being expression is divided into the data of original image four parts and takes out the figure in the part zone that is used to carry out repeated treatments from each cut zone.
Symbol description
1 image processing apparatus
2 shoot parts
3 control system portions
4 handling parts
5 recording portion
6 test sections
7 essential factor information storing section
8 display parts
The data of Io initial pictures (data of arbitrary image)
Io ' relatively uses data
G changes the data (data of deterioration essential factor information) of essential factor information
The data of the variation essential factor information that GS is reduced
The data of Img ' original image (image that is taken)
ISmg ' original image dwindles data
The δ differential data
The k distribution ratio
Io+n restored data (data of restored image)
ISo+n dwindles restored data
Img is the data of the former correct images of deterioration (base image) not
ISmg dwindles base image
G (x), g ' (x), g2 (x) transfer function (restoring the transfer function that big image is used)
G1 (x), g1 ' be transfer function (transfer function that is obtained by reduced data) (x)
Embodiment
Below, with reference to accompanying drawing the image processing apparatus 1 that first embodiment of the invention relates to is described.And, this image processing apparatus 1 is as civilian camera, but, also can be used as monitoring camera, TV camera, perhaps also can be used in the instrument beyond image diagnosing system that microscope, binoculars and then NMR take etc. etc., the camera with other purposes such as camera, endoscope cameras.
Shoot part 2 is to be provided with the part that the light that will pass through to have the photographic optical system of camera lens or camera lens is converted to the picture pick-up device of the CCD (Charge Coupled Devices) of the signal of telecommunication or C-MOS (Complementary Me tal Oxide Semiconductor) etc.Each one in control system portion 3 control shoot parts 2, handling part 4, recording portion 5, test section 6, essential factor information storing section 7 and display part 8 etc., the image processing apparatus 1.
Handling part 4 is made of image processor, and is made of the such hardware of ASIC (Application Specific Integ rated Circuit).Also preserving in this handling part 4 becomes the image that generates following basis when relatively using data.Handling part 4 also can be used as the formation of utilizing software to handle, rather than constitutes as the such hardware of ASIC.Recording portion 5 is made of semiconductor memory, but also can adopt the magnetic recording means of hard drive etc. or use the optical recording means etc. of DVD (Digi tal Versatile Disk) etc.
As shown in Figure 2, test section 6 is provided with two angular-rate sensors of rotary speed that detection is X-axis, the Y-axis of vertical direction with respect to the Z axle as the optical axis of image processing apparatus 1.But, rocking when utilizing camera to take pictures also can take place to all directions of directions X, Y direction, Z direction move or around the rotation of Z axle, with respect to each change suffered what have the greatest impact is around the rotation of Y-axis with around the rotation of X-axis.These two kinds of changes only change a little, and the image of shooting just can be very fuzzy.Therefore, in this embodiment, only dispose Fig. 2 around X-axis with around two angular-rate sensors of Y-axis.But, more perfect in order to expect, also can further add angular-rate sensor around the Z axle, perhaps additional detected is to the transducer that moves of directions X or Y direction.In addition,, also can not adopt angular-rate sensor, and adopt angular acceleration transducer as employed transducer.
Essential factor information storing section 7 is recording portion of the variation essential factor information of preserving known deterioration essential factor information etc., the aberration of for example optical system etc.And, in this embodiment, in essential factor information storing section 7, preserve the information of the deflection of the aberration of optical system or camera lens, but do not utilize these information when rocking take place fuzzy in recovery described later.Display part 8 is obtained by recording portion 5 and is dwindled restored data, and this is dwindled restored data shows as dwindling base image, and wherein dwindling restored data is the view data of correcting back (restoring the back) as the image that cuts pixel count from the image that is taken.As shown in Figure 2, be to be disposed on the face of opposition side of shoot part 2 under these display part 8 a lot of situations, still, also can be disposed on the face with shoot part 2 the same sides or on the face of side side.
Then, according to Fig. 3 the summary of the processing method of the handling part 4 of the image processing apparatus 1 that constitutes is as described above described.
Among Fig. 3, " Io " is initial pictures arbitrarily, is the data of image that are stored in the recording portion of handling part 4 in advance." Io ' " represents the data of deterioration image of the data I o of this initial pictures, be used to compare relatively use data." G " is the data by test section 6 detected variation essential factor information (=deterioration essential factor information (some transform)), and is stored in the recording portion of handling part 4." Img ' " refer to shooting image, be the data of deterioration image, be the data that in this processing, become the original image of process object.
" δ " is the data I mg ' of original image and the differential data of relatively using data I o '." k " is based on the distribution ratio of the data that change essential factor information." Io+n " is with the data (restored data) of differential data δ according to the data allocations newly-generated restored image behind the data I o of initial pictures that changes essential factor information." Img " be become as the basis of the data I mg ' of the original image of the deterioration image that is taken, the data of the former correct images of deterioration not.At this, the relation of Img and Img ' is represented with following formula (1).
Img’＝Img×G …(1)
In addition, differential data δ also can be the simple difference of the pixel of correspondence, still, and usually according to the difference of the data G that changes essential factor information and difference is represented with following formula (2).
δ＝f(Img’，Img，G) …(2)
The handling procedure of handling part 4 is at first from preparing the data I o (step S101) of arbitrary image.As the data I o of this initial pictures, can use the data I mg ' of the deterioration image that is taken, in addition, also can use the data of such image such as complete black, complete white, full ash, chequered with black and white tartan.In step S102, replace the Img of (1) formula and substitution becomes the data I o of the arbitrary image of initial pictures, and obtain and relatively use data I o ' as the deterioration image.Then, will and relatively use data I o ' to ask poor, calculate differential data δ (step S103) as the data I mg ' of the original image of the deterioration image that is taken.
Then, judge in step S104 whether this differential data δ is more than the setting, more than setting, in step S105, generate the processing of the data (=restored data) of new restored image.That is, with the data I o that differential data δ is allocated in arbitrary image, generate new restored data Io+n according to the data G that changes essential factor information.Then, repeating step S102, S103, S104.
In step S104, in differential data δ end process (step S106) during less than setting.Then, the restored data Io+n of the time point of handling being through with is inferred as correct image, i.e. the data I mg of the image of deterioration not, and with this data record in recording portion 5.In addition, also can in recording portion 5, write down the data I o of initial pictures or the data G of variation essential factor information, thereby carry out the transition to handling part 4 as required.At this, the approximate image of original image or correct images that should be taken originally or above-mentioned image is called " base image ".Therefore, data I mg or restored data Io+n become the data of " base image ".
The words that the viewpoint of above processing method is summarized are as described below.That is, in this processing method, separating as inverse problem of will not handling solves, but solves as obtaining the optimization problem of reasonably separating.When solving, also be possible in theory as the record of patent documentation 2, but be difficult as realistic problem as inverse problem.
Solving as optimization problem, is prerequisite with following condition.
That is, (1) is defined as unique value with respect to the output of input.
(2) if output is identical, then import identical.
(3), make and separate convergence by when upgrading input, carrying out repeated treatments in order to make output identical.
In other words, shown in Fig. 4 (A), (B), if can generate with as the data I mg ' of original image of the image that is taken approximate relatively use data I o ' (Io+n '), then become the data I o or the restored data Io+n of initial pictures of the basic data of this generation, be similar to the data I mg of correct images on the basis of the data I mg ' that becomes original image.
And in this embodiment, the angular velocity detection transducer just detects angular speed every 5 μ sec.In addition, become the value of the criterion of differential data δ, under with the situation of each data, form " 6 " in this embodiment with eight (bit) (0～255) expressions.That is, less than 6, just be 5 when following, end process.In addition, by the initial data of rocking that the angular velocity detection sensor goes out, when the calibration of transducer self is insufficient, be not corresponding with rocking of reality.Therefore, for corresponding, when transducer is not calibrated, must carry out the initial data that goes out by sensor be multiply by the rectification of regulation multiplying power with rocking of reality.
Then, according to Fig. 5, Fig. 6, Fig. 7, Fig. 8, Fig. 9, Figure 10, Figure 11 and Figure 12 the details of Fig. 3 and processing method shown in Figure 4 is described.
(the recovery algorithm that rocks)
When not shaking, the luminous energy corresponding to the pixel of stipulating concentrates on this pixel in the time for exposure.In addition, when shaking, luminous energy is scattered in the time for exposure in the pixel of rocking.And then, if know rocking in the time for exposure,, therefore can make unambiguous image by fuzzy image just know the dispersing mode of energy in the time for exposure.
Below, for oversimplifying, describe with horizontal one dimension.With pixel begin from a left side in turn to be made as n-1, n, n+1, n+2, n+3 ..., and observe a certain pixel n.Because the concentration of energy in the time for exposure is in this pixel when not shaking, so the energy concentration degree is " 1.0 ".This information slip is shown in Fig. 5.The shooting results of this moment is shown in the table of Fig. 6.Data shown in Figure 6 become the correct images data I mg when deterioration does not take place.And each data is with the data representation of eight (0～255).
Rock in time for exposure, and 50% the time in the time for exposure of being respectively shakes in n number pixel, time of 30% shakes in n+1 number pixel, time of 20% shakes in n+2 number pixel.The dispersing mode of energy table is as shown in Figure 7 put down in writing.This becomes the data G that changes essential factor information.
Because rocking all is identical in all pixels, if therefore there is not the top to rock (vertically rocking), the situation of rocking table is as shown in Figure 8 put down in writing.Conduct among Fig. 8 " shooting results " and the data of expression are the data I mg of former correct images is as " blurred picture " and the data of expression are the data I mg ' of the deterioration image taken.Specifically, " 120 " of the pixel of " n-3 " for example, according to the distribution ratio of " 0.5 " " 0.3 " " 0.2 ", in the pixel of " n-3 ", disperse to disperse in the pixel of " 60 ", " n-2 " to disperse " 24 " in the pixel of " 36 ", " n-1 " as the data G that rocks change in information essential factor information.Equally, as " 60 " of the data of the pixel of " n-2 ", in the pixel of " n-2 ", disperse to disperse in the pixel of " 30 ", " n-1 " to disperse " 12 " in the pixel of " 18 ", " n ".Data G by the data I mg ' of this deterioration image and variation essential factor information shown in Figure 7 calculates unambiguous shooting results.
Data I o as the arbitrary image shown in the step S101 can adopt data arbitrarily, uses the data I mg ' of the original image of taking when carrying out this explanation.That is, begin to handle as Io=Img '." input " in the table of Fig. 9 is equivalent to the data I o of initial pictures.By step S 102 at this data I o, be to multiply by the data G that changes essential factor information on the Img '.That is, for example " 60 " of the pixel of the data I o of initial pictures " n-3 " are distributed in the pixel of distribution " 18 ", " n-1 " in the pixel of distribution " 30 ", " n-2 " in the pixel of " n-3 " respectively " 12 ".Pixel for other is distributed similarly, generate as " output Io ' " and expression relatively use data I o '.Therefore, the differential data δ of step S103 is shown in one hurdle, bottom of Fig. 9.
Then, judge the size of differential data δ by step S104.Specifically, become 5 end process when following entirely at the absolute value of all differential data δ, still, differential data δ shown in Figure 9 does not meet this condition, therefore enters step S105.That is, use the data G that changes essential factor information that differential data δ is assigned to the data I o of arbitrary image, and generate the restored data Io+n that represents as " input next time " among Figure 10.At this moment, owing to be for the first time, therefore in Figure 10, be expressed as Io+1.
The distribution of differential data δ, for example will on the data " 30 " of the pixel of " n-3 ", multiply by the pixel that is allocated in " n-3 " as " 15 " of 0.5 gained of (pixel of=" n-3 ") distribution ratio of himself, be distributed in " 4.5 " of multiply by on the data " 15 " of the pixel of " n-2 " in addition as distribution ratio 0.3 gained in the pixel that should result from this " n-2 ", and then, be distributed in " 1.84 " of multiply by on the data " 9.2 " of the pixel of " n-1 " as distribution ratio 0.2 gained in the pixel that should result from this " n-1 ".The total amount that is allocated in the pixel of " n-3 " is " 21.34 ", and data I o (using the data I mg ' of the original image that the is taken here) addition with this value and initial pictures generates restored data Io+1.
As shown in figure 11, this restored data Io+1 becomes the data (the data I o of=initial pictures) of the input picture of step S102, implementation step S102 and to step S103 transition, thus obtain new differential data δ.Judge the size of the differential data δ that this is new by step S104, greater than setting the time, new differential data δ is allocated among the last restored data Io+1, generate new restored data Io+2 (with reference to Figure 12) by step S105.Then, by the carrying out of step S102, generate the new data I o+2 ' that relatively uses by restored data Io+2.Like this, after implementation step S102, S103, enter step S104, and the judgement by step S104 is to step S105 transition or to step S106 transition.Repeat such processing.
In this image processing apparatus 1, when handling, in step S104, can set any one party or both sides in the criterion value of number of processes and differential data δ in advance.For example, as number of processes, can set 20 times, 50 inferior number of times arbitrarily.In addition, can will make the value of handling the differential data δ stop be set at " 5 " in eight (0～255), the words that become 5 below finish processing, perhaps are set at " 0.5 ", and the words that become below 0.5 make the processing end.Can at random set this set point.Under the situation of having imported number of processes and criterion value both sides, when satisfying any one party, just stop to handle.And, in the time can carrying out both sides' setting, serve as preferential with the criterion value, when the processing by stipulated number does not enter in the criterion value, also can again repeat the processing of stipulated number.
In the explanation of this execution mode, utilize the information be stored in essential factor information storing section 7, but also can use the data of the deflection that is stored in known deterioration essential factor, for example optical aberration or camera lens in the essential factor information storing section 7 etc.Under this situation, for example in the processing method of before example (Fig. 3), make up and be treated to good with the information of the information of will rock and optical aberration as a deterioration essential factor, but, also can rock the rectification that utilizes the information of optical aberration after information processing finishes in utilization.In addition, also this essential factor information storing section 7 can be set, and only the dynamic essential factor when taking, for example only image revised or restored by rocking.
More than, the basic comprising of the image processing apparatus 1 that embodiment of the present invention is related to and restore the base program of handling and be illustrated, below, example or the variation of using this image processing apparatus 1 described.
For example, when generating restored data Io+n, also can not use distribution ratio k, and the differential data δ of respective pixel intactly is appended in the respective pixel of last restored data Io+n-1, or after becoming the differential data δ of respective pixel doubly, append, after becoming doubly, the data k δ after perhaps differential data δ being assigned with (as " renewal amount " among Figure 10, Figure 12 and the value of expression) is appended among the last restored data Io+n-1.Apply flexibly these processing methods well, then processing speed accelerates.
In addition, when generating restored data Io+n, also can calculate the center of gravity of the variation essential factor of deterioration etc., and only with the difference of this center of gravity, be appended among the last restored data Io+n-1 after maybe will this difference becoming doubly.Below, according to Figure 13 and Figure 14, the processing method that this viewpoint, i.e. utilization is changed the center of gravity of essential factor describes as first processing method of having utilized processing method shown in Figure 3.
As shown in figure 13, when the data I mg of correct images is made of pixel 11～15,21～25,31～35,41～45,51～55, shown in Figure 13 (A), observe pixel 33.Move to the position of pixel 33,43,53,52 owing to pixel 33 such as rock, in data I mg ', shown in Figure 13 (B), occur the influence of initial pixel 33 in the pixel 33,43,52,53 as the original image of deterioration image.
Under the situation of this deterioration,, deterioration, promptly change the center of gravity of essential factor, in the data I mg ' of original image, be in the position of pixel 43 about the pixel among the data I mg of correct images 33 if it is the longest to be positioned at time of position of pixel 43 when pixel 33 moves.Thus, differential data δ as shown in figure 14, as the data I mg ' of original image with relatively use the difference of data I o ' pixel 43 separately to calculate.This differential data δ is appended to the pixel 33 of the data I o or the restored data Io+n of initial pictures.
In addition, come word with before example, the center of gravity that " 0.5 " " 0.3 " " 0.2 " is three is the position of maximum " 0.5 ", becomes the position of self.Therefore, do not consider the distribution of " 0.3 " or " 0.2 ", only make the change of differential data δ " 0.5 " or " 0.5 " doubly partly be allocated in self-position.The situation of the concentration of energy that such processing is suitable for rocking.
And then, also can automatically select above-mentioned each processing method according to the content of the data G that changes essential factor information.That is, the data G that changes essential factor information can be classified in a plurality of kinds any one of handling part 4, and this each classification carried out different processing.For example, to can carry out (1) as Fig. 5～shown in Figure 12 as processing method, use distribution ratio k to distribute the method (by way of example) of differential data δ, difference or differential data δ change method (respective pixel mode), (3) doubly that (2) make respective pixel to detect the center of gravity of deterioration essential factor and utilize the program of these three kinds of methods of method (center of gravity method) of the data of this barycentric subdivision to be stored in the handling part 4, analyze the situation of deterioration essential factor, and select in these three kinds of methods any one according to this analysis result.In addition, also can select multiple arbitrarily in three kinds of methods, in each program, alternately utilize, perhaps in initial several, handle, otherwise handle again afterwards in a certain mode.
In this image processing apparatus 1, the image that will be restored is shown on the display part 8.But if desire is intactly restored the deterioration image of taking, the processing time can be elongated, can't be shown in immediately on the display part 8 after the shooting.Therefore, seek to restore the high speed of processing.Below, this high speed is described.
On the meaning of the high speed of seeking to restore processing, there is the method that makes up with inverse problem.That is, utilization is dwindled data and is carried out repeated treatments, and calculates by the transfer function of the original image that dwindles to the restored data of dwindling.Then, with the transfer function of calculating amplify, interpolation, and use that this is exaggerated, the transfer function of interpolation draws the restored data of original image.This processing method is favourable for big treatment of picture.
Below, describe for basic concept the favourable fast processingization of the recovery of big image.
Only by repeated treatments, in any case all can be in convergence spended time.This shortcoming is obvious under the situation of big image.On the other hand, because the deconvolution in the frequency interval can utilize fast Fourier transform (Fast Fourier Transform:FFT) to calculate fast, be very attractive therefore.Here legal being meant of said optical superposition removed this deflection etc. from the image that deterioration etc. takes place because of deflection or fuzzy etc., thereby the base image of deterioration etc. does not take place in recovery.
Under the situation of image, input is set in (x), output is set at ou (x), and when transfer function was set at g (x), exporting ou (x) in perfect condition became convolution integral, and sets up the relation of following formula (3).In addition, " ∫ " is the symbol of integration in formula (3).
ou(x)＝∫in(t)g(x-t)dt …(3)
This formula (3) becomes following formula (4) in frequency interval.
O(u)＝I(u)G(u) …(4)
The process of obtaining the input in (x) of transfer function g (x) or the unknown from this known output ou (x) is exactly a deconvolution, for this purpose, in frequency interval, can obtain I (u)=O (u)/G (u), can obtain unknown input in (x) by making it return the real space.
But in fact because noise etc., formula (3) becomes " ou (x)+α (x)=∫ in (t) g (x-t) dt+ α (x) ".At this, " ou (x)+α (x) " is known, and still each of " ou (x) " and " α (x) " is unknown naturally.For example, even approx " ou (x) " and " α (x) " answered as inverse problem, on reality, also be difficult to draw satisfactory separating fully.At this, the jn (x) that utilizes iterative method will become " ou (x)+α (x)=∫ in (t) g (x-t) dt+ α (x) ≈ ∫ jn (t) g (x-t) dt " restrains and the process that draws is exactly the handling process of above-mentioned Fig. 3.
At this, as long as " α (x) " ou (x) " (" " " be represent very less than the Japanese symbol), just can think " jn (x) ≈ in (x) ".
But,,, exist the shortcoming of data number change words spended time how though therefore can access satisfactory separating fully because this method is repeatedly and restrains the calculating total data zone in.On the other hand, under noiseless perfect condition, can utilize deconvolution in the frequency interval to calculate to obtain apace and separate.At this,, can draw satisfactory separating fully apace by making up this two processing.
As this processing method, can consider two kinds of methods.The firstth,, form the method for reduced data by data sampling.This method is described as second processing method of having utilized processing method shown in Figure 3.The method of data sampling, for example as shown in figure 15, be when the data I mg ' of original image is made of pixel 11～16,21～26,31～36,41～46,51～56,61～66, pixel is taken a sample every one, thereby generate the method that the original image of 1/4th sizes that are made of pixel 11,13,15,31,33,35,51,53,55 dwindles data I Smg '.
Like this, the data I mg ' of original image and the data G of variation essential factor information are taken a sample, generate the data GS that sampled original image dwindles data I Smg ' and reduced variation essential factor information, and the data GS that uses original image to dwindle data I Smg ' and reduced variation essential factor information carries out repeated treatments shown in Figure 3, thus obtain with dwindle to original image data I Smg ' before changing dwindle base image ISmg approximate, be carried out the sampling fully satisfied be similar to dwindle restored data ISo+n.
With this reduced approximate dwindle restored data ISo+n be inferred as to original image dwindle data I Smg ' before changing dwindle base image ISmg, be the downscaled images of correct images Img.Then, original image is dwindled data I Smg ' as the convolution integral of dwindling restored data ISo+n and transfer function g (x), can dwindle data I Smg ' and draw unknown transfer function g1 (x) from resulting restored data ISo+n and the known original image of dwindling.
Be complete satisfactory data though dwindle restored data ISo+n, be similar to all the time.Therefore, the transfer function g (x) of the original restored data Io+n and the data I mg ' of original image is not the transfer function g1 (x) that draws by the repeated treatments of utilizing reduced data.Therefore, will be by by dwindling restored data ISo+n and dwindling data I Smg ' as the original image of the data of the original image that dwindles and calculate transfer function g1 (x), interpolation, correction are carried out and the new transfer function g2 (x) that obtains in the interval that the transfer function g1 (x) that calculates is amplified and will amplify, as the transfer function g (x) for the data I mg ' of the original image that becomes basic data.New transfer function g2 (x) is, by resulting transfer function g1 (x) is formed original image dwindle data reduction magnification inverse doubly after, carry out the interpolation of linear interpolation or spline interpolation etc. and handle and obtain amplifying value at interval.For example, under 1/2 the situation of all taking a sample in length and breadth as shown in Figure 15, become 1/4 reduction magnification, therefore doubly become 4 times as inverse.
Then, use the new transfer function g2 (x) (=g (x)) of this correction, carry out deconvolution at frequency interval and calculate (by calculating the calculating of from contain the group of pictures that has a stain, removing stain), obtain the complete restored data Io+n of general image and it is inferred as the not former correct image I mg (base image) of deterioration.
The flow process of above processing is represented with flow chart shown in Figure 16.
In step S201, the data I mg ' of original image and the data G of variation essential factor information are reduced into 1/M.In the example of Figure 15, be reduced into 1/4.Use resulting original image to dwindle data I Smg ' and dwindle the data GS that changes essential factor information and the data I o of arbitrary image (image of regulation), repeat step S102 shown in Figure 3～step S105.Then, obtain image that differential data δ diminishes, promptly with dwindle to original image data I Smg ' before changing dwindle base image ISmg approximate dwindle restored data ISo+n (step S202).At this moment, " G, Img ', Io+n " quilt " GS, ISmg ', ISo+n " shown in Figure 3 replaces.
Dwindle data I Smg ' and calculate from original image and dwindle data I Smg ' by resulting restored data ISo+n and the known original image of dwindling to the transfer function g1 that dwindles restored data ISo+n (x) (step S203).Among step S204s, resulting transfer function g1 (x) zoomed into M doubly (in the example of Figure 15 be 4 times) and utilize the interpolating method of linear interpolation etc. the interval of this amplification carried out interpolation obtain new transfer function g2 (x) thereafter.Will this new transfer function g2 (x) be inferred as transfer function g (x) for base image.
Then, carry out deconvolution by the new transfer function g2 (x) that calculates and the data I mg ' of original image, thereby obtain restored data Io+n.With this restored data Io+n as base image (step S205).As mentioned above, by using reprocessing simultaneously and obtaining transfer function g1 (x), g2 (x) and use the processing of the new transfer function g2 (x) that this quilt obtains, can seek to restore the high speed of processing.
In image processing apparatus 1, to be shown on the display part 8 by the resulting restored data ISo+n that dwindles of reprocessing immediately after the shooting, and when image processing apparatus 1 is not operated, for example when being disconnected, the power supply of shoot part 2 utilizes handling afterwards of transfer function.Then, the base image that will be restored is recorded in recording portion 5.
In addition, under the situation of this processing, also can be with the resulting restored data Io+n that is inferred to be correct images as the data I o of the initial pictures of processing shown in Figure 3 and use, the data I mg ' that utilize to change the original image of the data G of essential factor information and deterioration further carries out reprocessing, and will handle resulting data by this and be stored in recording portion 5.
Utilizing second kind of method of reduced data, is that the data in the part zone of the data I mg ' by taking out original image form the method that original image dwindles data I Smg '.This method is described as the 3rd processing method of having utilized processing method shown in Figure 3.For example, as shown in figure 17, be when the data I mg ' of original image is made of pixel 11～16,21～26,31～36,41～46,51～56,61～66, taking-up is as zone its middle section, that be made of pixel 32,33,34,42,43,44, and generates the method that original image dwindles data I Smg '.
Utilize the flow chart of Figure 18 that this second method is at length described.
In second method, at first, in step S301, obtain original image as described above and dwindle data I Smg '.Then, the data G that uses this original image to dwindle data I Smg ' and change essential factor information with dwindle the data I o of the initial pictures of the same size of data I Smg ' (=identical pixel count) as the data of arbitrary image and with original image, repeat the processing of step S102 shown in Figure 3～step S105, thereby obtain dwindling restored data ISo+n (step S302).In this is handled, respectively " Img ' " among Fig. 3 replaced to " ISmg ' ", " Io+n " replaces to " ISo+n ".
Dwindle restored data ISo+n and known original image dwindles data I Smg ' by resulting, calculate from dwindling transfer function g1 ' that restored data ISo+n dwindles from data I Smg ' to original image (x) (step S303).Then, with the transfer function g1 ' that calculates (x) as (x) for the transfer function g ' of base image Img, and utilize this transfer function g1 ' (x) the data I mg ' of (=g ' (x)) and known original image obtain base image Img by contrary calculating.In addition, the data of obtaining are actually the data of the base image that is similar to base image Img.
As mentioned above, by using reprocessing simultaneously and obtaining transfer function g1 ' (x) and the transfer function g1 ' processing (x) of using this quilt to obtain, can seek to restore the high speed of processing.In addition, can be not yet with the transfer function g1 ' that obtained (x) intactly as a whole transfer function g ' (x), but utilize the data G that changes essential factor information to revise.
In the handling part 4 of this image processing apparatus 1, to be shown on the display part 8 by the resulting restored data ISo+n that dwindles of reprocessing immediately after the shooting, and will restore the processing of base image and show to handle by the contrary calculating of using transfer function and carry out simultaneously.Then, the base image that will be restored is stored in recording portion 5.In addition, be next to the reprocessing processing that utilizes transfer function afterwards, also can after show processing, carry out or when image processing apparatus 1 is not operated, carry out.
Like this, in above-mentioned second method that is used for high speed, not to utilize repeated treatments to restore image-region integral body, but the part in repeated treatments zone and obtain good restored image and use this restored image to obtain (x) for the transfer function g1 ' of this part, utilize this transfer function g1 ' (x) self or it is revised (amplify etc.) after function carry out the recovery of integral image.But taking out the zone must be fully greater than variable domain.In the example before shown in Fig. 5 waits, through three pixels and therefore change must take out the above zone of three pixels.
And, under the situation of taking out Figure 17, the method for dwindling the zone shown in Figure 180, also can be for example to be divided into four parts as shown in Figure 19 by data I mg ' with original image, take out the zone of a part from each cut zone, four original images as the zonule are dwindled data I Smg ' carry out repeated treatments respectively, to be formed one by four split images that four cut zone of cutting apart are restored respectively and will be restored, thereby form original general image.And, be divided into when a plurality of, with have must be through a plurality of zones and overlapping areas (overlapping region) for good.In addition, the overlapping region of the image that each restores, with use mean value or being treated to of in the overlapping region, connecting smoothly etc. good.
And then, adopt in reality under the situation of processing method of Fig. 3, for image etc., be judged as to the convergence of the restored image of good approximation slow with obvious changes in contrast.Like this, according to difference, thereby exist the convergence rate of repeated treatments must increase the situation of the number of occurrence slowly as the Properties of Objects of taking pictures of original image.Under the situation of this object of taking pictures, be inferred as and adopt following such processing method just can address this problem.
This method as described below.Promptly, the object desire of taking pictures with obvious changes in contrast is used the repeated treatments of the recovery utilize processing method shown in Figure 3 and is obtained words with the approximate image of original image, it is very many that repeat number becomes, and also can't generate after the processing of carrying out repeatedly simultaneously and the original approximate restored data Io+n of object that takes pictures.Therefore, use the data G of the variation essential factor information when taking to generate the data B ' of blurred picture, and these data B ' is superimposed among the data I mg ' of the original image (blurred picture) that is taken by the data B of known image, thereby generate " Img '+B '., utilize shown in Figure 3 processing the image of stack restored processing, remove the data B of known image, will want that the data I mg of the restored image obtained takes out from the result data C that becomes this restored data Io+n thereafter.
This method is applicable on the above-mentioned downscaled images,, also is shown on the display part 8 in the downscaled images that will be restored immediately after the shooting even for the object of taking pictures with obvious changes in contrast.On the other hand, the processing of big original image is to show that handling the transfer function of obtaining from this processing by handling part 4 utilizations the back restores processing.The base image of being restored is stored in recording portion 5 thereafter.In addition, in the method, the data I mg of correct images contains obvious changes in contrast, but can alleviate this obvious changes in contrast by the data B that appends known image, can reduce and restore the number of occurrence of handling.
In addition, as the processing method and the processing method at a high speed of the object of taking pictures that restores difficulty, also can adopt other processing method.For example, make the words that the number of occurrence that restore to handle increases can more approaching good restored image, go up spended time but handle.Therefore, the image that uses the repeated treatments number by to a certain degree to obtain is calculated the error percentage that it comprises, and by removing the error of calculating from the restored image that contains error, can access good restored image, be restored data Io+n.
Below, specifically describe this method.To want that the correct images of obtaining is made as A, the original image of taking will be made as A ', will be made as A+ γ, will fuzzy relatively be made as A '+γ ' by what this restored data generated with data by the image that original image A ' restores.The additional original image of taking in this " A '+γ ' " " A ', it is restored processing, become " A+ γ+A+ γ+γ ", this is " 2A+3 γ ", or is " 2 (A+ γ)+γ ".Because " A+ γ " handles by the recovery of last time to obtain, and therefore can calculate " 2 (A+ γ)+γ-2 (A+ γ) ", thereby obtain " γ ".Therefore, want the correct images A that obtains by removing " γ " from " A+ γ ", can accessing.With this method and above-mentioned similarly being applicable on the downscaled images, the original image that will be restored dwindles data and is shown in display part 8, utilizes the big base image of resulting transfer function recovery and is stored in recording portion 5.These controls or processing are carried out by handling part 4.
More than, suitable execution mode of the present invention is illustrated, but only otherwise breaking away from purport of the present invention can implement various changes.For example, the processing that utilizes handling part 4 to carry out is made of software, still, also can constitute by the hardware of being made up of the parts of bearing part processing respectively.
In addition, as the original image that becomes process object, except that photographic images, also can be for this photographic images being carried out complementary color or carrying out Fourier transform etc., implemented the image of processing.And then, use data as a comparison, except that the data that the data G that use to change essential factor information generates, also can be used as to using the data G that changes essential factor information the data after the data that generate applies complementary color or carry out Fourier transform.In addition, as the data that change essential factor information, not only be the data of deterioration essential factor information, also can be for containing the information that only makes image change or making the data of the information that image improves on the contrary with deterioration.
In addition, automatically or under the situation about being set regularly, also can change the number of times of this setting in image processing apparatus 1 side in the number of occurrence of handling by the data G that changes essential factor information.For example, owing to rocking under the situation that is scattered in a plurality of pixels, can increase the number of occurrence, disperse to reduce the number of occurrence under few situation in the data of certain pixel.
And then, in repeated treatments, also can when differential data δ disperses, when becoming big, just make to handle and end.About whether dispersing, if be judged as the method for dispersing greater than the last time just for example can adopt behind the mean value of observing differential data δ this mean value to become.In addition, the method for back abort process doublely takes place or disperses the method that continues abort process behind the stipulated number but also can adopt to disperse in abort process immediately after dispersing generation once.
In addition, in repeated treatments, when desire changes to exceptional value with input, also can make to handle and end.For example, under eight situation, be when surpassing 255 value in the value of desiring to be changed, make to handle and end.In addition, in the repeated treatments, when desire will change to unusual value as the input of new data, also can not use this value and form normal value.For example, eight 0～255 in, will be above 255 value during in desire as the input data, handle as maximum 255.Promptly, in restored data, contain when allowing in addition unusual numerical value of numerical value (being 0～255) (in above-mentioned example for surpassing 255 value) in above-mentioned example, can end this processing, perhaps, when in restored data, containing the unusual numerical value of allowing beyond the numerical value, this unusual numerical value is changed to allow numerical value and proceed and handle.
In addition, when generation becomes the restored data of output image, exist owing to change the data conditions in zone that the difference of the data G of essential factor information exceeds the image of desire recovery.Under this situation, the data that exceed the zone are transfused to opposition side.In addition, existing under the data conditions that outside the zone, to import, be good to be taken into these data from opposition side.For example, by be positioned at the zone below the data conditions of the data of pixel XN1 (capable 1 row of the N) pixel below being allocated in more under, this position is in outside the zone.Therefore, these data carry out being allocated in the processing that is positioned at uppermost pixel X11 (1 row, 1 row) directly over pixel XN1.For the adjacent pixels XN2 of pixel XN1 (capable 2 row of N), similarly directly over be allocated in the pixel X12 (1 row, 2 row on the next door of=pixel X11) that goes up the hurdle most.Like this, when generating restored data, become under the data conditions of restoring outside the subject area, be disposed at these data occurrence positions vertically, laterally or any one party in oblique to the recovery subject area of opposition side position in, can carry out certain recovery for the subject area that desire is restored.
In addition, also can be with each processing method of above explanation, promptly (1) uses distribution ratio k to distribute the method (by way of example) of differential data δ, (2) make the difference of respective pixel, or differential data δ becomes method (respective pixel mode) doubly, (3) detect the center of gravity of deterioration essential factor and utilize the method (center of gravity method) of the data of this barycentric subdivision, (4) data are taken a sample, method (inverse problem sampling method) with the inverse problem combination, (5) zone is dwindled in taking-up, method (inverse problem zone removing method) with the inverse problem combination, (6) image of overlapping regulation and carry out repeated treatments, remove the method (bad image countermeasure method of superposition) of the image of this regulation then, (7) program of each processing method of removing the method (error removing method) of the error of calculating from the restored image that contains error is stored in handling part 4, thereby can automatically select processing method according to user's the selection or the kind of image.
For example, in the reprocessing of downscaled images, be not only (4), (5) (utilizing (1)), also can utilize (2), (3), (6), (7) promptly to restore and be shown in display part 8, can adopt to the recovery of base image is handled from original image the transfer function of (4), (5) handle or from (2), (3), (6), (7) resulting downscaled images obtain transfer function and utilize the processing of this transfer function, maybe will utilize this transfer function and method that the image that obtains carries out reprocessing once more.In addition, also can utilize transfer function or do not utilize and carry out at leisure to restore and handle to the recovery of base image from original image.That is, with show to handle differently, the recovery of base image also can utilize above-mentioned (1), (2), (3), (6), (7) any one directly obtain base image from original image.
In addition, the recovery of base image is handled the processing that can adopt and will dwindle restored data to be shown in display part 8 and is carried out simultaneously, or be disconnected, carry out when taking, or carrying out after the demonstration of display part 8 is handled, waiting the whole bag of tricks at the power supply of shoot part 2.In addition, be stored in recording portion 5 by the base image of being restored, but also can with this preservations simultaneously or replace preservation and utilize internet etc. communication line and to transmissions such as other servers.
In addition, also the data G that changes essential factor information can be classified in a plurality of kinds any one of handling part 4, and this each classification carried out different processing (any one of above-mentioned each method), perhaps make the number of occurrence difference of this each classification.In addition, in the processing of handling part 4, whether display image to be formed to be divided into that to have carried out restoring the image of handling be good according to the data G that changes essential factor information.For example, only handling being judged as the recovery that deterioration is shown in the image of display part 8 when obvious, not have only to show intactly that it is good that the image of rectification has a little been carried out in downscaled images or demonstration when so big in deterioration.Thus, the load of handling part 4 is alleviated.
In addition, also can be with any a plurality of being stored in the handling part 4 in these (1)～(7), thus can automatically select processing method according to user's the selection or the kind of image.In addition, also can select a plurality of arbitrarily in these seven methods, alternately or in turn utilize in each program, perhaps initial several is handled, is otherwise handled then in a certain mode.And, image processing apparatus 1 in above-mentioned (1)～(7) any one or a plurality of, also can have the processing method different with them.
In addition, above-mentioned each processing method also can be by sequencing.In addition, also can will be deposited in recording medium, for example CD (Compact Disc), DVD, USB (Universal Serial Bus) memory by the processing method of sequencing, and can read by computer.Under this situation, image processing apparatus 1 has the means of reading in of reading in the program in this recording medium.And then, also the processing method of this sequencing can be deposited in image processing apparatus 1 external server, download as required and use.Under this situation, image processing apparatus 1 has the means of communication that the program in this recording medium is downloaded.
Claims (19)
1. image processing apparatus, be provided with handling part, this handling part restores from the original image that becomes process object and changes the image before or the approximate image of image that should be taken originally or above-mentioned image, and this approximate image is the image that is collectively referred to as base image, it is characterized in that
Above-mentioned handling part has:
1) obtain dwindling the module of data from the original image that above-mentioned original image cuts pixel count,
2) dwindle data recovery and the module of dwindling restored data that base image is similar to of dwindling that changes before according to this original image,
3) dwindle the module that restored data is shown in display part with resulting,
4) utilize to restore the above-mentioned module of the above-mentioned base image of resulting data recovery when dwindling restored data.
2. image processing apparatus as claimed in claim 1 is characterized in that,
The described approximate module of dwindling restored data of base image of dwindling before dwindling data recovery and change according to above-mentioned original image has: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate the module of relatively using data, 2) above-mentioned original image is dwindled data and this and relatively make the module that difference obtains differential data with data, 3) utilize resulting differential data to generate and dwindle restored data, and use this to dwindle restored data to replace the afore mentioned rules view data, replace the dwindling restored data of last time and repeat same processing, the module that when reaching the reprocessing number of times of prior setting, stops to handle with the resulting above-mentioned restored data of dwindling afterwards.
3. image processing apparatus as claimed in claim 1 is characterized in that,
The described approximate module of dwindling restored data of base image of dwindling before dwindling data recovery and change according to above-mentioned original image has: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate the module of relatively using data, 2) above-mentioned original image is dwindled data and this and relatively make the module that difference obtains differential data with data, 3) at resulting differential data during greater than setting, utilize resulting differential data to generate and dwindle restored data, and dwindle restored data with this and replace the afore mentioned rules view data, replace and last to dwindle restored data and repeat same processing with the resulting above-mentioned restored data of dwindling afterwards, when resulting differential data is the module that setting stops to handle when following.
4. image processing apparatus as claimed in claim 1 is characterized in that,
The described approximate module of dwindling restored data of base image of dwindling before dwindling data recovery and change according to above-mentioned original image has: the data that 1) will become the variation essential factor information of image change main cause multiply by the specified image data and generate the module of relatively using data, 2) above-mentioned original image is dwindled data and this and relatively make the module that difference obtains differential data with data, 3) be that setting is when above at resulting differential data, utilize resulting differential data to generate and dwindle restored data, and dwindle restored data with this and replace the afore mentioned rules view data, replace and last to dwindle restored data and repeat same processing with the resulting above-mentioned restored data of dwindling afterwards, the module that stops to handle during less than setting when resulting differential data.
5. as any described image processing apparatus of claim 2～4, it is characterized in that, describedly dwindle the approximate module of dwindling restored data of base image before dwindling data recovery and change according to above-mentioned original image, have by above-mentioned and dwindle restored data and above-mentioned original image dwindles the module that data computation goes out transfer function
The above-mentioned module of the above-mentioned base image of resulting data recovery when dwindling restored data is restored in described utilization, have with this transfer function form above-mentioned original image dwindle data above-mentioned original image reduction magnification inverse doubly after, the interval that is exaggerated is carried out interpolation and is obtained new transfer function as the transfer function of the deterioration state of representing above-mentioned original image, and use this new transfer function and generate the module of above-mentioned base image.
6. as any described image processing apparatus of claim 2～4, it is characterized in that, describedly dwindle the approximate module of dwindling restored data of base image before dwindling data recovery and change according to above-mentioned original image, have by above-mentioned original image and dwindle data and above-mentionedly dwindle the module that restored data is obtained transfer function
The above-mentioned module of the above-mentioned base image of resulting data recovery when dwindling restored data is restored in described utilization, have this transfer function transfer function of image as a whole intactly, and by the contrary module that generates the above-mentioned base image before above-mentioned original image changes of calculating.
7. image processing apparatus as claimed in claim 5 is characterized in that, it is to form by the data of above-mentioned original image are taken a sample that described original image dwindles data.
8. image processing apparatus as claimed in claim 6 is characterized in that, it is to form by the data of above-mentioned original image are taken a sample that described original image dwindles data.
9. image processing apparatus as claimed in claim 5 is characterized in that described original image dwindles data, is to form by intactly take out part zone from the data of above-mentioned original image.
10. image processing apparatus as claimed in claim 6 is characterized in that described original image dwindles data, is to form by intactly take out part zone from the data of above-mentioned original image.
11. image processing apparatus as claimed in claim 1 is characterized in that, the recovery of described base image is handled, and dwindles the processing that restored data is shown in above-mentioned display part and carries out simultaneously above-mentioned.
12. image processing apparatus as claimed in claim 1 is characterized in that, the recovery of described base image is handled and is carried out when power supply for shooting is disconnected.
13. image processing apparatus as claimed in claim 1 is characterized in that, the recovery of described base image is handled, and carries out after dwindling the processing that restored data is shown in above-mentioned display part above-mentioned carrying out.
14. an image processing method, the image processing apparatus that utilizes claim 1 to put down in writing is characterized in that may further comprise the steps:
1) original image that obtains cutting from original image pixel count dwindles data,
2) before dwindling data recovery and change according to this original image dwindle base image approximate dwindle restored data,
3) the resulting restored data of dwindling is shown in display part,
4) utilize resulting data recovery base image when recovery is above-mentioned dwindles restored data.
15. image processing method as claimed in claim 14 is characterized in that,
The described approximate step of dwindling restored data of base image of dwindling before dwindling data recovery and change according to above-mentioned original image may further comprise the steps:
1) data that will become the variation essential factor information of image change main cause multiply by the specified image data and generate and relatively use data,
2) above-mentioned original image is dwindled data and this and relatively makes difference and obtain differential data with data,
3) utilize resulting differential data to generate and dwindle restored data, and use this to dwindle restored data to replace the afore mentioned rules view data, replace the dwindling restored data of last time and repeat same processing with the resulting above-mentioned restored data of dwindling afterwards, when reaching the reprocessing number of times of prior setting, stop to handle.
16. image processing method as claimed in claim 14 is characterized in that,
The described approximate step of dwindling restored data of base image of dwindling before dwindling data recovery and change according to above-mentioned original image may further comprise the steps:
1) data that will become the variation essential factor information of image change main cause multiply by the specified image data and generate and relatively use data,
2) above-mentioned original image is dwindled data and this and relatively makes difference and obtain differential data with data,
3) at resulting differential data during greater than setting, utilize resulting differential data to generate and dwindle restored data, and dwindle restored data with this and replace the afore mentioned rules view data, replace and last to dwindle restored data and repeat same processing with the resulting above-mentioned restored data of dwindling afterwards, when resulting differential data is that setting stops to handle when following.
17. image processing method as claimed in claim 14 is characterized in that,
The described approximate step of dwindling restored data of base image of dwindling before dwindling data recovery and change according to above-mentioned original image may further comprise the steps:
1) data that will become the variation essential factor information of image change main cause multiply by the specified image data and generate and relatively use data,
2) above-mentioned original image is dwindled data and this and relatively makes difference and obtain differential data with data,
3) be that setting is when above at resulting differential data, utilize resulting differential data to generate and dwindle restored data, and dwindle restored data with this and replace the afore mentioned rules view data, replace and last to dwindle restored data and repeat same processing with the resulting above-mentioned restored data of dwindling afterwards, when resulting differential data stops to handle during less than setting.
18. any described image processing method as claim 15～17 is characterized in that,
Describedly dwindle the approximate step of dwindling restored data of base image before dwindling data recovery and change according to above-mentioned original image, comprise by above-mentioned and dwindle restored data and above-mentioned original image dwindles the step that data computation goes out transfer function,
The above-mentioned step of resulting data recovery base image when dwindling restored data is restored in described utilization, comprise with this transfer function form above-mentioned original image dwindle data above-mentioned original image reduction magnification inverse doubly after, the interval that is exaggerated is carried out interpolation and is obtained new transfer function as the transfer function of the deterioration state of representing above-mentioned original image, and use this new transfer function and generate the step of above-mentioned base image.
19. any described image processing method as claim 15～17 is characterized in that,
Describedly dwindle the approximate step of dwindling restored data of base image before dwindling data recovery and change according to above-mentioned original image, comprise by above-mentioned original image and dwindle data and above-mentionedly dwindle the step that restored data is obtained transfer function,
The above-mentioned step of resulting data recovery base image when dwindling restored data is restored in described utilization, comprise this transfer function transfer function of image as a whole intactly, and by the contrary step that generates the above-mentioned base image before above-mentioned original image changes of calculating.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2005267906A JP2007081905A (en) | 2005-09-15 | 2005-09-15 | Image processing apparatus |
| JP267906/2005 | 2005-09-15 | ||
| PCT/JP2006/314596 WO2007032148A1 (en) | 2005-09-15 | 2006-07-24 | Image processor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101305598A CN101305598A (en) | 2008-11-12 |
| CN101305598B true CN101305598B (en) | 2011-04-06 |
Family
ID=37864748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2006800415164A Expired - Fee Related CN101305598B (en) | 2005-09-15 | 2006-07-24 | Image processing apparatus and method |
Country Status (3)
| Country | Link |
|---|---|
| JP (1) | JP2007081905A (en) |
| CN (1) | CN101305598B (en) |
| WO (1) | WO2007032148A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011130061A (en) * | 2009-12-16 | 2011-06-30 | Fujifilm Corp | Method and device for adjusting positional relation between photographic lens and imaging device, and method and device for manufacturing camera module |
| JP5522068B2 (en) * | 2011-01-25 | 2014-06-18 | カシオ計算機株式会社 | Image display device, image display method, and program |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1448029A (en) * | 2000-08-18 | 2003-10-08 | 去来株式会社 | Method and system of revision for 3-D image |
| JP2004343483A (en) * | 2003-05-16 | 2004-12-02 | Acutelogic Corp | Device and method for correcting camera-shake and device for detecting camera shake |
| CN1819626A (en) * | 2004-12-28 | 2006-08-16 | 精工爱普生株式会社 | Imaging apparatus and portable device and portable telephone using the same |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002290811A (en) * | 2001-03-23 | 2002-10-04 | Minolta Co Ltd | Imaging device, method and program for image processing, and information recording medium |
| JP2002300459A (en) * | 2001-03-30 | 2002-10-11 | Minolta Co Ltd | Image restoring device through iteration method, image restoring method and its program, and recording medium |
| JP4179278B2 (en) * | 2004-12-28 | 2008-11-12 | セイコーエプソン株式会社 | IMAGING DEVICE AND MOBILE DEVICE AND MOBILE PHONE HAVING THE SAME |
-
2005
- 2005-09-15 JP JP2005267906A patent/JP2007081905A/en active Pending
-
2006
- 2006-07-24 CN CN2006800415164A patent/CN101305598B/en not_active Expired - Fee Related
- 2006-07-24 WO PCT/JP2006/314596 patent/WO2007032148A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1448029A (en) * | 2000-08-18 | 2003-10-08 | 去来株式会社 | Method and system of revision for 3-D image |
| JP2004343483A (en) * | 2003-05-16 | 2004-12-02 | Acutelogic Corp | Device and method for correcting camera-shake and device for detecting camera shake |
| CN1819626A (en) * | 2004-12-28 | 2006-08-16 | 精工爱普生株式会社 | Imaging apparatus and portable device and portable telephone using the same |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2007081905A (en) | 2007-03-29 |
| CN101305598A (en) | 2008-11-12 |
| WO2007032148A1 (en) | 2007-03-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9177368B2 (en) | Image distortion correction | |
| CN101198985B (en) | Image processing apparatus and image processing method | |
| CN101305598B (en) | Image processing apparatus and method | |
| CN101375591A (en) | Image processing device | |
| JP4965179B2 (en) | Image processing device | |
| JP4602860B2 (en) | Image processing device | |
| JP4598623B2 (en) | Image processing device | |
| CN101310520B (en) | Image processing device and method | |
| JP5133070B2 (en) | Signal processing device | |
| JP2007129354A (en) | Image processing apparatus | |
| CN101569176B (en) | Signal processing device | |
| JP4975644B2 (en) | Image processing device | |
| CN101960835A (en) | Method for generating change factor information data and signal processing device | |
| JP4629622B2 (en) | Image processing device | |
| JP4629537B2 (en) | Image processing device | |
| CN101960488B (en) | Method of creating data relating to change factor information and image processor | |
| JP4763419B2 (en) | Image processing device | |
| CN101632309A (en) | Image processing device and image processing method | |
| JP5007245B2 (en) | Signal processing device | |
| JP2008199305A (en) | Image processor and image processing method | |
| WO2006137309A1 (en) | Image processing apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110406 Termination date: 20170724 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |