CN101222584A - Apparatus and method for blur detection, and apparatus and method for blur correction - Google Patents

Abstract

A blur detection apparatus that detects blur contained in a first image acquired by shooting by an image sensor based on the output of the image sensor has a blur information creator adapted to create blur information reflecting the blur based on the first image and a second image shot with an exposure time shorter than the exposure time of the first image.

Description

Device for detecting rock, blur correcting device and method

Technical field

The present invention relates to rock the device for detecting rock that detects and rock detection method to what comprise in the image that obtains by photography.Have again, the present invention relates to rocking the blur correcting device of revising and rocking modification method.Also have, the present invention relates to utilize the camera head of these device and methods.

Background technology

It is the technology that the hand when alleviating photography is trembled that hand is trembled correction technique, comes into one's own as the differential technology in the camera heads such as digital camera.Hand tremble correction technique no matter revise to as if rest image or moving image, can consider to be divided into and detect key technologies that hand trembles and the key technologies of coming correction image according to this testing result.

In the method that the detection hand is trembled, following two kinds of methods are arranged: adopt hands such as angular-rate sensor or acceleration transducer to tremble the method for detecting sensor; Analysis image detects the electronic type method that hand is trembled.In the makeover process of image, the favourable optical profile type hand of revising with the driving of optical system is trembled the electronic type hand of revising and utilizing image processing to revise and is trembled correction.

Tremble modification method as hand, have: tremble detecting sensor detection hand with hand and tremble, and carry out the method that the optical profile type hand is trembled correction according to this testing result at rest image; Tremble detecting sensor detection hand with hand and tremble, and carry out the method that the electronic type hand is trembled correction according to this testing result; Analysis image and detect hand and tremble, and carry out the method that the electronic type hand is trembled correction according to this testing result.

Owing to if adopt hand to tremble detecting sensor, then cause cost to increase considerably, tremble modification method so seek to develop the hand that does not need hand to tremble detecting sensor.

Can carry out the method that hand is trembled correction as not adopting hand to tremble detecting sensor, sum operation formula hand is trembled revised law and just is practical.With reference to Figure 15, illustrate that simply sum operation formula hand trembles revised law.Sum operation formula hand is trembled in the revised law, and several that take continuously the time for exposure t2 common time for exposure t1 being cut apart and obtain are cut apart exposure image (short exposure image) DP1～DP4.Be made as P at the width of cloth number of several being cut apart exposure image _NUMSituation under, be t2=t1/P _NUM(P in this example _NUM=4).And, by to carry out sum operation on the basis of cutting apart exposure image DP1～DP4 contraposition synthetic making can eliminate the mode of respectively cutting apart the position deviation between exposure image, thereby generate 1 width of cloth rest image that desirable lightness was trembled less and can be guaranteed to hand.

Have again, following mode is also proposed, promptly tremble image according to 1 width of cloth hand that rocks that comprises that obtains by photography, the hand that hand in the deduction expression photography is trembled is trembled information (some expanded function or image restoration filter), tremble information and hand is trembled image according to this hand, generate the restored image that does not rock with Digital Signal Processing.In this mode, the mode that adopts the Fourier iterative method is disclosed.

Figure 16 represents to realize the block diagram of the formation of Fourier iterative method.In the Fourier iterative method, by carrying out Fourier transform and inverse fourier transform repeatedly, thereby infer final restored image according to the deterioration image by the correction of restored image and some expanded function (PSF).In order to carry out the Fourier iterative method, initial restored image (initial value of restored image) need be provided, as initial restored image, generally adopt random image or tremble the deterioration image of image as hand.

If adopt the Fourier iterative method, then can generate the image that has reduced the influence that hand trembles, and need not to adopt hand to tremble detecting sensor.But the Fourier iterative method is the nonlinear optimization method, needs a lot of iterationses in order to obtain suitable restored image.That is to say that hand is trembled detection and revise the required processing time very long.Therefore, the difficulty that becomes of the practicability in the digital camera etc.Reduction in processing time is an important topic aspect practicability.

Have again, tremble modification method, also proposed following method as the hand that does not adopt hand to tremble detecting sensor.In a certain existing method, before and after the photography of the master image that should revise, take several sub-pictures, the hand when inferring the master image photography according to these several sub-pictures is trembled information, rocks rocking of information correction master image according to this.Owing to infer rocking of master image according to the amount of exercise (amount of exercise that also comprises exposure interval) between the sub-picture that obtains before and after master image, accuracy of detection of rocking and correction precision reduce but in the method.Also have, in other existing methods, detect hand and tremble according to hand being trembled the image of image transform to two-dimensional frequency.Specifically be that the image projection that will obtain by conversion is in the circle at center to the initial point with frequency coordinate, asks for the size and the direction of rocking according to this data for projection.But, in the method, can only infer rocking of straight line and constant speed, in addition under the situation that the frequency content in the specific direction of body that is taken is little, rock the detection failure of direction sometimes and can't suitably revise and rock.Rocking the high precision int of correction self-evident also is important problem.

Summary of the invention

The device for detecting rock that the present invention relates to, its output based on image unit detects rocking that first image that the photography by described image unit obtains comprised, this device for detecting rock possesses the information generating unit of rocking, it is based on described first image, with second image that the time for exposure also shorter than time for exposure of described first image photographed, and generates with described and rocks the corresponding information of rocking.

Specifically be that for example described information of rocking is the image deterioration function that rocks of described first integral image of expression.

Have, for example described information generating unit of rocking possesses the extraction unit that extracts a part of image from described first image and second image respectively, generates the described information of rocking based on each a part of image again.

More particularly, the for example described information generating unit of rocking is according to first function that will obtain to the frequency domain based on the image transform of described first image and second function that will obtain to the frequency domain based on the image transform of described second image, temporarily ask for the described image deterioration function on the frequency domain, by the processing that the constraints of function utilization regulation that the described image deterioration functional transformation on the frequency domain of will be tried to achieve is obtained on the spatial domain is revised, finally try to achieve described image deterioration function.

Further concrete is, the for example described information generating unit of rocking will and be handled as deterioration image and initial restored image respectively based on the image of described second image based on the image of described first image, utilize the described information of rocking of Fourier iteration Method.

And, the for example described information generating unit of rocking possesses the extraction unit that extracts a part of image from described first image and second image respectively, by generating described deterioration image and described initial restored image by each a part of image, thereby make each picture size of described deterioration image and described initial restored image less than the picture size of described first image.

Also have, for example this device for detecting rock also possesses holding unit, it keeps with image the demonstration based on the output of the described image unit before or after the photography of described first image, and the described information generating unit of rocking is continued to use with image described demonstration as described second image.

In addition, the for example described information generating unit of rocking is in the process that generates described deterioration image and described initial restored image according to described second image and described second image, at based on the image of described first image and based at least one side of the image of described second image, carry out noise and remove processing, with brightness degree between described first image and described second image than corresponding luminance standard processing, edge extracting handle and the picture size standardization corresponding with the picture size between described first image and described second image in more than one processing.

In addition, for example this device for detecting rock also possesses holding unit, it will keep as the 3rd image with image based on the demonstration of the output of the described image unit before or after the photography of described first image, and the described information generating unit of rocking generates the described information of rocking according to described first image, described second image and described the 3rd image.

In addition, for example described information generating unit of rocking generates the 4th image by described second image and described the 3rd image are weighted sum operation, and generates the described information of rocking according to described first image and described the 4th image.

Replace, the described information generating unit of rocking comprises the selected cell that the either party in described second image and described the 3rd image is chosen as the 4th image, and generating the described information of rocking according to described first image and described the 4th image, described selected cell carries out selection between described second image and described the 3rd image based in the following condition at least one: each edge strength of described second image and described the 3rd image, described second image and each time for exposure of described the 3rd image and the external information that sets.

Have, for example described information generating unit of rocking will and be handled as deterioration image and initial restored image respectively based on the image of described the 4th image based on the image of described first image, utilize the Fourier iterative method to calculate the described information of rocking again.

And, the for example described information generating unit of rocking possesses the extraction unit that extracts a part of image from described first image, second image and the 3rd image respectively, by generating described deterioration image and described initial restored image by each a part of image, thereby make each picture size of described deterioration image and described initial restored image less than the picture size of described first image.

Also have, for example also can form following such blur correcting device.This blur correcting device is according to other image generation unit, and it utilizes the described information of rocking that is generated by described device for detecting rock, generates the described correction image of rocking that has reduced described first image.

Have again, first camera head that the present invention relates to, it possesses described device for detecting rock; With described image unit.

And then, what the present invention relates to rocks detection method, detect rocking that first image that the photography by described image unit obtains comprised based on the output of image unit, based on described first image, with second image of the time for exposure photography also shorter, generate with described and rock the corresponding information of rocking than time for exposure of described first image.

Also have, the blur correcting device that the present invention relates to comprises: image is obtained the unit, it obtains first image by the photography that has utilized image unit, and obtains several short exposure images by the time for exposure repeatedly photography also shorter than the time for exposure of described first image; Second image generation unit, it generates 1 width of cloth image by described several short exposure images, with as second image; With the correcting process unit, it is based on described first image and described second image, and rocking of being comprised of described first image revised.

Specifically be, for example described second image generation unit selects 1 width of cloth short exposure image based in the following condition at least one from described several short exposure images, with as described second image, described condition is: the contrast of each short exposure edge of image intensity, each short exposure image, and each short exposure image with respect to the anglec of rotation of described first image.

And then for example described second image generation unit also utilizes the photography time difference of each short exposure image and described first image to carry out the selection of described second image.

Perhaps, for example described second image generation unit is by synthesizing the short exposure image more than 2 width of cloth in described several short exposure images, thereby generates described second image.

Perhaps, for example described second image generation unit comprises: select processing unit, it comes to select 1 width of cloth short exposure image from described several short exposure images based in the following condition at least one, and described condition is: the contrast of described each short exposure edge of image intensity, each short exposure image, and each short exposure image with respect to the anglec of rotation of described first image; Synthetic processing unit, the composograph after its generation is synthesized the above short exposure image of 2 width of cloth in described several short exposure images; And switch control unit, it plays a role by only making the either party in described selection processing unit and the described synthetic processing unit, thereby generates described 1 width of cloth short exposure image or described composograph is used as described second image; Described switch control unit recently determine to make described selection processing unit and described synthetic processing unit based on the S/N of each short exposure image which side play a role.

Have, for example described correcting process unit generates and rocks the corresponding information of rocking with the described of described first image, and based on the described information of rocking, described the rocking of described first image revised based on described first image and described second image again.

Replace, for example described correcting process unit is by synthesizing the luminance signal of described second image and the color signal of described first image, thereby described the rocking of described first image revised.

Replace, for example described correcting process unit carries out sharpening by utilizing described second image to described first image, thereby described the rocking of described first image revised.

The camera head that the present invention relates to, it comprises: described blur correcting device; With described image unit.

Have again, what the present invention relates to rocks modification method, comprise: image is obtained step, obtains first image by the photography that has utilized image unit, and obtains several short exposure images by the time for exposure repeatedly photography also shorter than the time for exposure of described first image; Second image generates step, generates 1 width of cloth image by described several short exposure images, with as second image; With the correcting process step,, rocking of being comprised of described first image revised based on described first image and described second image.

By the explanation of execution mode shown below, can further to define implication of the present invention and effect.Wherein Yi Xia execution mode only is an embodiment of the invention, and the meaning of the term of the present invention and each constitutive requirements is not defined in the content that following execution mode is put down in writing.

Description of drawings

Fig. 1 is the entire block diagram of the camera head that relates to of first embodiment of the invention.

Fig. 2 is that the hand that relates to of expression first embodiment of the invention is trembled and detected and hand is trembled the flow chart of the motion flow of correction.

Fig. 3 is the concept map of a part of the motion flow of presentation graphs 2.

Fig. 4 is the detail flowchart of the Fourier iterative method of Fig. 2.

Fig. 5 is the block diagram of formation of realizing the Fourier iterative method of Fig. 2.

Fig. 6 is that the hand that relates to of expression second embodiment of the invention is trembled and detected and hand is trembled the flow chart of the motion flow of correction.

Fig. 7 is the concept map of a part of the motion flow of presentation graphs 6.

Fig. 8 is the figure of processing and amplifying in length and breadth of the filter factor of the image restoration filter that is used for illustrating that second embodiment of the invention is carried out.

Fig. 9 is that the hand that relates to of expression third embodiment of the invention is trembled and detected and hand is trembled the flow chart of the motion flow of correction.

Figure 10 is the concept map of a part of the motion flow of presentation graphs 9.

Figure 11 (a) and (b) be the figure of the meaning of the weighting summation calculation process that is used for illustrating that third embodiment of the invention is carried out.

Figure 12 is that the hand that relates to of expression fourth embodiment of the invention is trembled and detected and hand is trembled the flow chart of the motion flow of correction.

Figure 13 is the concept map of a part of the motion flow of expression Figure 12.

Figure 14 realizes that hand that fifth embodiment of the invention relates to is trembled to detect and hand is trembled the formation block diagram of correction.

Figure 15 is used to illustrate that existing sum operation formula hand trembles the figure of correction.

Figure 16 is the block diagram of the formation of existing realization Fourier iterative method.

Figure 17 is the entire block diagram of the camera head that relates to of second embodiment of the invention.

Figure 18 be that the expression second embodiment of the invention relates to, from revise object images with reference to the figure that extracts the form of a plurality of little images the image.

Figure 19 be that the expression second embodiment of the invention relates to, from revising object images and with reference to the figure of the little image in correspondence with each other that extracts the image.

Figure 20 is figure that the expression second embodiment of the invention relates to, can detect the form of the straight line that extends along edge direction by the little image that extracts being implemented edge extracting handle from the reference image.

Figure 21 is the figure of the straight line that second embodiment of the invention relates to, overlapping demonstration is extended along edge direction on the image of Figure 19.

Figure 22 is the figure that is used to represent the distribution of the brightness value on the orthogonal direction of straight line of Figure 21.

Figure 23 is the figure that is used to represent the distribution of the brightness value on the orthogonal direction of straight line of Figure 21.

Figure 24 be that the expression second embodiment of the invention relates to, as the figure of the spatial filter of the smooth function that generates based on the distribution of brightness value.

Figure 25 is that the hand that the expression second embodiment of the invention relates to is trembled the flow chart that detects motion flow.

Figure 26 is the entire block diagram of the camera head that relates to of third embodiment of the invention.

Figure 27 is the flow chart that hand that expression sixth embodiment of the invention camera head that relate to, Figure 26 carries out is trembled the motion flow of correction.

Figure 28 is the flow chart that hand that expression seventh embodiment of the invention camera head that relate to, Figure 26 carries out is trembled the motion flow of correction.

Figure 29 is the flow chart that hand that expression eighth embodiment of the invention camera head that relate to, Figure 26 carries out is trembled the motion flow of correction.

Figure 30 is light measuring circuit in the expression eighth embodiment of the invention camera head that relate to, that be located at Figure 26 and the figure of LUT.

Figure 31 be that the expression ninth embodiment of the invention relates to, be used to generate flow chart with reference to the calculating action of first evaluation of estimate of image.

Figure 32 be used to illustrate that ninth embodiment of the invention relates to, be used to generate figure with reference to the computational methods of first evaluation of estimate of image.

Figure 33 be that the expression ninth embodiment of the invention relates to, be used to generate flow chart with reference to the calculating action of second evaluation of estimate of image.

Figure 34 (a) and (b) be respectively the figure that represents short exposure image clearly and rock big and unsharp short exposure image; All be the figure that is used to illustrate the meaning of the action corresponding with Figure 33.

It (b) is respectively the figure of the expression brightness histogram corresponding with Figure 34 (a) and short exposure image (b) that Figure 35 (a) reaches.

Figure 36 be used to illustrate that ninth embodiment of the invention relates to, be used to generate figure with reference to the computational methods of the 3rd evaluation of estimate of image.

Figure 37 is the flow chart that expression tenth embodiment of the invention hand that relate to, first modification method is trembled the motion flow of correcting process.

Figure 38 is the flow chart that expression tenth embodiment of the invention hand that relate to, second modification method is trembled the motion flow of correcting process.

Figure 39 is the concept map that the hand corresponding with Figure 38 trembled correcting process.

Figure 40 is the flow chart that expression tenth embodiment of the invention hand that relate to, the 3rd modification method is trembled the motion flow of correcting process.

Figure 41 is the concept map that the hand corresponding with Figure 40 trembled correcting process.

Figure 42 is the figure of the Gaussian Profile of the one dimension that relates to of expression tenth embodiment of the invention.

Figure 43 is used to illustrate that the hand corresponding with Figure 40 tremble the figure of the effect of correcting process.

Figure 44 is the figure of the example of the light stream between that the expression eleventh embodiment of the invention relates to, each short exposure image and adjacent short exposure image.

Figure 45 is the figure of other examples of the light stream between expression eleventh embodiment of the invention short exposure image that relate to, adjacent.

Figure 46 is the figure of another example of the light stream between expression eleventh embodiment of the invention short exposure image that relate to, adjacent.

Embodiment

Below, specify embodiments of the present invention with reference to accompanying drawing.In each figure of institute's reference, to give identical mark with a part, in principle to omitting with the repeat specification of same part correlation.

" first execution mode "

At first, first execution mode of the present invention is described.Fig. 1 is the entire block diagram of the camera head 1 that relates to of first embodiment of the invention.The camera head 1 of Fig. 1 is the Digital Video that can take and write down the digital camera of rest image or can take and write down rest image and moving image.

Camera head 1 possesses image pickup part 11, AFE (Analog Front End) 12, master control part 13, internal storage 14, display part 15, recording medium 16, operating portion 17, exposure control part 18 and hand and trembles detection/correction portion 19.Have shutter release button 17a in the operating portion 17.

Image pickup part 11 has the driver (all not shown) that optical system, aperture, the imaging apparatus by formations such as CCD (Charge Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor) imageing sensors, control optical system or aperture are used.Driver is controlled the zoom ratio of optical system or the aperture of focal length and aperture according to the AF/AE control signal from master control part 13.Imaging apparatus carries out light-to-current inversion to the optical image of the subject of expression by optical system and aperture incident, will output to AFE12 by the signal of telecommunication that this light-to-current inversion obtains.

AFE12 amplifies the analog signal of exporting from image pickup part 11 (imaging apparatus), and amplified analog signal is converted to digital signal.AFE12 outputs to master control part 13 successively with this digital signal.

Master control part 13 possesses CPU (Central Processing Unit), ROM (Read OnlyMemory) and RAM (Random Access Memory) etc., also works as the vision signal handling part.Master control part 13 is according to the output signal of AFE12, generates the vision signal of the image that expression photographs by image pickup part 11 (below be also referred to as " photographs ").Have, master control part 13 also possesses the function as the indicative control unit that the displaying contents of display part 15 is controlled, and display part 15 is shown required control again.

Internal storage 14 is formed by SDRAM (Synchronous Dynamic Random AccessMemory) etc., temporarily stores the various data that generate in the camera head 1.Display part 15 is the display unit that are made of LCD etc., under the control of master control part 13, and image that photographs in the frame before showing or the image that is stored in storage medium 16 etc.Recording medium 16 is nonvolatile memories such as SD (Secure Digital) storage card, stores photographs etc. under the control of master control part 13.

The operation that operating portion 17 is accepted from the outside.Content of operation at operating portion 17 is communicated to master control part 13.Shutter release button 17a is used to indicate the photography of rest image and the button of record.

The time for exposure of each pixel of exposure control part 18 control imaging apparatuss is so that exposure the best of the imaging apparatus of image pickup part 11.Have, providing under the situation of exposure time control signal to exposure control part 18 from master control part 13, exposure control part 18 is controlled the time for exposure according to this exposure time control signal.

Comprise the photograph mode of the shooting that can carry out rest image or moving image and record in the pattern of camera head 1 and on display part 15 the regeneration displayed record in the rest image of recording medium 16 or the regeneration mode of moving image.According to operation, implement the conversion between each pattern at operating portion 17.

In photograph mode, image pickup part 11 was taken successively with the frame period (for example 1/60 second) of regulation.Master control part 13 generates straight-through (through) according to the output of image pickup part 11 and shows and use image in each frame, and makes the display part 15 straight-through demonstration image that obtains successively of update displayed successively.

In photograph mode, when supressing shutter release button 17a, master control part 13 will represent that the image data storage of 1 width of cloth photographs is to recording medium 16 and internal storage 14 (that is, making it to store).This photographs is possible comprise hand to tremble the image that rocks that is caused, then according to the correction indication that provides via operating portion 17 grades or tremble detection/correction portion 19 by hand automatically and revise.Therefore, below will follow above-mentioned 1 width of cloth photographs of pressing of shutter release button 17a to be called " correction object images " especially.Have again and since by hand tremble/correction portion 19 can detect and revise rocking that object images comprised, so also the correction object images can be referred to as " detected object image ".

Detection/the correction portion 19 of trembling hand does not adopt hands such as angular-rate sensor to tremble detecting sensor, but the view data that obtains according to output signal by image pickup part 11, detect and revise rocking that object images comprised, by revising revising object images, remove or reduced the correction image of rocking thereby generate according to this testing result.

Below, tremble the embodiment of the function of detection/correction portion 19 as describing hand in detail, first～the 5th embodiment is described.The short of contradiction of item of certain embodiment record just can be applicable to other embodiment.Have again, in the description of first～the 4th embodiment (and second execution mode described later), " memory " of so-called memory image etc. is meant that internal storage 14 or hand tremble the not shown memory that is provided with in detection/correction portion 19 (hand is trembled detection/correction portion 20 in second execution mode).

[first embodiment]

At first first embodiment is described.With reference to Fig. 2 and Fig. 3.Fig. 2 is that the hand that relates to of expression first embodiment is trembled and detected and hand is trembled the flow chart of the motion flow of correction.Fig. 3 is the concept map of the part of this motion flow.Describe according to the flow chart of Fig. 2 flow process this action.

In photograph mode, if press shutter release button 17a, the photography that then exposes usually is stored in (step S1 and S2) on the memory with the correction object images that generates thus.Below this correction object images among first embodiment be called revise object images A1.

Then, in step S3 to obtaining revising time for exposure T1 and the threshold value T when the object images A1 _THCompare, at time for exposure T1 less than threshold value T _THThe time, think to revise and do not contain (perhaps extremely slight contains) hand in the object images and tremble rocking of causing, do not carry out hand and tremble correction, finish the processing of Fig. 2.As threshold value T _TH, for example adopt hand to tremble the boundary time for exposure.It is to be judged as to ignore the boundary time for exposure that hand is trembled that hand is trembled the boundary time for exposure, can be according to focal distance f _DInverse calculate.

At time for exposure T1 greater than threshold value T _THThe time, move to step S4, then short time exposure photography is carried out in the exposure photography usually, will be stored on the memory as the reference image by the photographs that this short time exposure photography obtains.Below among first embodiment this is called with reference to image A 2 with reference to image.Though revise object images A1 and obtain (promptly in adjacent frame, obtaining) by sequence photography with reference to image A 2, but the exposure control part 18 of 13 couples of Fig. 1 of master control part is controlled, so that obtain with reference to the time for exposure when the image A 2 also shorter than time for exposure T1.For example the time for exposure with reference to image A 2 is T1/4.Have again, revise object images A1 and equate with picture size with reference to image A 2.

Then, in step S5, among revising object images A1, extract distinctive zonule, the image in this zonule that extracts is stored on the memory as little image A 1a.So-called distinctive zonule is meant: the rectangular area of marginal element many (in other words contrast is more intense) in the extraction source image, for example utilize Harris's (Harris) corner detection device, the zonule of 128 * 128 pixels is extracted as distinctive zonule.Like this, select distinctive zonule according to the size (perhaps contrast tolerance) of the edge of image composition in this zonule.

Next, in step S6, the zonule of extracting from reference image A 2 and extracting from revise object images A1 is the zonule of same coordinate, and the image in the zonule that will extract from reference image A 2 is stored on the memory as little image A 2a.Equate from the centre coordinate (centre coordinate of correction object images A1) of the zonule that correction object images A1 extracts with from the centre coordinate (with reference to the centre coordinate of image A 2) of the zonule that reference image A 2 extracts, revise object images A1 and also equate, so the picture size of two zonules is also equal with picture size with reference to image A 2.

Because shorter with reference to the time for exposure of image A 2, so the signal to noise ratio of little image A 2a (hereinafter referred to as the S/N ratio) is lower.Therefore, in step S7, carry out noise removal process at little image A 2a.Will except that the little image A 2a behind the denoising as little image A 2b.Noise is removed by utilizing linear filter (weighted average filter etc.) or nonlinear filter (median filter (medianfilter) etc.) that little image A 2a is carried out filtering and is undertaken.

Little image A 2b is low-light level, therefore in step S8, the brightness degree of little image A 2b is increased.Promptly, for example so that the mode that the brightness degree of little image A 2b equates with the brightness degree of little image A 1a (mode that the mean flow rate of little image A 2b is equated with the mean flow rate of little image A 2a) carries out the luminance standard processing that the brightness value of so-called each pixel with little image A 2b multiply by steady state value.To so make the little image A 2b after brightness degree increases be called little image A 2c.

The little image A 1a that will obtain like this as deterioration image and basis that little image A 2c is handled as initial restored image on (step S9), in step S10, implement the Fourier iterative method, ask for the image deterioration function.

When implementing the Fourier iterative method,, this initial restored image is called initial restored image though initial restored image (initial value of restored image) need be provided.

As the image deterioration function, ask for an expanded function (Point Spread Function; Hereinafter referred to as PSF).Because the rock operator (operator) or the spatial filter that are depicted in the track on the image and are weighted of camera head 1 are called as PSF, generally be used as the Mathematical Modeling that hand is trembled according to desirable some picture.The PSF that obtains at zonule A1a because hand is trembled entire image caused the same deterioration, so can be used as the PSF at whole correction object images A1.

The Fourier iterative method is to be removed or to have reduced the method (reference literature for example: G.R.Ayers and J.C.Dainty of the restored image of deterioration according to the deterioration image that comprises deterioration, " Iterativeblind deconvolution method and its applications ", OPTICS LETTERS, 1988, Vol.13, No.7, p.547-549).With reference to Fig. 4 and Fig. 5, describe this Fourier iterative method in detail.Fig. 4 is the detail flowchart of processing of the step S10 of Fig. 2.Fig. 5 is a block diagram of implementing the position of Fourier iterative method.

At first, in step S101 restored image is made as f ', f ' is made as initial restored image with this restored image.That is,, use above-mentioned initial restored image (being little image A 2c in the present embodiment) as initial restored image f '.Then, in step S102, deterioration image (being A1a in the present embodiment) is made as g.And, will be stored in (step S103) on the memory as G to the image that deterioration image g carried out Fourier transform.For example, when the picture size of initial restored image and deterioration image was 128 * 128 pixels, f ' and g can show as the matrix with matrix size of 128 * 128.

Then, in step S110, ask for the F ' that restored image f ' was carried out Fourier transform, and then in step S111, calculate H by following formula (1).H is equivalent to PSF was carried out the function of Fourier transform.In formula (1), F ' ^*Be the conjugate complex matrix of F ', α is a constant.

$H=\frac{G\·{F^{\′}}^{*}}{{\left|F^{\′}\right|}^2+\mathrm{\α}}...\left(1\right)$

Then, in step S112,, obtain PSF by H is carried out inverse Fourier transform.To be made as h at this PSF that obtains.Then, in step S113, after with the constraints of following formula (2a) PSF h being revised, revise with the constraints of formula (2b).

$h(x,y)=\left\{\begin{array}{cc}1& :h(x,y)>1\\ h(x,y)& :0\&le;h(x,y)\&le;1\\ 0& :h(x,y)<0\end{array}\right....\left(2a\right)$

∑h(x,y)=1 …(2b)

Therefore PSF h shows as the matrix of two dimension, and (x y) represents each key element of this matrix with h.Each key element of PSF should be for more than 0 and 1 following value.Therefore, in step S113, judging whether each key element of PSF is more than 0 and below 1, is more than 0 and the value of 1 following key element is directly continued to use.And be the value of this key element to be modified to 1 during greater than 1 key element, and for to be modified to 0 less than 0 o'clock value with this key element.The correction that this constraints that is based on formula (2a) is carried out.And, PSF is carried out standardization, so that the summation of each key element of this revised PSF is 1.This standardization is based on the constraints of formula (2b) and the correction carried out.

To be made as h ' according to the PSF that formula (2a) and (2b) constraints carried out revising.

Next, in step S114, ask for the H ' that PSF h ' was carried out Fourier transform, and then in step S115, calculate F according to following formula (3).F is equivalent to restored image was carried out the value of Fourier transform.In formula (3), H ' ^*Be the conjugate complex matrix of H ', β is a constant.

$F=\frac{G\&CenterDot;{H^{\&prime;}}^{*}}{{\left|H^{\&prime;}\right|}^2+\mathrm{\&beta;}}...\left(3\right)$

Then, in step S116, by F is carried out inverse fourier transform, thereby obtain restored image.To be made as f at the restored image of this acquisition.Then, in step S117, restored image f is revised, revised restored image is made as f ' again with the constraints of following formula (4).

$f(x,y)=\left\{\begin{array}{cc}255& :f(x,y)>255\\ f(x,y)& :0\&le;f(x,y)\&le;255\\ 0& :f(x,y)<0\end{array}\right....\left(4\right)$

Therefore restored image f shows as the matrix of two dimension, and (x y) represents each key element of this matrix with f.The current pixel value of representing each key element of deterioration image and restored image with 0 to 255 digital value that is made as.Like this, each key element (being each pixel value) of the matrix of expression restored image f should be got more than 0 and 255 following values.Therefore, judge that in step S117 whether each key element of the matrix of representing restored image f is more than 0 and below 255, be more than 0 and the value of 255 following key elements is directly continued to use, and when existing, the value of this key element is modified to 255, and when existing, the value of this key element is modified to 0 less than 0 key element greater than 255 key element.This is the correction of carrying out according to the constraints of formula (4).

Then, in step S118, by judging whether to satisfy the condition of convergence, thereby the convergence of carrying out iterative processing is judged.

For example, the index that the absolute value of the difference of the F ' that obtains before up-to-date F ' and one is judged as convergence.When this index, be judged as and satisfy the condition of convergence, and situation judges to not satisfying the condition of convergence in that no when following in the threshold value of regulation.

Satisfying under the situation of the condition of convergence, will be made as final PSF the function that up-to-date H ' has carried out inverse fourier transform.That is, this up-to-date H ' having been carried out the function behind the inverse fourier transform, should be the PSF that tries to achieve in the step S10 of Fig. 2.Do not satisfying under the situation of the condition of convergence, turning back to step S110, each of repeating step S110～S118 handled.In each repetitive process of handling of each step S110～S118, f ', F ', H, h, h ', H ', F and f (with reference to Fig. 5) are updated to last look successively.

Index as convergence is judged also can adopt other indexs.For example, the index that the absolute value of the difference of the H ' that obtains before up-to-date H ' and one can be judged as convergence, judging that the above-mentioned condition of convergence is set up still is false.Also have, for example also can be with the index of utilizing the correction among above-mentioned formula (2a) and the step S113 (2b) or utilizing the correction among the step S117 of above-mentioned formula (4) to judge as convergence, judge the establishment of the above-mentioned condition of convergence or be false.This is because if convergence is tended in iterative processing, then these corrections diminish.

Have again, also can reach under the situation of stipulated number, be judged as and restrain, end process and do not calculate final PSF in the number of repetition of the circular treatment that constitutes by step S110～S118.Do not revise the correction of object images this moment.

Turn back to the explanation of each step of Fig. 2.After in step S10, calculating PSF, move on to step S11.In step S11, each key element of the inverse matrix of the PSF that obtains among the step S10 is asked for as each filter factor of image restoration filter.This image restoration filter is the filter that is used for being obtained by the deterioration image restored image.In fact, be equivalent to each filter factor of image restoration filter, therefore can directly utilize the result in the computational process of the Fourier iterative method among the step S10 with each key element of the matrix of following formula (5) expression of the part that is equivalent to above-mentioned formula (3) the right.Wherein, the H ' in the formula (5) ^*And H ' is the H ' that obtained before the condition of convergence of step S118 is set up ^*And the H ' (H ' that promptly finally obtains ^*And H ').

$\frac{H^{{\&prime;}^{*}}}{{\left|H^{\&prime;}\right|}^2+\mathrm{\&beta;}}...\left(5\right)$

In step S11, try to achieve after each filter factor of image restoration filter, move on to step S12, utilize this image restoration filter to carry out filtering, thereby generation is removed or has been reduced and revised the filtering image that rocks that object images A1 is comprised to revising object images A1.Since comprise the bell signal (ringing) of following filtering in the filtering image, be removed by step S13, thus generate final correction image.

[second embodiment]

Then second embodiment is described.

As mentioned above, under photograph mode, image pickup part 11 was photographed successively with the frame period (for example 1/60 second) of regulation, and master control part 13 generates straight-through the demonstration according to the output of image pickup part 11 and uses image in each frame, made the display part 15 straight-through demonstration image that obtains successively of update displayed successively.

The straight-through demonstration with image is the image that moving image is used, its picture size liken to into the size of the correction object images of rest image also little.Revise object images and generate, and straight-through the demonstration with image is that picture element signal to each pixel of this effective camera watch region carries out extracting at interval and generation according to the picture element signal of the whole pixels in effective camera watch region of the imaging apparatus of image pickup part 11.Under the situation that photographs is thought of as the image that generates by the picture element signal of the whole pixels in effective camera watch region, the correction object images is according to pressing of shutter release button 17a and by the photographed images itself of exposing photography and writing down usually, the straight-through interval extraction image that is equivalent to the photographs of each frame with image that shows.

In a second embodiment, will be based on the straight-through demonstration of the photographs of the frame before or after the frame that take to revise object images with image as with reference to image.Below, the situation of the straight-through demonstration of the frame before the frame of revising object images with image taken in the illustration utilization.

With reference to Fig. 6 and Fig. 7.Fig. 6 is that the hand that relates to of expression second embodiment is trembled and detected and hand is trembled the flow chart of the motion flow of correction.Fig. 7 is the concept map of the part of this motion flow of expression.With reference to Fig. 6 this motion flow is described.

In photograph mode, as mentioned above, generate straight-through the demonstration at each frame and use image, at this image of updated stored and on display part 15, carry out update displayed (step S20) successively on the memory.And, if press shutter release button 17a, the photography that then exposes usually, the correction object images (step S21 and S22) that storage generates thus.Below this correction object images among second embodiment be called revise object images B1.Be stored in straight-through demonstration image on the memory this moment and be the straight-through demonstration image that photography obtains that passes through of taking the frame frame before of revising object images B1, below be referred to as with reference to image B 3.

Then, in step S23, time for exposure T1 and threshold value T when obtaining revising object images B1 _THCompare.At time for exposure T1 less than above-mentioned threshold value T _TH(for example be focal distance f _DInverse) situation under, think to revise and do not contain (perhaps containing extremely slightly) hand in the object images and tremble rocking of causing, do not carry out hand and tremble correction, finish the processing of Fig. 6.

At time for exposure T1 greater than threshold value T _THSituation under, move to step S24, to time for exposure T1 and obtain comparing with reference to the time for exposure T3 when the image B 3.Under the situation of T1≤T3, think and tremble greatly with reference to the hand in the image B 3, do not carry out hand and tremble correction and the processing of end Fig. 6.Situation at T1＞T3 is displaced downwardly to step S25, utilizes Harris's corner detection device etc., extracts distinctive zonule among reference image B 3, and the image in this zonule that extracts is stored on the memory as little image B 3a.That describes among the meaning of distinctive zonule and extracting method and first embodiment is same.

Then, corresponding with the coordinate of little image B 3a in step S26, from revise object images B1, extract the zonule.And, will according to the picture size of revising object images B1 and reference image B 3 than and image after the image in this zonule that extracts from revise object images B1 dwindled stores on the memory as little image B 1a.That is to say, when generating little image B 1a, carry out the standardization of picture size, so that little image B 1a equates with the picture size of B3a.

Suppose so that revise the mode that object images B1 equates with picture size with reference to image B 3 and amplify with reference to image B 3, then consistent with the centre coordinate (with reference to the centre coordinate of image B 3) of the zonule of extracting from reference image B 3 from the centre coordinate (revising the centre coordinate of object images B1) of revising the zonule that object images B1 extracts.Wherein, be actually different with picture size, so the picture size of two zonules is according to revising that object images B1 compares with the picture size of reference image B 3 and different with reference to image B 3 owing to revise object images B1.Therefore, make from the picture size of the zonule that reference image B 3 is extracted with from the ratio of the picture size of revising the zonule that object images B1 extracts and consistent with the ratio of the picture size of revising object images B1 with reference to the picture size of image B 3.And, finally dwindle image in the zonule of correction object images B1 extraction, thereby obtain little image B 1a by the mode that equates with the picture size of B3a with little image B 1a.

Then, in step S27, little image B 1a and B3a are implemented the edge extracting processing, obtain little image B 1b and B3b.For example, adopt rim detection operator arbitrarily, thereby generate the edge extracting image of little image B 1a, this edge extracting image is made as little image B 1b by each pixel to little image B 1a.For little image B 3b too.

Then, in step S28, little image B 1b and B3b are carried out the luminance standard processing.That is to say, the brightness value of each pixel of little image B 1b and/or B3b be multiply by steady state value, so that little image B 1b equates (mean flow rate of little image B 1b equates with the mean flow rate of little image B 3b) with the brightness degree of B3b.Little image B 1b after this luminance standard processing and B3b are made as little image B 1c and B3c.

Because as the straight-through demonstration image of reference image B 3 are images that moving image is used, so for example so that the image processing that its mode with the tone that is applicable to moving image is used by moving image and obtaining.On the other hand, be the rest image of following pressing of shutter release button 17a to photograph owing to revising object images B1, so its image processing of using by rest image and obtaining.Because the difference between two image processing, even subject is identical, tone etc. also can be different between little image B 1a and B3a.Because this difference can be handled by edge extracting and remove, and handles so carry out edge extracting in step S27.And then, to revise object images B1 and roughly remove owing to can handle with reference to the luminance difference of image B 3 by edge extracting, though so can suppress the influence (promptly improve and rock accuracy of detection) of luminance difference by the edge extracting processing, but owing to can not remove fully, so next in step S28, carry out the luminance standard processing.

The little image B 1c that will obtain as mentioned above as deterioration image and basis that little image B 3c is handled as initial restored image on (step S29), move to step S10, each of execution in step S10, S11, S12 and S13 handled successively.

Same among each contents processing of step S10～S13 and first embodiment.Wherein, (and the PSF that obtains by step S10) is applicable to the picture size of moving image because each filter factor of the image restoration filter that obtains by step S10 and S11, so so that the mode of the picture size of its suitable rest image is carried out processing and amplifying in length and breadth.

For example, show that straight-through picture size ratio with image and correction object images is 3: 5 and is of a size of by the image restoration filter that step S10 and S11 obtain under 3 * 3 the situation, when each filter factor shown in the mark 101 that calculates Fig. 8, by processing and amplifying in length and breadth, generate each filter factor of the image restoration filter of 5 * 5 sizes shown in the mark 102 of Fig. 8.And each filter factor of the image restoration filter of 5 * 5 sizes is made as each filter factor that obtains by step S11 the most at last.Also have, in the example corresponding with the mark 102 of Fig. 8, though will be made as 0 by the filter factor of interpolation by processing and amplifying in length and breadth, also can calculate by linear interpolation etc. should be by the filter factor of interpolation.

In step S11, try to achieve after each filter factor of image restoration filter, move to step S12, by utilizing this image restoration filter to carry out filtering, remove or reduced and revised the filtering image that rocks that object images B1 is comprised thereby generate to revising object images B1.Since may contain the bell signal of following filtering in the filtering image, remove this bell signal by step S13, thus generate final correction image.

[the 3rd embodiment]

Then, the 3rd embodiment is described.With reference to Fig. 9 and Figure 10.Fig. 9 is that the hand that relates to of expression the 3rd embodiment is trembled and detected and hand is trembled the flow chart of the motion flow of correction.Figure 10 is the concept map of the part of this motion flow of expression.This motion flow of flowchart text according to Fig. 9.

In photograph mode, generate straight-through the demonstration at each frame and use image, update stored in it on memory successively and update displayed in display part 15 (step S30).And, if press shutter release button 17a, the photography that then exposes usually, the correction object images (step S31 and S32) that storage generates thus.Below this correction object images among the 3rd embodiment be called revise object images C1.This straight-through demonstration image that is stored on the memory constantly is the straight-through demonstration image that photography obtains that passes through that take to revise frame before the frame of object images C1, and below is referred to as with reference to image C 3.

Then, in step S33, time for exposure T1 and threshold value T when obtaining revising object images C1 _THCompare.At time for exposure T1 less than above-mentioned threshold value T _TH(focal distance f for example _DInverse) situation under, think to revise and do not contain (or containing extremely slightly) hand in the object images and tremble rocking of causing, do not carry out hand and tremble and revise and finish the processing of Fig. 9.

At time for exposure T1 greater than threshold value T _THSituation under, time for exposure T1 and the time for exposure T3 when obtaining with reference to image C 3 are compared, under the situation of T1≤T3, think and tremble greatly, below carry out to tremble and detect and hand is trembled correction (promptly carrying out the processing same with S4～S13 of Fig. 2) with the same hand of first embodiment with reference to the hand in the image C 3.On the other hand, be displaced downwardly to step S34, carrying out short time exposure photography after the exposure photography usually, will be stored on the memory as reference image C 2 by this short time exposure photographs that obtains of photographing in the situation of T1＞T3.In Fig. 9, the record of omission and the more relevant contents processing of T1 and T3 below describes the situation of T1＞T3.

Though revise object images C1 and obtain (promptly in adjacent frame, obtaining) by sequence photography with reference to image C 2, but the exposure control part 18 of 13 couples of Fig. 1 of master control part is controlled, so that the time for exposure when obtaining with reference to image C 2 is also shorter than time for exposure T1.For example, the time for exposure of establishing with reference to image C 2 is T3/4.Have again, revise object images C1 and equate with picture size with reference to image C 2.

Move to step S35 after the step S34, utilize Harris's corner detection device etc., among reference image C 3, extract distinctive zonule, the image in this zonule that extracts is stored on the memory as little image C 3a.That describes among the meaning of distinctive zonule and extracting method and first embodiment is identical.

Then, corresponding with the coordinate of little image C 3a in step S36, from revise object images C1, extract the zonule.And, will be according to revising the picture size ratio of object images C1 with reference image C 3, the image after the image in this zonule that extracts from revise object images C1 is dwindled stores on the memory as little image C 1a.That is to say, when generating little image C 1a, carry out the standardization of picture size, so that little image C 1a equates with the picture size of C3a.Equally, corresponding with the coordinate of little image C 3a, from reference image C 2, extract the zonule.And, will be according to the picture size ratio of reference image C 2 with reference image C 3, the image after the image in this zonule that extracts from reference image C 2 dwindled stores on the memory as little image C 2a.Obtain by revising object images C1 (or with reference to image C 2) that illustrated among the method for little image C 1a (or little image C 2a) and second embodiment to obtain the method (the step S26 of Fig. 6) of little image B 1a same by revising object images B1.

Then, in step S37, be benchmark with little image C 3a, little image C 2a is carried out the luminance standard processing.That is to say, so that the mode that little image C 3a equates with the brightness degree of C2a (mode that the mean flow rate of little image C 3a equates with the mean flow rate of C2a), the brightness value of each pixel of little image C 2a be multiply by steady state value.Little image C 2a after this luminance standard processing is made as little image C 2b.

After the processing of step S37, move to step S38.In step S38, at first generate the difference image of little image C 3a and little image C 2b.Only the pixel value of difference image is taken as value beyond 0 at the part that there are differences between little image C 3a and the C2b.And, the pixel value of each pixel of difference image as weight coefficient, by little image C 3a and C2b are weighted sum operation, thereby is generated little image C 4a.

With I _D(p, the q) pixel value of each pixel of expression difference image is with I ₃(p, the q) pixel value of each pixel of the little image C 3a of expression is with I ₂(p, the q) value of each pixel of the little image C 2b of expression is with I ₄(p, q) pixel value of each pixel of the little image C 4a of expression, then I ₄(p q) can be represented by following formula (6).At this, k is a constant, and p and q represent horizontal coordinate and the vertical coordinate in difference image or each the little image.

I ₄(p，q)＝k·I _D(p，q)·I ₂(p，q)+(1-k)·I _D(p，q)·I ₃(p，q) …(6)

By explanation described later as can be known, little image C 4a is used as the image that rocks corresponding PSF of calculating and correction object images C1.In order to obtain good PSF, need in little image C 4a, suitably preserve the marginal portion in advance.Certainly, the S/N of little image C 4a can obtain good PSF more than high more.Generally,, then can improve the S/N ratio,, certainly obtain little image C 4a, but by this sum operation, the marginal portion thickens, and can't obtain good PSF therefore by little image C 3a and C2b are carried out sum operation if a plurality of images are carried out sum operation.

Therefore, as mentioned above, generate little image C 4a by the weighting summation computing corresponding with the pixel value of difference image.Reach the meaning of this weighting summation computing that (b) remarks additionally with reference to Figure 11 (a).Because little image C 3a is also longer than the time for exposure of little image C 2b, thus shown in Figure 11 (a), when having taken same border body the former rock common rocking greatly than the latter.Therefore, if merely two little images are carried out sum operation, then shown in Figure 11 (a), edge part thickens, but shown in Figure 11 (b), if the pixel value according to the difference image between two little images is weighted sum operation to two little images, then can preserve edge part more well.This be because: rocking in the different part 110 (parts that the edge part deterioration is different) that produces greatly I because of little image C 3a _D(p q) becomes big, and the weighting of little image C 2b increases, and is difficult to reflect the big edge part deterioration of little image C 3a among the little image C 4a.On the contrary, in non-different part 111 because long little image C 3a of time for exposure to add adaptability in tactics big, so also can obtain to improve the effect (reduction noise effects) of S/N ratio.

Then, in step S39, be benchmark with little image C 1a, little image C 4a is carried out the luminance standard processing.That is to say, so that the mode that little image C 1a equates with the brightness degree of C4a (mode that the mean flow rate of little image C 1a equates with the mean flow rate of C4a), the brightness value of each pixel of little image C 4a be multiply by steady state value.Little image C 4a after this luminance standard processing is made as little image C 4b.

The little image C 1a that will obtain as mentioned above as deterioration image and basis that little image C 4b is handled as initial restored image on (step S40), move to step S10, each of execution in step S10, S11, S12 and S13 handled successively.

Same among each contents processing of step S10～S13 and first embodiment.Wherein, (and the PSF that obtains by step S10) is applicable to the picture size of moving image because each filter factor of the image restoration filter that obtains by step S10 and S11, so so that the mode of the picture size of its suitable rest image is carried out processing and amplifying in length and breadth.It is same that this illustrates among processing and amplifying and second embodiment in length and breadth.

In step S11, try to achieve after each filter factor of image restoration filter, move to step S12, by utilizing this image restoration filter to carry out filtering, remove or reduced and revised the filtering image that rocks that object images C1 is comprised thereby generate to revising object images C1.Since may contain the bell signal of following filtering in the filtering image, remove this bell signal by step S13, thus generate final correction image.

[the 4th embodiment]

Then the 4th embodiment is described.With reference to Figure 12 and Figure 13.Figure 12 is that the hand that relates to of expression the 4th embodiment is trembled and detected and hand is trembled the flow chart of the motion flow of correction.Figure 13 is the concept map of the part of this motion flow of expression.Describe according to the flow chart of Figure 12 flow process this action.

In the 4th embodiment, at first each of implementation step S50～S56 handled.Each contents processing of step S50～S56, since same with the contents processing of step S30～S36 (with reference to Fig. 9) among the 3rd embodiment, the explanation that repeat the Therefore, omited.Wherein, the correction object images C1 among the 3rd embodiment and replace with in the 4th embodiment with reference to image C 2 and C3 and to revise object images D1 and with reference to image D2 and D3.Have, for example the time for exposure with reference to image D2 is made as T1/4 again.

Through step S50～S56, obtain based on revising object images D1 and, moving to step S57 then with reference to little image D1a, D2a and the D3a of image D2 and D3.

In step S57, the little image of either party among little image D2a and the D3a is chosen as little image D4a.Selection can be carried out based on various indexs.

For example, the edge strength of little image D2a and the edge strength of little image D3a are compared, the little image that edge strength is bigger is chosen as little image D4a.Little image D4a is as the basis of the initial restored image of Fourier iterative method, and this is because consider: the big image of edge strength, the deterioration of its edge part is few more, and is therefore preferred as initial restored image.For example, by each pixel of little image D2a is suitable for the rim detection operator of regulation, thereby generate the edge extracting image of little image D2a, with the summation of the pixel value of this edge extracting image edge strength as little image D2a.Equally also can calculate the edge strength of little image D3a.

Also have, for example also can compare, the little image corresponding with short exposure time more is chosen as little image D4a to time for exposure of reference image D2 with reference to time for exposure of image D3.This is because consider: short image of time for exposure, the deterioration of its edge part is few more, preferably as initial restored image.Have again, for example also can from little image D2a and D3a, select little image D4a according to predefined selection information (external information) such as operating portion 17 by Fig. 1.Also have, also can carry out this selection according to the desired value that has made up above-mentioned edge strength, time for exposure and the information of selection.

Then, in step S58, be benchmark with little image D1a, little image D4a is carried out the luminance standard processing.That is to say, so that the mode that little image D1a equates with the brightness degree of D4a (mode that the mean flow rate of little image D1a equates with the mean flow rate of little image D4a), the brightness value of each pixel of little image D4a be multiply by steady state value.Little image D4a after this luminance standard processing is made as little image D4b.

The little image D1a that will obtain as mentioned above as deterioration image and basis that little image D4b is handled as initial restored image on (step S59), move to step S10, each of execution in step S10, S11, S12 and S13 handled successively.

Same among each contents processing of step S10～S13 and first embodiment.Wherein, (and the PSF that obtains by step S10) is applicable to the picture size of moving image because each filter factor of the image restoration filter that obtains by step S10 and S11, so so that the mode of the picture size of its suitable rest image is carried out processing and amplifying in length and breadth.It is same that this illustrates among processing and amplifying and second embodiment in length and breadth.

In step S11, try to achieve after each filter factor of image restoration filter, move to step S12, by utilizing this image restoration filter to carry out filtering, remove or reduced and revised the filtering image that rocks that object images D1 is comprised thereby generate to revising object images D1.Since may contain the bell signal of following filtering in the filtering image, remove this bell signal by step S13, thus generate final correction image.

[the 5th embodiment]

Then the 5th embodiment is described.In the 5th embodiment, tremble and detect and formation that hand is trembled correction describes being implemented in the hand of having described among first～the 4th embodiment.Figure 14 is the block diagram of this formation of expression.The correction object images of describing in the present embodiment is the correction object images (A1, B1, C1 or D2) among first～the 4th embodiment, describe in the present embodiment with reference to image be among first～the 4th embodiment with reference to image (A2, B3, C2 and C3 or D2 and D3).

Memory 31 among Figure 14 is realized by the internal storage 14 of Fig. 1, perhaps is located at hand and trembles in detection/correction portion 19.Deterioration image among Figure 14/initial restored image configuration part 32, Fourier iterative processing portion 33, filtering portion 34 and bell signal are removed the hand that portion 15 is located at Fig. 1 and are trembled detection/correction portion 19.

Memory 31 storage is revised object images and with reference to image.Deterioration image/initial restored image configuration part 32 is set deterioration image and initial restored image with the method for describing among first～the 4th embodiment, and these images is offered Fourier iterative processing portion 33 according to the recorded content of memory 31.For example, adopting under the situation of first embodiment, each of step S1～S8 that will be by Fig. 2 handled the little image A 1a and the A2c that obtain and offered Fourier iterative processing portion 33 as deterioration image and initial restored image respectively.

In addition, be located at the little image extraction unit 36 of deterioration image/initial restored image 32 from the correction object images and with reference to each the little image (C1a of the A1a of Fig. 3, A2a, Figure 10, C2a, C3a etc.) that extracts the image as the basis of deterioration image and initial setting image.

Fourier iterative processing portion 33 implements to wait the Fourier iterative method that illustrated with reference to Fig. 4 according to deterioration image that is provided and initial restored image.The image restoration filter itself is installed in the filtering portion 34, and Fourier iterative processing portion 33 handles by the step S10 of execution graph 2 grades and each of S11, thereby calculates each filter factor of this image restoration filter.

Filtering portion 34 carries out filtering to revising object images, thereby generates filtering image by the image restoration filter with each filter factor that calculates being applicable to each pixel of revising object images.Though the size of image restoration filter is littler than the picture size of revising object images, but can cause the same deterioration to entire image owing to consider in one's hands trembling, so pass through whole correction object images is suitable for this image restoration filter, thereby can remove rocking of whole correction object images.

Bell signal is removed portion 35 by the filtering image that generated is weighted on average with revising object images, thereby generates final correction image.For example, weighted average is carried out according to each pixel, and the average weighted ratio of each pixel can be determined according to the edge strength of each pixel of revising object images.

The final correction image that generated be remove or reduced revise that object images comprised rocked and removed or reduced the image of following behind the bell signal of filtering.Wherein, because the filtering image that filtering portion 34 generates also is to remove or reduced the image that rocks, therefore can be interpreted as filtering image also is a kind of of correction image.

In addition, be known owing to remove the method for above-mentioned bell signal, detailed.As its method, for example can adopt the spy to open the disclosed method of 2006-129236 communique.

What obtain than the common also short photography of exposure photography by the time for exposure is low-light level with reference to image, so the hand amount of trembling that is comprised is few, so its marginal element approaches the edge of image composition that do not have hand to tremble.Therefore, as mentioned above, will be according to this image that obtains with reference to image as the initial restored image in the Fourier iterative method (initial value of restored image).

By repetition based on the circular treatment of Fourier iterative method, restored image (f ') move closer in removed the image that hand is trembled as far as possible, but because the image that initial restored image itself has been trembled near no hand, so can be than the rapid convergence (the shortest in 1 circular treatment, restrain) more of such method that random image or deterioration image are made as initial restored image in the past.As a result, the generation hand be can shorten and the processing time of information (filter factor of PSF or image restoration filter) usefulness and the processing time that hand is trembled correction usefulness trembled.Have again, if initial restored image falls far short with the image that should restrain, then converge to local solution (with the different image of the image that should indeed restrain) probability increase, but by setting initial restored image as described above, thereby the probability that converges to local solution reduces (be hand tremble revise failed probability reduce).

Also have, entire image is caused the same deterioration,, these information can be applicable to entire image so, tremble information (filter factor of PSF or image restoration filter) by the view data generation hand of each zonule by extracting the zonule in each image because hand is trembled.Thus, can reduce the operand that needs, can shorten the hand information of trembling and generate the processing time of usefulness and the processing time that hand is trembled correction usefulness.Certainly, also can predict the scale down of necessary circuitry or follow the effect that reduces in this cost.

At this moment, as described in each embodiment, be made as the distinctive zonule that automatic extraction comprises a lot of marginal elements.The increase of the marginal element in the calculating source images of PSF means the increase of signal component with respect to the noise components in proportions, and therefore by the distinctive zonule of extraction, thereby The noise reduces, and can detect hand more accurately and tremble information.

Also have, need not to be used to obtain special-purpose photography in a second embodiment with reference to image, even in the first, the 3rd and the 4th embodiment, also only once be used to obtain special-purpose photography (short time exposure photography), the load in the time of therefore can increasing photography hardly with reference to image.Have again, need not many speeches, owing to do not use angular-rate sensor to wait to carry out hand to tremble to detect and hand is trembled correction, so can realize the cost degradation of camera head 1.

In addition,, the processing based on Fig. 4 of Fourier iterative method has been described more than as the processing example that is used to ask for PSF, but subsidiary supplementary notes and the variation (also with reference to Fig. 5) that mentions this.In the processing of Fig. 4, by utilizing Fourier transform that deterioration image g on the spatial domain and restored image f ' are transformed on the frequency domain, thereby can be in the hope of the function G of the deterioration image g on the expression frequency domain and the function F of the restored image f ' on the expression frequency domain ' (wherein so-called frequency domain is meant the frequency domain of two dimension certainly).According to function G of being tried to achieve and F ', ask for the function H of the PSF of expression on the frequency domain, by inverse fourier transform this function H is transformed to function on the spatial domain, is PSF h.Utilize the constraints of regulation that this PSF h is revised, try to achieve revised PSF h '.The processing that below will revise this PSF is called " first correcting process ".

By Fourier transform PSF h ' is transformed on the frequency domain once more, ask for function H ', ask for the function F of the restored image on the expression frequency domain according to function H ' and function G.By this function F is carried out inverse fourier transform, thereby obtain the restored image f on the spatial domain, utilize the constraints of regulation that this restored image f is revised, ask for revised restored image f '.The processing that below will revise this restored image is called " second correcting process ".

In above-mentioned example, utilize revised correction image f ' to carry out above-mentioned processing repeatedly after having described, till satisfying the condition of convergence among the step S118 at Fig. 4.Have again, also described: consider if iterative processing is tended to restrain, the characteristic that reduces of correction then, also can judge the establishment of this condition of convergence/be false according to the correction among the step S113 corresponding or with the correction among the corresponding step S117 of second correcting process with first correcting process.Carrying out according to correction under the situation of this judgement, preestablish the benchmark correction, correction among correction among the step S113 or the step S117 and benchmark correction are compared, judging less than the latter's situation at the former is that the condition of convergence is set up, but if set the benchmark correction enough big, the then not processing of repeated execution of steps S110～S117.That is to say, only carry out one time first correcting process under this situation and the PSF h ' that obtains just becomes the final PSF that derive with the step S10 of Fig. 2 etc.Like this, even adopted the processing of Fig. 4, also not necessarily repeat first and second correcting process.

The increase that repeats number of times of first and second correcting process helps to improve the precision of the PSF that finally tries to achieve, but in the present embodiment, because initial restored image itself has approached not have the image that hand is trembled, the precision of the PSF h ' that obtains so only carry out one time first correcting process on practicality also up to no problem degree.If consider these, then also can omit the judgment processing of step S118 itself.Under this situation, the PSF h ' that asks for by the processing of only carrying out a step S113 becomes the final PSF that should derive with the step S10 of Fig. 2 etc., according to the function H ' that the processing of only carrying out a step S114 is asked for, ask for each filter factor of the image restoration filter that to derive with the step S11 of Fig. 2 etc.Therefore, under the situation of the processing of omitting step S118, the processing that also can omit step S115～S117.

Variation or note item as at first execution mode below are designated as note 1～note 6.The short of contradiction of the content of putting down in writing in each note just can make up arbitrarily.

[note 1]

In the first, the 3rd or the 4th embodiment (with reference to Fig. 3, Figure 10 or Figure 13), illustrated by the short time exposure photography after the common exposure photography that is used to obtain to revise object images to obtain with reference to image A 2, C2 or D2, even but by this usually short time exposure photography before the exposure photography obtain these and also be fine with reference to image.Under this situation, if with among the 3rd or the 4th embodiment with reference to image C 3 or D3 as the straight-through demonstration in the frame of taking after the frame of revising object images with image.

[note 2]

In each embodiment, in that little image generates in the process at the deterioration image of Fourier iterative method and initial restored image by each, to each little image be suitable for that noise is removed processings, luminance standard processing, edge extracting is handled and picture size standardization (with reference to Fig. 3, Fig. 7, Figure 10 and Figure 13) in any one more than processing.The usability methods of these processing of describing among each embodiment is an example, can carry out various changes.In the deterioration image in each embodiment and the generative process of initial restored image, extreme is whole (wherein the picture size standardization is insignificant in first embodiment) that also can implement 4 above-mentioned processing to each zonule.

[note 3]

As from the correction object images or with reference to the method for extracting the characteristic zonule that comprises a lot of marginal elements the image, can adopt various methods.For example, also can divert the AF evaluation of estimate of in focusing is controlled automatically, calculating and carry out this extraction.Automatically the Contrast Detection method of TTL (Through The Lends) mode is adopted in focusing control.

Be provided with AF evaluation portion (not shown) in the camera head 1, AF evaluation portion is divided into a plurality of cut zone with each photographs (or each straight-through demonstration use image), according to the corresponding AF evaluation of estimate of contrast of the image in each cut zone calculating and the cut zone.The master control part 13 of Fig. 1 is with reference to the AF evaluation of estimate at the arbitrary cut zone in above-mentioned a plurality of cut zone, utilize mountain-climbing to control the position to focus lens of image pickup part 11, so that this AF evaluation of estimate is got maximum (or maximum), make optical image imaging on the shooting face of imaging apparatus of subject thus.

Under the situation of carrying out this automatic focusing control, when extracting the characteristic zonule from the correction object images or with reference to image, reference is at the AF evaluation of estimate of each cut zone of this extraction source image.For example, determine, will extract as the characteristic zonule with this maximum AF evaluation of estimate corresponding divided areas (or be the zone of benchmark with this cut zone) at the maximum AF evaluation of estimate in the AF evaluation of estimate of each cut zone of extraction source image.The AF evaluation of estimate follow the image in the cut zone contrast tolerance (perhaps marginal element) increase and increase, therefore also can extract as the characteristic zonule containing the more zonule of marginal element in view of the above.

[note 4]

Concrete numerical value shown in the above-mentioned description only is illustration, certainly it is changed to various numerical value.

[note 5]

Have, the camera head 1 of Fig. 1 can be realized by the combination of hardware, software or hardware and software again.Especially, the function of each position shown in Figure 14 (wherein removing memory 31) can realize by the combination of hardware, software or hardware and software, can also realize each function at these positions by the external device (ED) (computer etc.) of camera head 1.

Utilizing software to constitute under the situation of camera head 1, is the functional block diagram at this position of expression at the block diagram at the position of realizing by software.All or part of of the function that can be realized by each position (wherein removing memory 31) of Figure 14 with program description by going up this program of execution at program executing apparatus (for example computer), thereby realized this function all or part of.

[note 6]

In Figure 14, deterioration image/initial restored image configuration part 32 forms device for detecting rock with Fourier iterative processing portion 33, and blur correcting device constitutes filtering portion 34 and the bell signal portion 35 of comprising.Wherein, also can from this blur correcting device, omit bell signal portion 35.Have again, also can consider to contain above-mentioned device for detecting rock at this blur correcting device.Also have, above-mentioned device for detecting rock also can comprise memory 31 (holding unit).In addition, in Fig. 1, hand is trembled detection/correction portion 19 and is worked as device for detecting rock, and also works as blur correcting device.

The unit that information (filter factor of PSF or image restoration filter) is trembled as the generation hand by Fourier iterative processing portion 33 or deterioration image/initial restored image configuration part 32 and Fourier iterative processing portion 33 works.

" second execution mode "

Then, second embodiment of the invention is described.Second execution mode is equivalent to the variation of first execution mode, and the specified particular of first execution mode only otherwise contradiction just goes for second execution mode.Figure 17 is the entire block diagram of the camera head 1a that relates to of second execution mode.Camera head 1a is made of each position of reference marker 11～18 and 20.That is, be replaced into hand and tremble detection/correction portion 20 and form camera head 1a by the hand of Fig. 1 being trembled detection/correction portion 19.Two camera heads are same in other respects, therefore omit the repeat specification of same part.

In camera head 1a, under photograph mode,, the photographs that obtains is thus stored on the memory as correction object images E1 if press the shutter release button 17a photography that then exposes usually.Time for exposure when representing to obtain revising object images E1 (length of time for exposure) with T1.Have again, before or after the common exposure photography that obtains revising object images E1, carry out short time exposure photography, will store on the memory as reference image E2 by the photographs that this short time exposure photography obtains.Though by sequence photography to revise object images E1 with reference to image E2 (promptly obtaining) at adjacent frame, master control part 13 is by exposure control part 18 control image pickup parts 11, so that the time for exposure when obtaining with reference to image E2 is also shorter than time for exposure T1.For example, the time for exposure with reference to image E2 is made as T1/4.Also have, revise object images E1 and equate with picture size with reference to image E2.

In addition, also can be to the T that described in the time for exposure T1 and first execution mode _TH(hand is trembled the boundary time for exposure) compares, and under the former situation less than the latter, thinks to revise and do not contain (or containing extremely slightly) hand among the object images E1 and tremble rocking of being caused, and do not carry out hand and trembles correction.Have again, can not carry out the short time exposure that obtains using under this situation and photograph with reference to image E2.

Obtain revising object images E1 with reference to image E2 after, from reference image E2, extract distinctive zonule, from revise object images E1, extract on the other hand and corresponding district territory, zonule from reference image E2 extracts.The zonule of being extracted is made as for example zonule of 128 * 128 pixels.The meaning of distinctive zonule and extracting method are as described in first execution mode.In the present embodiment, from reference image E2, extract a plurality of distinctive zonules.Therefore, from revise object images E1, also extract zonule with its equal number.At present, as shown in figure 18, from reference image E2, extract 8 zonules, the image in these 8 zonules (image in the hatched example areas) is called little image GR ₁～GR ₈On the other hand, will with little image GR ₁～GR ₈Corresponding and images (image in the hatched example areas) in 8 zonules that extract from revise object images E1 are called little image GL ₁～GL ₈

I is being made as under the situation of the integer below 8 more than 1 little image GR _iWith little image GL _iPicture size equate.Ignoring under correction object images E1 and the situation, according to the little image GR that extracts from reference image E2 with reference to the position deviation between the image E2 _iCentre coordinate (with reference to the centre coordinate in the image E2) and with this little image GR _iThe little image GL that from revise object images E1, extracts accordingly _iThe equal mode of centre coordinate (revising the centre coordinate in the object images E1), carry out the extraction of zonule.Under the situation that can't ignore this position deviation, also can utilize template matches (template matching) method etc. to carry out the search (this also goes for first execution mode in addition) in respective cell territory.That is, for example with little image GR _iAs template, adopt known template matching method, search and the highest zonule of this template similarity are made as little image GL with the image in the zonule that searches from revise object images E1 _i

Figure 19 illustrates little image GL ₁And GR ₁Enlarged drawing.In Figure 19, the part that brightness is high shows as white, and the part that brightness is low shows as black.Have, brightness changes rapid marginal existence in little image GL on imagination horizontal direction and the vertical direction again ₁In and little image GR ₁Interior situation.And then imagination is comprising little image GL ₁The exposure period of correction object images E1 between in the hand of horizontal direction tremble the situation that acts on camera head 1a.Therefore, at little image GR based on short time exposure photography ₁Inward flange can not blur, but at the little image GL based on common exposure photography ₁Inward flange is fuzzy in the horizontal direction.

By to this little image GR ₁Implement to adopt the edge extracting of any rim detection operator to handle, thereby generate edge extracting image ER shown in Figure 20 ₁Edge extracting image ER at Figure 20 ₁In, the part that edge strength is stronger shows as white, and the part that edge strength is more weak shows as black.Along little image GR ₁The part at interior linearity edge is presented in edge extracting image ER as the stronger part of edge strength ₁In.By to this edge extracting image ER ₁Be suitable for (huff) conversion of known Hough and extract straight line along the edge.Right-hand straight line and the little image GR that is extracted that illustrate at Figure 20 ₁Overlapping part.In this example, at little image GR ₁, extract the straight line HR that vertically extends ₁₁Straight line HR with the along continuous straight runs extension ₁₂

Then, from little image GL ₁The middle extraction and straight line HR ₁₁And HR ₁₂Corresponding straight line HL ₁₁And HL ₁₂In Figure 21 with the straight line HL that is extracted ₁₁And HL ₁₂With little image GL ₁Overlapping demonstration.In Figure 21, also illustrated straight line HR overlapping ₁₁And HR ₁₂Little image GR ₁The direction of corresponding straight line is identical.That is straight line HL, ₁₁With straight line HR ₁₁Bearing of trend identical, straight line HL ₁₂With straight line HR ₁₂Bearing of trend identical.

After extracting each straight line, ask for the distribution of the brightness value in the little image on the orthogonal direction of each straight line.This distribution is to ask at each the little image that is extracted.About little image GL ₁And GR ₁, straight line HL ₁₁With straight line HR ₁₁Vertical direction at image is parallel, and straight line HL ₁₂With straight line HR ₁₂Horizontal direction parallel at image.Therefore, paying close attention to straight line HL ₁₁With straight line HR ₁₁Situation under, ask for the distribution of the brightness value on the horizontal direction of image, paying close attention to straight line HL ₁₂With straight line HR ₁₂Situation under, ask for the distribution of the brightness value on the vertical direction of image.

With reference to Figure 22 and Figure 23, specify the acquiring method that brightness value distributes.In Figure 22, little image GL ₁Each solid arrow shown in interior is illustrated in and straight line HL ₁₁The form that brightness value is scanned on the direction of quadrature.Because straight line HL ₁₁Orthogonal direction be horizontal direction, so with little image GL ₁Certain point of left end scans from left to right for starting point, obtains little image GL simultaneously ₁The brightness value of each interior pixel is asked for straight line HL thus ₁₁Orthogonal direction on the distribution of brightness value.Wherein, establish with crosscut and straight line HL ₁₁The mode of the part of corresponding marginal existence scans.That is, ask for the distribution of brightness value at the rapid part of the gradient of brightness value.Therefore along the dotted arrow of Figure 22, do not scan (in Figure 23 described later too).The influence of noise contribution is bigger in only paying close attention to 1 row (be horizontal line under this situation) distribution of trying to achieve, so at little image GL ₁In multirow ask for same distribution, average as to straight line HL with a plurality of distributions of being tried to achieve ₁₁The distribution 201 that finally should try to achieve.

Equally, to straight line HR ₁₁Also ask for distribution.In Figure 22, little image GR ₁Each solid arrow shown in interior is illustrated in and straight line HR ₁₁The form that brightness value is scanned on the direction of quadrature.Because straight line HR ₁₁Orthogonal direction be horizontal direction, so with little image GR ₁Certain point of left end scans from left to right for starting point, obtains little image GR simultaneously ₁The brightness value of each interior pixel is asked for straight line HR thus ₁₁Orthogonal direction on the distribution of brightness value.Wherein, establish with crosscut and straight line HR ₁₁The mode of the part of corresponding marginal existence scans.That is, ask for the distribution of brightness value at the rapid part of the gradient of brightness value.Therefore, the dotted arrow along Figure 22 does not scan (in Figure 23 described later too).The influence of noise contribution is bigger in only paying close attention to 1 row (be horizontal line under this situation) distribution of trying to achieve, so at little image GR ₁In multirow ask for same distribution, average as to straight line HR with a plurality of distributions of being tried to achieve ₁₁The distribution 202 that finally should try to achieve.

In the expression of Figure 22 distributes each curve chart of 201 and 202, the horizontal level of transverse axis remarked pixel, the longitudinal axis is represented brightness value.By distributing 201 and 202 as can be known, though brightness value is that the border sharply changes with the marginal portion of extending along the vertical direction of image, in the distribution 201 corresponding with common exposure photography, because the hand between exposure period is trembled, the variation of this brightness value is slower.With straight line HL ₁₁Corresponding little image GL ₁In the interior marginal portion, with WL ₁₁The expression brightness value begins to change to the pixel count that changes the horizontal direction till finishing, with straight line HR ₁₁Corresponding little image GR ₁In the interior marginal portion, with WR ₁₁Expression begins to change to the pixel count of the horizontal direction that changes end position from brightness value.Have again, with the WL that tries to achieve like this ₁₁Or WR ₁₁Be called border width.In this example, be made as " WL ₁₁＞WR ₁₁", ignoring under the situation of rocking that is comprised with reference to image E2, can think the poor " WL of border width ₁₁-WR ₁₁" be to be the value of the hand amount of trembling of the horizontal direction that produces between the exposure period of unit representation correction object images E1 with the pixel.

To from little image GL ₁And GR ₁Each straight line that extracts is carried out the above-mentioned processing of asking for border width.In the case of this example, even to from little image GL ₁And GR ₁The straight line HL that extracts ₁₂And HR ₁₂Also ask for above-mentioned border width.

In Figure 23, little image GL ₁Each solid arrow shown in interior is illustrated in and straight line HL ₁₂The form that brightness value is scanned on the direction of quadrature.With crosscut and straight line HL ₁₂The mode of the part of corresponding marginal existence vertically scans, and obtains little image GL simultaneously ₁The brightness value of each interior pixel is asked for straight line HL thus ₁₂Orthogonal direction on the distribution of brightness value.Scan average as to straight line HL with a plurality of distributions of being tried to achieve at multirow (under this situation for vertical row) ₁₂The distribution 211 that finally should try to achieve.In Figure 23, little image GR ₁Each solid arrow shown in interior is illustrated in and straight line HR ₁₂The form that brightness value is scanned on the direction of quadrature.With crosscut and straight line HR ₁₂The mode of the part of corresponding marginal existence vertically scans, and obtains little image GR simultaneously ₁The brightness value of each interior pixel is asked for straight line HR thus ₁₂Orthogonal direction on the distribution of brightness value.Scan average as to straight line HR with a plurality of distributions of being tried to achieve at multirow (under this situation for vertical row) ₁₂The distribution 212 that finally should try to achieve.

And, according to distributing 211 and 212, ask for border width WL ₁₂Or WR ₁₂Border width WL ₁₂Be illustrated in and straight line HL ₁₂Corresponding little image GL ₁Brightness value begins to change to the pixel count that changes the vertical direction till finishing, border width WR in the interior marginal portion ₁₂Be illustrated in and straight line HR ₁₂Corresponding little image GR ₁Begin to change to the pixel count that changes the vertical direction till finishing from brightness value in the interior marginal portion.In this example, be " WL ₁₂ WR ₁₂".This situation is trembled corresponding to the hand that vertical direction does not almost take place between the exposure period of revising object images E1.

Though pay close attention to little image GL ₁And GR ₁, the computational methods of border width have been described, but at other little image GL ₂～GL ₈And GR ₂～GR ₈Ask for the poor of border width and border width too.The variable of representing the numbering of little image is made as i, the variable of representing the numbering of straight line is made as j (i and j are integer).Like this, by little image GL _iAnd GR _iIn extract straight line HL _IjAnd HR _IjAfter, ask for and straight line HL _IjAnd HR _IjCorresponding border width WL _IjAnd WR _IjThen, according to formula " D _Ij=WL _Ij-WR _Ij" calculate the D of difference of expression border width _IjSuppose from little image GL ₁～GL ₈Each little image in respectively extract 2 straight lines, then ask for the altogether poor D of 16 border widths _Ij(at this, i is the integer below 8 more than 1, and j is 1 or 2).

In the present embodiment, will with a plurality of poor D that is obtained _IjThe straight line of maximum correspondence to tremble detection right with straight line to being defined as hand, tremble detection with difference and the rectilinear direction of straight line according to this hand to pairing border width, ask for and the whole corresponding PSF that revises object images E1.

For example, consider a plurality of poor D obtained _IjIn, the poor D corresponding with Figure 22 ₁₁(=WL ₁₁One WR ₁₁) maximum situation.Under this situation, with straight line HL ₁₁And HR ₁₁Being defined as hand, to tremble detection right with straight line, will with straight line HL ₁₁And HR ₁₁Corresponding poor D ₁₁The variables D of maximum difference is represented in substitution _MAXIn.And, be created on straight line HL ₁₁Orthogonal direction on image is carried out the smooth function that smoothing is used.With shown in Figure 24, straight line HL ₁₁Orthogonal direction on tap number (filter size) be D _MAXSpatial filter 220 represent this smooth function.In this spatial filter 220, only to along straight line HL ₁₁The key element of orthogonal direction constant filter factor beyond 0 is provided, provide filter factor 0 to other key elements.The spatial filter of Figure 24 has 5 * 5 filter size, and only each key element to the row of vertical direction center provides filter factor 1, and the filter factor of other key elements is 0.In addition, be that 1 mode is carried out standardization in fact with the summation of whole filter factors.

And hand is trembled detection/correction portion 20 by this smooth function is handled as the PSF that revises object images E1 integral body, thereby revises revising rocking of object images E1.Under the direction that the hand that between the exposure period of revising object images E1 camera head 1a is worked is trembled and the assumed condition of constant airspeed, above-mentioned PSF plays a role well.If the correct and above-mentioned smooth function of this hypothesis can correctly represent to revise the PSF of object images E1, then by the ideal image of not rocking is implemented to adopt the space filtering of spatial filter 220, thereby can obtain and revise the equal image of object images E1.

Figure 25 is the flow chart that hand that expression comprises above-mentioned processing action is trembled the motion flow of detection.Each of step S151 shown in Figure 25～S155 handled, and trembles detection/correction portion 20 by hand and carries out.

Obtain and revise object images E1 and, in step S151, from reference image E2, extract a plurality of distinctive zonules with reference to behind the image E2, with the image in each zonule as little image GR _iStore on the memory.In following step S152, from revise object images E1, extract and each little image GR _iThe corresponding district territory, the image in the zonule that will extract from revise object images E1 is as little image GL _iStore on the memory.This for example generates little image GL shown in Figure 180 constantly ₁～GL ₈With little image GR ₁～GR ₈

Move to step S153 after the processing of step S152.In step S153, carry out circular treatment repeatedly at variable i, comprise circular treatment in this circular treatment at variable j.In step S153, generate little image GR _iEdge extracting image ER _i, from this edge extracting image ER _iMiddle extraction 1 or many straight line HR _Ij, and then from little image GL _iThe middle extraction and straight line HR _IjCorresponding straight line HL _IjAnd, pay close attention to straight line HL in correspondence with each other _IjAnd HR _Ij, calculate the border width WL of these straight lines _IjAnd WR _Ij, ask for its difference D _Ij(=WL _Ij-WR _Ij).In step S153, each of each of the value that can get at variable i and the value that can get at variable j is carried out same processing.As a result, in the moment of shifting to step S154 from step S153, can calculate poor D at all combinations of i and j _IjFor example, if in step S151, generate little image GR by extracting 8 zonules ₁～GR ₈, and from little image GR ₁～GR ₈Each in respectively extract 2 straight lines, then ask for the altogether poor D of 16 border widths _Ij(at this, i is the integer below 8 more than 1, and j is 1 or 2).

In step S154, the maximum D among the poor Dij of whole border widths of obtaining among the determining step S153 _MAx, will with this maximum D _MAXTo tremble detection right with straight line to being defined as hand for corresponding straight line.And then, in step S155, according to this hand tremble detection with straight line to maximum D _MAX, computational chart is shown the PSF of smooth function.For example, the poor D corresponding in the whole poor Dij that obtains with Figure 22 ₁₁(=WL ₁₁-WR ₁₁) be maximum D _MAXSituation under, with straight line HL ₁₁And HR ₁₁Being defined as hand, to tremble detection right with straight line, calculates the PSF by spatial filter 220 expressions of Figure 24.

The hand of calculating behind the PSF is trembled describe in the corrective action and first execution mode same.Promptly, hand is trembled detection/correction portion 20 each key element of the inverse matrix of the PSF that obtains among the step S155 is asked for as each filter factor of image restoration filter, utilizes the image restoration filter with this each filter factor that the integral body of revising object images E1 is carried out filtering.And, implemented bell signal with this filtered image or to this and removed the image of processing as final correction image.This correction image is to remove or reduced and revised the image that rocks that object images E1 is comprised.

In the present embodiment, owing to be to act on the PSF (in other words folding degradation function) that asks under the assumed condition of direction that the hand of camera head 1a trembles and constant airspeed as the image deterioration coefficient between the exposure period of revising object images E1, so its correction effect is lower for the hand of inapplicable this assumed condition is trembled.Yet, owing to can ask for PSF simply with less treating capacity, so be practical.

Have again, present embodiment is suitable for above-mentioned second embodiment (with reference to Fig. 7), also can generate with reference to image E2 (wherein need to make this straight-through show with the time for exposure of image shorter than the time for exposure of revising object images E1) with image according to obtaining revising the straight-through demonstration that obtains before or after the common exposure photography that object images E1 uses.This straight-through show that picture size with image is than the also little situation of the picture size of revising object images E1 under, to the straight-through image processing and amplifying of implementing to make both identical usefulness of picture size with image that shows, generate with reference to image E2 and get final product.On the contrary, dwindle processing, also can realize the same of picture size by the image that obtains by common exposure photography is implemented image.

Also have, present embodiment is suitable for above-mentioned the 4th embodiment (with reference to Figure 13), also can according to obtain revising before the common exposure photography that object images E1 uses and obtain afterwards two generate with reference to image E2 with reference to image with reference to the either party in the image.These two can be to be straight-through demonstration image with reference to one in the image.Certainly, these two each time for exposure with reference to image are also shorter than the time for exposure of revising object images E1.

In addition, the record content of note 3～note 5 of describing at first execution mode equally also can be applicable to second execution mode.The hand of Figure 17 is trembled detection/correction portion 20 when working as device for detecting rock, also can work as blur correcting device.Hand is trembled in detection/correction portion 20 and is also comprised: the information generating unit of rocking that generates the PSF corresponding with whole correction object images; Or extract as revising object images or with reference to the extraction unit of the little image of a part of image of image.

" the 3rd execution mode "

Then, third embodiment of the invention is described.The image (following also be referred to as " short exposure image ") that obtains by short time exposure photography is comprised rock also littler than rocking of being comprised of the image that obtains by common exposure photography (following also be referred to as " exposure image usually "), so above-mentioned hand to tremble modification method be very useful.But, the influence that hand is trembled the short exposure image is not have fully, because the hand between the exposure period of short time exposure photography trembles or subject is rocked (activity of subject in real space), also can comprise in the short exposure image and can't ignore rocking of degree.Therefore, in the 3rd execution mode, obtain a plurality of short exposure images by the exposure photography of short time repeatedly, by these a plurality of short exposure images generate the hand that is used for common exposure image tremble correction with reference to image.

Figure 26 is the entire block diagram of the camera head 1b that relates to of the 3rd execution mode.Camera head 1b possesses each position of reference marker 11～18 and 21.Each position of reference marker 11～18 is identical with each position of Fig. 1, omits the repeat specification with a part.Be replaced into hand and tremble correction portion 21 if the hand in the camera head 1 of Fig. 1 is trembled detection/correction portion 19, just can obtain camera head 1b.

When under photograph mode, pressing shutter release button 17a, the photography that exposes usually, master control part 13 will represent that the image data storage of 1 width of cloth photographs that obtains by this photography is to recording medium 16 and internal storage 14 (that is, it being stored).This photographs is can comprise hand to tremble the caused image that rocks, then according to the correction indication that provides by operating portion 17 grades or tremble correction portion 21 by hand automatically and revise.Therefore, same with first execution mode, will especially be called " correction object images " by the 1 above-mentioned width of cloth photographs that common exposure photography obtains.Hand is trembled correction portion 21 and is not utilized hand such as angular-rate sensor to tremble to detect and use transducer, but the view data that obtains according to the output signal by image pickup part 11, and object images comprised rocks and revise to correction.

Below, tremble the embodiment of the function of correction portion 21 as describing hand in detail, the 6th～the 11 embodiment is described.The item that is recorded in certain embodiment only otherwise contradiction just goes for other embodiment.In addition, in the following description, under the situation that singly is called " memory ", it is meant internal storage 14 or is located at hand and trembles not shown memory in the correction portion 21.

[the 6th embodiment]

At first the 6th embodiment is described.In the 6th embodiment, by selecting 1 width of cloth for example to be speculated as the contained minimum short exposure image that rocks in a plurality of short exposure images.Then, with selected short exposure image as the reference image and will handle as revising object images, according to revising object images and trembling correction with reference to the hand that image is revised object images by the image that common exposure photography obtains.Figure 27 is a flow chart that expression camera head 1b carries out, that this hand is trembled the motion flow of correction.According to this flow chart, the action of camera head 1b is described.

If press shutter release button 17a, the photography that then exposes is usually stored the common exposure image that generates thus on the memory (step S201 and S202) as revising object images Lw under photograph mode.Then, in step S203, time for exposure T1 and threshold value T when obtaining revising object images Lw _THCompare, at time for exposure T1 less than threshold value T _THSituation under, think to revise and do not contain (or containing extremely slightly) hand among the object images Lw and tremble rocking of being caused, do not carry out hand and tremble correction, finish the processing of Figure 27.As threshold value T _TH, for example adopt according to focal distance f _DThe inverse hand of calculating tremble the boundary time for exposure.

At time for exposure T1 greater than threshold value T _THSituation under, under the control of master control part 13, after common exposure photography, carry out N short time continuously and expose and photograph, obtain short exposure image C w ₁～Cw _NHand is trembled the processing of correction portion 21 by execution in step S206～S209, thereby calculates and short exposure image C w ₁～Cw _NCorresponding evaluation of estimate K ₁～K _N, and according to evaluation of estimate K ₁～K _N, from short exposure image C w ₁～Cw _NMiddle 1 width of cloth short exposure image of selecting is as the reference image.At this, N is the integer more than 2, for example is 4.Though revise object images Lw and short exposure image C w ₁～Cw _NObtain by sequence photography, but 13 pairs of master control part exposure control part 18 controls, so that the time for exposure when obtaining each short exposure image is also shorter than time for exposure T1.For example, the time for exposure of each short exposure image is T1/4.Have again, revise object images Lw and equate with the picture size of each short exposure image.

Be described more specifically the contents processing of each step.At time for exposure T1 greater than threshold value T _THSituation under, shift to step S204 from step S203.In step S204, introduce variable i and with initial value 1 substitution variable i.Then, move to step S205, carry out the exposure photography of 1 short time, with the short exposure image that obtains thus as short exposure image C w _iStore on the memory.This memory is the short exposure image memory that can store the view data of 1 width of cloth part short exposure image.Therefore, for example when i=1 with short exposure image C w ₁Store the short exposure image into on the memory, when i=2 with short exposure image C w ₂Store the short exposure image into on the memory.

Among the step S206 after being next to step S205, hand is trembled correction portion 21 and is calculated and short exposure image C w _iCorresponding evaluation of estimate K _iAs principle, evaluation of estimate K _iGet and short exposure image C w _iThe value that the size of rocking that is comprised (following also be referred to as " rolling momentum ") is corresponding, short exposure image C w _iThe rolling momentum more little, evaluation of estimate K _i(comprise exception, evaluation of estimate K greatly more _iComputational methods in the 9th embodiment, describe in detail).

Then, shift to step S207, to the up-to-date evaluation of estimate K that calculates among the step S206 _iThe evaluation of estimate of calculating before with expression (is K ₁～K _I-1) peaked variable K _MAXCompare, the former greater than the latter's situation under or under the situation of variable i=1, in step S208 with short exposure image C w _iRw stores on the memory as the reference image, and in step S209 with evaluation of estimate K _iSubstitution variable K _MAXIn, move to step S210 then.On the other hand, at " i ≠ 1 " and " K _i≤ K _MAX" under the situation about setting up, directly shift to step S210 from step S207.In step S210, whether judgment variable i is consistent with the value of N, shifting to step S212 under the situation of " i=N " from step S210, and shifting to step S211 from step S210 under the situation of " i ≠ N ", turn back to step S205 after variable i added 1, repeated execution of steps S205 later above-mentioned each handle.

As a result, each that carry out N step S205 and S206 handled, in the moment that arrives step S212, with short exposure image C w ₁～Cw _NCorresponding evaluation of estimate K ₁～K _NAll calculated, with evaluation of estimate K ₁～K _NInterior maximum substitution variable K _MAX, and short exposure image that will be corresponding with this maximum stores on the memory as reference image Rw.For example, if evaluation of estimate K ₁～K _NIn evaluation of estimate K _N-1Maximum is then with short exposure image C w _N-1The state that stores on the memory as reference image Rw arrives step S212 down.In addition, storage with reference to the memory of image Rw be can store 1 width of cloth part with reference to the view data of image Rw with reference to the image memory, needs with new image data storage to the reference image with the situation on the memory under, cover the new view data of storage at the memory area of having stored old view data.

In step S212, hand tremble correction portion 21 according to be stored in reference to image with on the memory with reference to the correction object images Lw that obtains among image Rw and the step S202, enforcement is trembled correction at the hand of revising object images Lw, generates to have reduced the correction image Qw that rocks (describing modification method later in the tenth embodiment) that revises among the object images Lw.This correction image Qw is recorded in recording medium 16 and shows with display part 15.

As mentioned above, by generating with reference to image Rw, even big hand is trembled or subject is rocked thereby produced in during the part in for example exposing photography in a plurality of short time, also hand can be trembled the less short exposure image of influence and be chosen as with reference to image Rw, therefore can carry out hand accurately trembles correction.Generally, though,, then can carry out hand and tremble correction to the frequency content of high-frequency domain more if hand is trembled the minimum short exposure image of influence as with reference to image owing to hand is trembled the radio-frequency component decay that causes image.Have again, by as step S205～S211, handling and cover storage short exposure image and, thereby the short exposure image can be suppressed to 1 width of cloth image part with memory and with reference to image with the required separately memory span of memory with reference to image.

[the 7th embodiment]

Then the 7th embodiment is described.In the 7th embodiment,, selected short exposure image is synthesized and generate 1 width of cloth with reference to image by selecting for example to be speculated as the contained less short exposure image that rocks more than 2 width of cloth in a plurality of short exposure images.Then, according to generate with reference to the image and the correction object images that will obtain by common exposure photography, revise the hand of object images and tremble correction.Figure 28 is a flow chart that expression camera head 1b carries out, that this hand is trembled the motion flow of correction.According to this flow chart, the action of camera head 1b is described.

If press shutter release button 17a, the photography that then exposes is usually stored the common exposure image that generates thus on the memory (step S221 and S222) as revising object images Lw under photograph mode.Then, in step S223, time for exposure T1 and threshold value T when obtaining revising object images Lw _THCompare, at time for exposure T1 less than threshold value T _THSituation under, think to revise and do not contain (or containing extremely slightly) hand among the object images Lw and tremble rocking of being caused, do not carry out hand and tremble correction, finish the processing of Figure 28.

At time for exposure T1 greater than threshold value T _THSituation under, under the control of master control part 13, after common exposure photography, carry out N short time continuously and expose and photograph, obtain short exposure image C w ₁～Cw _NHand is trembled the processing of correction portion 21 by execution in step S226 and S227, thereby calculates and short exposure image C w ₁～Cw _NCorresponding evaluation of estimate K ₁～K _N, and according to evaluation of estimate K ₁～K _N, from short exposure image C w ₁～Cw _NThe middle M width of cloth short exposure image of selecting.At this, M is the integer more than 2, and inequality " N＞M " is set up.Therefore, N is a integer more than 3 in the 7th embodiment.N=4 for example.Though revise object images Lw and short exposure image C w ₁～Cw _NObtain by sequence photography, but 13 pairs of master control part exposure control part 18 controls, so that the time for exposure when obtaining each short exposure image is also shorter than time for exposure T1.For example, the time for exposure of each short exposure image is T1/4.Have again, revise object images Lw and equate with the picture size of each short exposure image.

Be described more specifically the contents processing of each step.At time for exposure T1 greater than threshold value T _THSituation under, shift to step S224 from step S223.In step S224, introduce variable i and with initial value 1 substitution variable i.Then, move to step S225, carry out the exposure photography of 1 short time, with the short exposure image that obtains thus as short exposure image C w _iStore on the memory.This memory is the short exposure image memory that can store the view data of 1 width of cloth part short exposure image.Therefore, for example when i=1 with short exposure image C w ₁Store the short exposure image into on the memory, when i=2 with short exposure image C w ₂Store the short exposure image into on the memory.

Among the step S226 after being next to step S225, hand is trembled correction portion 21 and is calculated and short exposure image C w _iCorresponding evaluation of estimate K _i(computational methods describe in detail in the 9th embodiment).This evaluation of estimate K _iIdentical with the evaluation of estimate of calculating among the step S206 of Figure 27.

Then, move to step S227, with the evaluation of estimate K that calculates up to now ₁～K _iArrange in order from big beginning, from i width of cloth short exposure image C w ₁～Cw _iIn select the evaluation of estimate corresponding short exposure image big with the 1st～the M, with the M width of cloth short exposure image selected as being synthesized image Dw ₁～Dw _MRecord on the memory.For example, at i=3 and M=2 and inequality " K ₁＜K ₂＜K ₃" under the situation about setting up, from 3 width of cloth short exposure image C w ₁～Cw ₃Select 2 width of cloth short exposure image C w ₂And Cw ₃, with this short exposure image C w ₂And Cw ₃As being synthesized image Dw ₁And Dw ₂Be recorded on the memory.Certainly because the not enough M width of cloth of total width of cloth number of the short exposure image obtained of the stage that the little and inequality " i＜M " of variable i is set up, so under this situation with short exposure figure Cw ₁～Cw _iDirectly as being synthesized image Dw ₁～Dw _iBe recorded on the memory.In addition, the memory that storage is synthesized image is synthesized the image memory for what can store several view data that are synthesized image, needing under the state that stores the multiple image data to store under the situation of new view data, the memory area that has write down unwanted old view data is being covered this new view data of record.

Among the step S228 after the processing of step S227, whether judgment variable i is consistent with the value of N, shifting to step S230 under the situation of " i=N " from step S228, and shifting to step S229 from step S228 under the situation of " i ≠ N ", turn back to step S225 after variable i added 1, repeatedly execution in step S225 later above-mentioned each handle.As a result, carry out each processing of N step S225～S227, in the moment that arrives step S230, with short exposure image C w ₁～Cw _NCorresponding evaluation of estimate K ₁～K _NAll calculated, with evaluation of estimate K ₁～K _NThe pairing M width of cloth of the evaluation of estimate short exposure image conduct that the 1st～the interior M is big is synthesized image Dw ₁～Dw _MStore into and be synthesized image with on the memory.

In step S230, hand is trembled correction portion 21 by to being synthesized image Dw ₁～Dw _MCarry out synthesizing after the contraposition, thereby generate 1 width of cloth with reference to image Rw.For example, will be synthesized image Dw ₁As benchmark image and will be synthesized image Dw ₂～Dw _MEach as non-benchmark image, synthesize after making each non-benchmark image and benchmark image contraposition.Wherein " contraposition " is identical with the implication of " position deviation correction " described later.

Explanation is to 1 width of cloth benchmark image and 1 processing that non-benchmark image carries out contraposition and synthesizes.For example, utilize Harris's corner detection device from benchmark image, to extract distinctive zonule (for example zonule of 32 * 32 pixels).Distinctive zonule is meant more (in other words contrast is more intense) rectangular area of marginal element in the extraction source image, for example is the zone of containing distinctive pattern.So-called distinctive pattern for example is meant that the bight of object etc. has brightness and changes more than both direction, changes the pattern of the position (position on the image) that can easily detect this pattern according to this brightness.And, will be from the image in this zonule that benchmark image extracts as template, the employing template matching method extracts the highest zonule of similar degree with this template from non-benchmark image.And, calculate as positional offset amount Δ d with the position (position on the non-benchmark image) of the zonule that searches out with from the departure of the position (position on the benchmark image) of the zonule that benchmark image extracts.Positional offset amount Δ d is the two dimension amount that comprises horizontal composition and vertical composition, shows as so-called motion vector.It is the image that benchmark has produced the position deviation that is equivalent to positional offset amount Δ d part that non-benchmark image can be considered as with the benchmark image.Therefore, in the mode of offsetting this positional offset amount Δ d non-benchmark image is carried out coordinate transform (affine (affine) conversion etc.), thereby non-benchmark image is carried out the position deviation correction.For example, ask for and carry out the geometric transformation parameter that this coordinate transform is used, by with non-benchmark image coordinate transform to the coordinate that has defined benchmark image, thereby carry out the position deviation correction.The pixel that is positioned at coordinate (x+ Δ dx, y+ Δ dy) in the non-benchmark image before the position deviation correction is transformed to by the position deviation correction and is positioned at coordinate (x, pixel y).Δ dx and Δ dy are respectively horizontal composition and the vertical composition of Δ d.And, benchmark image and the revised non-benchmark image of position deviation are synthesized.By being positioned at coordinate (x in the image that is synthesized into, the picture element signal of pixel y), be equivalent to be positioned at coordinate (x, (picture element signal of the pixel in the x, the revised non-benchmark image of position deviation y) has carried out the signal after the sum operation to the picture element signal of the pixel in the benchmark image y) with being positioned at coordinate.

Each non-benchmark image is carried out above-mentioned contraposition and synthetic the processing.Thus, obtain being synthesized image Dw ₁With the revised image Dw that is synthesized of position deviation ₂～Dw _MImage after synthesizing stores resulting image on the memory into as reference image Rw.In addition, also can from benchmark image, extract a plurality of distinctive zonules, utilize template matching method, the search a plurality of zonules corresponding from non-benchmark image with these a plurality of zonules, position according to a plurality of zonules that search out in the position of a plurality of zonules that extract in the benchmark image and the non-benchmark image, ask for above-mentioned geometric transformation parameter, carry out above-mentioned position deviation correction.

In step S230, generate with reference to behind the image Rw, in step S231, hand tremble correction portion 21 according to generated with reference to the correction object images Lw that obtains among image Rw and the step S222, enforcement is trembled correction at the hand of revising object images Lw, generates to have reduced the correction image Qw (describing modification method later in the tenth embodiment) that revises after the rocking among the object images Lw.This correction image Qw is recorded in recording medium 16 and is shown by display part 15.

As mentioned above by generating with reference to image Rw, even big hand is trembled or subject is rocked thereby produced in during the part in for example exposing photography in a plurality of short time, be not synthesized image yet, tremble correction so can carry out hand accurately because the short exposure image that obtains during this period can not become by the evaluation of estimate comparison operation.Have again, carry out contraposition and synthetic the generation by short exposure image with reference to image Rw to the M width of cloth, so it is not only equal with reference to the rolling momentum of the short exposure image of the rolling momentum of image Rw and 1 width of cloth, and synthetic by the pixel value sum operation, also can be with reference to the S/N of image Rw than the S/N of 1 width of cloth short exposure image than taller than (signal to noise ratio).Therefore, can realize that more high-precision hand trembles correction.Have again, by as step S225～S229, handling and cover storage short exposure image and being synthesized image, thereby the short exposure image can be suppressed to 1 width of cloth image part with the required memory span of memory, be suppressed to M width of cloth image part with the required memory span of memory being synthesized image.

[the 8th embodiment]

Then, the 8th embodiment is described.The 8th embodiment by switch to carry out the 6th embodiment with reference to image generating method (following 1 width of cloth selection mode that also is referred to as) and the 7th embodiment with reference to image formation mode (following several synthesis modes that also are referred to as), tremble correction thereby carry out hand.The S/N of this switching controls by inferring the short exposure image is than carrying out.Figure 29 is the flow chart that this hand that expression camera head 1b carries out is trembled the motion flow of correction.According to this flow chart, the action of camera head 1b is described.Have again, also with reference to Figure 30.Be arranged on light measuring circuit 22 and LUT (look-up table) 23 in the camera head 1b shown in Figure 30.

Under photograph mode, if press shutter release button 17a, then master control part 13 obtains lightness information from light measuring circuit 22, according to the pairing optimum exposure time of imaging apparatus (step S241 and S242) of this lightness information calculations image pickup part 11.Light measuring circuit 22 utilizes the output signal of photometry sensor (not shown) or imaging apparatus, measures the lightness incident light quantity of image pickup part 11 (in other words to) of subject.Lightness information is the information of this measurement result of expression.Then, in step S243, master control part 13 is according to optimum exposure time and predefined program line chart, actual time for exposure of decision (below be referred to as real exposure time).Store the list data of representation program line chart among the LUT23 in advance, if with lightness information input LUT23, according to this list data, from the amplification degree of LUT23 output real exposure time, f-number and AFE12.Master control part 13 decides real exposure time according to the output of LUT23.And then according to from the f-number of LUT23 output and the amplification degree of AFE12, regulation is the f-number (aperture of the aperture of image pickup part 11) in exposure photography and the short time exposure photography and the amplification degree of AFE12 usually.

In following step S244, the photography that exposes usually in the real exposure time that determines in by step S243 is stored in the common exposure image that generates thus on the memory as revising object images Lw.Wherein, real exposure time than the also short situation of optimum exposure time under, will each pixel value of common exposure image be multiply by certain value and the pixel value enlarged image that obtains stores on the memory as revising object images Lw in the mode of the compensation under-exposed part corresponding with the ratio of real exposure time and optimum exposure time.At this moment, also can carry out noise to the pixel value enlarged image as required and remove processing, will be stored on the memory as revising object images Lw except that the pixel value enlarged image behind the denoising.Noise is removed by utilizing linear filter (weighted average filter etc.) or nonlinear filter (median filter etc.) that the pixel value enlarged image is carried out filtering and is undertaken.

Then, in step S245, real exposure time when obtaining revising object images Lw and above-mentioned threshold value T _THCompare, at real exposure time less than threshold value T _THThe time, think to revise and do not contain (or containing extremely slightly) hand among the object images Lw and tremble rocking of being caused, do not carry out hand and tremble correction, finish the processing of Figure 29.

At real exposure time greater than threshold value T _THThe time, in step S246, the short exposure time Topt that master control part 13 is calculated based on optimum exposure time.In following step S247, the short exposure time Treal that master control part 13 is calculated based on real exposure time.So-called short exposure time is meant the time for exposure in the short time exposure photography.For example, short exposure time Topt is made as optimum exposure time 1/4 and short exposure time Treal is made as 1/4 of real exposure time.Then, in step S248, master control part 13 judges whether inequality " Treal＜Topt * kro " is set up.Coefficient k ro is the pre-determined factor that satisfies inequality " 0＜kro＜1 ", for example kro=0.8.

And, under the invalid situation of inequality " Treal＜Topt * kro ", because it is higher frequently to be inferred as the S/N of the short exposure image that short exposure time Treal planted agent obtains, therefore move to step S249, hand is trembled correction portion 21 and is adopted and can carry out 1 width of cloth selection mode that hand is trembled correction by fairly simple processing, generates with reference to image Rw.That is, in step S249, carry out each processing of step S205～S211 of Figure 27, generate with reference to image Rw.

On the other hand, under the situation that inequality " Treal＜Topt * kro " is set up, because it is lower to be inferred as the S/N of the short exposure image that short exposure time Treal planted agent obtains, so move to step S250, hand is trembled correction portion 21 and is adopted several synthesis modes that can reduce noise effect to generate with reference to image Rw.That is, in step S250, carry out each processing of step S225～S230 of Figure 28, generate with reference to image Rw.In addition, no matter in step S249, still in step S250, the time for exposure of the reality in the photography that will expose the short time all is made as Treal.

After in step S249 or step S250, generating with reference to image Rw, in step S251, hand tremble correction portion 21 according to this with reference to the correction object images Lw that obtains among image Rw and the step S244, generation correction image Qw (aftermentioned modification method in the tenth embodiment).This correction image Qw is recorded in the recording medium 16 and with display part 15 shows.

In order to suppress that dark place when photography hand is trembled or subject is rocked rocking of the image that causes, the mostly also short time for exposure of optimum exposure time that goes out according to the measurement result simple computation of light measuring circuit 22 with the beguine photography that exposes usually, after each pixel value of the image that obtains by this photography implemented to multiply by the processing (promptly improving sensitivity) of certain value, recording image data.In this case, inequality " Treal＜Topt * kro " is set up easily, and the S/N of obtained short exposure image is lower than very.Therefore, adopt several synthesis modes that can suppress noise effect to generate in this case with reference to image Rw.On the other hand, causing inequality " Treal＜Topt * kro " untenable because the peripheral illumination of camera head 1b is bigger, the S/N that is inferred as the short exposure image is than under the condition with higher, employing can be carried out 1 width of cloth selection mode that hand is trembled correction by fairly simple processing, generates with reference to image Rw.Like this,, switch generation method, thereby can tremble and do one's utmost to suppress to assess the cost when revising precision keeping hand with reference to image Rw by S/N ratio according to the short exposure image.What is called assesses the cost and is meant the load that calculating brings, and assesses the cost to increase to make processing time and power consumption increase.In addition, also can implement noise to the short exposure image and remove processing, the short exposure image of removing after the processing according to noise generates with reference to image RW, even but above-mentioned in this case switching controls also can play a role effectively.

[the 9th embodiment]

Then, the 9th embodiment is described.In the 9th embodiment, the evaluation of estimate K that utilizes in the processing to the 6th～the 8th embodiment _iComputational methods describe.Evaluation of estimate K _iBe to decide, that is: based on the first evaluation of estimate Ka of short exposure edge of image intensity according to the evaluation of estimate more than in following four evaluations of estimate any _iThe second evaluation of estimate Kb based on the contrast of short exposure image _iBased on the three evaluation of estimate Kc of short exposure image with respect to the rotation amount of revising object images Lw _iAnd based on the four evaluation of estimate Kd of short time exposure photography with the photography time difference of the photography that exposes usually _iAt first, to the first evaluation of estimate Ka _i～the four evaluation of estimate Kd _iComputational methods describe respectively.

(1) first evaluation of estimate Ka _iComputational methods

With reference to Figure 31 and Figure 32, to evaluation of estimate Ka as first evaluation of estimate _iComputational methods describe.Figure 31 is expression evaluation of estimate Ka _iThe flow chart of flow process of calculating action.Figure 32 is the figure of the relation of the expression image etc. that is used for this action.According to evaluation of estimate Ka _iCalculate evaluation of estimate K _iSituation under, in the step S226 of the step S206 of Figure 27 and Figure 28, carry out step S301～S305 of Figure 31.

At first, in step S301, whether judgment variable i is 1, be displaced downwardly to step S302 in the situation of i=1, and the situation in i ≠ 1 is displaced downwardly to step S303.In step S302, extraction is positioned at short exposure image C w _iCentral authorities or central authorities near the zonule, the image in this zonule is made as little image C s _iThe zonule of being extracted for example is made as the zonule of 128 * 128 pixels.Only under the situation of i=1, arrive step S302, therefore in step S302 from the 1st short exposure image C w ₁The little image C s of middle extraction ₁

Move to step S304 after the processing of step S302, in step S304, to little image C s _iThe enforcement edge extracting is handled, and obtains little image Es _iFor example pass through little image C s _iEach pixel be suitable for rim detection operator arbitrarily, thereby generate little image C s _iThe edge extracting image, with this edge extracting image as little image Es _iThen, in step S305, calculate little image Es _iThe summation of pixel value, this summation is made as evaluation of estimate Ka _i

Under the situation of i ≠ 1 from the step S303 that step S301 shifts, from short exposure image C w _i(≠ Cw ₁) in extract with from short exposure image C w ₁In the corresponding district territory, zonule that extracts, will be from short exposure image C w _iImage in the zonule that extracts is as little image C s _iThe search in respective cell territory is undertaken by the image processing of having utilized template matching method etc.Promptly for example, will be from short exposure image C w ₁In the little image C s that extracts ₁As template, adopt known template matching method, from short exposure image C w _iIn the search with this template the highest zonule of similar degree, with the image in the zonule that searches out as little image C s _iIn step S303, extract little image C s _iAfter, at this little image C s _iEach of implementation step S304 and S305 handled.By above-mentioned contents processing as can be known, evaluation of estimate Ka _iFollow little image C s _iEdge strength increase and increase.

If the image of same patterned, it is clear more that the hand that produces in then between exposure period is trembled more little edge, and the edge strength in the image is high more.Have again, because hand is trembled entire image is caused the same deterioration, so short exposure image C w _iEdge strength in the integral body and little image C s _iIn the edge strength correspondence.Therefore, above-mentioned evaluation of estimate Ka _iBig more, then can be inferred as the little image C s corresponding with it _iAnd short exposure image C w _iThe rolling momentum just more little.Be used to generate the as far as possible little viewpoint of rolling momentum from hope, utilize evaluation of estimate Ka with reference to the short exposure image of image _iBe useful.For example, also can be with evaluation of estimate Ka _iThe evaluation of estimate K that should ask among step S226 itself as the step S206 of Figure 27 and Figure 28 _i

In addition, general in order to ask for the rolling momentum of this image by 1 width of cloth image, it is disclosed to open flat 11-27574 communique as the spy, need carry out that image is carried out Fourier transform and generate the changing image on the frequency domain, instrumentation is trembled the frequency that decays owing to hand the big processing that assess the cost such as interval.With respect to this, if utilize the relation of edge strength and rolling momentum, infer the rolling momentum according to edge strength, then compare with existing methods such as utilizing Fourier transform, can reduce to infer assessing the cost that the rolling momentum uses.Have again, calculate evaluation of estimate by paying close attention to the little image that is extracted, rather than entire image, thereby the effect that assesses the cost can further be reduced.And then owing to adopt template matches etc., comparative evaluation value between the corresponding district territory is so even composition changes in a plurality of short time exposure photographies, the influence that this variation caused also is slight.

(2) second evaluation of estimate Kb _iComputational methods

With reference to the evaluation of estimate Kb of Figure 33 explanation as second evaluation of estimate _iComputational methods.Figure 33 is expression evaluation of estimate Kb _iThe flow chart of flow process of calculating action.According to evaluation of estimate Kb _iCalculate evaluation of estimate K _iSituation under, in the step S206 of Figure 27 and carry out step S311～S315 of Figure 33 among the step S226 of Figure 28.

Each contents processing of step S311 among Figure 33～S313 is identical with each contents processing of step S301～S303 among Figure 31 respectively, and therefore the repetitive description thereof will be omitted.Wherein be made as and after the processing of step S312 or S313, move to step S314.

In step S314, extract little image C s _iThe luminance signal of each pixel.Certainly, for example when i=1, extract little image C s ₁The luminance signal of each pixel, extract little image C s during i=2 ₂The luminance signal of each pixel.And, in step S315, generate little image C s _iThe histogram of brightness value (being the value of luminance signal), calculate this histogrammic dispersion variance, this dispersion variance is made as evaluation of estimate Kb _i

If the image of same patterned, the hand that produces in then between exposure period is trembled big more, and the brightness between neighbor is level and smooth more, and the ratio of the pixel of middle gray grade increases, and the distribution in the histogram of brightness value concentrates on the middle gray grade.Above-mentioned level and smooth degree is big more, and then the dispersion variance in the histogram is more little, evaluation of estimate Kb _iMore little, so evaluation of estimate Kb _iBig more, then can be inferred as corresponding therewith little image C s _iAnd short exposure image C w _iThe rolling momentum just more little.Be used to generate the as far as possible little viewpoint of rolling momentum from hope, utilize evaluation of estimate Kb with reference to the short exposure image of image _iBe useful.For example, also can be with evaluation of estimate Kb _iThe evaluation of estimate K that should ask among step S226 itself as the step S206 of Figure 27 and Figure 28 _i

As the example of short exposure image, short exposure image 261 shown in Figure 34 (a); And short exposure image 262 shown in Figure 34 (b).Short exposure image 261 is a distinct image, trembles and produced big hand between the exposure period of short exposure image 262, thereby contains big rocking in the short exposure image 262.Have, Figure 35 (a) illustrates respectively at short exposure image 261 and 262 histograms that generate in step S315 in reaching (b) again.With the histogrammic contrast of short exposure image 261 in (with reference to Figure 35 (a)), in the histogram (with reference to Figure 35 (b)) of short exposure image 262, can see that distribution concentrates to the middle gray grade.Owing to should concentrate, dispersion variance (and standard deviation) diminishes.

Pay close attention to image about certain, the dispersion variance in the histogram is little low corresponding to this contrast of paying close attention to image, and the dispersion variance in the histogram is paid close attention to the contrast height of image greatly corresponding to this.Therefore, in above-mentioned method, infer the contrast of paying close attention to image by the dispersion variance in the compute histograms, the contrast that goes out is by inference inferred the rolling momentum of paying close attention to image.And, with the guess value of contrast as evaluation of estimate Kb _iDerive.

In this evaluation calculation method, utilize the relation of contrast and rolling momentum, infer the rolling momentum according to contrast.Therefore, compare, can reduce to infer assessing the cost that the rolling momentum uses with the existing method of utilizing Fourier transform etc.Have again, calculate evaluation of estimate by paying close attention to the little image that is extracted, rather than entire image, thereby the effect that assesses the cost can further be reduced.And then owing to adopt template matches etc., comparative evaluation value between the corresponding district territory is so even composition changes in a plurality of short time exposure photographies, the influence that this variation caused also is slight.

(3) the 3rd evaluation of estimate Kc _iComputational methods

To evaluation of estimate Kc as the 3rd evaluation of estimate _iComputational methods describe.Evaluation of estimate Kc _iBe according to short exposure image C w _iCalculate with respect to the anglec of rotation of revising object images Lw.With reference to Figure 36, be described more specifically computational methods.

At first, from revise object images Lw, extract a plurality of distinctive zonules (for example zonule of 32 * 32 pixels).The implication of characteristic zonule and extracting method are as (even in other embodiment described later too) as described in the 7th embodiment.As shown in figure 36, from revise object images Lw, extract 2 zonules 281 and 282.Zonule 281 and 282 central point be reference marker 291 and 292 respectively.In example shown in Figure 36, the direction of line segment that connects central point 291 and 292 is consistent with the horizontal direction of correction object images Lw.

Then, from short exposure image C w _iIn extract and 2 zonules 281 that from correction object images Lw, extract and 282 2 corresponding zonules.The search in respective cell territory can be adopted and to have utilized the said method of template matching method etc. to carry out.Shown in Figure 36 from short exposure image C w ₁In 2 zonule 281a extracting and 282a and from short exposure image C w ₂In 2 the zonule 281b and the 282b that extract.Zonule 281a and 281b are corresponding to zonule 281; Zonule 282a and 282b are corresponding to zonule 282.Have, the central point of zonule 281a, 282a, 281b and 282b is reference marker 291a, 292a, 291b and 292b respectively again.

Calculating and short exposure image C w ₁Corresponding evaluation of estimate Kc ₁Situation under, ask for the line segment that connects central point 291a and 292a the anglec of rotation (being gradient) θ with respect to the line segment that connects central point 291 and 292 ₁Equally, calculating and short exposure image C w ₂Corresponding evaluation of estimate Kc ₂Situation under, ask for the line segment that connects central point 291b and 292b the anglec of rotation (being gradient) θ with respect to the line segment that connects central point 291 and 292 ₂At other short exposure image C w ₃～short exposure image C w _NAnglec of rotation θ ₃～θ _NAsk for too, as evaluation of estimate Kc _i, ask for anglec of rotation θ _iInverse.

Because different with the photography time that generates the short exposure image of usefulness with reference to image (photography constantly), so can produce the deviation of composition at both camera booth as the common exposure image of revising object images.Tremble detection in order to carry out high-precision hand, need carry out contraposition, so that eliminate correction object images that this deviation causes and with reference to the position deviation between the image.Though this contraposition can realize by coordinate transform (affine transformation etc.),, then can increase circuit scale and assess the cost if comprise the image rotation processing in this contraposition.Therefore, reduce, utilize evaluation of estimate Kc in order only to make the anglec of rotation that generates the short exposure image of usefulness at the reference image _iBe useful.For example, also can be with evaluation of estimate Kc _iThe evaluation of estimate K that should ask among step S226 itself as the step S206 of Figure 27 and Figure 28 _iLike this, the preferential use with respect to the little short exposure image of the anglec of rotation of revising object images Lw generates with reference to image Rw, therefore not only can obtain parallel mobile contraposition effect, can also obtain better hand and tremble correction effect, also help dwindling of circuit scale.

In addition, tremble under the situation of correction utilizing Fourier iterative method described later to carry out hand, by to revising object images Lw and carrying out with reference to image Rw carrying out linear operation (in the tenth embodiment, describing in detail) between image on the frequency domain that Fourier transform obtains.Under this situation, if revise object images Lw with reference to image Rw between have the deviation of direction of rotation, then on the characteristic of Fourier transform, hand is trembled and is detected and hand is trembled the correction precision and significantly descended.Therefore, tremble under the situation of correction utilizing the Fourier iterative method to carry out hand, by utilizing evaluation of estimate Kc _iSelect with reference to image Rw, hand is trembled detection and hand is trembled the correction precision thereby can increase substantially.

(4) the 4th evaluation of estimate Kd _iComputational methods

To evaluation of estimate Kd as the 4th evaluation of estimate _iComputational methods describe.Evaluation of estimate Kd _iFor revising object images Lw and short exposure image C w _iThe inverse of photography time difference.Revise object images Lw and short exposure image C w _iThe time for exposure of photography time difference when revising object images Lw photography in the middle of constantly and short exposure image C w _iThe time difference in the moment in the middle of between the exposure period during photography.After the photography of revising object images Lw according to short exposure image C w ₁, Cw ₂..., Cw _NThe order situation of photographing under, Kd certainly ₁＞Kd ₂＞...＞Kd _NSet up.

Revise object images Lw and short exposure image C w _iThe photography time difference big more, the probability of subject activity therebetween is just high more, and the probability that photography conditions such as illumination changes is also high more.The activity of subject or the variation of photography conditions act on and hand are trembled detect and hand is trembled and revised the direction that precision reduces.Therefore, according to preferential utilization and big evaluation of estimate Kd _iCorresponding short exposure image generates with reference to the mode of the Rw of image and utilizes evaluation of estimate Kd _iFor good.Like this, can alleviate the influence of the variation of the activity of subject or photography conditions, can carry out more high-precision hand and tremble and detect and hand is trembled correction.

(5) final evaluation of estimate K _iComputational methods

The evaluation of estimate K that should ask among the step S206 of Figure 27 and the step S226 of Figure 28 _iBy evaluation of estimate Ka _i, Kb _i, Kc _i, Kd _iIn any above evaluation of estimate decide.For example, calculate evaluation of estimate K by following formula (A-1) _iAt this, ka, kb, kc and kd be have 0 or on the occasion of the weight coefficient of regulation.Utilize evaluation of estimate Ka _i, Kb _i, Kc _iAnd Kd _iIn 2 or 3, calculate evaluation of estimate K _iSituation under, be 0 as long as make necessary weight coefficient.For example, do not considering to revise object images Lw and short exposure image C w _iThe situation of photography time difference under, as long as on the basis that makes " kd=0 ", calculate evaluation of estimate K _iGet final product.

Ki＝ka×Ka _i+kb×Kb _i+kc×Kc _i+kd×Kd _i ...(A-1)

In addition, as mentioned above, preferred only basis generates with reference to image Rw with the short exposure image that the photography time difference of revising object images Lw is lacked as far as possible, but is calculating evaluation of estimate K _iThe time, should utilize evaluation of estimate Kd auxiliaryly _iThat is, should not make weight coefficient ka, kb and kc all is 0.

[the tenth embodiment]

Then, the tenth embodiment is described.In the tenth embodiment, to describing based on correction object images Lw and with reference to modification method image Rw, that revise object images Lw.Be used to carry out the processing of this correction, in the step S251 of the step S231 of step S212, Figure 28 of Figure 27 and Figure 29, carry out.As the modification method of revising object images Lw, following illustration first～the 3rd modification method.First, second, third modification method is based on the modification method of image restoration mode, image synthesis mode, image sharpening mode respectively.

(1) first modification method

With reference to Figure 37, first modification method is described.Figure 37 is the flow chart of expression based on the flow process of the correcting process of first modification method.Under the situation that adopts first modification method, the processing of the step S251 of the step S212 of Figure 27, the step S231 of Figure 28 and Figure 29 is formed by respectively the handling of step S401～S409 of Figure 37 respectively.

At first, in step S401, from revise object images Lw, extract distinctive zonule (for example zonule of 128 * 128 pixels), the image in this zonule that extracts is stored on the memory as little image Ls.

Then, in step S402, the zonule of the zonule same coordinate of extracting from reference image Rw and extracting from revise object images Lw will be stored on the memory as little image Rs from the image in the zonule that reference image Rw extracts.Equate from the centre coordinate (centre coordinate of correction object images Lw) of the zonule that correction object images Lw extracts with from the centre coordinate (with reference to the centre coordinate of image Rw) of the zonule that reference image Rw extracts, revise object images Lw and also equate, so the picture size of two zonules is also equal with picture size with reference to image Rw.

Since shorter with reference to the time for exposure of image Rw, so the S/N of little image Rs is lower.Therefore, in step S403, little image Rs is carried out noise and remove processing.To be made as little image Rsa except that the little image Rs behind the denoising.Noise is removed by adopting linear filter (weighted average filter etc.) or nonlinear filter (median filter etc.) that little image Rsa is carried out filtering and is undertaken.Little image Rsa is low-light level, therefore in step S404, the brightness degree of little image Rsa is increased.Promptly, for example, so that the mode that the brightness degree of little image Rsa equates with the brightness degree of little image Ls (mode that the mean flow rate of the mean flow rate of little image Rsa and little image Ls equates), carry out the luminance standard processing that the brightness value to each pixel of little image Rsa multiply by steady state value.To make the little image Rsa after brightness degree increases be made as little image Rsb like this.

The little image Ls that obtains as mentioned above as the deterioration image, little image Rsb is handled (step S405) as initial restored image, is asked for PSF as the image deterioration function by implementing the Fourier iterative method in step S406.Based on the computational methods of the PSF of Fourier iterative method, described same with first execution mode.That is, in step S406,, thereby ask for corresponding PSF with little image Ls by the step S101～S103 of execution graph 4 and the processing of S110～S118 formation.Hand is trembled integral image is caused the same deterioration, and therefore the PSF that obtains at zonule Ls can be as the PSF at whole correction object images Lw.In addition, described in first execution mode, also can delete the processing of step S118, ask for final PSF with 1 time correcting process.

In step S407, ask for as each filter factor of image restoration filter in each key element of the inverse matrix of the PSF that step S406 is obtained.This image restoration filter is to obtain the filter that restored image is used according to the deterioration image.In fact, as described in first execution mode,, thereby can ask for each filter factor of image restoration filter by result in the computational process of directly utilizing the Fourier iterative method among the step S406.

After in step S407, obtaining each filter factor of image restoration filter, move to step S408, utilize this image restoration filter to carry out filtering (space filtering) revising object images Lw.That is, the image restoration filter that will have each filter factor of being obtained is applicable to each pixel of revising object images Lw, carries out filtering to revising object images Lw, can generate thus to remove or reduced and revised the filtering image that rocks that object images Lw is comprised.The size of image restoration filter is also littler than the picture size of revising object images Lw, but hand is trembled entire image is caused the same deterioration, therefore by this image restoration filter being applicable to whole correction object images Lw, thereby can remove rocking of whole correction object images Lw.

May contain the bell signal of following filtering in the filtering image.Therefore, in step S409, remove processing by filtering image being implemented remove the bell signal that this bell signal uses, thereby generate final correction image Qw.Because it is known removing the method for bell signal, the explanation that the Therefore, omited is detailed.As this method, for example as long as adopt the special method that the 2006-129236 communique is put down in writing of opening.

Revise object Qw and be and remove or reduced and revised the image that has rocked and removed or reduced the bell signal of following filtering that object images Lw is comprised.Wherein, because filtering image also is to remove or reduced the image that rocks, so also can be with filtering image as correction image Qw.

Because the rolling momentum that comprised with reference to image Rw is few, so its marginal element approaches not have the marginal element of the ideal image that hand trembles.Therefore, as mentioned above, will be made as initial restored image in the Fourier iterative method by this image that obtains with reference to image Rw.Thus, can obtain the described various effects of first execution mode (shortening the effect that hand is trembled the calculating treatmenting time of information (filter factor of PSF or image restoration filter)).

(2) second modification methods

Then, with reference to Figure 38 and Figure 39, second modification method is described.Figure 38 is the flow chart of expression based on the correcting process flow process of second modification method.Figure 39 is the concept map of this correcting process flow process of expression.Under the situation that adopts second modification method, the processing of the step S251 of the step S212 of Figure 27, the step S231 of Figure 28 and Figure 29 is made of respectively the handling of step S421～S425 of Figure 38 respectively.

The image that the photography of the image pickup part 11 by Figure 26 obtains is to comprise the information relevant with brightness and the coloured image of relevant information with color.Therefore, picture element signal that form to revise each pixel of object images Lw is made of the color signal of the color of the luminance signal of the brightness of remarked pixel and remarked pixel.At present, show the picture element signal of each pixel with the YUV form.Under this situation, color signal is become by two color difference signal U and V-arrangement.And picture element signal that form to revise each pixel of object images Lw is become by two the color difference signal U and the V-arrangement of the color of the brightness signal Y of the brightness of remarked pixel and remarked pixel.

Like this, as shown in figure 39, revising object images Lw can be decomposed into: the image Lw that only comprises brightness signal Y as picture element signal _YThe image Lw that only comprises color difference signal U as picture element signal _UThe image Lw that only comprises color difference signal V as picture element signal _VEqually, also can be decomposed into reference to image Rw: the image Rw that only comprises brightness signal Y as picture element signal _YThe image Rw that only comprises color difference signal U as picture element signal _UThe image Rw that only comprises color difference signal V as picture element signal _v(pictorial images Rw only among Figure 39 _Y).

In the step S421 of Figure 38, at first revise luminance signal and the color difference signal of object images Lw, thereby generate image Lw by extraction _Y, Lw _UAnd Lw _vIn following step S422,, thereby generate image Rw by the luminance signal of extraction with reference to image Rw _Y

Because image Rw _YBe low-light level, so in step S423, make image Rw _YBrightness degree increase.That is, for example, so that image Rw _YBrightness degree and image Lw _YMode (the image Rw that equates of brightness degree _YMean flow rate and image Lw _YThe mode that equates of mean flow rate), carry out image Rw _YThe luminance standard processing of multiply by steady state value of the brightness value of each pixel.And then, to the image Rw after this luminance standard processing _YEnforcement has adopted the noise of median filter etc. to remove processing.Luminance standard processing and noise are removed image Rw after the processing _YAs image Rw _Y' store on the memory.

Then, in step S424, by to image Lw _YPicture element signal and image Rw _Y' picture element signal compare, thereby computed image Lw _YWith image Rw _Y' between positional offset amount Δ D.Positional offset amount Δ D is the two dimension amount that comprises horizontal composition and vertical composition, shows as so-called motion vector.Can adopt known representative point matching method or template matching method, carry out the calculating of positional offset amount Δ D.For example, will be from image Lw _YIn image in the zonule that extracts as template, utilize template matching method, from image Rw _Y' in the search and the highest zonule of similar degree of this template.And, with position (the image Rw of the zonule that searches _Y' on the position) with from image Lw _YIn position (the image Lw of the zonule that extracts _YOn the position) bias calculate as positional offset amount Δ D.In addition, hope should be from image Lw _YThe zonule of extracting is made as above-mentioned distinctive zonule.

With image Lw _YConsider that as benchmark positional offset amount Δ D is image Rw _Y' with respect to image Lw _YPositional offset amount.Can be with image Rw _Y' be considered as with image Lw _YProduced the image of the position deviation of the amount that is equivalent to positional offset amount Δ D for benchmark.Therefore, in step S425, in the mode of offsetting this positional offset amount Δ D to image Rw _Y' implement coordinate transform (affine transformation etc.), thus to image Rw _Y' carry out the position deviation correction.Image Rw before the position deviation correction _Y' in the pixel that is positioned at coordinate (x+ Δ Dx, y+ Δ Dy), be transformed to by the position deviation correction and be positioned at coordinate (x, pixel y).Δ Dx and Δ Dy are respectively horizontal composition and the vertical composition of Δ D.

In step S425, and then to image Lw _UAnd image Lw _vWith the revised image Rw of position deviation _Y' synthesize, will export as correction image Qw by this image that is synthesized into.(x, the picture element signal of pixel y) is by to being positioned at coordinate (x, image Lw y) for the coordinate that is positioned in the correction image Qw _UIn pixel picture element signal and be positioned at coordinate (x, image Lw y) _vThe picture element signal of interior pixel constitutes.

In coloured image, rocking mainly of being seen cause by rocking of brightness, if the marginal element of the ideal image that the marginal element of brightness approaches not rock, then the observer can feel rock few.Therefore, in this modification method,, tremble correction effect thereby obtain doubtful hand by shaking fewer the synthesizing of momentum with reference to the luminance signal of image Rw and the color signal of correction object images Lw.According to this method,, in eyes, can rock few image with the low generation that assesses the cost though near the edge, produce misalignment.

(3) the 3rd modification methods

Then, with reference to Figure 40 and Figure 41, the 3rd modification method is described.Figure 40 is the flow chart of expression based on the correcting process flow process of the 3rd modification method.Figure 41 is the concept map of this correcting process flow process of expression.Under the situation that adopts the 3rd modification method, the processing of the step S251 of the step S212 of Figure 27, the step S231 of Figure 28 and Figure 29 is formed by respectively the handling of step S441～S447 of Figure 40 respectively.

At first, in step S441,, thereby generate little image Ls, in step S442, from reference image Rw, extract and little image Ls corresponding district territory, generate little image Rs thus by the distinctive zonule of extraction from revise object images Lw.The processing of this step S441 and S442 is identical with the processing of the step S401 of Figure 37 and S402.Then in step S443, implemented to adopt the noise of median filter etc. to remove processing to little image Rs, and then the brightness degree that makes noise remove the little image Rs after the processing increase.Promptly, for example so that the mode that the brightness degree of little image Rs equates with the brightness degree of little image Ls (mode that the mean flow rate of the mean flow rate of little image Rs and little image Ls equates) carries out the luminance standard processing that the brightness value to each pixel of little image Rs multiply by steady state value.With noise remove handle and the luminance standard processing after little image Rs store on the memory as little image Rs '.

Then, in step S444, little image Rs ' is carried out filtering, thereby generate 8 different level and smooth little image Rs of level and smooth degree by utilizing mutually different 8 kinds of smoothing filters _G1, Rs _G2..., Rs _G8At present, what use as 8 smoothing filters is mutually different 8 Gaussian filters (Gaussian filter), with σ ²Represent Gaussian Profile by the Gaussian filter performance.

Pay close attention to the one dimension image, during location of pixels in represent this one dimension image with x, known is σ by following formula (B-1) expression average out to 0 and dispersion variance ²Gaussian Profile (with reference to Figure 42).If this Gaussian Profile is applicable to Gaussian filter, then can use h _g(x) represent each filter factor of Gaussian filter.That is to say, when Gaussian filter is applicable to the pixel of position 0, with h _g(x) represent the filter factor at x place, position.In other words, with h _g(x) represent to carry out the influence degree of the pixel value of the pixel value of filtered position 0 position x corresponding, before the filtering with Gaussian filter.

$h_g\left(x\right)=\frac{1}{\sqrt{2\mathrm{\&pi;}}\mathrm{\&sigma;}}\exp (-\frac{x^2}{2{\mathrm{\&sigma;}}^2})\&CenterDot;\&CenterDot;\&CenterDot;(B-1)$

This idea is expanded to two dimension, so that (x, y) during the locations of pixels of expression in the two dimensional image, (B-2) represents two-dimentional Gaussian Profile by following formula.In addition, x and y represent the position of horizontal direction and the position of vertical direction respectively.If should the two dimension Gaussian Profile be applicable to Gaussian filter, then can be with h _g(x y) represents each filter factor of Gaussian filter, when Gaussian filter being applicable to the pixel of position (0,0), with h _g(x y) represents position (x, y) filter factor in.That is to say, with h _g(x y) represents to carry out with Gaussian filter position (x, the influence degree of pixel value y) the pixel value correspondence, before the filtering of filtered position (0,0).

$h_g(x,y)=\frac{1}{2\mathrm{\&pi;}{\mathrm{\&sigma;}}^2}\exp (-\frac{x^2+{\mathrm{<figure-callout\; id="2"\; label="y"\; filenames="S2008100030251E00071.png"\; state="\{\{state\}\}">y</figure-callout>}}^2}{2{\mathrm{\&sigma;}}^2})\&CenterDot;\&CenterDot;\&CenterDot;(B-2)$

In step S444,, adopt σ=1,3,5,7,9,11,13,15 Gaussian filter as 8 Gaussian filters.In following step S445, at little image Ls and level and smooth little image Rs _G1～Rs _G8Each between carry out images match, determine level and smooth little image Rs _G1～Rs _G8The level and smooth little image (promptly the highest level and smooth little image) of middle matching error minimum with the correlation of little image Ls.

Pay close attention to level and smooth little image Rs _G1, simple declaration contrasts little image Ls and level and smooth little image Rs _G1The time the computational methods of matching error (coupling residual error).Little image Ls and level and smooth little image Rs _G1Picture size identical, the pixel count of the horizontal direction of these images and the pixel count of vertical direction are made as M respectively _NAnd N _N(M _NAnd N _NBe the integer more than 2).Pass through V _Ls(x y) represents that (x, y) pixel value of the pixel in passes through V to the interior position of little image Ls _Rs(x y) represents little image Rs _G1(x, y) (wherein x and y satisfy 0≤x≤M to the pixel value of the pixel in interior position _N-1 and 0≤y≤N _N-1 integer).Like this, calculate the R of the SAD (Sum of Absolute Difference) between the expression contrast images according to following formula (B-3) _SAD, calculate the R that represents the SSD (Sum of Square Difference) between contrast images according to following formula (B-4) _SsD

$R_{\mathrm{SAD}}=\underset{y=0}{\overset{N_N-1}{\mathrm{\&Sigma;}}}\underset{x=0}{\overset{M_N-1}{\mathrm{\&Sigma;}}}|V_{\mathrm{Ls}}(x,y)-V_{\mathrm{Rs}}(x,y)|\&CenterDot;\&CenterDot;\&CenterDot;(B-3)$

$R_{\mathrm{SSD}}=\underset{y=0}{\overset{N_N-1}{\mathrm{\&Sigma;}}}\underset{x=0}{\overset{M_N-1}{\mathrm{\&Sigma;}}}\{V_{\mathrm{Ls}}(x,y)-V_{\mathrm{Rs}}(x,y){\}}^2\&CenterDot;\&CenterDot;\&CenterDot;(B-4)$

With this R _SADOr R _SsDBe made as little image Ls and level and smooth little image Rs _G1Between matching error.Equally, also ask for little image Ls and level and smooth little image Rs _G2～Rs _G8Each between matching error, determine the level and smooth little image of matching error minimum.At present, determine the level and smooth little image Rs corresponding with σ=5 _G3In step S445, will with level and smooth little image Rs _G3Corresponding σ is made as σ '.That is, the value of σ ' is 5.

In following step S446, by will handling, thereby realize removing the deterioration of revising object images Lw by the image deterioration function that the Gaussian Blur of this σ ' expression is revised the deterioration state of object images Lw as expression.

Specifically be in step S446,, to revise rocking of object images Lw thereby take out by being suitable for unsharp mask filter (unsharp mask filter) to revising object images Lw integral body according to σ '.Image before the suitable unsharp mask filter is made as input picture I _INPUT, the image behind the suitable unsharp mask filter is made as output image I _OUTPUT, the contents processing of unsharp mask filter is described.At first as the unsharp mask filter, adopt the Gaussian filter (being the Gaussian filter of σ=5) of σ ', the Gaussian filter by utilizing this σ ' is to input picture I _INPUTCarry out filtering, thereby generate blurred picture I _BLURThen, by from input picture I _INPUTEach pixel value in deduct blurred picture I _BLUREach pixel value, thereby generate input picture I _INPUTWith blurred picture I _BLURBetween difference image I _DELTAAt last, will be by to input picture I _INPUTEach pixel value add difference image I _DELTAEach pixel value and the image that obtains as output image I _OUTPUTAt input picture I shown in the formula (B-5) _INPUTWith output image I _OUTPUTRelational expression.In formula (B-5), (I _INPUTGauss) expression utilizes the Gaussian filter of σ ' to input picture I _INPUTCarry out the result of filtering.

I _OUTPUT＝I _INPUT+I _DELTA

＝I _INPUT+(I _INPUT-I _BLUR)

＝I _INPUT+(I _INPUT-(I _INPUT·Gauss) …(B-5)

In step S446, by revising object images Lw as input picture I _INPUTHandle, thereby can obtain as output image I _OUTPUTFiltering image.And, in step S447, generate correction image Qw (processing of step S447 is identical with the processing of the step S409 of Figure 37) after removing the bell signal of this filtering image.

By employing unsharp mask filter, thereby can emphasize input picture (I _INPUT) the edge, obtain the effect of image sharpening.Wherein, if generate image (I _BLUR) time the actual fuzzy quantity that fog-level and input picture comprised differ widely, then can't obtain suitable fuzzy correction effect.Output image (I _OUTPUT) become factitious extremely clearly image.On the other hand, if the fog-level when generating blurred picture is littler than actual fuzzy quantity, then the sharpening effect excessively a little less than.In this modification method, as the unsharp mask filter, adopt the Gaussian filter of having stipulated fog-level with σ, as the σ of this Gaussian filter, adopt the σ ' corresponding with the image deterioration coefficient.Therefore, best sharpening effect can be obtained, fuzzy correction image can be obtained removing well.That is, can generate with low assessing the cost and rock few image in the eyes.

Among Figure 43 with as input picture I _INPUTHand tremble image 300 and illustrate together: the image (correction image originally) 302 that obtains when adopting the Gaussian filter of best σ; The image 301 that obtains when adopting the Gaussian filter of too small σ; The image 303 that obtains when adopting the Gaussian filter of excessive σ.Hence one can see that: if σ is too small, then a little less than the sharpening effect, if the excessive factitious image that then can generate extreme sharpening of σ.

[the 11 embodiment]

In the 9th embodiment, to generating the short exposure image of usefulness and first～the 4th evaluation of estimate Ka that utilizes with reference to image in order to select _i, Kb _i, Kc _iAnd Kd _iComputational methods describe.Wherein, described: from short exposure image C w _iThe little image C s of middle extraction _i, according to little image C s _iEdge strength or contrast, infer and short exposure image C w _iWhole corresponding rolling momentum, calculate evaluation of estimate Ka thus _iAnd Kb _i(with reference to Figure 31 and Figure 33).At this moment, enumerated from short exposure image C w _iCentral authorities near extract little image C s _iExample, but might not be from short exposure image C w _iCentral authorities near extract little image C s.For example also can be so as described below.In addition, in order to specialize explanation, consider that the situation of N=5 promptly obtains 5 width of cloth short exposure image C w ₁～Cw ₅Situation.

At first, utilize BMA etc., ask for 2 width of cloth short exposure image C w of shooting continuous in time _I-1With short exposure image C w _iBetween light stream (optical flow).Figure 44 represents the example of the light stream obtained.Light stream is the motion vector between contrast images.Then, according to the light stream of obtaining, detect a series of short exposure image C w ₁～Cw ₅In little image extract with the zone.Little image extracts and is defined within each short exposure image with the zone.And, from short exposure image C w _iLittle image extract with extracting little image C s in the zone _i

For example, in the photography of 5 width of cloth short exposure images, in camera 1 basic fixed and be positioned near the photography zone central authorities personage under the situation of activity on the real space, on the position corresponding, can detect deliberate motion vector with this personage, and in occupying the most peripheral part of short exposure image, can't detect this motion vector.So-called deliberate motion vector is meant the motion vector with the above size of prescribed level, refers to that merely size is not 0 motion vector.Figure 44 represents light stream in this case.Under this situation, the zone that does not detect deliberate motion vector is the static subject zone of describing static subject on real space, should detect as little image extraction zone in static subject zone.The short exposure image C w of Figure 44 ₁～Cw ₅In extracted with the zone corresponding to detected little image by the intra-zone of dotted line.

In addition, for example in the photography of 5 width of cloth short exposure images, near the personage who is positioned at the photography zone central authorities is moving right on the real space and the framework (not shown) of camera head 1 is followed under the situation that this personage shakes to the right, as shown in figure 45, on the position corresponding, deliberate motion vector can't be detected, in occupying the most peripheral part of short exposure image (background parts), deliberate motion vector can be detected on the other hand with this personage.And the size of the deliberate motion vector of detected this and direction are homogeneous.Under this situation, can detect this deliberate motion vector the zone, be that mastery moving region in the image is extracted with the zone as little image and detected (result can detect with the same little image of situation shown in Figure 44 and extract with the zone).

In addition, for example in the photography of 5 width of cloth short exposure images all subjects be camera head 1 being under the static situation on the real space, in the whole zone of short exposure image, can't detect deliberate motion vector.Under this situation, the whole zone of each short exposure image is static subject zone, should detect with the zone as little image extraction in static subject zone.Have again, for example in the photography of 5 width of cloth short exposure images, whole subjects static on the real space and under the situation that framework camera head 1 is shaken to the right or camera head 1 static on the real space and under the situation that whole subjects are moved to the left equably, as shown in figure 46, in whole short exposure images, can detect the deliberate motion vector of size and direction equalization.Under this situation, be the mastery moving region, this mastery moving region extracted as little image detect with the zone with the whole region decision of short exposure image.

Like this, carry out statistical disposition, thereby can determine that little image extracts with the zone by a plurality of motion vectors to the formation light stream.

Perhaps, also can detect motion subject (personage etc.) movable on real space, the zone that this motion subject is not in is detected with the zone as little image extraction.If adopt tracking technique, then can detect the motion subject, the line trace of going forward side by side according to the output of the image pickup part 11 of the view data that comprises the short exposure image based on the known motion subject of image processing.

If in the zone of describing the motion subject of irregular activity in the photography zone, extract little image C s _i, according to this little image C s _iCalculate evaluation of estimate (Ka _iOr Kb _i), then this evaluation of estimate is subjected to the influence of the activity of motion subject, at little image C s _iAnd short exposure image C w _iThe supposition precision deterioration of rolling momentum.As a result, the selection of the short exposure image that the rolling momentum is little failure can't generate the suitable possibility with reference to image Rw and increase.Yet, as mentioned above, extract with the little image C s of extraction the zone with zone (static subject zone or mastery moving region) and from little image if detect little image extraction _iEven, short exposure image C w then _iIn contain the motion subject of irregular activity, the short exposure image that the momentum of also can correctly selecting to shake is few and generate suitable to image Rw.

Have again, even calculating based on short exposure image C w _iThe evaluation of estimate Kc of the anglec of rotation _iWhen (with reference to Figure 36) also from revise object images Lw, extract the zonule, but in order to avoid evaluation of estimate Kc _iBe subjected to the influence of the activity of motion subject, also can extract with extracting this zonule the zone from little image.Under this situation, as long as at revising object images Lw and 5 width of cloth short exposure image C w ₁～Cw ₅The a series of sequence photography image sets that constitute are asked for light stream as mentioned above like that, by a plurality of motion vectors that form this light stream are carried out statistical disposition, get final product with the zone thereby set little image extraction in revising object images Lw.

Variation or note item as at the 3rd execution mode below are designated as note 7 and note 8.The content of each note record only otherwise contradiction just can combination in any.Have again, for the 3rd execution mode, also can be suitable at the record content of the above-mentioned note 2～note 5 of first execution mode statement.

[note 7]

In the above-mentioned action of the 6th, the 7th and the 8th embodiment, after obtaining revising the common exposure photography that object images Lw uses, carry out N short time exposure photography, expose and photograph but can before exposure photography usually, carry out this short time of N time.Have again, also can before exposure is photographed usually, carry out Na time short time exposure photography and after exposure is photographed usually, carry out Nb time short time exposure photography, add up to the short time exposure of carrying out N time to photograph (wherein, N=Na+Nb)

[note 8]

For example can consider as follows.Comprise blur correcting device in the camera head 1b of Figure 26, it has: image is obtained the unit, and it is obtained as common exposure image of 1 width of cloth and the N width of cloth short exposure image of revising object images; With reference to image generation unit (second image generation unit), the method that it adopts the 6th, the 7th or the 8th embodiment to be put down in writing is generated with reference to image by N width of cloth short exposure image; The correcting process unit, the processing of the step S231 of step S212, Figure 28 of its execution Figure 27 or the step S251 of Figure 29 generates correction image.This blur correcting device is mainly trembled by hand that correction portion 21 forms or is trembled correction portion 21 and master control part 13 forms by hand.Particularly under the situation of the action that realizes the 8th embodiment, have: the selection processing unit of the processing of the step S249 of execution Figure 29 with reference to image generation unit (second image generation unit); The synthetic processing unit of the processing of the step S250 of execution Figure 29; With the branch process of step S248 by carrying out Figure 29, thus the either party's of execution in step S249 and S250 switch control unit.

Claims (24)

1. device for detecting rock, its output based on image unit detect rocking that first image that the photography by described image unit obtains comprised,

This device for detecting rock possesses the information generating unit of rocking, and it is based on described first image, with second image that the time for exposure also shorter than time for exposure of described first image photographed, and generates with described and rocks the corresponding information of rocking.

2. device for detecting rock according to claim 1 is characterized in that,

The described information of rocking is the image deterioration function that rocks of described first integral image of expression.

3. device for detecting rock according to claim 1 is characterized in that,

The described information generating unit of rocking possesses the extraction unit that extracts a part of image from described first image and second image respectively, generates the described information of rocking based on each a part of image.

4. device for detecting rock according to claim 2 is characterized in that,

The described information generating unit of rocking is according to first function that will obtain to the frequency domain based on the image transform of described first image and second function that will obtain to the frequency domain based on the image transform of described second image, temporarily ask for the described image deterioration function on the frequency domain, by the processing that the constraints of function utilization regulation that the described image deterioration functional transformation on the frequency domain of will be tried to achieve is obtained on the spatial domain is revised, finally try to achieve described image deterioration function.

5. device for detecting rock according to claim 1 is characterized in that,

The described information generating unit of rocking will and be handled as deterioration image and initial restored image respectively based on the image of described second image based on the image of described first image, utilize the described information of rocking of Fourier iteration Method.

6. device for detecting rock according to claim 5 is characterized in that,

The described information generating unit of rocking possesses the extraction unit that extracts a part of image from described first image and second image respectively, by generating described deterioration image and described initial restored image by each a part of image, thereby make each picture size of described deterioration image and described initial restored image less than the picture size of described first image.

7. device for detecting rock according to claim 1 is characterized in that,

Also possess holding unit, it keeps with image the demonstration based on the output of the described image unit before or after the photography of described first image,

The described information generating unit of rocking is continued to use with image described demonstration as described second image.

8. device for detecting rock according to claim 1 is characterized in that,

Also possess holding unit, it will keep as the 3rd image with image based on the demonstration of the output of the described image unit before or after the photography of described first image,

The described information generating unit of rocking generates the described information of rocking according to described first image, described second image and described the 3rd image.

9. device for detecting rock according to claim 8 is characterized in that,

The described information generating unit of rocking generates the 4th image by described second image and described the 3rd image are weighted sum operation, and generates the described information of rocking according to described first image and described the 4th image.

10. device for detecting rock according to claim 8 is characterized in that,

The described information generating unit of rocking comprises the selected cell that the either party in described second image and described the 3rd image is chosen as the 4th image, and generates the described information of rocking according to described first image and described the 4th image,

Described selected cell carries out selection between described second image and described the 3rd image based in the following condition at least one: each edge strength of described second image and described the 3rd image, described second image and each time for exposure of described the 3rd image and the external information that sets.

11. device for detecting rock according to claim 9 is characterized in that,

The described information generating unit of rocking will and be handled as deterioration image and initial restored image respectively based on the image of described the 4th image based on the image of described first image, utilize the Fourier iterative method to calculate the described information of rocking.

12. device for detecting rock according to claim 11 is characterized in that,

The described information generating unit of rocking possesses the extraction unit that extracts a part of image from described first image, second image and the 3rd image respectively, by generating described deterioration image and described initial restored image by each a part of image, thereby make each picture size of described deterioration image and described initial restored image less than the picture size of described first image.

13. a camera head wherein possesses:

The described device for detecting rock of claim 1; With

Described image unit.

14. one kind is rocked detection method, detects rocking that first image that the photography by described image unit obtains comprised based on the output of image unit,

Based on described first image, with second image of the time for exposure photography also shorter, generate with described and rock the corresponding information of rocking than time for exposure of described first image.

15. a blur correcting device, comprising:

Image is obtained the unit, and it obtains first image by the photography that has utilized image unit, and obtains several short exposure images by the time for exposure repeatedly photography also shorter than the time for exposure of described first image;

Second image generation unit, it generates 1 width of cloth image by described several short exposure images, with as second image; With

The correcting process unit, it is based on described first image and described second image, and rocking of being comprised of described first image revised.

16. blur correcting device according to claim 15 is characterized in that,

Described second image generation unit selects 1 width of cloth short exposure image based in the following condition at least one from described several short exposure images, with as described second image, described condition is: the contrast of each short exposure edge of image intensity, each short exposure image, and each short exposure image with respect to the anglec of rotation of described first image.

17. blur correcting device according to claim 16 is characterized in that,

Described second image generation unit also utilizes the photography time difference of each short exposure image and described first image to carry out the selection of described second image.

18. blur correcting device according to claim 15 is characterized in that,

Described second image generation unit is by synthesizing the short exposure image more than 2 width of cloth in described several short exposure images, thereby generates described second image.

19. blur correcting device according to claim 15 is characterized in that,

Described second image generation unit comprises:

Select processing unit, it comes to select 1 width of cloth short exposure image from described several short exposure images based in the following condition at least one, and described condition is: the contrast of described each short exposure edge of image intensity, each short exposure image, and each short exposure image with respect to the anglec of rotation of described first image;

Synthetic processing unit, the composograph after its generation is synthesized the above short exposure image of 2 width of cloth in described several short exposure images; With

Switch control unit, it plays a role by only making the either party in described selection processing unit and the described synthetic processing unit, thereby generates described 1 width of cloth short exposure image or described composograph is used as described second image;

Described switch control unit recently determine to make described selection processing unit and described synthetic processing unit based on the S/N of each short exposure image which side play a role.

20. blur correcting device according to claim 15 is characterized in that,

Described correcting process unit generates and rocks the corresponding information of rocking with the described of described first image, and based on the described information of rocking, described the rocking of described first image revised based on described first image and described second image.

21. blur correcting device according to claim 15 is characterized in that,

Described correcting process unit is by synthesizing the luminance signal of described second image and the color signal of described first image, thereby described the rocking of described first image revised.

22. blur correcting device according to claim 15 is characterized in that,

Described correcting process unit carries out sharpening by utilizing described second image to described first image, thereby described the rocking of described first image revised.

23. a camera head, comprising:

The described blur correcting device of claim 15; With

Described image unit.

24. one kind is rocked modification method, comprising:

Image is obtained step, obtains first image by the photography that has utilized image unit, and obtains several short exposure images by the time for exposure repeatedly photography also shorter than the time for exposure of described first image;

Second image generates step, generates 1 width of cloth image by described several short exposure images, with as second image; With

The correcting process step based on described first image and described second image, is revised rocking of being comprised of described first image.

Images