JP2009182576A - Image processing apparatus, imaging apparatus, printer and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a correction image whose blurring is sufficiently reduced in a short processing time. <P>SOLUTION: The image processing apparatus 104 has a processing means for receiving an original image acquired by an imaging apparatus and repeating image calculation processing for calculating a correction image whose burring components are reduced more than those of the original image so as to make the blurring components less, and a storing means 105 for storing information about blurring obtained by measuring blurring that occurs in the imaging apparatus or other imaging apparatus. The processing means performs image calculation processing on the basis of the information about the blurring stored in the storing means. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing technique for generating a still image in which a blur component is reduced from an image including a blur component caused by vibration of an imaging apparatus such as a digital camera (a frame image constituting a still image or a moving image). .

  When the imaging apparatus vibrates due to camera shake or the like within the exposure time of a frame image constituting a still image or a moving image, a captured image (hereinafter referred to as a blurred image) in which subject images shifted from each other are subjected to multiple exposure is generated.

  As a method of generating a corrected image with reduced blur components from such a blurred image, there are a method using an inverse filter, a Wiener inverse filter, and a Lucy-Richardson method. Furthermore, as a method improved from the Lucy-Richardson method, a method of using “deconvblind” (manufactured by MathWorks), which is a blur removal function using “MATLAB” (registered trademark), is also known.

  In the method using an inverse filter, f is a non-blurred image that does not include a blur component, h is a degradation process (information about blur), and g is an image that includes a blur component.

Assuming the corrected image

The

It is the method of calculating by.

  The method using the winner inverse filter is:

This is a method for calculating a corrected image.

The Lucy-Richardson method fixes the degradation process h,

Corrected image with maximum likelihood

It is a method of searching by iteratively calculating. Specifically, an index e indicating good correction (remaining amount of blur component)

Represented by The index e is a pre-correction image g including a blur component, and a corrected image.

Obtained by convolution of degradation process h

This is equivalent to the square of the difference (residual). And for each iteration operation using Jacobi method, steepest descent method, conjugate gradient method or gradient projection method

And the index (residual) e becomes less than the predetermined value.

Is the final corrected image.

In FIG.

An index that is a function of

Shows the concept of decreasing as the number of iterations i increases. Vertical axis is indicator

The horizontal axis represents the number of iterations i.

The residual

Change in the direction of decreasing,

When the value falls below the specified value

Is a corrected image.

FIG. 11 shows a flowchart of the iterative calculation by the Jacobi method. In step S1310, an uncorrected image (for example, a captured image) g in the equation (3)

And the deterioration process h is fixed (for example, all elements of h are set to 1). And the i-th corrected image from the image g

The residual when subtracting the image obtained by convolution with the degradation process h

Is calculated. i = 0, 1, 2, 3,...

In S1320, the residual calculated in S1310

Whether or not is less than or equal to a predetermined value, and the residual

If the value is equal to or smaller than the predetermined value, the process proceeds to S1340 to end the correction process.

Meanwhile, the residual

If is larger than the predetermined value, the process proceeds to S1330 and the correction process is continued.

In S1330

I-th corrected image

(I + 1) th corrected image added to

Is calculated.

In S1340, the residual

Corrected image when is below the specified value

Is output as a final corrected image.

  Details of the Lucy-Richardson method are described in Non-Patent Document 1. Details of the Jacobi method, steepest descent method, conjugate gradient method, and gradient projection method are described in Non-Patent Document 2.

The function “deconvblind” changes the deterioration process h,

Corrected image with maximum likelihood

This is a method of searching for the above by iterative calculation. The function “deconvblind” is a function for removing image blur using a blind deconvolution algorithm. The blind deconvolution algorithm is an algorithm that performs an iterative operation so as to maximize the likelihood of an image obtained by convolving a point spread function (PSF) with an image including assumed Poisson noise.

  Japanese Patent Application Laid-Open No. 2004-133867 discloses an imaging apparatus that previously holds camera shake states of photographers (adults, children, men, and women) as information relating to camera shake, and performs image shake correction using the information related to camera shake. It is disclosed. However, this imaging apparatus does not repeat the shake correction calculation.

  Note that Non-Patent Document 3 discloses a specific method for measuring the tendency of blurring occurring in an imaging apparatus. However, Non-Patent Document 3 does not mention correcting the image using the measured tendency of blurring.

JP 2006-238098 A B. Lucy: “An interactive method for the rectification of observed distributions”, Astron. J 79, pp. 745-754 (1974) “Introduction to Data Processing for Scientific Measurements Chapter 6 6.8 Iterative Solution and Square Error Sum Map p134-p137” (CQ Publisher, edited by Shigeo Minami, edited by Kei Ikeda) “3D measurement and quantification of camera shake” (Technical Report of IEICE Technical Report Pattern Recognition / Media Understanding January 19, 2007, p49-p54)

  In the case of using the inverse filter or the winner inverse filter, the deterioration process h needs to be known. That is, if the degradation process h is different from the actual degradation process h of the captured image, the expected corrected image cannot be calculated.

  Further, in the Lucy-Richardson method, since h of the captured image is unknown, iterative calculation is normally performed with all the elements of h as 1. Since iterative calculation is performed using the Jacobi method, the steepest descent method, the conjugate gradient method, and the gradient projection method, an image closer to an expected correction image than when using an inverse filter or a Wiener inverse filter can be obtained. However, since the degradation process h is different from h of the photographed image, as shown in FIG. 13, the residual is likely to fall into an extreme value by increasing the number of iterations i, and as a result, it is difficult to obtain a satisfactory corrected image. Become. Moreover, the processing time increases as the number of iterations i increases.

  In addition, the function “deconvblind” also performs iterative calculation with the initial value of the degradation process h as the initial value of the degradation process h, usually with all the elements of h as 1, as in the Lucy-Richardson method. Therefore, there is a problem similar to the Lucy-Richardson method.

  As described above, when the deterioration process h is unknown, there arises a problem that it is difficult to obtain a good corrected image and the processing time is increased.

  The present invention provides an image processing apparatus and an image processing method capable of obtaining a corrected image in which blurring is sufficiently reduced in a short processing time.

  An image processing apparatus according to one aspect of the present invention has an image calculation process in which an original image acquired by an imaging apparatus is input and a corrected image in which a blur component is reduced as compared with the original image is calculated. And a storage means for storing information relating to the shake obtained by measuring the shake generated in the image pickup apparatus or another image pickup apparatus. The processing means performs an image calculation process based on information relating to blur stored in the storage means.

  Note that an imaging apparatus and a printing apparatus including the image processing apparatus also constitute another aspect of the present invention.

  Furthermore, an image processing method according to another aspect of the present invention includes a step of inputting an original image acquired by an imaging device, and an image calculation process for calculating a corrected image in which a blur component is reduced from the original image. A processing step that is repeated so as to reduce the blur component, and a measurement step that measures blurring generated in the imaging device or another imaging device, and stores information relating to blurring obtained by the measurement in a storage unit. . In the processing step, an image calculation process is performed based on information relating to blur stored in the storage unit.

  According to the present invention, by inputting an original image including a blur component caused by the blur of the photographing apparatus, it is possible to obtain a corrected image in which the blur component is better reduced than the original image in a short time.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows the flow of an image processing method that is Embodiment 1 of the present invention.

  The image processing method of this embodiment is executed according to a computer program installed in a digital still camera or a video camera. According to the present embodiment, it is possible to reduce blurring of one frame image in a still image acquired by a digital still camera or a moving image acquired by a video camera and obtain a clear still image. The same applies to image processing methods of other embodiments described later.

  In S (step) 110 of FIG. 1, the deterioration process h is set based on the information (blurring information) indicating the tendency of blur stored in advance in the memory M as the storage unit. A method for generating information indicating the tendency of shaking will be described later.

  Next, in S120, a corrected image is obtained by performing a deconvolution operation (deconvolution) using an original image (hereinafter referred to as a captured image) g acquired by an imaging device (not shown) and the degradation process h set in S110. The image calculation process to be calculated is repeated. The deconvolution operation in this embodiment is also called blind deconvolution. The corrected image is an image in which the blur component is reduced compared to the captured image. A specific flow of the image calculation process is shown in FIG.

In FIG. 2, in S210, the image obtained by convolving the degradation process h with the i-th corrected image (i = 0, 1, 2, 3,...) Is subtracted from the photographed image g in the equation (3) described above. Residual when

Is calculated.

The i-th corrected image is a corrected image calculated by the i-th image calculation process among the repeated image calculation processes. Also residual

Corresponds to a blur component included (remaining) in the i-th corrected image. Residual error by repeating the image calculation process

Becomes smaller (less).

In the first image calculation process, the captured image g is used as the initial corrected image.

And The deterioration process h is fixed in the image calculation process that is repeatedly performed.

In S220, the residual calculated in S210

Is determined to be less than or equal to a predetermined value. Residual error

If is less than or equal to the predetermined value, the process proceeds to S240 and the residual

If is larger than the predetermined value, the process proceeds to S230.

In S230, the corrected image

Residual

Is

(I + 1) th corrected image by adding

Is calculated. In the first image calculation process, the initial corrected image

Residual

Is added to calculate the second corrected image. Then, the process returns to S210. Thus, the residual at S220

Until it is determined that is equal to or less than the predetermined value, the processes of S210 to S230 are repeated.

On the other hand, in S240, the residual

Corrected image when is below the specified value

Is output as a final corrected image.

  In S210 to S240, the repetitive calculation is performed by the Jacobi method, but the steepest descending method, the conjugate gradient method, or the gradient projection method may be used.

FIG. 3 shows the residual as the number of repeated image calculation processes (hereinafter referred to as the number of iterations) i increases.

And the residual

Corrected image when is below the specified value

Conceptually. In FIG. 3, the vertical axis is the residual

The number of iterations i is shown on the horizontal axis.

  Information indicating the blur generated in the imaging apparatus and the tendency of the blur (hereinafter, also simply referred to as a blur tendency) is measured and generated according to the flow shown in steps 1 to 6 below.

  Procedure 1: A plurality of measurement images are sequentially displayed on a monitor (display means) at predetermined time intervals. The plurality of measurement images are images having different patterns.

  Procedure 2: One still image imaging process (shutter longer than the predetermined time interval) of the imaging apparatus held by the user at a position away from the monitor by a plurality of measurement images displayed on the monitor in Procedure 1 (Second time). The imaging device used here is the same as the imaging device used for acquiring the original image described above. However, the measurement image may be captured using an imaging device (or other imaging device) different from the imaging device used to acquire the original image.

Step 3: The plurality of measurement images displayed on the monitor in step 1 are compared with the captured image acquired by one still image capturing process in step 2, and each measurement image ( Pan angle, Tilt angle, and Roll angle of (pattern) are calculated. Further, an index indicating the intensity of each pattern in each measurement image in the captured image (hereinafter referred to as the intensity of each measurement image) is calculated. Step 4: Pan angle of each measurement image calculated in Step 3; From the tilt angle and the roll angle, a change amount of each angle in time series order (hereinafter referred to as time series data of the change amount of each angle) is calculated.

  Procedure 5: The intensity of each measurement image calculated in Procedure 3 is arranged in time series, and the intensity of the second and subsequent measurement images (hereinafter referred to as intensity time-series data) is obtained.

  Procedure 6: The time series data of the change amount of each angle calculated in Procedure 4 and the time series data of the intensity obtained in Procedure 5 are associated in time series order. By converting the time-series data of the amount of change in each angle and the time-series data of the intensity, which are associated in time-series order in this way, into a two-dimensional coordinate trajectory, the blur when the user performs imaging with the imaging device Information (in other words, a habit related to the user's hand shake). Then, the tendency of blurring is stored in the memory M shown in FIG.

  Note that steps 1 to 6 may be performed either before or after the original image is captured by the imaging device.

  Here, FIG. 12 shows a specific example of a method for measuring the shake generated in the imaging apparatus in the procedure 1. This measurement method is described in Non-Patent Document 3 shown above.

  The user holds the camera at a position away from the monitor by a predetermined distance L (for example, 310 cm), and a plurality of (M) pattern images (measurements) are displayed on the monitor at predetermined time intervals (for example, 1 / M second: 1/60 second). Image) is taken at a shutter speed (1 / S seconds: 1/10 seconds, for example). Here, 1 / M seconds <1 / S seconds. As a result, a plurality of pattern images appear in an image captured by the camera (for example, one still image of 700 × 700 pixels). The plurality of pattern images are images having patterns that do not overlap each other, and have a size of, for example, 30 × 30 cm.

  Then, matching between the captured image and each pattern image is performed, and a Pan angle, a Tilt angle, and a Roll angle at each time point are calculated to obtain time-series data of the change amount of each angle. Further, the time series data of the intensity is obtained by using the degree of matching at each time point as the intensity of each measurement image. Furthermore, information indicating the tendency of blurring is generated from these data.

  Perform the measurement in steps 1 to 6 described above a plurality of times, calculate the representative value (center value or average value) of the Pan angle, Tilt angle, Roll angle, and intensity change amount due to shaking. The result of conversion into a locus may be stored in the memory M as a tendency of blurring.

Through the above processing, the corrected image in which the blur component is more favorably reduced than the original image g including the blur component due to the blur of the photographing apparatus.

Can be obtained in a short time.

  Note that the shake measurement method and the method for generating information indicating the tendency of shake described in the present embodiment are examples, and other measurement methods and generation methods may be employed. For example, a shake sensor (an angular velocity sensor, an acceleration sensor, or the like) mounted on the imaging apparatus may be used to measure the shake generated in the imaging apparatus, and information indicating the tendency of the shake may be generated based on the measurement result.

  An image processing method that is Embodiment 2 of the present invention will be described.

The image processing method of the present embodiment is basically the same process as S110 to S120 (see FIG. 1) of the first embodiment. However, the processing in S210 shown in FIG. That is, in this embodiment, in S210, the initial corrected image in the expression (3) is used.

Is an inverse filter,

The image is calculated using. The deterioration process h is a tendency of blurring, and g is a captured image (original image).

Also in the present embodiment, as in the first embodiment, a corrected image in which the blur component is reduced more favorably than the original image g including the blur component due to the blur of the photographing apparatus.

Can be obtained in a short time. This is the same in other embodiments described later.

  An image processing method that is Embodiment 3 of the present invention will be described.

The image processing method of the present embodiment is basically the same process as S110 to S120 (see FIG. 1) of the first embodiment. However, the processing in S210 shown in FIG. That is, in this embodiment, in S210, the initial corrected image in the expression (3) is used.

Is a winner inverse filter,

The image is calculated using. Deterioration process h is a tendency of blurring, and g is a photographed image (original image).

  An image processing method that is Embodiment 4 of the present invention will be described.

The image processing method of the present embodiment is basically the same process as S110 to S120 (see FIG. 1) of the first embodiment. However, the processing in S210 shown in FIG. That is, in this embodiment, in S210, the initial corrected image in the expression (3) is used.

Is an appropriate pattern image, for example, a flat image of one white color.

  An image processing method that is Embodiment 5 of the present invention will be described.

  The image processing method of the present embodiment is basically the same process as S110 to S120 (see FIG. 1) of the first embodiment. However, the processing corresponding to S220 and S230 shown in FIG.

In Example 1, the residual in S220

Corrected image when is below the specified value

Has been described as a corrected image to be output (S230). On the other hand, this example shows the residual

After the value becomes less than or equal to the predetermined value, the residual is repeated a predetermined number

An operation (image calculation process) is performed to make the image size smaller (reduce the blur component). And the residual

A plurality of corrected images having a value equal to or smaller than a predetermined value are set as candidates for a corrected image to be output.

FIG. 4 conceptually shows the image calculation processing in this embodiment. In FIG. 4, the vertical axis is the residual

The horizontal axis represents the number of iterations i. Figure 4 shows the residual

After the value becomes less than or equal to the specified value, the calculation is repeated two more times to obtain the residual

This shows a case in which three corrected images having a value equal to or smaller than a predetermined value are set as corrected image candidates.

The processing at S120 in this embodiment is shown in FIG. In S710, the residual when the image obtained by convolving the degradation process h with the i-th corrected image (i = 0, 1, 2, 3,...) Is subtracted from the photographed image g in the expression (3).

Is calculated. Also in this embodiment, the deterioration process h is fixed in the image calculation process that is repeatedly performed.

In S720, the residual calculated in S710

Is less than or equal to a predetermined value and the residual

It is determined whether or not the number of computations (number of times of image calculation processing) after the value becomes equal to or less than a predetermined value is equal to or less than the predetermined number. Residual error

Is less than or equal to the specified value and residual

If the number of computations after the value becomes equal to or less than the predetermined value has reached the predetermined number, the process proceeds to S740. Also residual

Is greater than a given value or residual

If the number of computations after the value becomes less than the predetermined value has not reached the predetermined number, the process proceeds to S730.

In S730, the corrected image

Residual

(I + 1) th corrected image by adding

Is calculated. Then, the process returns to S710. Thus, the residual at S720

Is determined to be less than or equal to a predetermined value and the residual is

The processing of S710 to S730 is repeated until the number of computations after the value becomes equal to or less than the predetermined value reaches the predetermined number.

On the other hand, in S740, the residual

As a candidate for a corrected image to be output, a corrected image at the time when becomes equal to or less than a predetermined value, and a corrected image obtained by a predetermined number of operations (image calculation processing) thereafter. Among these corrected image candidates, the most residual error

A corrected image with a small is output.

  In S710 to S740, the repetitive calculation is performed by the Jacobi method, but the steepest descending method, the conjugate gradient method, or the gradient projection method may be used.

  An image processing method that is Embodiment 6 of the present invention will be described.

The image processing method of this embodiment is basically the same flow as that of the fifth embodiment. However, the processing in S710 illustrated in FIG. That is, in this embodiment, in S710, the initial corrected image in the expression (3)

Is an inverse filter,

The image is calculated using. The deterioration process h is a tendency of blurring, and g is a captured image (original image).

  An image processing method that is Embodiment 7 of the present invention will be described.

The image processing method of this embodiment is basically the same flow as that of the fifth embodiment. However, the processing in S710 illustrated in FIG. That is, in this embodiment, in S710, the initial corrected image in the expression (3)

Is a winner inverse filter,

The image is calculated using. Deterioration process h is a tendency of blurring, and g is a photographed image (original image).

  An image processing method that is Embodiment 8 of the present invention will be described.

  The image processing method of this embodiment is basically the same flow as that of the fifth embodiment. However, the processing in S710 illustrated in FIG.

That is, in this embodiment, in S710, the initial corrected image in the expression (3)

Is an appropriate pattern image, for example, a flat image of one white color.

An image processing method that is Embodiment 9 of the present invention will be described with reference to FIGS. FIG. 6 shows the residual due to the corrected image changing as the number of iterations i increases.

Shows a decrease. The vertical axis in Fig. 6 is the residual

The horizontal axis represents the number of iterations i. FIG. 7 shows a decrease in the residual e (h) due to the deterioration process h changing as the number of iterations i increases. In FIG. 7, the vertical axis indicates the residual e (h), and the horizontal axis indicates the number of iterations i.

In this embodiment, as the first process, the deterioration process h is fixed to the first deterioration process (first filter) h1, and the residual is set.

The image calculation process described in the first to eighth embodiments is repeated until becomes smaller than the predetermined value 1. The first deterioration process h1 is determined on the basis of the blurring tendency, similarly to the deterioration process h described in the first to eighth embodiments. Thus, the residual

The first corrected image at the time when becomes a predetermined value 1 or less

Get.

Next, as a second process, the corrected image

The first corrected image

To fix. Then, the residual e (h) becomes smaller (the first corrected image in calculation).

The deterioration process h by the deconvolution operation is repeated until the blur component included in is less than or equal to a predetermined value A smaller than the predetermined value 1. In this way, a second deterioration process (second filter) h2 is obtained when the residual e (h) becomes equal to or less than the predetermined value A.

Further, as a third process, the deterioration process h is fixed to the second deterioration process h2, and a residual is obtained.

The image calculation process is repeated until becomes smaller than the predetermined value 2 which is smaller than the predetermined value 1. Thus, the residual

Second corrected image at the time when becomes a predetermined value 2 or less

Get.

  Thereafter, the same processing as the second and third processing is repeated, and the corrected image (when the residual e becomes equal to or smaller than a predetermined value (a value smaller than the predetermined value 2) corresponding to a good shake correction state ( Here, it is assumed that the corrected image is finally output).

  In the first to third processes, when calculating the corrected image or the degradation process, any of the Jacobi method, the steepest descent method, the conjugate gradient method, and the gradient projection method can be used.

  In this way, the blur component is reduced more satisfactorily by alternately performing the repeated calculation of the corrected image performed with the deterioration process fixed and the repeated calculation (estimation) of the deteriorated process performed with the corrected image fixed. A corrected image can be obtained in a short time.

An image processing method that is Embodiment 10 of the present invention will be described with reference to FIGS. FIG. 8 shows the residual due to the corrected image changing as the number of iterations i increases.

Shows a decrease. The vertical axis in FIG.

The horizontal axis represents the number of iterations i. FIG. 9 shows a decrease in the residual e (h) due to the deterioration process h changing as the number of iterations i increases. In FIG. 9, the vertical axis represents the residual e (h), and the horizontal axis represents the number of iterations i.

  In the present embodiment, the first to third processing (repetition of the second and third processing) in the ninth embodiment provides a residual by the j-th image calculation processing, that is, a residual equal to or less than a predetermined value. A j-th corrected image at the time is obtained. Thereafter, image calculation processing is performed a predetermined number of times (here, twice), and the j-th corrected image and two corrected images calculated thereafter are set as candidates for a corrected image to be output. A corrected image having the smallest residual among these three corrected image candidates is output.

  An image processing method that is Embodiment 11 of the present invention will be described. The image processing method according to the present embodiment is used particularly in an imaging apparatus or a personal computer to generate a corrected image in which a blur component is reduced from a captured image (original image) acquired by the imaging apparatus.

  In this embodiment, an area designation operation unit (area designation means) for designating a partial area of the entire captured image is provided in the imaging apparatus or personal computer. Perform the process described. The “partial region” here is a region of interest including the main subject. For example, a captured image obtained by so-called panning that captures a moving subject while rotating (panning or tilting) the imaging device. Corresponds to a region including the subject. The main subject is, for example, a person (face) that runs in an athletic meet or a car that runs in motor sports.

  By performing the processing described in the first to tenth embodiments on the attention area of the panning image, a good image with reduced blur is obtained for the attention area, while blurring indicating the background flow around the area. Since it remains, a panning effect can be secured.

  Also, when small characters such as those used in timetables and newspapers are blurred and difficult to read in the photographed image, make the characters you want to read the area of interest and reduce the blurring to make clear characters easier to read. Can do.

  In the first to eleventh embodiments described above, the case where the corrected image in which the blurring in the image due to the blurring of the imaging device is reduced is generated has been described. This is also effective when generating a corrected image in which image blur due to blur is reduced.

  FIG. 14 illustrates a configuration of an imaging apparatus that executes the image processing method described in the first to eleventh embodiments.

  In FIG. 14, reference numeral 101 denotes an imaging optical system that forms a subject image. Reference numeral 102 denotes an image sensor composed of a CCD sensor, a CMOS sensor, or the like, which photoelectrically converts a subject image. An image generation unit 103 generates a captured image (original image) based on an output from the image sensor 102. The imaging device 102 and the image generation unit 103 constitute an imaging system.

  An image processing unit (processing unit) 104 operates according to the computer program so that the image processing method described in the first to eleventh embodiments is applied to the captured image generated by the image generation unit 103.

  Reference numeral 105 denotes a memory (storage means) for storing information indicating the tendency of shaking, and corresponds to the memory M shown in FIG. Reference numeral 109 denotes a shake measurement unit, which measures the shake generated in the imaging apparatus by the method described in the first embodiment, and generates information indicating the tendency of the shake. Information indicating the tendency of shaking generated here is stored (held) in the memory 105.

  Reference numeral 106 denotes an area designation operation unit described in the eleventh embodiment. The user designates a part of the photographed image displayed on the display 107, which will be described later, in order to designate a part of the area to reduce blur by a touch panel method or a button operation method. Is operated by.

  The image processing unit 104, the memory 105, and the area designation operation unit 106 constitute an image processing apparatus.

  The display 107 displays the captured image and the corrected image output from the image processing unit 104. Reference numeral 108 denotes a recording medium composed of a semiconductor memory, an optical disk, and the like, and records the corrected image output from the image processing unit 104.

  Note that the image processing unit 104 (or the image generation unit 103) may be provided with a function of adding information indicating the tendency of blur read from the memory 105 to the captured image data and outputting the information to the recording medium 108. Information indicating the tendency of blur added to the photographed image is used when the photographed image is printed by a printing apparatus described later or processed by a personal computer.

  In addition to the captured image data, the imaging apparatus may be provided with a function of outputting information indicating blurring tendency stored in the memory 105 to the outside.

  FIG. 15 shows the configuration of a printing apparatus such as a printer that executes the image processing method described in the first to eleventh embodiments.

  In FIG. 15, reference numeral 201 denotes an image input unit, to which captured image data (still images and moving images) acquired by the imaging apparatus is input. The image input unit 201 is provided with a slot in which a recording medium in which captured image data is recorded is mounted, and a communication unit that captures captured image data by communication (wired and wireless) with the imaging apparatus.

  Information indicating the blurring tendency described in the twelfth embodiment may be added to the captured image data input via the image input unit 201. In addition, information indicating the tendency of blurring may be captured from the imaging apparatus separately from the captured image data via the image input unit 201.

  Reference numeral 202 denotes a memory that temporarily stores data of a captured image input via the image input unit 201 and information indicating the tendency of blurring. The memory 202 corresponds to the memory M shown in FIG.

  An image processing unit (processing unit) 203 operates according to the computer program so that the image processing method described in the first to eleventh embodiments is applied to the captured image read from the memory 202. At this time, information indicating the tendency of shaking is also read from the memory 202. When a moving image is input via the image input unit 201, the image processing unit 203 applies the image processing method described in the first to eleventh embodiments to a frame image constituting the moving image, A corrected image as a still image is output.

  Reference numeral 206 denotes a frame / area specifying operation unit, which is a frame image to be printed out of a captured image (moving image) displayed on a display (not shown), or to reduce blur in a captured image (frame image or still image). It is operated by the user to specify the area of the copy. As an operation method, a touch panel method, a button operation method, or the like is adopted.

The image processing unit 203, the memory 202, and the area designation operation unit 206 constitute an image processing apparatus.
A printing unit 207 prints the corrected image (frame image or still image) output from the image processing unit 203 on paper or a sheet.

  FIG. 16 shows a personal computer as an image processing apparatus that executes the image processing methods described in the first to eleventh embodiments.

  Information relating to the captured image and blur acquired by the imaging device 1401 is transmitted to the personal computer 1402. The transmission method may be either a cable method or a wireless method, and may be transmitted via the Internet or a LAN.

  Although not shown, the personal computer 1402 is provided with a storage unit that temporarily stores captured images received from the imaging device 1401 and information regarding blurring.

  Then, the personal computer 1402 generates the corrected image by executing the processes described in the first to eleventh embodiments. The corrected image is displayed on the display of the personal computer 1402, recorded on a hard disk in the personal computer 1402, or recorded on a recording medium attached to the personal computer 1402. Further, the corrected image is printed by a printer connected to the personal computer 1402.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

3 is a flowchart illustrating a procedure of an image processing method that is Embodiment 1 of the present invention. The flowchart which shows the procedure in S120 in FIG. FIG. 6 is a diagram showing a decrease in residual e in the first embodiment. The figure which shows the reduction | decrease of the residual e in Example 5 of this invention. 10 is a flowchart illustrating a procedure of an image processing method in Embodiment 5. The figure which shows the reduction | decrease of the residual e with the change of the correction | amendment image in Example 9 of this invention. The figure which shows the reduction | decrease of the residual e with the change of the degradation progress h in Example 9. FIG. The figure which shows the reduction | decrease of the residual e accompanying the change of the correction | amendment image in Example 10 of this invention. The figure which shows the reduction | decrease of the residual e with the change of the degradation progress h in Example 10. FIG. The figure which shows the reduction | decrease of the residual e in Lucy-Richardson method. The flowchart of the iterative calculation by Jacobi method. FIG. 3 is a diagram illustrating a shake measurement method according to the first embodiment. The figure which shows the change of the residual e in the conventional image processing method. FIG. 16 is a block diagram illustrating a configuration of an imaging apparatus that is Embodiment 12 of the present invention. FIG. 20 is a block diagram illustrating a configuration of a printing apparatus that is Embodiment 13 of the present invention. 18 is a diagram showing an image processing apparatus (personal computer) that is Embodiment 14 of the present invention. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 102 Image pick-up element 104,203 Image processing part 105,202 Memory 109 Blur measuring part 207 Printing part

Claims (9)

Processing means for repeating an image calculation process for calculating a corrected image in which an original image acquired by an imaging device is input and a blur component is reduced as compared with the original image so as to reduce the blur component;
Storage means for storing information relating to blurring obtained by measuring blurring occurring in the imaging device or another imaging device;
The image processing apparatus characterized in that the processing means performs the image calculation processing based on information on the blur stored in the storage means. An area designating unit for designating a part of the original image;
The image processing apparatus according to claim 1, wherein the processing unit performs the image calculation process on an area specified by the area specifying unit.   The said processing means calculates the said correction | amendment image by performing the deconvolution operation using the convolution filter determined based on the said original image and the information regarding the said blur in the said image calculation process, The said correction | amendment image is calculated. The image processing apparatus according to 1 or 2. The processing means includes
A first process of fixing the convolution filter to a first filter determined based on the information on blur and repeating the image calculation process so that the blur component is reduced;
A second process of fixing the corrected image to the first corrected image calculated in the first process and repeating the calculation of the convolution filter so that the blur component is reduced;
The third process is performed by fixing the convolution filter to the second filter calculated in the second process and repeating the image calculation process so as to reduce the blur component. The image processing apparatus according to 3. The measurement of the shake generated in the imaging device is performed by imaging a plurality of measurement images having different patterns sequentially displayed on the display unit by one still image imaging process of the imaging device,
5. The image processing according to claim 1, wherein the information on the blur is generated based on the plurality of measurement images and an image acquired by the still image capturing process. apparatus.   The image processing apparatus according to claim 5, wherein a blur generated in the imaging apparatus is measured a plurality of times, and a representative value of information relating to the blur generated for each measurement is used for the image calculation process. An imaging system that captures a subject image to generate an original image;
An imaging apparatus comprising: the image processing apparatus according to claim 1, wherein the original image is input. The image processing device according to any one of claims 1 to 4, wherein an original image acquired by an imaging device is input;
A printing apparatus comprising: a printing unit that prints an image output from the image processing apparatus. A step of inputting an original image acquired by an imaging device;
A processing step of repeating an image calculation process for calculating a corrected image in which a blur component is reduced from that of the original image so as to reduce the blur component; and
A measurement step of measuring shake occurring in the imaging device or other imaging device, and storing information on shake obtained by the measurement in a storage unit,
An image processing method characterized in that, in the processing step, the image calculation process is performed based on information on the blur stored in the storage unit.